FFmpeg x AWS Lambda : convert .ts files from .m3u8 playlist to .mp4

Lambda is probably not the best way to do this, given that transcoding is very compute intensive, and a lambda function would require a lot of compute and end up costing you more money than provisioned capacity.

To answer your question on being unable to read from S3. from your comment it looks like you are trying to treat lambda like it is a virtual machine, by simply calling the lambda function directly. This would require you to make your S3 bucket publically accessible. This is likely not the best way to do this with Lambda.

You don't mention what your Lambda run-time is, however, for sake of example, assuming this is Python. If you want to use ffmpeg in a lambda function consider using something like ffmpeg-python and putting that in your layer. This provides Python bindings for ffmpeg and will make it easier to use. You could then use the AWS Boto3 SDK for S3 to stream from S3 directly to ffmpeg - with Boto3 handling authentication and interaction with S3.

For this to work the Lambda execution role that your function is using needs to have permissions to be able to access S3 where the playlist files are stored. An example of such a permission policy is:

{
      "arn": "arn:aws:iam::123123123123:policy/lambda-limited-s3-access",
      "document": {
        "Version": "2012-10-17",
        "Statement": [
          {
            "Sid": "allowListBuckets",
            "Effect": "Allow",
            "Action": [
              "s3:ListAllMyBuckets"
            ],
            "Resource": "arn:aws:s3:::*"
          },
          {
            "Sid": "allowList",
            "Effect": "Allow",
            "Action": [
              "s3:List*",
              "s3:get*",
              "s3:put*"
            ],
            "Resource": [
              "arn:aws:s3:::mpeg-files",
              "arn:aws:s3:::mpegfiles/*"
            ]
          }
        ]
      }

Now moving beyond lambda, if you would like to do transcoding like this - I would suggest that you take a look at Elemental Media Convert in AWS. This is a hardware accelerated transcoding service for exactly this type of use-case, where you need to process millions of files.

You should download and parse the .ts files and convert each one like ffmpeg -i input.ts -c:v libx264 -c:a aac output.mp4

You should use a lambda function to read the m3u8 file and parse each input ts file. Trigger the transcoding using AWS Elemental MediaConvert (https://aws.amazon.com/mediaconvert/) to convert each .ts file into MP4. Or you can just use MediaConvert to use the .m3u8 manifest as the input and output a single MP4 from all the .ts segments

You probably have a permissions issue here - it's unlikely that ffmpeg is adding any credentials to the S3 request so it will be anonymous; and if the bucket and objects are not set up for public access then the request will be denied. If your bucket is set up to only allow authenticated access and the Lambda function has appropriate permissions then you need to find a way to add authentication to the S3 request.

One way of doing this is to use the inbuilt libraries for whatever language you're using in Lambda to download the file and then process it with ffmpeg. Of course, if you can get ffmpeg to sign the request using the appropriate AWS credentials then that will also work but I think that will be much harder than doing the download.