Why Does S3 Return 403 Instead of 404 When the Object Doesn’t Exist?

• With s3:ListBucket Permission: Amazon S3 returns an HTTP status code 404 (Not Found) if the object does not exist.
• Without s3:ListBucket Permission: Amazon S3 returns an HTTP status code 403 (Access Denied) if the object does not exist.

This distinction arises because the s3:ListBucket permission allows you to list the objects within a bucket. Without this permission, Amazon S3 cannot disclose whether the object does not exist or if you simply do not have access to it, thereby returning a 403 error to prevent information leakage.

The following steps illustrate how the s3:ListBucket permission affects the message S3 returns:

1. User IAM Policy with Only s3:GetObject Permission:

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Sid": "GetObject",
               "Effect": "Allow",
               "Action": "s3:GetObject",
               "Resource":"arn:aws:s3:::mybucketnamehere/*"
           }
       ]
   }

2. Bucket Contents: The S3 bucket contains only one object, "1.JPG".

3. Successful Object Retrieval: The user can successfully retrieve "1.JPG" using the AWS CLI command.

   $ aws s3 cp s3://mybucketnamehere/1.JPG .
   download: s3://mybucketnamehere/1.JPG to ./1.JPG

4. Attempt to Retrieve Non-Existent Object: S3 returns a 403 error when the user attempts to get "2.JPG", which does not exist.

   $ aws s3 cp s3://mybucketnamehere/2.JPG .
   fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden

5. Update User IAM Policy with s3:ListBucket Permission:

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Sid": "ListBucket",
               "Effect": "Allow",
               "Action": "s3:ListBucket",
               "Resource": "arn:aws:s3:::mybucketnamehere"
           },
           {
               "Sid": "GetObject",
               "Effect": "Allow",
               "Action": "s3:GetObject",
               "Resource":"arn:aws:s3:::mybucketnamehere/*"
           }
       ]
   }

6. Attempt to Retrieve Non-Existent Object Again: S3 returns a 404 error when the user attempts to get "2.JPG", which does not exist.

   $ aws s3 cp s3://mybucketnamehere/2.JPG .
   fatal error: An error occurred (404) when calling the HeadObject operation: Key "2.JPG" does not exist

By understanding and configuring permissions appropriately, you can control the type of error responses returned by Amazon S3, ensuring better security and clarity in your applications.

References:

• Amazon S3 GetObject API
• AWS CLI S3API Get-Object