Category Archives: EC2

How to access S3 Bucket from application on Amazon EC2 without access credentials

Assumptions

  1. You know the use of “AWS S3” and how to access the S3 bucket through the application with the help of Secret Key/Access Key
  2. In this Blog, We will use S3 Bucket – “parthicloud-test” as the bucket name where the static images like photos are stored for the application
  3. Developers usually use the Access Key/Secret Key for accessing the S3 Bucket in the application through SDK’s or AWS API.
  4. Managing the Access Key/Secret Key and keeping it secure becomes pain of the Developers and the Administrators

Use case

Developers want to Read/Write/List files in the “parthicloud-test” – S3 bucket programmatically from an EC2 instance without managing or configuring  the AWS secret key/Access Key.

Solution

We can use IAM role to manage temporary credentials for applications that run on an EC2 instance. When we use a role, we don’t have to distribute long-term credentials to an EC2 instance. Role supplies temporary permissions that application can use when they make API calls to S3 storage.

Advantages

  • Since role credentials are temporary and rotated automatically.
  • Developers don’t have to manage the credentials
  • We don’t have to worry about long-term security risks.
  • Flexibility to assign single role to multiple EC2 instances where application requires access to S3 storage
  • We can change the Role policy any time and the change is propagated automatically to all the instances.

Caution

  • IAM role cannot be assigned to an instance that is already running.
  • If we need to add a role to the running instance, We have the only option to create an image of the instance and then launch a new instance from the image with the desired role assigned.

How does it work?

Developer runs an application in EC2 instance that requires access to the S3 Bucket named “parthicloud-test”. AWS administrator creates the “ParthiCloud-S3” role. The role contains the policies that grant read/write/list permissions for the bucket.

When the application runs on the instance, it can use the role’s temporary credentials to access the parthicloud-test S3 bucket. AWS administrator doesn’t have to grant the developer permission to access the parthicloud-test bucket. The developer never has to share or manage credentials which is very risky in terms of security  compliance.

There is an other application running in EC2 instance which doesn’t have an IAM role attached. When the application in that instance tries to access the parthicloud-test bucket, Access will be denied because of secret Key/Access Key was missing. Refer the illustration below.

S3-Access-IAM-Role-EC2

Lets discuss the steps involved in detail from Creating VPC, Subnet, S3 Bucket, IAM Role and Policy, launching an Instance with IAM role and access to S3 bucket from an instance

Step 1 – Create VPC

Let’s create a VPC with a single subnet for the illustration purpose.

VPC

Step 2 – Create Key Pair

Create Key Pair by providing the friendly name. It will be used for accessing the instances using putty.

Key-Pair

<key pair name>.pem file will be downloaded. In our case it is ParthiCloud.pem. We need to have PuTTYgen to convert .PEM file to .PPK file.

PuttyGen

Click File–>Load Private Key and then Click Save private Key. We can save the .PPK in the desired location. We also have an option to provide password for the .PPK file. Password can be assigned in Key passphrase box as given in the above image.

 Step 3 – Create S3 Bucket

Create a S3 bucket named “parthicloud-test” in US Standard Region.

S3-bucket-create

Upload a test file – “Test.txt” in the S3 Bucket

S3-Upload

S3-Upload-1

Step 4 – Create IAM Policy and Role

Create policy to access S3 bucket. Select “Create Your Own Policy”

Policy-1

Enter Policy Name, Description and the Policy Document as given below

{
 "Version": "2012-10-17",
 "Statement": [
 {
 "Effect": "Allow",
 "Action": [
 "s3:ListBucket"
 ],
 "Resource": [
 "arn:aws:s3:::parthicloud-test"
 ]
 },
 {
 "Effect": "Allow",
 "Action": [
 "s3:PutObject",
 "s3:GetObject",
 "s3:DeleteObject",
 "s3:ListObject"
 ],
 "Resource": [
 "arn:aws:s3:::parthicloud-test/*"
 ]
 }
 ]
}

Policy-Create-1

Policy-Create-Success

 

Create Role by giving  the name

IAM_Role

Select Role Type as Amazon EC2

IAM_Role_Type

Then attach a policy – “ParthiCloud-S3-Policy”

Policy-Attach

Attach_Policy

Now IAM Role and Policy is ready. Let’s Launch the instance with IAM Role

Step 4 – Launching Instance

Launch a Ubuntu instance  – used Micro Instance for illustration.

Instance-Launch

Select the VPC, Subnet and IAM role which was created earlier.

Add Storage-

Storage

Tag Instance.  It will be very helpful during billing analysis.

Tag

Create Security Group. Don’t give 0.0.0.0/0, Instance can be accessed from anywhere which is not recommended.

SG-1

Click Review & Launch. It will ask you select the Key pair, select the previously created Key pair and click Launch Instances.

Select-KP

ParthiCloud instance is launched.

Access Instance

Give ubuntu@<Elastic IP> in the Host Name, attach the Private key in the “Auth” section to connect to the instance.

Putty

Access S3 bucket from Instance

We had already uploaded the file named Test.txt in the parthicloud-test S3 bucket. Type the below command to verify the access and list the files in the bucket. We have not specified Access Key / Secret Key in the instance.

$ aws s3 ls s3://parthicloud-test

S3-Access-Ubuntu

Let’s try to upload a file to S3 Bucket

$ aws s3 cp newfile.txt s3://parthicloud-test/ --region us-east-1

S3-Copy

$ aws s3 ls s3://parthicloud-test

S3-Newfile

The new file was successfully uploaded to S3 bucket.

We had discussed in detail on how to use IAM policy in EC2 Instance where the application is running, which requires access to the S3 bucket.

Share this: