🚀 Mastering Amazon S3 Using the AWS Command Line Interface (CLI)

Step 1: Installing and Setting Up AWS CLI

To interact with AWS from your terminal, you need the AWS CLI installed.

✅ Installation

Windows / macOS / Linux: Use the official guide to install: AWS CLI Installation Docs

Quick Install for macOS/Linux:

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
        

Verify Installation:

aws --version
        

🔐 Step 2: Configuring AWS CLI Credentials

Before performing S3 operations, you need to configure your AWS credentials.

aws configure
        

You'll be prompted for:

• AWS Access Key ID
• AWS Secret Access Key
• Default region (e.g., us-east-1)
• Output format (json, text, or table)

These credentials are stored in ~/.aws/credentials and ~/.aws/config.

Amazon S3 CLI Operations

Once your CLI is set up, you can perform a variety of operations with S3. Below is a categorized list with examples for each.

1. Bucket Operations

Create a Bucket

aws s3 mb s3://my-bucket-name
        

Delete a Bucket

aws s3 rb s3://my-bucket-name
        

To Delete non empty Buckets

aws s3 rb s3://my-bucket-name --force
        

List Buckets

aws s3 ls
        

📂 2. File Upload & Download

Upload a File

aws s3 cp myfile.txt s3://my-bucket-name/
        

Download a File

aws s3 cp s3://my-bucket-name/myfile.txt ./local-folder/
        

Sync Local Directory with Bucket

aws s3 sync ./local-folder/ s3://my-bucket-name/
        

Sync Bucket to Local

aws s3 sync s3://my-bucket-name/ ./local-folder/
        

3. Listing Bucket Contents

List Files in a Bucket

aws s3 ls s3://my-bucket-name/
        

List Files Recursively with Timestamps and Sizes

aws s3 ls s3://my-bucket-name/ --recursive --human-readable --summarize
        

4. Delete Files & Objects

Delete a Single File

aws s3 rm s3://my-bucket-name/myfile.txt
        

Delete All Files in a Folder

aws s3 rm s3://my-bucket-name/my-folder/ --recursive
        

5. Move and Copy Files

Copy File Within Buckets or Between Buckets

aws s3 cp s3://source-bucket/file.txt s3://destination-bucket/file.txt
        

Move File

aws s3 mv s3://my-bucket-name/old-name.txt s3://my-bucket-name/new-name.txt
        

6. Manage Permissions & Public Access

Make a File Public

aws s3api put-object-acl --bucket my-bucket-name --key myfile.txt --acl public-read
        

Get Object ACL

aws s3api get-object-acl --bucket my-bucket-name --key myfile.txt
        

📦 7. Advanced: Using s3api for More Control

While aws s3 is simple, aws s3api provides full control over low-level S3 operations.

Create Bucket (API level)

aws s3api create-bucket --bucket my-bucket-name --region us-east-1
        

Get Bucket Policy

aws s3api get-bucket-policy --bucket my-bucket-name
        

Set Bucket Policy

aws s3api put-bucket-policy --bucket my-bucket-name --policy file://policy.json
        

🛠️ Use Cases in Real World

• DevOps: Automate backups and restores via shell scripts
• Data Engineering: Batch upload datasets to S3 buckets
• Security: Audit and update bucket policies from scripts
• CI/CD Pipelines: Push static files to S3 during deployment

🔚 Conclusion

Using the AWS CLI to manage S3 gives you precision, speed, and automation power. Whether you're a DevOps engineer, developer, or data scientist, mastering these commands will significantly boost your productivity.

👉 Have you used AWS CLI for automating S3 tasks? What are your favorite tricks or scripts?

Let’s connect and share knowledge! 💬

#AWS #S3 #CloudComputing #DevOps #AWSCLI #Automation #LinkedInLearning #CloudStorage