S3 Auditor

AWS S3 Bucket Auditor: Automating Security Checks and Macie Classification

← GitHub Repository, please contribute or provide feedback.

The AWS S3 Bucket Auditor is a Go-based command-line tool that performs security audits on your AWS S3 buckets. This tool was built to simplify and automate multiple security checks, including detecting sensitive data using AWS Macie. It is a work in progress and will continue to evolve based on feedback and further development.

Project Overview

As a software engineer focused on cloud infrastructure, I’ve created this tool to automate the tedious process of manually auditing S3 buckets. With security being a major concern, especially when it comes to cloud storage, this tool makes it easier for developers and security teams to check for potential vulnerabilities, such as:

• Publicly accessible buckets
• Buckets without encryption
• Buckets without versioning enabled
• Presence of sensitive data using AWS Macie

I wanted to create something more streamlined than manually executing a list of AWS CLI commands, something that could perform all these checks at once, saving time and effort.

Features

Here are some of the key features of the AWS S3 Bucket Auditor:

• List S3 Buckets: Quickly retrieves and lists all S3 buckets in your AWS account.
• Public Access Check: Flags buckets that are publicly accessible and may pose a security risk.
• Encryption Status: Verifies whether server-side encryption is enabled on the bucket.
• Versioning Status: Checks if versioning is enabled for added data protection.
• Sensitive Data Detection: Leverages AWS Macie to check for any sensitive data present in your S3 buckets, such as personally identifiable information (PII) or financial data.

The goal is to consolidate security checks into one command-line tool, and eventually, make it more sophisticated with better error handling, logging, and reporting capabilities.

Why I Built This

As part of my journey to automate as much as possible in my work, I often find myself writing tools that help me simplify recurring tasks. The AWS S3 Bucket Auditor is one such project. While AWS CLI offers a lot of flexibility, it requires chaining together multiple commands to get a complete picture of a bucket’s security posture.

The need to perform audits regularly, especially for projects that handle sensitive data, inspired me to create this automated solution. Moreover, the tool integrates AWS Macie to detect sensitive data, which is a game-changer in terms of ensuring compliance with data privacy regulations such as GDPR or HIPAA.

It’s still a work in progress, but I hope to continue refining it and adding features to make it even more robust.

The Development Process

Developing this tool was quite interesting. Here’s a breakdown of the steps I took:

1. Initial Concept: The idea was to create a single Go-based command-line tool that could automate various checks on S3 buckets, including security and compliance checks.
2. Go SDK Integration: I integrated AWS SDK for Go to interact with S3 and Macie. This allowed the tool to list buckets, check their encryption status, public access settings, versioning, and trigger Macie classification jobs.
3. Concurrency for Performance: The tool leverages Go’s concurrency model to perform checks faster by running audits on multiple buckets simultaneously.
4. User Interface: I used libraries like promptui for a user-friendly command-line interface and fatih/color to add a bit of visual flair to the output.

How It Works

After installing and configuring the tool, you simply run it in the command line, select a bucket to audit, and the tool will handle the rest. It will perform checks and provide a clean report of its findings.

Example Workflow

Here’s how the interaction looks:

   _____   _____      ___                __    _    __
  / ___/  |__  /     /   |  __  __  ____/ /   (_)  / /_  ____    _____
  \__ \    /_ <     / /| | / / / / / __  /   / /  / __/ / __ \  / ___/
 ___/ /  ___/ /    / ___ |/ /_/ / / /_/ /   / /  / /_  / /_/ / / /
/____/  /____/    /_/  |_|\__,_/  \__,_/   /_/   \__/  \____/ /_/

Welcome to the AWS S3 Bucket Auditor!
✔ Audit a Bucket

Select an S3 bucket to audit...
✔ sens-data-bucket
Auditing bucket: sens-data-bucket

Performing security checks...
Auditing bucket: sens-data-bucket
🔍 Macie classification job created with Job ID: 38994cf77e0222d282417b8379597cce
✅ Macie job completed for bucket: sens-data-bucket                                               
🛑 Finding ID: b58b28a550e1aa4392917bd7876ed106
Details: {classification details...}

Returning to the main menu...

S3 Bucket Security Audit Report:
=====================================================================
Bucket Name      : sens-data-bucket
Region           : us-east-1
Public Access    : false
Encryption       : AES256
Versioning       : Disabled
Sensitive Data   : true
---------------------------------------------------------------------

What’s Next?

While the current version is functional, there are many improvements I plan to make in future iterations:

• Improved Logging: Add more robust logging and log files for error tracking and debugging.
• Enhanced Reporting: Generate reports in multiple formats like JSON or CSV for easier integration with other systems.
• Notifications: Integrate notifications (e.g., email or Slack) when sensitive data is found or when audits complete.
• UI Enhancements: Continue improving the command-line interface to make it even more intuitive and visually appealing.
• Expanded Checks: Add more security checks and integrate additional AWS services like GuardDuty or CloudTrail for a comprehensive audit.

Conclusion

Building the AWS S3 Bucket Auditor has been an exciting project. It streamlines security audits for S3 buckets, integrating powerful AWS services like Macie to ensure that no sensitive data is left unsecured. While it’s still a work in progress, I’m proud of what it’s achieved so far and excited for where it’s headed.

You can follow the progress of this project on my GitHub, and feel free to contribute or provide feedback. Stay tuned for more updates as the tool evolves!

GitHub Repository: AWS S3 Bucket Auditor