This article discusses cloud storage buckets, highlighting potential consequences of bucket misconfiguration. It also demonstrates a real-world example of how easy it can be to accidentally enable public bucket access in a manner that is not obvious to the un-trained eye.
Cloud providers such as Amazon AWS, Google Cloud Platform and Microsoft Azure have seen phenomenal growth over the last five years, as many organisations make the transition to cloud from on-prem hosted services.
Cloud providers offer flexible, scalable and cost-effective services that allow their customers to provision a wide range of IT services at the simple click of a button. However, a simple click of a button can also have dramatic and undesirable outcomes – such as taking down business critical services or exposing sensitive data to the entire planet.
In general, cloud hosted services tend to default to a safe and secure mode of operation, however the configuration of many services is complex and it is easy for them to be mis-configured, either by accident or by users not fully understanding the knock-on effect of a configuration update.
Public S3 Buckets
Amazon’s Simple Storage Service (S3) is a cloud-based data storage service where data objects are stored in collections called ‘buckets’. Buckets can be locked down so that data within them is protected, or they can be configured (or maybe ‘mis-configured’) to permit public access to anyone on the internet. There are valid use-cases for both, for example companies may use bucket storage to hold personal details of customers - which requires strict privacy to be enforced. On the flip side, a company could use bucket storage to host static content for their website, which would naturally require public access.
There are many recent examples of data breaches as a result of incorrectly configured permissions on S3 buckets, for example this article covers how Sennheiser exposed the personal data of 28,000 customers via an incorrectly configured S3 bucket.
When creating an S3 bucket, AWS defaults to blocking all public access, and provides some clear and simple indications that this is the case. However, the flexibility and complexity of the AWS permissions system can mask the true configuration, and these simple indications can be very misleading – read on for an example.
Is this bucket private?
Here is an example: I have created an S3 bucket in Red Maple Technologies’ test AWS account that I want to be private – and left all security settings set at default to block all public access. The main ‘Buckets’ dashboard is shown below :
The label Bucket and objects not public is reassuring for our newly created private bucket.
Examining the content of the bucket, I have put the following files in it:
Let’s quickly double check the permissions on an individual file…
… all seems OK here, the group Everyone (public access) does not have permission to access the file.
Moving back and examining the permissions tab on the bucket itself, we find the following:
Reading the above, things are looking good for our ‘private’ bucket. The label Block all public access, with a confidence-inspiring green tick next to it, appears fairly plain and simple?
However, would you be surprised if I said that the contents of this bucket were actually publicly accessible?
If you do not believe me – take a look here…
Why is this bucket publicly accessible?
What is going on here? Why are the contents of my seemingly private bucket available for anyone on the planet to access? There is a clue in the URL posted above – CloudFront.
CloudFront is AWS’ Content Delivery Network (CDN) service. What I haven’t mentioned so far is that I have also configured a CloudFront distribution to proxy http requests to our private bucket. I had to explicitly go to the CloudFront services page and create the distribution and direct it to our private bucket. The config of the CloudFront distribution is shown below:
And if we look at the ‘Origins’ section, we can see that it is configured to proxy data from our private bucket:
And, if we look back at the Bucket Policy on our private bucket, we get a hint that something like this is going on:
And interestingly, there is even a banner saying Public access is blocked right above the policy that specifically grants permission to a service that inherently provides public access.
Although this might seem like a slightly contrived example, I should highlight that this was all configured with a few simple mouse clicks – I did not need to craft some special AWS IAM JSON magic to make this happen, the CloudFront config process very helpfully generated and inserted the bucket policy for me. This would make it very easy for someone to configure something like this without fully appreciating the consequences, or accidentally apply the CloudFront distribution to an incorrect bucket. The use of CloudFront to serve content from an S3 bucket is a common practice, and I often see this configuration in use by our customers, but I always feel fact that a bucket so clearly labelled as private when it is not could catch many people out.
Should you be worried?
Should you be worried?… I would say ‘maybe’. Hopefully the above highlights some of the complexities of configuring cloud services and the consequences that misconfiguration can have. It also highlights that some settings and indications can be very misleading – they do not always tell the full story.
For those of you that are worried, Red Maple Technologies are on hand to help. We are specialists in cloud provider infrastructure and offer a cloud hosting audit service to help organisations fully understand their architecture and discover potential issues like those highlighted in this article.
Want to find out more… then look at our services page, or get in touch.
Update
I raised this issue with AWS and they responded with the following…
Hello,
Thank you for bringing your security concern to our attention. We greatly appreciate and encourage reports from the security community worldwide.
We do not believe the behavior you describe in this report presents a security concern, rather, a feature request that our internal team is working on.
If you discover or become aware of other concerns specific to AWS products and services, please do not hesitate to contact us again.
Best regards
AWS Security
This response is good - it sounds as though the AWS team are looking to make improvements to the service to provide a more detailed assessment of public access on S3 storage buckets.
- What policies do I need in place as a small business?
- Getting Started with Microsoft Intune - Part 2
- Getting Started with Microsoft Intune - Part 1
|
Martin is an experienced technical consultant and engineering lead, having worked across the Defence, Government and Commercial sectors for the past 18 years. Martin has a strong background in Cyber Security, IT security architecture, systems engineering, software engineering and Agile development practices. |
