Buckets & Objects
Buckets :
I like to think of them as directories which can contain files in them and the files are known as "Objects" and directories are known as "Buckets".
Bucket name must be unique across the whole world and it can't have uppercases or underscores, etc.
Buckets need to be defined at which region they are, so buckets are regional thing. but you must know that S3 is global.
You can configure the bucket for website hosting and host a static file on S3 if you upload the website files. To access the website, you're bucket must be public otherwise you get a 403 (Forbidden) error.
Can enable versioning to create multiple versions of your file, to protect against accidental deletes and to roll back to a previous version of a file.
You can use S3 access logs to log all the access to S3 buckets and objects, so you can see causes of issues and any suspicious activity.
Can't change its region or name after creation.
Can't have buckets within buckets.
Objects :
Consists of a file and some metadata that describes the file.
Key is the full file path; it identifies the object within a bucket
Max size for an object is 5TB
Amazon S3 Security
User Based Policies
IAM Policies : -> Attached to an IAM user or group to allow or deny what the person can do with the S3 Bucket, e.g. someone might have write and read access and another person might have only read access or something like that.
Resource Based Policies :
Bucket Policy : -> JSON Based Policy ( similar to IAM policies) Contains :
Resources -> Which objects and buckets is this policy for.
Actions -> Which permission you want to allow or deny.
Effect -> Whether you want to "Allow" or "Deny" it.
Principle -> Who are you defining all of this for, e.g. an account, a user, a service, etc.
Define who can access the bucket -> You can grant public access with the bucket policy or grant another AWS account access to the bucket (Known as Cross Account Access).
ACL - Access Control Lists :
-> Much less common and is not recommended as much as bucket policies.
Encryption :
-> Can encrypt objects in S3 with encryption keys to ensure that you can read the data.
Amazon S3 Replication
You must enable versioning in the source bucket and the destination bucket otherwise Replication will not work.
The buckets used can be in different AWS accounts.
Can enable Cross Region Replication (CRR) or Same Region Replication (SRR) : -> CRR - Useful or lower latency access or replication across accounts. -> SRR - Useful for live replication between production and test account.
Amazon S3 Storage Classes
S3 Standard :
Used for general purpose storage of frequently accessed data, e.g. mobile and gaming apps, big data analysis, etc.
99.99% Availability.
Low latency and high throughput.
S3 Standard - Infrequent Access (IA) :
Used for data infrequently accessed but needed quickly when accessed, e.g. disaster recovery, backups, etc..
99.9% Availability.
Stores the data across multiple Availability Zones.
Cheaper than S3 Standard.
S3 One Zone - Infrequent Access (IA)
Basically, the same as STANDARD_IA but it only stores the data in one Availability Zone, e.g. it can be used for secondary backups or data that you can recreate.
99.5% Availability.
Less expensive than STANDARD_IA.
Data needs to be resilient to the failure of one Availability Zone.
S3 Intelligent-Tiering :
Used for people who want to be cost-optimized and their data access patterns are unknown/random.
99.9% Availability.
No retrieval fee.
S3 Glacier
Suitable for long-term use, e.g. backups or archives.
Takes very long to access the data stored : -> Expedited = 1-5 minutes. -> Standard = 3-5 hours. -> Bulk = 5-12 hours.
S3 Glacier Deep Archive :
Cheapest out of all the S3 options.
Also, takes an even longer time to access the data : -> Standard = 12 hours. -> Bulk = 48 hours.
S3 Lifecycle Rules :
-> Transition objects between S3 storage classes.
AWS Storage Gateway :
Hybrid Storage Service
Allows you to seamlessly bridge between on-premises data and cloud data ( to get the best of both worlds.)
AWS - Snow Family
Snowball Edge - for data transfers : -> Physical data transport solution : move TBs or PBs of data in or out of AWS. -> Alternative to moving data over the network -> Pay per data transfer job -> Provide block storage and Amazon S3-compatible objects storage. -> Snowball Edge Storage Optimized : 80TB of HDD capcity -> SnowBall Edge Compute Optimized : 42 TB of HDD capacity -> Use cases : large data cloud migrations, DC decommission, disaster recovery..
AWS Snowcone : -> Small, portable computing, Anywhere, rugged & secure, withstands harsh environments. -> Light (4.5 pounds, 2.1kg) -> Device used for edge computing, storage, and data transfer. -> 8TBs of usable storage. -> Use Snowcone where snowball does not fit. ->Must provide your own battery / cables. -> can be sent back to AWS offline, or connect it to internet and use AWS DataSync to send data.
AWS Snowmobile : -> Transfer exabytes of data ( 1 EB = 1,000 PB = 1,000,000 tbS° -> Each Snowmobile has 100PB of capacity (use multiple in //) ->High security : temperature controlled, GPS, 24/7 video surveillance. -> Better than Snowball if you transfer more than 10PB.
Amazon S3 - Summary
Buckets vs Objects -> Global unique name, tied to a region.
S3 Security -> IAM policy, S3 bucket policy ( public access ), S3 Encryption.
S3 Websites -> Host a static website on Amazon S3.
S3 Versioning -> Multiple Versions for files, Prevent accidental deletes.
S3 Access Logs -> Log requests made within your S3 Bucket.
S3 Replication -> Same-Region or Cross-Region, must enable versioning.
S3 Storage Classes -> Standard, IA, 1Z-IA, Intelligent, Glacier, Glacier Deep Archive.
S3 Lifecycle Rules -> Transition objects between classes.
S3 Glacier-Vault Lock / S3 Objects Lock -> WORM ( Write once Read Many )
Snow Family -> Import Data onto S3 through a physical device, EDGE computing.
OpsHub -> Desktop application to manage Snow Family devices.
Storage Gateway -> Hybrid Solution to extend on-premises storage to S3.
Last updated
