Lifecycle rules
Lifecycle rules delete data automatically when it's no longer needed, so you don't pay to store it forever. Typical uses: delete logs after 30 days, drop old backup versions after 90 days, and clean up uploads that never finished.
:::info API only, expiration only Lifecycle rules are set with the S3 API. They can delete data, filtered by prefix (folder). Moving data to another storage class and filtering by tag aren't supported. :::
What rules can do
| Rule | What it deletes |
|---|---|
Expiration (days or date) | Current objects older than the given age, or after a date. On a versioned bucket it only adds a delete marker. The data stays as an older version. |
NoncurrentVersionExpiration | Older versions, a set number of days after they were replaced or deleted |
ExpiredObjectDeleteMarker | Delete markers that no longer have any versions behind them |
AbortIncompleteMultipartUpload | Parts of uploads that weren't completed within a set number of days |
Apply rules
Write the rules to a file, for example lifecycle.json, then apply them:
cd ~
aws s3api put-bucket-lifecycle-configuration --bucket <bucket> \
--lifecycle-configuration file://lifecycle.json --profile bamboozle
This replaces all existing rules on the bucket. Put every rule you want in one file.
Check or remove the rules:
cd ~
aws s3api get-bucket-lifecycle-configuration --bucket <bucket> --profile bamboozle
aws s3api delete-bucket-lifecycle --bucket <bucket> --profile bamboozle
Examples
Expire objects after N days
Delete everything under logs/ 30 days after upload:
{
"Rules": [
{
"ID": "expire-logs-after-30-days",
"Status": "Enabled",
"Filter": { "Prefix": "logs/" },
"Expiration": { "Days": 30 }
}
]
}
Use "Prefix": "" to apply a rule to the whole bucket.
Expire old versions
On a versioned bucket, delete versions 90 days after they stop being the current version:
{
"Rules": [
{
"ID": "expire-noncurrent-after-90-days",
"Status": "Enabled",
"Filter": { "Prefix": "" },
"NoncurrentVersionExpiration": { "NoncurrentDays": 90 }
}
]
}
Abort incomplete multipart uploads
Remove leftover parts of uploads that weren't finished within 7 days:
{
"Rules": [
{
"ID": "abort-incomplete-uploads",
"Status": "Enabled",
"Filter": { "Prefix": "" },
"AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
}
]
}
Clean up expired delete markers
Remove delete markers once all versions behind them have been deleted, so listings stay clean:
{
"Rules": [
{
"ID": "remove-expired-delete-markers",
"Status": "Enabled",
"Filter": { "Prefix": "" },
"Expiration": { "ExpiredObjectDeleteMarker": true }
}
]
}
A complete set for a backup bucket
{
"Rules": [
{
"ID": "expire-noncurrent-after-90-days",
"Status": "Enabled",
"Filter": { "Prefix": "" },
"NoncurrentVersionExpiration": { "NoncurrentDays": 90 }
},
{
"ID": "remove-expired-delete-markers",
"Status": "Enabled",
"Filter": { "Prefix": "" },
"Expiration": { "ExpiredObjectDeleteMarker": true }
},
{
"ID": "abort-incomplete-uploads",
"Status": "Enabled",
"Filter": { "Prefix": "" },
"AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
}
]
}
Good to know
- Rules run in the background. Objects may be removed some time after they become due, not at an exact minute.
- Expired data is deleted permanently. Test a new rule on a test bucket or a narrow prefix first.
- Lifecycle rules don't delete versions that are still under Object Lock retention or legal hold. They are removed once the lock has ended.