Amazon S3

Fetch a config or secret blob from an S3 bucket (or any S3-compatible store: MinIO, Cloudflare R2).


Scheme	`s3://`
Module	`github.com/xavidop/mamori/providers/s3`
Sensitive	no (opt-in with `WithSensitive`)
Watch	poll (ETag)
Auth	default AWS credential chain (`WithRegion`)

Install

go get github.com/xavidop/mamori/providers/s3

import _ "github.com/xavidop/mamori/providers/s3"

Using the ref

An s3:// ref points at one object in a bucket, fetched with a single GetObject.

s3://<bucket>/<key>[#json-key]

Part	Required	What it means
`<bucket>`	yes	The S3 bucket name.
`<key>`	yes	The object key. It may contain slashes - everything after the bucket segment is the key.
`#json-key`	no	Treat the object as a JSON object and return one field of it.

Examples

s3://my-bucket/config/app.json fetches the whole object - decode it with flatten:"json".
s3://my-bucket/config/app.json#database returns just the database field of that JSON object.
s3://my-bucket/tls/app.crt fetches a raw blob (a certificate) - pair it with a []byte field.

type Config struct {
	AppConfig AppConfig `source:"s3://my-bucket/config/app.json" flatten:"json"`
	Cert      []byte    `source:"s3://my-bucket/tls/app.crt"`
}

The object key may contain slashes, so config/prod/app.json is a single key. Value.Version is the object ETag (or version id), so change detection is cheap: mamori compares the ETag before downloading again. Objects are not marked sensitive by default; because buckets often hold secret bundles (credential JSON, PEM chains, dotenv files), pass WithSensitive(true) to redact resolved values downstream.

Watch

mamori polls (WithPollInterval + jitter) using the ETag. For push, wire S3 Event Notifications to SQS/EventBridge and reload on demand.

Error classification

Failures are classified so mamori.ErrorKind can distinguish them:

S3 error code	mamori kind
`NoSuchKey`, `NoSuchBucket`, `NoSuchVersion`, `NotFound`	`not_found`
`AccessDenied`, `AllAccessDisabled`	`permission_denied`
`InvalidAccessKeyId`, `SignatureDoesNotMatch`, `ExpiredToken`, `InvalidToken`, `TokenRefreshRequired`	`unauthenticated`
`SlowDown`	`rate_limited`
`ServiceUnavailable`, `InternalError`	`unavailable`
`InvalidRequest`, `InvalidArgument`, `MalformedXML`	`invalid`
anything else	`unknown`

Codes not listed above report unknown rather than being guessed at. SlowDown and ServiceUnavailable share a 503 status but mean different things (throttling vs. overload), so they map to different kinds. NoSuchVersion is defensive: Resolve never sets GetObjectInput.VersionId, so this provider cannot actually trigger it today; it is included in case a future code path requests a specific object version. The original SDK error stays reachable with errors.As.

Configuration

import s3prov "github.com/xavidop/mamori/providers/s3"

mamori.WithProvider(s3prov.New(s3prov.WithRegion("eu-west-1")))
// S3-compatible (MinIO / R2):
mamori.WithProvider(s3prov.New(s3prov.WithEndpoint("https://<accountid>.r2.cloudflarestorage.com")))

Verified with an in-memory fake; live behavior is covered by //go:build integration tests.