Skip to the content.

@tgmc/media-aws

AWS adapters and SQS worker for @tgmc/media.

Package README: packages/media-aws/README.md.

Role

Concern Owner
DynamoDB catalog / jobs / audit This package
S3 blobs under m/ on AssetsBucket This package
Presigned PUT ingest (reserved key) createPresignedIngest, PRESIGN_REQUIRED_HEADERS
SQS enqueue + worker (materialize jobs, S3 upload events, re-runs) This package (upload triggers)
CloudFront invalidation on delete/replace invalidateMediaPaths / MediaLibrary hooks
HTTP routes @tgmc/media-nuxt /api/media/* (Bearer ADMIN_TOKEN)
IaC infra/sam Media table, queue, worker, Nitro IAM

Requirements: docs/plans/2026-09-22-002-feat-media-aws-backend-plan.md.

HTTP API (Bearer admin)

Method Path Notes
POST /api/media/ingest/presign { kind, mime, filename, tenant?, preSanitized? } → reserves m/{tenant}/{kind}/{name}.{ext}, returns { assetId, path, name, expiresAt, uploadUrl, headers }; 409 name_conflict with a suggested name; 415 sanitize_required for every SVG (body route) and for PDF / MP4 unless preSanitized; 415 unsupported_type outside the kinds allowlist
POST /api/media/ingest/complete { assetId, path, kind, mime, filename?, meta, sha256?, preSanitized?, preSanitizedSig? } (an admin PDF / MP4’s flag is checked against the HMAC, see media-nuxt); path must be the key reserved for assetId (400 otherwise); sanitizes the object in place (If-Match) before the row is written
PATCH /api/media/assets/:id { meta?, tags? }, validated (400 invalid_meta)
POST /api/media/assets/:id/rename { filename } → new keys for the original and variants; old paths invalidated on CloudFront (afterRename)
GET /api/media/assets list (kind, tenant, tag, limit, offset)
GET /api/media/assets/:id asset + children + processing (on-ingest step status)
POST /api/media/assets/:id/process Re-run processing: queues { type: 'on-ingest', assetId } for the worker (enqueueOnIngestRecipes); idempotent
POST /api/media/assets/:id/materialize { role?, variant?, format?, recipe?, useDefaultThumb? } → enqueue (or sync locally); default is the t thumbnail
GET /api/media/jobs/:id job status
DELETE /api/media/assets/:id hard cascade + invalidation when configured (a failed purge is audited asset.invalidation_failed, not a 500; DELETE again retries it)

Keys and DynamoDB layout

Object keys are the catalog path (m/{tenant}/{kind}/{name}.{ext}, variants …@{variant}.{ext}); toS3Key is the identity for m/ keys and toCloudFrontPath is / + key. See media for the scheme.

Item Key
Asset pk = ASSET#{id}, sk = META; gsi1pk = T#pf#K#i (tenant + kind code), gsi2pk = T#pf#ALL
Child pk = ASSET#{id}, sk = CHILD#{childId} (+ CHILDLOOKUP#{childId})
Name / reservation pk = NAME#m/{tenant}/{kind}/{name}, sk = META → { assetId, path, pending, expiresAt?, ttl? }
Usage row pk = USE#{assetId}, sk = PAGE#{pageType}#{pageId} → the MediaUsageRecord fields (no GSI)
Page mirror pk = PAGEUSE#{tenant code}#{pageType}#{pageId}, sk = USE#{assetId}; gsi2pk = T#pf#USAGE, gsi2sk = {pageType}#{pageId}#{assetId}
Usage counter pk = USE#{assetId}, sk = COUNT → { n, rev, tenant }; gsi2pk = T#pf#USAGECOUNT, gsi2sk = {assetId}
Page head pk = PAGEUSE#{tenant code}#{pageType}#{pageId}, sk = HEAD → { rev } (no GSI)

The NAME# key has no extension, so hero.png and hero.jpg share one item and the second is a 409 name_conflict with a -2 suggestion (the suggestion search reads NAME# items, up to 1000 consistent reads in the worst case). insertAsset writes the asset and its NAME# item in one transaction (attribute_not_exists on both; the name item may also be the asset’s own pending reservation or another asset’s expired one). A pending reservation carries expiresAt and a numeric epoch-seconds ttl; the completed claim is permanent (pending = false) and carries neither.

DynamoDB TTL. Enable TTL on attribute ttl for the media table so lapsed reservations are deleted (infra/sam/template.yaml sets TimeToLiveSpecification: { AttributeName: ttl, Enabled: true } on MediaTable; docker/dynamodb-init.sh enables it on the docker:test Media table too). TTL deletion is lazy (up to ~48 h), so reads still compare expiresAt and treat an expired reservation as free. The store’s deleteAsset removes both in one transaction, so a deleted name is reusable at once; deleteAssetCascade(id, { holdNameUntil }) (what MediaLibrary.deleteAsset calls) instead replaces the NAME# item, in the same transaction, with a pending discarding claim of <id>#delete (with ttl) until the objects are deleted (media § Name reservation). listAssets queries the tenant-scoped GSI partitions.

Usage (DynamoUsageStore, src/dynamo-usage.ts; createDynamoMediaStores returns it as usage and createAwsMediaLibrary passes it to MediaLibrary). It lives in MediaTable with the items above: no new table, no IAM change. Fields are stored as attributes and the keys are never parsed (keys.ts: usagePk, usageSk, USAGE_COUNT_SK, pageUsagePk, pageUsageSk, PAGE_USAGE_HEAD_SK, usageGsiPk, usageCountGsiPk). See media § Usage list and delete safeguards for the port.

Presign and S3 headers

createPresignedIngest({ path, assetId, mime, bucket, byteSize? }) signs a PUT for an already reserved key, at its private pending key pending/<assetId>/<path> (s3Key; media § Name reservation).

Clients. S3BlobStore, createPresignedIngest, invalidateMediaPaths and the SQS helpers share one client per configuration for the process (sharedS3Client, sharedCloudFrontClient, created on first use). Every media AWS client, the DynamoDB document client of createDynamoMediaStores included, starts from mediaClientConfig(region, endpoint) in src/client-config.ts (no SDK import, so a DynamoDB- or SQS-only path does not load the S3 and CloudFront clients), which applies MEDIA_AWS_CLIENT_DEFAULTS: maxAttempts: 3, a 3 s connection timeout and a 30 s socket idle timeout, so a stalled endpoint fails the request instead of hanging it. Pass client to use your own. With byteSize it also signs content-length (ContentLength, kept in the signed headers), so S3 refuses a body of any other length, and echoes byteSize in the result; the uploader must send exactly that many bytes. Without it no length is signed (the unsized expand phase, see media-nuxt § Upload size cap). Content-Length is not in PRESIGN_REQUIRED_HEADERS: browsers and fetch set it from the body. PRESIGN_REQUIRED_HEADERS (Content-Type, If-None-Match, Cache-Control, Content-Disposition) is the single list of headers the uploader must send, each with exactly the value returned in headers (they are signed into the URL); the bucket CORS AllowedHeaders must cover it, which scripts/sam-template.test.js checks by importing packages/media-aws/src/presign-headers.ts directly (Node type stripping, no build step; the file has no imports on purpose).

S3BlobStore.put sets ContentType, CacheControl and ContentDisposition when given (server-side writes: ingest and variants). Presigned uploads carry the same headers (Cache-Control: public, max-age=31536000, immutable, Content-Disposition: inline; filename="<name>.<ext>", the asset mime), so browser and admin remote-forward uploads match server-side writes.

put(key, data, { ifMatch }) sends IfMatch (instead of IfNoneMatch: '*' for createOnly): upload sanitizing rewrites a presigned object at its own key only if it is still the object it read; 412, 409 or 404 then throw MediaBlobChangedError, which upload complete reports as a 409 conflict. createAwsMediaLibrary({ sanitize }) takes the sanitizer; the web server passes the raster one (upload sanitizing) and the MediaWorker none, so svgo, pdf-lib and ffmpeg never ship in the worker (tests/worker-boundary.spec.ts walks the worker’s static and dynamic imports through the workspace packages and fails if any of them, or @tgmc/image/sanitize, is reachable).

copy(from, to, { createOnly, ...headers }) is a server-side CopyObject with the new object headers (MetadataDirective: REPLACE); createOnly sends If-None-Match: * on the CopyObject, so S3 refuses atomically to write over an existing target (a 412 / 409 becomes MediaBlobExistsError); a HeadObject first only skips the copy when the target is plainly there. Renames therefore never stream objects through the Lambda. getPrefix(key, length) is a ranged GetObject (an empty object answers 416, read as empty bytes), used by the upload markup check.

Variant replacement, rename and delete call invalidateMediaPaths (/m/… paths) when ASSETS_DISTRIBUTION_ID is set. IAM for the worker and Nitro grants m/* (and s3:ListBucket on prefix m/*, m/) in infra/sam/template.yaml.

Endpoint overrides (DynamoDB Local + MinIO)

S3BlobStoreOptions, DynamoMediaStoresOptions (endpoint), the presigner and createAwsMediaLibrary (dynamoEndpoint, s3Endpoint, s3ForcePathStyle) accept custom endpoints; forcePathStyle is needed for MinIO. createPresignedIngest also takes publicEndpoint: the URL is signed against the host the uploader can reach (http://localhost:4420 in docker:test), not the server’s own http://minio:9000. mediaAwsEndpointsFromEnv() maps AWS_ENDPOINT_URL_DYNAMODB, MEDIA_S3_ENDPOINT / AWS_ENDPOINT_URL_S3, MEDIA_S3_FORCE_PATH_STYLE=1 and MEDIA_S3_PUBLIC_ENDPOINT; the SQS worker uses it too. See media-nuxt and docker. scripts/media-smoke.js (npm run docker:test:smoke, CD with MEDIA_SMOKE_ENABLED=1) exercises presign → PUT → complete → on-ingest processing → materialize → delete end to end.

Upload triggers

On SAM stacks the on-ingest recipes (media § On-ingest recipes) run in the media worker, started by S3 rather than by complete (one owner; see media-nuxt § On-ingest processing).

Transfer (seed and backup)

transferMedia from @tgmc/media works on these adapters. createDynamoMediaStores plus S3BlobStore form one MediaEndpoint, and S3BlobStore.digestSha256 re-hashes each copied object. insertAsset writes the NAME# claim in the same transaction as the asset, and objects get the immutable cache and inline disposition headers. npm run media:transfer (scripts/media-transfer.js) wires aws:<table>@<bucket> endpoints with the same --dynamodb-endpoint / --s3-endpoint / --path-style overrides. See media § Transfer between backends and the production runbook.

Env (Lambda)

MEDIA_TABLE / NUXT_MEDIA_TABLE, ASSETS_BUCKET, MEDIA_QUEUE_URL, ASSETS_DISTRIBUTION_ID, ADMIN_TOKEN, NUXT_AWS_REGION.

When the data backend is not dynamodb (host npm run dev, or DATA_BACKEND=sqlite|postgres), media uses SQLite/FS under MEDIA_LOCAL_ROOT (default .data/media; sync materialize) even if the table and bucket are set, and presign / complete switch to local presign mode (a signed PUT to /api/media/_local-s3/<key> into FsBlobStore, same headers and checks; see media-nuxt).