Skip to content
T1KTheOneKit

managed-datastore-write-cost-discipline

Managed Datastore Write-Cost Discipline — Aggregate Before You Write

Section titled “Managed Datastore Write-Cost Discipline — Aggregate Before You Write”

Managed row-billed stores (Cloudflare D1, Postgres-per-row insert paths, DynamoDB write units) bill per row or per operation written and scanned, not per byte stored. Raw event payloads written 1:1 are the single most expensive thing you can do with them.

Rule

Section titled “Rule”
  1. Aggregate before you write. Only aggregated, calculated, or otherwise reduced data goes into a row-billed datastore. The raw event stream never does. If you cannot name the aggregation or reduction a row represents, it does not belong there.
  2. Raw data stays in object storage or is computed at query time. Object storage (Cloudflare R2 equivalent: no egress, cheap ops) holds raw event payloads keyed by ID/timestamp; anything re-derivable at query time is not persisted at all.
  3. Watch the read-scan bill, not just the write bill. Row-billed stores charge rows scanned by the query planner, not rows returned. One unindexed WHERE over an N-row table bills N rows read per query — a “cheap” read path is how a future surprise bill arrives.
  4. Index every filtered column, LIMIT every scan, avoid SELECT * over wide tables. Never COUNT(*) a growing table on a hot path — maintain a counter column/row instead.
  5. Warn before you write or scan, not after. Before any command that writes to or scans a managed datastore (D1 INSERT/UPDATE/batch, wrangler d1 execute, bulk R2 ops, a migration), state the estimated row/operation count and its dollar cost at the current published rate — then proceed. Unknown volume is itself the warning: say “volume unknown, could be unbounded” rather than staying silent. Pairs with modules/t1k-base/rules/preview-first-batch.md (smoke a small N, surface it, confirm before scaling) — cite that rule’s mechanics rather than repeating them here.

Decision table

Section titled “Decision table”
Data shapeHome
Raw event / payload / blob (analytics, logs, fails)Object storage (R2-class), or compute at query time
Rollup / daily aggregate / KPI / configRow-billed store (D1-class)
Hot counter (user count, DAU, fails)Maintained counter column — never COUNT(*) on a hot path
Short-lived ephemeralLocal compute — do not persist

Why: D1 bills write rows beyond the included 50M/mo at ~$1.00/M while read rows beyond 25B/mo bill ~$0.001/M — writes are ~1000× the per-row price of reads, and an unindexed scan bills every row it touches. Worked example, incident, and full pricing table: docs/datastore-cost-discipline.md.

Section titled “Related”
  • rules/docker-volume-discipline.md — the sibling incident: an un-audited store pattern that silently became the expensive default
  • skills/t1k-datastore-cost/SKILL.md — the operational playbook (store choice, aggregate pipeline, index review, pre-deploy cost check)