Skip to main content

A File Format That Survives Reality

intermediate18 min readLesson 146 of 148

MiniKV's on-disk layout: magic, version, count, fixed-width records, checksum โ€” written byte by byte, not struct-dumped. Formats outlive code.

The format is the contract that outlives you

Code gets recompiled; files do not. The on-disk layout must be defined byte by byte, independent of struct padding, endianness habits, or compiler whims.

MiniKV's layout (fixed, documented)

offset  size  field
0       4     magic  = 'C','J','K','V'
4       1     version = 1
5       4     count  (u32 LE) โ€” number of LIVE records (informational)
9       656   all KV_CAP = 16 slots, 41 bytes each:
                key:   32 bytes, NUL-padded
                value: 8 bytes, i64 little-endian
                live:  1 byte (1 = live, 0 = free or tombstone)
last    4     checksum (u32 LE): byte-sum of bytes 9..664

Total size = 9 + 41*16 + 4 = 669, always. Writing every slot (not just live ones) makes the file a snapshot of the table โ€” a deleted key round-trips as the tombstone it is, and the loader never has to guess where records end. Every field is written with explicit byte stores โ€” fputc, shifts, & 0xFF โ€” never fwrite(&struct). M12's lesson applied: the struct layout belongs to the compiler; the format belongs to you.

Tombstones are part of the state

A deleted key is not nothing โ€” in this format it is a slot with live = 0. Why keep it? Because save/load must round-trip the logical state: "key k was absent" is information. Compaction (dropping dead slots on save) is a legitimate alternative design; what is not legitimate is being vague about which one you chose. This course's format states it plainly: snapshot, all 16 slots.

Checksum: cheap, honest, non-cryptographic

The checksum is a byte sum mod 2^32 over the record region. It detects accidental corruption โ€” a truncated write, a flipped bit โ€” and nothing else. It is not a hash, not a signature, and defends against nothing adversarial. Saying exactly what a mechanism does not do is part of specifying what it does; an over-claimed checksum is a bug waiting for an auditor to find.

static unsigned long kv_sum(const unsigned char *p, size_t n) {
    unsigned long s = 0;
    for (size_t i = 0; i < n; i++) s = (s + p[i]) & 0xFFFFFFFFUL;
    return s;
}

Loading is validation

kv_load never trusts the file: check magic, check version, read exactly 656 record bytes and 4 checksum bytes (any short read โ†’ KV_ETRUNC), verify the checksum before touching the table (mismatch โ†’ KV_EBADSUM). Each failure maps to a distinct KVError โ€” the M13 taxonomy, applied to bytes. A loader that returns "success" with a half-populated table is lying, and M13 taught you what lies cost. One more contract detail: kv_load resets the destination store first โ€” load is "become the state in this file", not "merge".

Now practice

Persistence & the Hostile FileThe byte-exact format: save a snapshot, load it back, and survive a file that lies โ€” wrong magic, truncated, corrupted checksum.2 challenges ยท ยท ~34 min