Auditing zero-knowledge claims with zkaudit

zkaudit is a small go cli that checks whether a "zero-knowledge" claim actually holds up, instead of just trusting the marketing copy on the landing page.

the idea: capture a real HAR (HTTP Archive, the export chrome/firefox devtools give you of every request a page made) from actually using the thing, point zkaudit at it with the secret value that should never leave your browser in plaintext (the message you typed, the encryption key, whatever), and it scans every request url, header, cookie, and body across the whole capture looking for that value showing up somewhere it shouldnt. real traffic, not a code read. i built it specifically to run against my own CipherDrop, since that one makes a zero-knowledge claim and i wanted to actually check it instead of just asserting it.

what it checks

plaintext leaks: the secret value itself, plus url-encoded/base64/case-folded variants of it, matched against request url, request/response headers, request/response body, cookies.

ciphertext plausibility: a base64-density check plus byte-level shannon entropy, so a report can say "this thing thats claimed encrypted actually looks like real ciphertext" instead of just "i didnt find the plaintext" (which could just mean i didnt look hard enough).

redaction: strips cookies and auth-looking headers, writes a sanitized copy of the HAR you can actually hand to someone else without leaking your own session.

multi-flow merge: point it at several HAR captures (upload flow, view flow, whatever) with -har a.har,b.har, it merges and scans them together as one audit.

markdown report + pass/fail SVG badge generation too, so an audit result is something you can drop straight into a README.

the entropy bug, this is the one worth reading

shannon entropy is just: how random do the bytes in this string look, on a scale that tops out at 8 bits/byte for a uniform distribution. real ciphertext should score high. real english text should score noticeably lower. simple enough idea.

first version measured entropy over the string as unicode runes (go's default range over a string). ran it against my actual CipherDrop capture expecting the ciphertext blob to obviously light up as high entropy. it never did. tried a synthetic test to sanity check the function directly: fed it some genuinely random binary bytes on one side and plain english on the other, expecting binary to score way higher. binary scored 3.312. english scored 4.138. binary scored lower than english. thats not a subtle miscalibration, thats backwards.

root cause: real encrypted binary data is not valid utf-8, its just random bytes. when you range over a go string as runes, invalid utf-8 sequences dont raise an error, they silently decode as repeated U+FFFD (the replacement character). so a chunk of genuinely high-entropy random bytes was collapsing into a long run of the identical replacement rune before the entropy calculation ever saw it, which understates entropy about as badly as you can. the fix under measured the exact thing it was supposed to measure.

fixed by switching to byte-level entropy, []byte(s) and a 256-bucket histogram instead of ranging as runes, recalculated the threshold (4.5) off real values afterward instead of a number id guessed going in.

other real bugs found via actually using it

duplicate findings. if a secret happens to be all-lowercase already, the "check case-folded variant too" logic produces two identical strings, and both matched, so the same leak got reported twice for no reason. fixed by deduplicating the variant list before scanning, not after.

short-secret false positives, found live. tested with the plaintext secret "hi" against real CipherDrop traffic just to see what happened. got 6 "findings." all coincidental: "hi" matched inside "while", "This", "hides", and a couple of hits inside raw binary font bytes in a response. none of them were real leaks, all of them were noise from picking too short a secret. added a floor, MinReliableSecretLength = 8, and the cli warns if you go under it instead of silently producing garbage findings.

the actual audit

ran it against my own CipherDrop upload flow, real capture from chrome://net-export, real distinctive test secret typed into the actual textbox. result: PASS, the secret never showed up anywhere in the HAR outside the ciphertext blob itself. also ran -redact-out on the same capture and confirmed by hand that this particular flow doesnt even set a session cookie on that endpoint, so there was nothing to strip.

16 tests, all real assertions against real behavior, not mocked. mit licensed, zkaudit on github if you want to point it at something of your own.

whats left

only audited CipherDrop's upload flow so far, not the view/decrypt-an-existing-drop flow. want to capture that one too and merge both into one combined report before calling the CipherDrop side of this fully done.

update, ran it through detsim

zkaudit is entirely single threaded, no goroutines anywhere, so the goroutine-scheduling half of detsim's rt/rewrite path has nothing to actually exercise here. what it does have is real file i/o: LoadHAR and SaveHAR are plain os.ReadFile/os.WriteFile, and thats exactly the boundary a HAR export crosses on its way from chrome's devtools to this cli, a real file someone dragged onto disk. so instead of forcing a concurrency story that isnt there, i pointed detsim.FaultyStorage at that boundary directly, the same primitive that found the wal bug in toil.

crash_sim_test.go: build one clean, realistic HAR fixture, write it through FaultyStorage with torn writes, byte corruption, and dropped syncs turned on, write whatever comes out the other side to a real temp file, then run it through the actual unmodified pipeline, LoadHAR, Scan, Redact, WriteMarkdownReport, WriteBadgeSVG, the same functions the cli itself calls. 3000 seeds. a second test replays the identical corrupted bytes through LoadHAR/Scan twice and requires the exact same report both times, proving a corrupted-but-parseable HAR produces deterministic output instead of anything order- or timing-dependent sneaking in.

all 3000 seeds passed clean, first run, no fix required. worth saying plainly instead of finding a bug to make the post more interesting: LoadHAR either successfully unmarshals a corrupted file or json.Unmarshal itself returns an error, there's no code path in between where a torn or bit-flipped HAR produces a half-built struct that then panics deeper in Scan or Redact. thats mostly go's own json decoder doing the right thing by construction, not some clever defensive code i wrote, but it's still the kind of guarantee i'd rather have proven against 3000 seeded corruption patterns than just assumed.

wired in the same way toil is. detsim's rt package (and the rest of it, the rewriter, the CLI tools, the docs) was sitting uncommitted on disk when this post first went up, so both projects pointed at it through a local replace in go.mod. it's actually published now, tagged v1.0.0, so zkaudit and toil both depend on the real module like any other dependency.