Transparency — how you check the numbers yourself

Level 1 — Integrity: does the data tie out? (0)

Level 2 — Semantics: does the account mean what the calculation assumes? (0)

Level 3 — Economic: is the result possible in the real world? (0)

Other product tests (15)

Access gate, navigation and the MCP connector — they test whether the product works, not whether a number is right, so they don't fit the 3 levels above by design. Still counted, still listed:

• AUD01 — The coordinates we hand an external auditor really do lead to the line. Three of the five external audit attempts died on the same rock: the reviewer's tooling cannot open a zip archive, so they read the published cases and could never compare against the source. Serving our own extracted CSV would have been worthless, since the auditor would then be checking our number against our own extraction, so we publish coordinates instead: the exact file inside the archive and the three-column filter that isolates the row. This test opens the archive, applies the published filter, and checks that exactly one row comes back and that its raw value times its declared scale equals what we serve. A wrong coordinate does not return nothing, it returns a real row belonging to a different account, and the auditor then reports a discrepancy against a correct database, which is the worst possible outcome of an audit.
• AUD02 — The instructions do not promise a check that the source makes impossible. The first draft said to compare the archive checksum and that it must match. Running our own instruction before publishing showed it failing: the regulator REPUBLISHES these archives, and the one for 2024 was modified two days after we read it. Any auditor downloading today would get a different checksum and conclude we had tampered with the data. Publishing a check that fails by design is worse than publishing nothing, because it carries the appearance of rigor while manufacturing a false accusation against our own database. The checksum is still published, but as a statement of which snapshot we read, never as a byte-for-byte proof.
• AUD03 — The route states its own limit instead of leaving it implicit. Two admissions have to survive any rewrite: the currency-scale conversion is OURS, and it is the one step in the chain the auditor cannot see; and anyone whose tooling can open the archive should read the original file rather than trust our coordinates. Without the second, a convenience for limited tooling quietly becomes an invitation to audit our own copy, which is the circularity this whole effort exists to avoid.
• AUD04 — End-to-end check on the route external auditors actually use, which builds its coordinates by a different code path than the lineage does, so one passing does not imply the other. It found a real defect on its first run: the statement of changes in equity carries an extra dimension, so the three-column filter isolates one row in the balance sheet and income statement but returned SIX there, one per equity column. We would have published an ambiguous coordinate for every item sourced from it, the auditor would have picked one of six in the dark, found a number that genuinely exists, and reported a discrepancy against a correct database.
• AUD05 — The methodology says what happens WHEN the reporting basis changes. I claimed the rule was undocumented and I was wrong: I searched the HTML shell and a content directory that does not exist, while the text lives in the sources-and-standardization page and has been live all along. Concluding absence from my own bad search is the same mistake an external reviewer made with a cached page on the same day. What was genuinely missing is the CONSEQUENCE at the boundary: that comparing two years there compares two different consolidation perimeters. This test now requires the rule, the flag name, and the concrete case to all appear in both languages, because an abstract rule with no example is exactly what let this boundary go unnoticed for months.
• AUD06 — A comparison that straddles a change of reporting basis is marked as such. One company published a 268 percent revenue increase because its consolidated figure for one year was compared against its individual figure for the year before; on a consistent basis revenue had actually fallen by a quarter, and the number carried no flag at all. The instructive part is why the existing protection missed it: the routine that picks the restated comparative denominator deliberately matches on the aggregation level, but that only guarantees the DENOMINATOR is internally consistent and never compares the denominator's basis against the NUMERATOR's. The guard was correct and in the wrong place. We mark rather than recompute, because recomputing on the individual basis would discard the consolidated statements, which are the right basis for the year they exist in.
• AUD07 — A browser gets a page; an agent that asks for JSON gets JSON. This is the fourth external audit attempt stopped by the same kind of door: the reviewer read the 47 KB home page and could not read the audit sample route at all. The route answered correctly in under two seconds — the problem was that it was 142 KB of raw JSON, a format browsing tools truncate or refuse. Half that weight was my own defect from the same day: the coordinate block repeated all twenty-odd accounts of the company for an indicator that consumes three, and a coordinate for an account the formula never touches does not help anyone audit, it just pushes what matters past the tool's reading limit. Trimming brought it to 60 KB; serving HTML closes it. Same remedy as the connector endpoint that used to answer 406 in a browser: someone arriving by browser is not speaking the protocol, they are trying to READ.
• AUD08 — The audit sample route is named in llms.txt and the home page links to the audit protocol. The fifth external attempt produced the sharpest diagnosis yet: the reviewer picked its own seed, composed the URL, and its own reading tool REFUSED it, because that tool only opens URLs that already appeared in the conversation or in a search result, and no search engine had indexed the domain, so there was no path by link. Checking that turned up two defects of ours: the home page did not link to the audit protocol at all, and llms.txt — the file we publish precisely so that AI agents can find their way around — never mentioned the audit route, which lives only in the sitemap, a file reading tools do not consult. Publishing a door and hanging no sign on it is the same as having no door, for anyone who does not already know it is there. The file now also lists a concrete indexed URL, because a tool that cannot compose a URL cannot use a template.
• AUD09 — Amortization of debt issuance cost is kept out of the depreciation figure that feeds EBITDA. This is the first data finding to come from an external auditor since the earlier successful one, and it started from a single case worth 0.34 percent: one company added a line labelled amortization of transaction cost to its depreciation and amortization, and the reviewer flagged it as a methodological caveat rather than declaring a discrepancy, because he could not open the source file to show the account. He was right, and the cause was systemic: the matcher accepted any label containing the word amortization. Measured across the base: 279 company-years affected, 226 inflated EBITDA figures, 107 of them by more than one percent and 35 by more than five, with the worst at nearly 39 percent. EBITDA is operating earnings plus depreciation and amortization OF ASSETS, and financing cost is not that, so this is a correction rather than a mark: the rule about marking instead of rewriting applies when the source is ambiguous, and a line that says cost of raising debt is not ambiguous. Half of this test guards the opposite error: amortization of the fair-value step-up from a business combination IS asset amortization and belongs in EBITDA, and the first exclusion pattern I wrote would have silently removed it from 89 lines.
• AUD10 — The audit protocol declares the conflict of interest instead of pretending it away. The prompt used to open with a sentence saying the person asking is not the owner of the database and has no stake in the result — and the people who paste this prompt are overwhelmingly us. The fifth external auditor caught it: the audit opened with a claim of independence and the outcome arrived in the first person, ours, admitting fault and announcing the fix. His finding survives the conflict, because the account he checked sits in an audited financial statement rather than in his trust of whoever asked. But a protocol whose entire thesis is honesty cannot begin with a false sentence, and that sentence was mine, written to sound neutral. Declaring the tie is stronger than hiding it: an auditor who knows who asked calibrates his skepticism, while one who finds out afterwards discounts the whole result, and is right to.
• AUD11 — The sample cache makes a retry instant and never serves a sample built from an older database. Six external attempts have now ended on transport rather than arithmetic: measured in production, the same route ranged from under two seconds to nearly fifty across six consecutive calls, with one exceeding a sixty-second ceiling, while locally it answers in 136 milliseconds. That is not the algorithm, it is contention on a half-CPU instance serving a 1.8 GB database from network-attached disk. Every audit tool retries the URL after a timeout, and the three that tried reported the same symptom under different names: a gateway error, a cache miss, an unavailable endpoint. Caching turns the second attempt into five milliseconds. The danger of the cache is precisely the defect it imitates: one auditor spent twelve reads looking at a frozen snapshot from the previous day and reported that the correction was not live. If WE served a stale sample after a rebuild we would be the cause of that, with the aggravation that the number would be wrong rather than merely old — so the database version is part of the cache key, and this test attacks by removing it.
• AUD12 — For a bank, the generic list of source accounts is not presented as though it applied. The sixth external auditor found that one bank case declared one pair of account codes while the accounts actually used were a different pair entirely: declared did not match used. He classified it precisely as an audit-trail defect rather than a numerical one, and refused to turn a documentation inconsistency into a claim that the ratio was wrong — the accounts actually used are the correct banking structure. But 1,407 indicators, 2.2 percent of the base, were published that way, and the placement is the worst part: the banking chart of accounts is exactly the surface our own prompt tells auditors to attack. A field called declared that does not describe what was used looks auditable and is not. The chart varies even between banks, so swapping in another fixed list would only change whose statement is wrong; the honest output says the generic list does not apply here and points at the field that does.
• AUD13 — An auditor can redo the ORDERING, not merely the individual scores. This is the finest objection we have received: having checked every published sample score and found them all correct, the auditor observed that this proves the key reproduces the scores shown, but not that those are the smallest scores in the whole population, because he never received the population to sort. He is right, and the distinction is subtle — checking each published score detects a fabricated key, but it cannot detect OMISSION. A case with a smaller score could have been left out and nothing in what he received would reveal it, so the claim that the sample is the smallest N rested on our word. A new route publishes the population keys, and this test does exactly what he would do: download them, score all of them, sort, and compare. Its first version had the same blind spot one level up — an attack that dropped one arbitrary line PASSED, because an arbitrary line is almost never in the top N — so the count of published keys is now checked against the table itself.
• AUD14 — The published population declares an arithmetic that has to reconcile against a different route. The external auditor had just proved, independently, that the sample is the top of the ordering over the population file — but the SIZE of that population was still a bare assertion: the health endpoint published one indicator count and the population file delivered a smaller number of keys, with a gap of several thousand that appeared nowhere. He could verify that the sample was the top, and could not verify that the population was the population. Publishing the count of suppressed indicators closes the arithmetic across the two routes. This does not remove the dependency on us, since both numbers come from here, but it replaces an assertion with a SUM, and a sum that has to reconcile is attackable: dropping a key from the population now requires editing the health endpoint too. The attack truncates the population file without touching the other route.
• AUD15 — The excluded depreciation lines are published, not only the ones that were summed. Without this the new rule was not falsifiable from outside: we published which accounts ENTERED the depreciation figure and nothing about which stayed out, so an auditor wanting to attack the exclusion of debt-issuance cost — a change that moved 232 EBITDA values — could see neither what was excluded nor why. The rule matches the NATURE of the label rather than the account code, since the same numeric code is transaction cost at one company and amortization of a sales stand at another, and a rule like that can only be defended by showing the label that triggered it, which means it can only be ATTACKED the same way. The auditor named three companies as targets; checking them showed the rule does not even fire at two of them, so the route exists for him to find the cases where it does fire on his own, rather than depending on the audited party to point at the battlefield.