What a Security Audit Does Not Tell You
An audit report describes one version of the code, on one date, as it was handed to the auditor.
Privacy services like to announce that they have been independently audited. The announcement usually arrives with a badge on the home page, a blog post, and a line in every comparison chart from then on. Email providers do it, VPN companies do it, and so do password managers and messaging apps.
Audits have real engineering value. A competent outside reviewer finds bugs the authors missed, and the authors fix them. We have nothing against that. This article is about the other use of an audit, the one aimed at you: the suggestion that because a firm reviewed the code, you can now trust the service.
That suggestion does not hold up, for three reasons. The audit covers a moment that has already passed. Nobody can show you that the audited code is the code in use. And being examined, even repeatedly, has a poor record of exposing a vendor that is working against its customers.
What an Audit Actually Is
A company hires a security firm and pays it. The company decides what the firm will look at: one mobile app, one browser extension, one protocol, a set of servers. This is the scope, and anything outside it is not examined. The firm gets a fixed number of days. At the end it writes a report listing what it found.
The company then decides what to publish. Sometimes that is the full report. Often it is a summary, or a letter of attestation, or a blog post describing a report you cannot read.
Read a full report when you can find one and you will see how carefully the auditors limit their own claims. They name the version or the commit they reviewed. They give the dates of the engagement. They list what was out of scope. They say that the work was time-limited and that the absence of findings is not proof of the absence of flaws. The auditors know what they are selling. The marketing department removes those limits later.
A Snapshot of a Moving Target
An audit applies to the exact code that was reviewed. When that code changes, the audit describes something that no longer exists.
Software changes constantly, and it has to. A service that is being maintained ships fixes and features every week, in some cases every day. Each change is new, unreviewed code. So is every update to the libraries underneath it, which were probably out of scope in the first place.
Security patches are the clearest case. When a vulnerability is disclosed in a service's own code, or in a library it depends on, the service has two choices. It can patch, and run code that no auditor has seen. Or it can hold still to keep its audit valid, and run code with known holes. No responsible operator picks the second, which means every well-run service is running unaudited code almost all of the time.
The same applies to the audit's own findings. The auditor reports a flaw, the company writes a fix, and the fix is new code. Unless the firm is paid to come back and retest, the published report certifies a version that the company itself has already replaced.
A badge that says “audited in 2024” is therefore accurate and nearly useless. It tells you that a version from 2024 was looked at. It says nothing about the version you are using today.
Nobody Can Show You the Audited Code Is the Running Code
Suppose the code never changed. You would still have no way to confirm that what the auditor read is what the company runs.
The auditor is given a repository, a build, or access to an environment the company prepared. The production servers are somewhere else. You cannot log in to them, compare what is installed against the reviewed commit, read the configuration, or see what is being logged. The auditor usually cannot either, and certainly cannot after the engagement ends.
Apps have the same gap one step removed. The binary in the app store was compiled by the company, and unless the build is reproducible and somebody actually reproduces it, its link to the audited source is a promise. A web client is worse, because the server sends the code fresh each time you load the page. It can send a different version tomorrow, or a different version to one particular account.
“No-logs” audits deserve a specific mention. An auditor who inspects a logging configuration during one week in March can report what the configuration was during that week. Logging is a setting. It can be turned on the day the auditor leaves, by the company or because a court ordered it, and the audit report will read exactly the same afterward.
None of this requires a dishonest company. An honest one cannot prove these things to you either. We made the same point about published source code in Why Commercial Open Source Is Not a Trust Model, and an audit inherits the whole problem: it is a review of source, and source is not the running system.
Crypto AG: Examined for Decades
The strongest case against treating examination as proof is a Swiss company named Crypto AG.
Crypto AG made cipher machines in Zug and sold them to governments. Over the second half of the twentieth century more than 120 countries bought them to protect diplomatic and military traffic. It was the obvious choice. Switzerland was neutral, the company was privately held, and the founder, Boris Hagelin, had built the machine American soldiers carried in the Second World War.
In February 2020 the Washington Post and the German broadcaster ZDF, working with Swiss television, published the CIA's own internal history of the company. In 1970 the CIA and the BND, West Germany's foreign intelligence service, had secretly bought Crypto AG for CHF 25 million, about US$7 million at the time, through a Liechtenstein holding structure. The BND sold its share to the CIA in the early 1990s. The CIA kept the company until 2018. Throughout, the machines sold to most customers used ciphers designed so that the NSA and its German counterpart could read the traffic. According to that history, Crypto AG machines carried roughly 40 percent of the foreign diplomatic traffic American codebreakers were processing in the 1980s. The CIA's authors called the operation “the intelligence coup of the century.”
We went looking for who audited Crypto AG, and the answer is the useful part. There was no audit firm and no published report, because that industry did not exist yet. The company was checked in the ways available at the time, and it was checked often. Every one of those checks either failed or was turned into reassurance.
The designs were reviewed by the people breaking them
The weakened ciphers were not crude. Crypto Museum's reconstruction of the operation, drawn from the CIA and BND histories, describes NSA cryptographers and the BND's cipher office supplying the designs and reviewing them before release. The requirement was that the output look random to anyone testing the machine from outside while remaining solvable by an agency that knew the structure. The most thorough technical review these products ever received was performed by the party that needed them to be weak.
Customers tested the machines, found flaws, and were sold new ones
The customers were governments with their own cryptographers, and some of them did the work. By 1976 Italy and Egypt had each found the weakness in the H-460 and demanded an explanation. Austria and Yugoslavia found a weakness in another model. Syria found one in a third. In 1979 the Argentine Navy found the flaw in its HC-500 machines on its own and summoned Crypto AG's chief executive to Buenos Aires.
Each of these was an audit that succeeded. The outcome each time was a corrected cipher, delivered by the vendor. Where the agencies got their way, the correction was one they had written and could still read. Argentina accepted its fix. The customers had no means to check the replacement beyond repeating the whole effort, and most stopped there. A finding leads to a patch, and the patch is new code.
A credible expert vouched for the product
Around 1980 Crypto AG brought in Kjell-Ove Widman, a Swedish professor of mathematics, as its scientific advisor. He was working with the intelligence services. When Argentina concluded after the 1982 Falklands War that its communications had been read, Widman went to Buenos Aires. He told the Argentines that what had been broken was their analogue voice scramblers, and that their HC-500 cipher machines used true digital encryption and were secure. They believed him and remained customers.
An official investigation found nothing
In 1992 Iran arrested Crypto AG salesman Hans Bühler and held him for nine months. After his release he said publicly that the machines were rigged, and former employees supported him. One of them, Peter Frutiger, the company's former head of development, filed a complaint. Swiss federal police opened an investigation, questioned company officers and examined equipment. They reported no evidence of manipulation, and the file was closed and never published.
Crypto AG then used the closed investigation as its defense. When the Baltimore Sun laid out the evidence in 1995, the company's chief executive attributed the allegations to disgruntled former employees and called the rest “pure invention.” In later statements the company pointed to the Swiss inquiry as having cleared it. Customers kept buying for another twenty years.
The 2020 Swiss parliamentary inquiry added one more detail. Switzerland's own intelligence service had known who owned Crypto AG since 1993, and had kept that knowledge to a handful of people. The country whose reputation customers were relying on knew, and the customers did not.
The engineers inside could not fix it either
Several Crypto AG engineers worked out for themselves that the ciphers they were handed were weak, and wrote strong ones. One was Frutiger, who as head of development had replaced a weak design with a secure one for Syria. The CIA history records that he was dismissed for it in 1977. Frutiger said in a 2020 interview that he left by his own decision. Another designed a cipher the NSA could not read, and about fifty of those machines were built before the NSA found out. People with full access to the designs, working in the building, saw the problem and were overruled.
What the Crypto AG Record Shows
Crypto AG's customers were not careless. They were national governments protecting their most sensitive traffic, and they did more verification than any consumer of a privacy service will ever do. They tested the equipment with their own experts. They challenged the vendor when something looked wrong. They had the word of a respected academic, the reputation of a neutral country, and eventually a police investigation that came up empty.
All of it produced confidence, and none of it produced security. The reviews that passed the machines were done by people who could not see what the designers saw. The reviews that caught a flaw were answered with a new version nobody reviewed. The investigation that found nothing became the company's best sales argument.
Formal certification has failed the same way more recently. The random number generator Dual_EC_DRBG was published as a NIST standard in 2006, and implementations of it appeared on NIST's own validation lists. Researchers showed in 2007 that its design left room for a backdoor. It stayed in the standard until September 2013, when the New York Times reported, from documents leaked by Edward Snowden, that the NSA had inserted one. NIST advised against using it that month and removed it from its guidance in April 2014. In Switzerland again, the broadcaster SRF reported in November 2020 that a second manufacturer, Omnisec, had ties to US intelligence and had sold manipulated encryption devices to several Swiss federal agencies and to private companies. That report rests on SRF's sources and has not been officially confirmed.
What an Audit Is For
If an audit expires at the next patch, cannot be tied to the running system, and cannot be relied on to catch a vendor acting in bad faith, the question is why privacy companies pay for them and promote them so heavily.
Part of the answer is that they do find bugs. The larger part is that an audit is something to point at. A customer who cannot evaluate cryptography can recognize the name of a security firm. Reviewers and comparison sites can tick a box. The phrase “independently audited” does the same job that “Swiss” did for Crypto AG: it stands in for an inspection the customer cannot perform.
An audit tells you what one reviewer saw in one version during one engagement. The trust built on it reaches much further than that.
So when a service leads with its audit, read the report itself instead of the announcement. Check the date. Check which version was reviewed and how many releases have shipped since. Check the scope, and note whether the servers were in it. Check whether the findings were retested after they were fixed. Then ask the question the audit cannot answer: what happens to you if the company, or someone with power over the company, decides to change the code tomorrow?
The services worth using are the ones where that question has a tolerable answer, because the design limits what the provider is able to do in the first place. If your private key is generated and kept on your own device, and decryption happens in software the provider does not deliver to you, then the provider's code can be wrong, or changed, or hostile, and your encrypted mail stays encrypted. That property comes from the architecture and does not depend on anyone's report. We cover it in The Truth About Zero Knowledge / Zero Trust / Zero Access and Why You Should Never Let a Provider Generate or Store Your Private Key, and How to Self-Evaluate Privacy Services lists the questions to ask of any provider, including us.
Published November 2026. Sources: Greg Miller, Washington Post, “The intelligence coup of the century” (2020); Crypto Museum, Operation RUBICON; Crypto Museum, Hans Bühler; Electrospaces, summary of the Swiss parliamentary (GPDel) report (2020); Swissinfo on the Swiss investigations; Wikipedia, Crypto AG; Scott Shane and Tom Bowman, Baltimore Sun, “Rigging the Game” (December 10, 1995); NIST, removal of Dual_EC_DRBG (April 2014); Wikipedia, Dual_EC_DRBG; Swissinfo on SRF's Omnisec report (November 2020).
