Security benchmark of messaging applications

Signal, WhatsApp, Telegram, Matrix-based solutions (such as Element, Tchap or Citadel), Threema, SimpleX and Olvid: what do messaging apps that call themselves "secure" actually guarantee? A criterion-by-criterion review, for everyone to verify.

Download the abstract

This benchmark evaluates the main consumer messaging apps against a single set of security criteria. It is intended for the curious user as much as for the technical reader, and it opens with the table below, which gives the overall picture.

To read this table properly, you need to understand each criterion, which we summarize further down by stating what it guarantees and what you are exposed to without it. Each summary is complemented by a dedicated page that explains in more detail why the criterion is essential to the security of a messaging app. A separate page lists some of the criteria we excluded and explains why we did so. Finally, each application is analyzed, with a criterion-by-criterion justification of the results we give it.

The results

How each messaging application scores against each criterion
Criterion	Olvid	Signal	WhatsApp	Telegram	Matrix-based^*	SimpleX	Threema
End-to-end encryption	✅Good	✅Good	✅Good	❌Poor	🟠Partial	✅Good	✅Good
End-to-end authentication	✅Good	❌Poor	❌Poor	❌Poor	❌Poor	❌Poor	❌Poor
End-to-end security	✅Good	❌Poor	❌Poor	❌Poor	❌Poor	❌Poor	❌Poor
No identity substitution	✅Good	❌Poor	❌Poor	❌Poor	❌Poor	✅Good	✅Good
Open source client	✅Good	✅Good	❌Poor	✅Good	✅Good	✅Good	✅Good
End-to-end multi-device	✅Good	✅Good	✅Good	❌Poor	❌Poor	➖Not applicable	✅Good
Minimal personal data	✅Good	❌Poor	❌Poor	❌Poor	❌Poor	✅Good	🟠Partial
No contact discovery and spam	✅Good	🟠Partial	❌Poor	❌Poor	❌Poor	✅Good	🟠Partial
Post-quantum encryption	❌Poor planned for 2027	✅Good	❌Poor	❌Poor	❌Poor	🟠Partial	❌Poor

Legend: ✅ Good 🟠 Partial ❌ Poor ➖ Not applicable

* Element, Tchap, Citadel…

Our approach

“Secure.” “Private.” “End-to-end encrypted.” These words now appear on many consumer messaging apps. But what do they really mean? Behind a seemingly identical promise lie radically different security models: not all of them protect the user against the same adversaries.

Some solutions simply assert their security; others, such as Signal or Olvid, back it up with detailed technical justifications. These justifications are necessary — they allow a community of experts to analyze, discuss, and even challenge or refute the vendors’ claims — but they are not sufficient: security concerns all users, while only the most knowledgeable are in a position to judge it.

These arguments must therefore be made accessible to the widest audience without distorting them: keep it simple without being simplistic. Misleading the end user would not only be dishonest; it would be potentially dangerous, since nobody knows in advance their profile, their context, or the risks they are exposed to.

We do not aim to be exhaustive, nor to catalog every messaging app on the market, but to shed light on structural weaknesses, too often overlooked, specific to certain operating models. We therefore immediately ruled out the messaging apps that have no end-to-end encryption at all (Teams, Google Chat, Slack or Discord), where the operator has direct access to all exchanged data.

As the designers of Olvid, one of the solutions examined here, we obviously have a bias. We will therefore stick, as far as possible, to objective and measurable arguments that anyone can verify for themselves.

An attentive reader will notice that Olvid checks almost every box in the table. It is tempting to see this as a tailor-made benchmark. We accept that suspicion, and we invite the reader to dispel it themselves: the relevance of each criterion is argued independently of whichever solution satisfies it. If Olvid meets almost all of them, it is because we first defined the fundamental security criteria, and then designed Olvid to meet them. Not the other way around.

The criteria

End-to-end encryption

What this criterion guarantees: Your messages are encrypted before they leave your device, and only those who hold the keys they were encrypted for can read them. You still need to make sure that those keys really belong to your correspondents (that is what the End-to-end authentication criterion is about).

Without it: Anyone with access to the server (operator, hacker, authority) can read all conversations in full.