Excluded criteria

Countless messaging app benchmarks exist on the internet, each with different criteria and its own angle of analysis. Ours focuses on a few essential security aspects that, in our view, are too often ignored by the others. In doing so, and to keep the result readable, we also had to set aside many criteria, either because they are unrelated to security or because their impact is smaller. That does not mean those criteria are without interest, so we list a few of them here.

Features: polls, disappearing messages, chat themes, calls, etc.

The messaging apps studied here all offer the ability to exchange messages and attachments in direct or group chats. But some features may be missing from one or another of them, or come with different limits: polls, disappearing messages, shared chat themes, group calls, number of members in a group, maximum attachment size, etc. It would be hard to draw up an exhaustive list, and there is no real reason to highlight some features over others. Above all, features mostly have no direct impact on the security of exchanges: they simply improve the user experience and make it possible to avoid resorting to other applications (which could indirectly improve security).

Moreover, new features are added to all these messaging apps very regularly, and a comparison based on a snapshot frozen at a given moment would not make much sense. It was therefore natural for us not to judge the selected messaging apps on features. Too bad for Telegram, which would certainly have stood out on that front!

The business model

The business model of an application’s vendor is certainly something to take into account when choosing a solution. Yet it has no direct impact on the security of the application itself: security rests only on technical elements, not on the price you pay or the trust you may place in a vendor. Of course, it is hard to believe in a vendor’s willingness to really protect its users’ exchanges when it lives off the resale of personal data. But if its application met the technical criteria presented in this benchmark, that would offer sufficient guarantees.

On the other hand, it is essential that the business model of a solution’s vendor be clear: developing a quality application and operating its servers is expensive (on the order of a few cents per month per user for Signal^archived or for Telegram^archived, the latter in Russian), and a sustainable business model is necessary to guarantee the application’s longevity. It also ensures that a forced change of business model does not risk calling technological choices into question.

Besides, there is no quantitative reason to favor a foundation over a private company, or the reverse, and it is extremely hard to judge the sustainability of a given business model. The recent financial troubles of the Session messaging app^archived illustrate this point perfectly.

Protection against malware

The Pegasus affair^archived led many people to look for communication tools that protect them from malware. For Pegasus, this was justified, since as we explain in our criterion on contact discovery, a zero-click attack on WhatsApp was one of the main vectors of remote infection by this spyware.

However, malware generally exploits not just a vulnerability in an application, but a vulnerability in the OS, the operating system itself, on which the application runs. Indeed, exploiting an application vulnerability alone would not let the malware break out of the sandbox in which mobile applications are confined, whereas exploiting an OS vulnerability gives immediate access to all the data on the phone.

But what is true for malware also applies to a messaging application: it is confined in an application sandbox that greatly limits its ability to detect malware and protect itself from it. Some good practices help limit the risks, such as encrypting databases, blocking screenshots (on Android), blocking non-official keyboards (on iOS) or detecting the presence of accessibility services. But absolute protection is not possible for a mere application, installed from app stores. And even a “secure” OS has to trust the hardware it runs on: a vulnerability in a component^archived cannot be ruled out.

A criterion on resistance to malware would therefore necessarily be debatable. It is better to consider that a phone infected with malware as advanced as Pegasus is totally compromised, and so is all the data it contains. The end-to-end security that the messaging apps in this benchmark aim to guarantee protects exchanges from everything that lies between the devices of the correspondents, but does not protect them from the phones themselves if they are compromised. No application can claim to build real security on a foundation that is not “sound.”