Matrix-based

Official website (opens in a new tab)

Not a messaging app but an open, federated protocol on which messaging apps such as Element, Tchap or Citadel are built. End-to-end encryption is only an option, and each user’s “homeserver” remains an unavoidable trusted third party.

At a glance
• End-to-end encryption — 🟠 Partial
• End-to-end authentication — ❌ Poor
• End-to-end security — ❌ Poor
• No identity substitution — ❌ Poor
• Open source client — ✅ Good
• End-to-end multi-device — ❌ Poor
• Minimal personal data — ❌ Poor
• No contact discovery and spam — ❌ Poor
• Post-quantum encryption — ❌ Poor

Matrix is not a messaging app but a protocol: an open specification for federated communication, on which several messaging apps are built, including Element^archived (the reference client), Tchap^archived (deployed by the French government) and Citadel^archived. The specification is public, the main clients and several server implementations are open source, and the federated architecture lets each organization run its own server (its “homeserver”) while remaining connected to the rest of the network: all of them verifiable properties that serve the sovereignty we advocate elsewhere.

This diversity calls for a ground rule: we assess here what the protocol guarantees across all of its deployments, in other words what a user can take for granted whichever Matrix-based messaging app they use. Our concern is with a fundamental design point: Matrix was first conceived as a system of chat rooms replicated between servers, to which end-to-end encryption was added later, as an extension (it is still an optional module^archived of the specification today). Matrix was launched in September 2014 without end-to-end encryption; that encryption (Olm/Megolm) arrived in beta in late 2016^archived; and it only became enabled by default for new private conversations in May 2020^archived.

This history can still be read in the architecture: a user’s account, devices and keys stay anchored to their homeserver, which remains a trusted third party; and, as Matrix implements it, federation — the technology that lets users on different servers communicate — replicates room data on the servers of all participants. The result is a model in which the effective security of a conversation depends on operators the user did not choose.

Notable points

When the vendor is not the operator

Most of the applications presented in this benchmark are developed and operated by the same entity: the software vendor is also the operator of the relay and directory servers. On this point, Matrix is an exception, and the official Element application can be used on servers operated by third parties, for example on the Tchap servers operated by the French state.

As we point out regarding publishing the server code, if the security guaranteed by the application is truly end-to-end, then the server plays no role in security, and neither the operator of the servers nor their location matters. However, Matrix cannot guarantee complete end-to-end security, so it is essential that the server operator be trustworthy. In the end, you have to trust both the application vendor and the server operator.

This duality also exists for Telegram, with its many unofficial applications (some of which inspire little confidence!), and for SimpleX, which lets you host your own relay server^archived.

Federation, and the weakest link

Federation, which makes it possible to build a network spanning several servers, is Matrix’s central argument, and a double-edged one. The promise of sovereignty is real: choosing your server means choosing who hosts your account, and under which jurisdiction. We subscribe to that goal. But federation, Matrix-style, has a downside, which can be summed up as follows: in a federated, interoperable system, effective security tends to align with the weakest link.

This weakest-link effect operates at three levels.

First, at the protocol level: for heterogeneous servers and clients to interoperate, strong guarantees can only be optional; encryption is not required, nor is cross-signing, and each deployment chooses its own level of stringency.
Next, at the room level: since a room’s data is replicated on the homeservers of all participants, it only takes a single member registered on a poorly administered, or malicious, server for the room’s event history and metadata to be persistently stored there; everyone chooses their own server, but nobody chooses other people’s servers.
Finally, at the level of usage: bridges to unencrypted systems, encouraged by the ecosystem’s culture of interoperability, silently reintroduce decryption points.

The problem is not federating as such; it is federating servers that play a role in security. When the server holds the account, declares the devices and keeps the history, every server added to the system is one more trusted third party, and federation multiplies points of failure instead of removing them. This is the opposite of an architecture in which servers, reduced to the role of relays with no security function, can multiply without security depending on them: in that second model, federating weakens nothing, because there is nothing on the server to weaken.

End-to-end encryption 🟠 Partial

Matrix has end-to-end encryption (Olm/Megolm, derived from Signal’s Double Ratchet), and the main clients enable it by default for private conversations. But it is not systematic in the sense of our criterion: the protocol does not require it, public rooms are not encrypted, activation depends on the client and on the server configuration, and a Matrix-based messaging app may very well not offer it at all. On top of that come bridges^archived to other systems (IRC, Slack, etc.), widely used in the ecosystem: a room bridged this way is decrypted by the bridge, by construction. Encryption therefore exists, but its actual scope varies from one deployment to another.

End-to-end authentication ❌ Poor

A Matrix user’s identity is their account (@user:server), held by their homeserver, which distributes the list of their devices and their keys to others. Manual verification exists (comparing a short sequence of emojis, a SAS-type protocol), as does a cross-signing mechanism that makes it possible to validate all of a user’s devices at once. While verifying your own devices is becoming mandatory^archived in the ecosystem, verifying your correspondent remains optional, and nothing prevents you from chatting without ever having done it. This is the directory model again, in federated form: trust rests on the correspondent’s homeserver, which distributes their identity key.

End-to-end security ❌ Poor

Encryption that is not systematic, authentication delegated to the server: neither component reaches the required level.

No identity substitution ❌ Poor

A Matrix account can be recovered like any other online account, by password or email reset, depending on the homeserver’s policy. Cross-signing keys can be reset, and correspondents then see their previous verifications invalidated^archived, with an alert that can simply be ignored. The identity is the account, not the key: everything the criterion describes applies, with authority over the account simply shifting from the phone number to the homeserver and the recovery email.

Open source client ✅ Good

The protocol specification is public, the main clients are open source (Element, the reference client, is published under AGPLv3^archived), and so are several server implementations^archived. The cryptography has also undergone independent audits^archived.

End-to-end multi-device ❌ Poor

The protocol provides an effective mechanism (cross-signing: a single certification key signs all of the user’s devices), but for a long time did not require it^archived: depending on the client and its configuration, verification might not be enabled, and the homeserver then became, in effect, the authority declaring which devices belong to the user, with the risk, described in the criterion, of a ghost device being added. The ecosystem is fixing this: following a change to the specification announced in late 2025, Element will make device verification mandatory^archived, and unverified devices will no longer be able to send or read encrypted messages, at a date not yet set (the initial April 2026 deadline has been postponed). As of September 2026, this guarantee is therefore neither in effect nor imposed on all deployments; this criterion will need to be re-examined once it is.

Minimal personal data ❌ Poor

Registration depends on the homeserver, and real-world deployments most often require an email address (that is the very principle of Tchap, reserved for the work addresses of French public servants^archived). But the real problem is that the homeserver keeps a complete history of the events in its users’ rooms, and that history is replicated on the homeservers of all participants^archived. Metadata (who is a member of which room, who wrote when) is thus not only collected, but disseminated.

No contact discovery and spam ❌ Poor

Users are discoverable through their server’s directory and through federation, and anyone can invite anyone into a room. Message content is generally only delivered once the invitation has been accepted, but the solicitation has already taken place, and it carries text freely chosen by its sender: the room topic is displayed to the recipient before any acceptance. Invitation spam is a documented phenomenon^archived on public servers.

Post-quantum encryption ❌ Poor

No post-quantum encryption: Olm and Megolm rely on classical cryptography. The integration of PQXDH into vodozemac, the reference cryptographic library, began in 2024, but the Matrix team was still stating in 2025 that this work was not a priority^archived.

Back to the benchmark