Chat control decoded: An interdisciplinary perspective

The “Regulation of the European Parliament and of the Council laying down rules on preventing and combating the sexual abuse of children”, colloquially referred to as chat control, has still not been adopted after more than three years of negotiations. This welcome goal is to be pursued through various measures, such as removal orders for known material, but also, as has received particular attention, through the scanning of interpersonal messages. Chat control in its original form from 2022¹ would have required communication providers to search their users’ data for abusive material involving children – both photos and text messages would have been affected.

Since the first legislative proposal on chat control, there have been various modifications, but ultimately none of them won the support of the majority of Member States. In light of the ongoing efforts to implement the proposal, it is worth taking a closer look at the difficulties of its possible implementation.

Whether, as initially planned, text messages and images should be made mandatory or, as in the current proposal, only voluntary scanning should be standardised, the question of technical implementation remains a challenge. The measures also face legal hurdles, especially as they are directed against all users without specific cause. In addition, such measures could be unpopular with the general population.

Technical implementation

Particularly in the context of encrypted communication, it remains unclear how scanning communication content can be technically implemented. The fundamental problem is as simple to describe as it is difficult to solve: in order for encrypted content to be checked at all, the encryption would have to be removed or circumvented at least at one point. At that moment, however, the content would also be more vulnerable to external, non-governmental attacks.
It is possible to access the data before it is encrypted “on device”, i.e. on the end device, before it is sent. In such a scenario, however, depending on the system, it is questionable to what extent this would even be technically possible in terms of the storage space required.² There is also a risk that scanning could be more easily evaded in this case.³ This raises concerns that the burden on all users would be disproportionately high, while the individuals sought by law enforcement agencies would be able to circumvent the scanning.

Various technical solutions were also considered in a leaked discussion paper by the European Commission, without ultimately being able to settle on a suitable approach: Compared to the baseline of monitoring unencrypted communications, there is no method that is equally effective, feasible and secure^.

The subsequent problem arises from scanning the content once access to it has been gained. Scanning photos and text messages involves different requirements.

Scanning text messages

Scanning text messages was originally intended to be a new way of detecting more complex abusive behaviour towards children, for example in the form of “grooming”, i.e. adults deliberately contacting minors with the intention of abusing them, and forwarding it directly to law enforcement agencies. The EU had planned to implement this using an AI application, which was probably unavoidable. However, the much greater context dependency of text messages compared to images poses a major challenge for AI models⁵.

According to estimates, an effective system for detecting abusive content in text messages should have a false positive rate of no more than about 0.001 per cent.⁶ This would affect around 100,000 messages out of the ten billion messages sent daily in the EU, which would then have to be reviewed by human staff.
However, in order to even reach this point, such a system would first have to be developed, which the EU does not expect to happen – it internally estimated a value of between 5 and 10 per cent⁷ in the case of grooming in 2022. A recent report on voluntary scanning by providers in 2023 and 2024 points out that machine learning methods could achieve a correction classification rate of up to 92 per cent in the best case scenario⁸. As a result, 8 per cent of all cases would still be misclassified. Even assuming that only chat sessions with minors are scanned for grooming, a significant number of false positives can be expected. How this volume of reports, possibly numbering in the hundreds of millions, could be processed seems questionable. It would also mean that a large number of private, non-criminal chats could potentially be read by moderators, and this knowledge could lead to a chilling effect, a kind of self-censorship due to the knowledge of surveillance⁹.

Scanning photos

In the case of photos, the question arises as to whether the images should be compared with known representations of abusive material or whether the system should also be designed to recognise previously unknown images.

While the former is already being carried out on a voluntary basis by social media companies and has¹⁰ already led to more than 19 million reports in 2024, the search for new abusive material is very challenging given the current state of technology.
How, for example, can a system automatically distinguish between a holiday photo of a child on the beach in a family group and abusive content? Among other things, artificial intelligence would have to be used here, but with the foreseeable consequence that the number of false positives would increase accordingly.¹¹ Age assessment in particular, which is often difficult even for humans, carries a considerable risk of error here: if the age of teenagers is misjudged, for example, legal communication between young people can be wrongly classified as abusive.¹²

Legal classification

In addition to technical feasibility, the question arises as to how such a measure should be classified legally. According to the Federal Commissioner for Data Protection and Freedom of Information, comprehensive scanning of private messages would violate fundamental rights under the European Charter of Fundamental Rights¹³, the Basic Law and the principle of proportionality¹⁴
This would affect, on the one hand, the right to respect for private and family life (Art. 7 of the EU Charter of Fundamental Rights), the right to data protection (Art. 8 of the EU Charter of Fundamental Rights), the secrecy of telecommunications (Art. 10 (1) of the Basic Law) and the fundamental right to IT systems (as an expression of the general right of personality, Art. 2(1) in conjunction with Art. 1(1) of the Basic Law). As a result of scanning private messages, there are also fears that chilling effects will have an impact on freedom of information and expression (Art. 11 of the EU Charter of Fundamental Rights).¹⁵

Chat monitoring is therefore considered by various voices in legal scholarship to be incompatible with fundamental rights.¹⁶

Acceptance

In addition to technical feasibility and legal classification, social acceptance of state interference in the privacy of citizens is of central importance. This is because the acceptance of state action, especially in areas sensitive to fundamental rights, is of considerable importance from a political perspective.

To our knowledge, there are no research findings on social acceptance of chat monitoring. However, there is research on the acceptance of other state measures for monitoring communication. This allows us to make informed assumptions about the social acceptance of chat monitoring.

Various studies show that general and unwarranted monitoring of communication behaviour directed at the entire population is much less socially acceptable than targeted measures directed at individual suspects.¹⁷ For example, representative surveys in Germany show that data retention, i.e. the general storage of telephone and internet connection data by telecommunications providers, is much more strongly rejected by society than the unnoticed online search of personal computers of suspects. ¹⁸

It can therefore be assumed that indiscriminate and mass chat monitoring would also meet with little social acceptance. However, the extent of actual social resistance would also depend heavily on media coverage and public attention to the issue.

Outlook

Despite several attempts, chat monitoring has repeatedly failed to reach agreement in the Council of the EU. Now, the Committee of Permanent Representatives (a sub-body of the Council of Ministers) has agreed on a draft proposal. This means that the proposal can be negotiated in a trilogue procedure (an informal procedure for reaching agreement on legislative proposals between the Parliament, the Council and the Commission). The outcome of this remains to be seen.

The Council Presidency´s proposal¹⁹ now under discussion differs significantly from the original version in terms of the points of criticism. The most striking difference is that there is no obligation to scan automatically. However, scanning text messages and images remains possible on a voluntary basis under Regulation (EU) 2021/1232 and can be considered a risk-reducing measure. This provides incentives for providers to continue scanning, albeit on a voluntary basis. The proposal also includes a clause mandating the Commission to examine the necessity and feasibility of a future detection order.

While the move away from mandatory scanning can be seen as a response to criticism, voluntary scanning is not without its problems.²⁰

As of 18 December 2025

The blog posts published by the bidt reflect the views of the authors; they do not reflect the position of the institute as a whole.

---