Opportunities, limitations and specific applications of AI in the criminal justice system – these topics were the focus of a workshop on AI and criminal law, at which experts from the judiciary, the legal profession, psychological research and the technology sector came together to discuss these issues. The workshop took place as part of the bidt-funded research project “How ‘human’ should justice be? Psychological determinants of trust in co-creation with generative AI in the context of the legal system (Gerechte KI)”, which investigates which psychological factors promote or inhibit trust in generative AI within the legal system. The project forms part of the bidt’s research priority “Humans and Generative Artificial Intelligence: Trust in Co-Creation’”. This initiative is dedicated to exploring the conditions under which people trust – or could trust – interactions with generative AI and the resulting products – and when they do not.
The workshop did not focus solely on what is already technically possible. Equally central to the discussion were the legal framework, societal acceptance and the question of human responsibility. This is because generative AI is already being used increasingly worldwide for legal research, information gathering and the drafting and revision of legal documents. However, its use in judicial decisions themselves faces particular challenges.
Trust in AI – but not without human involvement
The opening address by Dr Martin Wachter, Head of the Legal Tech and AI Division at the Bavarian State Ministry of Justice, highlighted the conditions the judiciary must establish to ensure the responsible use of AI. In addition to data protection, technical capabilities and the requirements of the European AI Regulation, he particularly emphasised the development of AI expertise. Psychological phenomena such as confirmation bias or anchoring effects make it clear that technical support does not automatically lead to better decisions. Indeed, an AI that appears convincing can also lead to a deterioration in the quality of decisions if flawed AI results are accepted uncritically. It is therefore crucial to embed knowledge of how these systems work, as well as their capabilities and limitations, within the justice system.
You need to understand how it all works to use it properly. Efficiency gains only arise when I no longer have to check everything. I need to be able to rely on it, and to do that, I need to understand it
Dr. Martin Wachter
Dr Marvin Walczok, a research fellow at LMU (social psychology with a focus on legal psychology) working on the bidt-funded “Fair AI” (Gerechte KI) project, presented psychological research findings on the perception of AI-supported judicial decisions. Several studies within the project examine, amongst other things, how people assess purely human, hybrid and fully AI-based decisions. The findings show that decisions generated purely by AI are perceived as less trustworthy and are met with less acceptance – particularly in the case of emotionally charged criminal offences. Hybrid models, in which humans and generative AI work together, are, by contrast, rated more positively, provided that a recognisable human influence remains. This brings a fundamental question to the fore: how human must decision-making processes in the judiciary be in order to be perceived as legitimate and trustworthy?
Human actors must not be replaced entirely. We need a justice system that includes everyone involved.
Dr Sebastian Dötterl, a judge at the Higher Regional Court of Munich, also placed human decision-making authority at the centre of his presentation. Using a practical experiment, he demonstrated how generative AI is already capable of analysing investigation files and drafting final decisions. The technical possibilities thus range from supportive analyses and draft texts to, in theory, automated decision-making. This is precisely why it is necessary to clarify what is technically possible, legally permissible and appropriate for a state governed by the rule of law. His conclusion: the formula “Support: yes, replace: no” falls short because it overlooks the basis of decisions and the decision-making process.
The systems would need to be designed to require users to engage with the case, the case file and the law. The task for the judicial administration is to organise training so that AI literacy becomes widespread and the applications can be thoroughly tested.
Dr. Sebastian Dötterl
From chatbots to AI agents: what is already possible today
Lawyer Nico Kuhlmann provided an overview of the practical use of AI in everyday legal practice. The spectrum ranges from general chatbots through specialised legal research and workspace applications to automated workflows and AI agents. AI can, for example, serve as a sparring partner in brainstorming, draft initial documents, analyse extensive documents or support learning.
However, Kuhlmann does not currently see any suitable area of application for the complete handling of entire work processes without human oversight. At the same time, generative AI is likely to fundamentally transform legal work: amongst other things, the number and volume of legal submissions could increase, and the range of their quality could widen.
We need to ask ourselves: How do I oversee AI? And what steps do I need to take to ensure I have the ability to exercise that oversight?
Nico Kuhlmann
AI competence is becoming a key skill
The possibilities and limitations of the systems, as well as the teaching of the necessary know-how, also shaped the concluding plenary discussion with Prof. Dr Friederike Funk (LMU, Head of the “Fair AI” (Gerechte KI) research project), Dr Marvin Walczok, Dr Sebastian Dötterl, Nico Kuhlmann and Dr Martin Wachter. In addition to technical AI expertise, there remains a need for sound legal expertise – and, increasingly, a strong capacity for ‘oversight’: anyone using AI must be able to critically examine and assess its results. This critical examination of AI-generated results requires not only technical AI expertise but also specialist legal expertise.
Legal training also plays an important role in this regard. Discussions centred on how AI expertise can be more firmly embedded in undergraduate studies and legal training programmes without compromising fundamental professional skills. This is because time pressure and high complexity can increase the risk of potentially flawed AI outputs being adopted without scrutiny.
The interdisciplinary workshop made it clear that the central question for the criminal justice system is no longer whether generative AI will transform legal work. Rather, the crucial issue is how its use can be structured – in such a way that its potential is harnessed without losing sight of human judgement, responsibility and the principles of the rule of law. Psychological research can identify potential sources of error here and, through its findings, contribute to possible solutions. Dr Martin Wachter summed it up as follows:
We all need to take responsibility for making sure we don’t forget how to think. With every day we use AI, we lose some of our existing skills, and that will happen very quickly. We need to make sure we retain our core competencies
Dr. Martin Wachter
