What is … ChatGPT?

On 30 November 2022, the US software company OpenAI released ChatGPT (GPT-3), a chatbot based on artificial intelligence (AI).

The possible uses are manifold: from summaries of novels to job applications to poems and, since the latest version (GPT-4), even image descriptions – ChatGPT can generate any text imaginably. What is initially a technological feat that holds immense potential, poses challenges for some areas, such as science and the examination system: How is the output of AI to be evaluated? How comprehensible and transparent are the results of the system? Which sources are used?

In our bidt dictionary, we explain what ChatGPT is, how the programme works and what challenges and potentials are associated with it.

What does ChatGPT “know”?

Language models like ChatGPT have billions of texts as their database. The development company is keeping quiet about the extent and curation of the database fed into it. However, it is known that most of the underlying training data come from the years before 2021, and the chatbot can, therefore, not make any information-based statements about the world after 2021. It is assumed that the complete content of Wikipedia makes up about three per cent of the texts in such a model. The neural networks must be correspondingly large to generate meaningful text output for the readers.[1]

In the case of ChatGPT, there is no question of knowledge in the conventional sense. The AI system is not capable of understanding semantic constructs, i.e. the meaning of words and sentences. This is due to the nature of the language model: it is trained to calculate the connections between words – not their meaning. Put simply, ChatGPT works with an equation with billions of variables that only analyse the statistical relationships of words. In this way, each subsequent word is calculated anew.[2]

How does ChatGPT work?

ChatGPT is an interactive language model that can generate any text based on an extensive database. When asked, “What is a language model?”, ChatGPT answers the question:

A language model is a mathematical representation of natural language that enables a chatbot to understand and respond to natural language. A language model can be a machine learning model, a statistical model or a neural network.

ChatGPT to the question "What is a language model?"

This answer provides a good insight. The three models mentioned by ChatGPT all fall within the spectrum of AI and aim to learn from data and make predictions. Although they differ in their technical approaches, they often overlap in practice.[3] [4]

The ChatGPT language model belongs to the latter category: it is based on neural networks. The networks are able to recognise and learn complex data patterns. To do this, the neural networks must be fed with a database, representing the “knowledge” of the model. On this basis, the system learns how words in a language are connected. It then creates a model that maps the relationships and uses it to predict which words or phrases are most likely to follow based on the previously input words. The neural networks are so multilayered and complexly interconnected that the learned models are ultimately black boxes; it is, therefore, no longer possible to understand exactly how an output comes about.[5]

In the case of ChatGPT, this model was supplemented by so-called supervised learning. For this, the system was initially fed with a gigantic database without supervision. To counter problems such as racism, sexism or hate speech, the language model was trained with human feedback in a second step. Humans thus evaluated the system’s output and sensitised it to politically correct answers.[6] [7]

Risks and potentials of ChatGPT

While the chatbot is celebrated in media discourse as a “marvel“[8] of artificial intelligence on the one hand, criticism of the speech system is voiced on the other. Some of the points of criticism, such as the lack of transparency of answers or the danger of blind trust in technology, are already known from similar AI applications. However, ChatGPT is the most potent speech system to date, which adds further weight to the criticism.

Ethical considerations

Sources and transparency

One criticism of ChatGPT is its handling of sources. For example, ChatGPT is often (not always) able to provide references when asked. However, the decision as to whether this source information is complete, correct or trustworthy is left to the users. When training the language model, the sources were not sufficiently taken into account. For example, the source information from ChatGPT leads nowhere with regard to the question, “What is a language model? The problem of the so-called framing effect makes this even more difficult. While Google, for example, outputs various sources and relevant information for a search query, ChatGPT only provides a single answer. This bears the risk of automation bias – blind trust in the machine answers. These are without context, classification and alternatives and – if not explicitly asked – without any source. Thus, relevant aspects could be omitted or emphasised in the presentation of an issue.

ChatGPT as a plagiarism machine?

Precisely because of its sometimes unheard-of performance, further problems arise: The tool poses immense challenges for the education and examination system of schools or research institutions in particular, as it is a “highly effective plagiarism machine“[9], as bidt director Professor Julian Nida-Rümelin emphasises in a WELT article. Examination results in schools or universities often consist of seminar papers and theses. Here, it can be difficult to prove to what extent new technologies such as ChatGPT were used, which could call into question the entire examination form of text submissions.[10] [11]

Legal challenges

Some challenges arise not only on an ethical and moral level but also on a legal level. Questions such as responsibility, liability or the handling of intellectual property – both on the part of the OpenAI producer and the users who continue to use the machine-generated output – must be considered. These new developments affect fields such as justice, medicine, journalism, auditing, research and software development. The respective case sizes differ drastically. For example, a birthday speech generated by ChatGPT might have failed, but the effects of a wrong doctor’s letter, a faulty indictment or a damaged code block are much more severe. Depending on the use case and how it is handled, this can have civil or even criminal consequences, in addition to ethical questions.

With instead of against ChatGPT

Despite all the criticism, it must be acknowledged that ChatGPT is a powerful AI with an output of unprecedented quality, which holds immense potential. In education, for example, the chatbot could be understood as a supplementary assistance system and used in lessons and exams, argued bidt the director, Professor Ute Schmid, in an interview.

Likewise, ChatGPT can catalyse discussions about the usefulness of exams that only ask for factual knowledge. If an AI can pass exams, is it a contemporary form of examination? This question is discussed by bidt directors Professor Dirk Heckmann and Professor Alexander Pretschner and Dr. Jan Gogoll, a researcher at the bidt, in an article in the Frankfurter Allgemeine Zeitung [paid content].

The chatbot could also support in terms of accessibility on the web. Through the image recognition and description implemented in version GPT-4, the programme potentially makes access much easier for people with visual impairments. Other contributions of the language model to improving accessibility would also be conceivable. For example, text that is difficult to understand for people with cognitive disabilities could be automatically converted into “easy language”.[12]

The ChatGPT language model is in the world, and now it is up to users, schools and universities, employers and the state to act accordingly. The consensus is that a boycott of the technology makes little sense. Instead, the appropriate digital and media competence for the use of the new tool as well as conscious handling of the output, should be taught and internalised. This technology can have a massive effect on productivity in many fields. Expectations of the assistance system should remain realistic, and the generated texts should serve as a basis for human revision, find bidt directors Professor Alexander Pretschner, Professor Eric Hilgendorf, Professor Ute Schmid and Professor Hannah Schmid-Peri in a guest contribution in the Frankfurter Allgemeine Zeitung [paid content].

---