The project team of the bidt project “Ethics in agile software development” is developing a concept to integrate ethical considerations already in software development. The project aims to enable a desirable normative design of software systems—an interview with Dr. Jan Gogoll, a researcher at the bidt.
Research Project
What criteria must software meet to be considered “morally good”?
Jan Gogoll: At the risk of disappointing you with the first answer, there is no checklist for morally good software because software and its use are far too context-specific for that.
The best way to approach an answer is to turn the question around: What are signs that software is morally questionable? And here, too, there will be no concrete list. However, based on various points and consistent reflection, it is often possible to determine that problems exist.
For example, software is morally unacceptable as soon as the freedom of individuals is curtailed by the use of the software; when people are not respected in their humanity, i.e. when their self-determination and autonomy are disregarded. Furthermore, some signs indicate that software is morally questionable, i.e. that moral risks are present: When machines suggest humanity or aliveness – i.e. when machines take over interpersonal relationships, when they creep into or take over the emotionality of people – then it is necessary to pay attention, pause and take action if necessary.
Another question is when it is still acceptable to be manipulated by software: for example, in the area of nudging or dark patterns – in other words, whenever human behaviour is exploited.
Dr. Jan Gogoll To the profile
But here, too, the situation is more complex than we think. Is it enough, for example, to make it transparent that nudging is taking place? Is this then already an impermissible encroachment on autonomy? This will always have to be answered in a specific context.
This context is important because some software products impact society or even bring about changes. Software operation unintentionally changes social structures (predictive privacy), undermines democratic processes (fake news) or entices people to delegate responsibility to machines; then we need to pay attention.
Philosopher Shannon Vallor sums it up quite well when she s, “n “Is Twitter good or evil?” ly meaningless. Instead of such sweeping assessments, we should look at the concrete context. What positive things does Twitter promote (e.g. social participation, freedom of expression), what negative effects does it have (e.g. fake news, cyberbullying) – and how do to preserve the former and avoid the latter through good design?
What incentives do companies have to incorporate ethical factors into software development?
Unfortunately, the impression often arises that ethics is primarily a tool for worriers and preventers that costs a lot and tries to slow down changes and hinder companies in their business activities. In my opinion, this is the wrong way of looking at things. Doing the right thing is certainly also an important motivation for most companies. However, one must not forget that companies are caught in a tension between different normative demands.
It is seldom easy to weigh up, for example, between safeguarding jobs or deliberately preceding a feature that may be dubious. However, if this view seems too naïve: ethics can also be easily formulated as a business case – namely, if it is viewed economically as an investment: In the best case, initial costs are offset by later profits or at least the minimisation of later costs. The most obvious incentive is certainly risk management.
Companies and especially their brands invest a lot of money and effort in building their reputation with customers and in society – and for reputation, it is indeed hard to build but easy to lose.
Dr. Jan Gogoll To the profile
Especially in affluent regions, interest is growing in products with which customers can also identify on a value level, which increases the potential for customer loyalty.
Following this, comprehensive normative support in software development makes sense to reduce the risk of bad investments. After all, if the developed product is deemed unsustainable in hindsight, development costs have still been incurred that are now not generating positive returns. in 2018, for example, Amazon stopped developing an AI that was supposed to make hiring decisions but disadvantaged female applicants. Amazon did not get this problem solved and eventually discontinued the project after incurring costs of around $50 million.
In addition, there are also other incentives, such as regulation or the avoidance of law, which is also always driven by perceived misconduct.
A final point that has become much more important in recent years is the issue of refinancing. Many lenders and investors increasingly assess companies according to the criteria of ESG (Environmental, Social and Governance. Even if the concrete design of these certifications is questionable in some cases, they are already incentivising companies to pay attention to these facts. After all, the question of financing is essential for all companies.
Is there such a thing as greenwashing on the topic of sustainability with regard to ethics in software development?
There certainly is. Here we would better use the somewhat broader term of ethics washing. Some software applications influence questions of ecological sustainability, i.e. classic greenwashing – one thinks above all of data centres etc. – but other problems are discussed more often. The field of AI, for example, is currently focusing strongly on the issues of transparency, bias and privacy.
Ethics washing, in general, would be understood to mean, as a first approximation, a possible discrepancy between the statements made by companies about their activities regarding ethical software and their actual implementation.
Dr. Jan Gogoll To the profile
If this delta is significant, then in the end, it is just lip service, for example, to marketing or to prevent possible regulation. Under certain circumstances, this can be a successful strategy for companies, but – to take up the point from above – it is not without risk. Reputation can be built above all with a “walk the walk” strategy – and an only apparent connection between the promises and reality can thus become a reputation killer if it is conspicuous. However, it should also be noted that reality is very complex and exaggerating one’s abilities is almost necessary for marketing. Whether the delta is too large in individual cases, for example, because the promises have not been implemented, or whether it is merely within the scope of the exaggerations customary in the industry, must be determined on a case-by-case basis.
Thank you very much for the interview!
The interview was conducted by Nadine Hildebrandt.
