Disclaimer: Asymmetrical Haircuts is produced as a podcast, meaning it is meant to be listened to and not read. Because of this, we recommend that you listen to the episode while reading, because the written word does not do justice to the emotion or tone used by our speakers. However, because we recognise there might be bandwidth issues or you might be using a hearing aid, we have provided written transcripts for all our available episodes.

[INTRO TUNE]

Steph 00:47 Hi Janet.

Janet 00:50 Hi, Steph. So unfortunately or fortunately, depending on your point of view, this really feels like the artificial intelligence year to me. I can now see it sort of at the top of all of my emails, even though I decide not to use it. I mean, it’s in searches. It’s being heavily discussed and what kind of effect it’s going to have on kind of everything in the world. Yeah. And we thought this would be a very good moment to dip our very human toes into this water and ask some basic questions  related to really our field of accountability for international crimes and how AI works in that space. So we’ve prepped a couple of podcasts that we’re running in May, we hope, and one which is coming up later on is going to be on how artificial intelligence is being deployed in the accountability world, how it’s being used in terms of the managing of databases and recognizing photos and that kind of thing. 

Steph 01:25 But today we have our first pod, and that’s going to be on use of artificial intelligence in targeting, and then we mean targeting technology. Of potential military targets and identifying targets, because we hear a lot about that and the laws of war that apply to that. And we are joined for this by Jessica Dorsey. Hi, Jessica.

Jessica Dorsey 02:05 Hi, great to be with you again.

Steph 02:08 Jessica is a friend of the pod. We’ve had her on before, also on targeting and drones. And she is at Utrecht University, where she teaches international law.

Janet 02:13 And our other contributor is Elke Schwarz.

Elke Schwarz 02:16 Hello, good morning.

Janet 02:18 And Elke is Professor of Political Theory at Queen Mary University. And Jessica said they make a bit of a double act, sort of complementing with an E each other. So we’ll see how that goes and whether we can maybe get them to disagree on something. But otherwise, I’m sure we will just get our best out of both of them. And what we wanted to kick off with for either of you is a bit of a definition of politics. In this subject, what are we talking about when we mean AI? What is it? So who would like to start?

Elke Schwarz 02:48 I think perhaps I start with a very basic idea of what artificial intelligence is and perhaps what it is not. In the broadest context, artificial intelligence, aside from being an industry and a whole field of study, it is a technique of data processing and data analysis. And the latest iteration, when we talk about artificial intelligence, we tend to talk about machine learning as a technique, a subset of artificial intelligence. Intelligence as a technique, which basically means that an extremely large curated data set is analyzed for statistical correlations. It’s then iteratively adjusted, and we make sense out of the output. So that’s the bare bones of artificial intelligence.

Elke Schwarz 03:34 In the context of military applications, artificial intelligence can be all kinds of things. It can be used for streamlining logistics operations, getting assets or equipment from point A to point B in a more efficient manner, or it can be used for predictive maintenance. It could be used to coordinate drone swarms. And many of these applications where only objects are at stake are less problematic. When it comes to using artificial intelligence for targeting, this is where we get into the thorny arenas, ethically speaking and legally speaking.

Jessica Dorsey 04:14 Yeah, and just to add to that, I think it’s important to note that it’s in the context of warfare on the sharp end of the stick. So when we’re talking about targeting, when these kinds of systems are used to generate or recommend targets or nominate them, so rank them hierarchically, this is something we’re seeing, and we’ll probably get more in depth about this. But why this is problematic is that it’s sort of a cognitive offloading. We are asking machines to do work that humans used to do solely. The proprietors were humans, and they require cognitive skills that only humans possess. So that’s where the tension arises. The idea is that they’re sold in military terms to be more efficient. They can do so much more and so much faster than humans can. Maybe that’s true, but at what cost? And I think those are the questions that we try to interrogate from a legal or ethical perspective.

Steph 05:04 We’re going to get more into the, at what point does it become offloading and what are you doing? But we wanted to also have an idea of the playing field, you know, who are the big players in this that we’ll be discussing. There are militaries like the U.S. Department of Defense, or I think it’s now the Department of War, the Israeli Defense Forces say they use AI in targeting. Who else? Is it all militaries that are doing that?

Steph 05:24 Are we looking more at state armies that are doing this? Or should we also assume that big rebel forces like Sudan, the RSF does this as well? And on top of that, and maybe in additional questions, who is providing this kind of military AI? Who are we talking about there?

Jessica Dorsey 05:56 I think a short answer is everybody is trying to get their hands on it, right? So we’re seeing reports now in current conflicts in Gaza, Ukraine, Iran, Lebanon, where the U.S. and Israel are. Well, I think we’re recording this on the morning that there has been a ceasefire that’s been announced, but we’ll see how long that holds. But we’ve seen this and we will continue to see this. And we’re also seeing militaries, Western militaries, developing, procuring, exploring ways to acquire this kind of technology. So it’s quite a pervasive issue. So I think yes to your question, all militaries probably all non-state actor groups are also going to try to get their hands on it in some way. It’s sort of the great equalizer, right? This is software that we’re getting.

Elke Schwarz 06:45 Perhaps to add to that as well, that this is not necessarily a brand new development, right? So this is an onward trajectory from the algorithmic infrastructures that we have already seen in the context of drone warfare. And many, if not most, Western militaries have started to develop their AI infrastructures from 2017, 18, 19 onwards. There was a push in 2020, 2021, towards finding principles around the ethical application and the use of artificial intelligence, an initiative that the U.S. back in the day had spearheaded. The UK had adopted its own responsible AI principles in 2022. And so this trajectory has a longer history ultimately. But really, as Jessica was saying, everybody’s in on it, right? So I think it was in 2017 when Putin uttered the infamous words now when he said, whoever is dominant in the field of artificial intelligence will dominate the globe. And I’m paraphrasing, but it was really seen as this ultimate technology that really everybody needs to focus their attention towards in the military domain. And so we have seen a significant shift in the last five years towards acquiring capabilities. And here we pivot to the companies that have been involved. There’s obviously the big, most prominently discussed company, which is Palantir and Maven Smart System, which is an AI targeting system. It’s a system that works with other systems. But we have big companies, neo-prime companies that are valued very highly, that have been very prominent in this space and have carved out almost a monopoly position in this environment. We also have to acknowledge the fact that these are dual-use technologies, meaning AI can be used for non-military purposes and then repurposed for defense applications. So we can only assume that everybody who has the technical chops to develop AI systems can ultimately make an AI system for defense purposes.

Steph 08:47 When we talk, you’re talking very much about this Palantir and Maven. These are very military applications. We saw them also at this technology in military congress that was in the Netherlands, a couple of the REAIM conference that was in the Netherlands. But we’re also talking partly about these big companies that we all know in AI, like, for example, Anthropic with Claude and OpenAI. And there was this big spat a couple of weeks ago where Anthropic and Claude kind of pulled back from the Department of the U.S., or the U.S. administration, and saying they wouldn’t let their AI be used for some of the military aims, and then OpenAI did make a deal with the Department of Defense. that also for targeting, or was that just more generally for defense use? Or is that because these kind of large language models like Claude and OpenAI with ChatGPT are being used for these Palantir and Maven systems, or is that a separate track?

Elke Schwarz 09:42 So without actually working with the systems, it’s very difficult to say this is exactly how these large language models are being used. So we assume that the press reports and the media reports about the use of Claude in conjunction with Maven Smart Systems are correct.  I do think there was quite a big media spectacle around this combination of this large language model, Claude, and Maven Smart System and Palantir’s software, data analytics software. I would assume that the use of large language models isn’t extremely pervasive at this time, but we are led to believe that it is the future of using an AI model and targeting.  Now, I would probably say that it is just one AI component amongst many. These systems are usually not just one specific delineated system, but tend to work with different AI models. Some of these AI models might be directed towards object recognition. Some might be large language models.  So Claude is likely just one component. There’s something else to be said about why Claude was so prominently presented as this huge problem and in contrast to the Pentagon, and what that does to really lowering the ethical guardrails or ethical standards that we had raised for many years in terms of pointing out the myriad of problems that arise with the use of large language models. But, you know, that’s perhaps for a little bit later in the conversation.

Jessica Dorsey 11:30 And just to dovetail off of that, I think it’s important talking about combining large language models. So these are, like you mentioned, statistical probabilities about how words are combined, right? So this idea that they are generating information based on sequences of words, etc., predictively, it doesn’t mean that they are doing it with the same kind of context that we as humans possess, right? What I keep seeing, and just recently in Dutch media reporting over this past weekend, was that the Ministry of Defense here, who also has a contract with Palantir and Maven Smart Systems, we’re not quite sure how they’re using it, but one quote from a piece in the NRC this weekend was that the CIO of the Dutch Ministry of Defense mentioned that Maven smart systems may not, as a toolbox, may not be that illustrious until you start combining it with the large language model. So understanding that what large language models do is they introduce what I think Elke and I share as two of the major issues in this space. They introduce the speed and the scale of information generation, and that speeds up operational tempo to a point where, from a legal perspective, this raises a lot of questions about the way that we are able to actually interrogate information that’s coming out of these systems. How do we make sense of it if we’re working and operating at machine speed? And then how do you then ensure that you can comply with the precautionary principle, for example, and rules around precautions to avoid, or to any extent minimize, civilian harm to the extent feasible before we even get into the problems that it raises with quantifying proportionality assessments? So lots of problems wrapped up into that idea of speed and scale introduced by large language models.

Janet 13:04 I want to just go back a couple of moments, Jessica, and pick up your cognitive offloading way of expressing this, because that’s all about removing humans from making decisions, essentially. And I’m wondering, because we’re in the accountability business, is the problem with that really that you can’t put a computer in the dock and that’s why humans need to be involved? Is it because the ultimate issue is who’s made a decision on this? Or is it something deeper than that that you mean?

Jessica Dorsey 13:38 I think it’s more multifaceted than that. Yes, that is one aspect of it. Accountability in the dock, legal accountability is one avenue. But I, in contrast to many lawyers, don’t see accountability solely as the purview of law. Like there are many more ways that accountability ought to be upstreamed within these systems. So the idea that we can build in compliance with IHL or build in legal and ethical guardrails at a much earlier stage in the design and development of these systems is actually something we’re working on, is that we want to get it to the point where everyone’s thinking about it at such an earlier stage that we don’t get to the dock, right? The idea is that if you start engaging with stakeholders at the earliest stages. So investors, you know, the R&D money has to come from somewhere. So you’re talking to maybe venture capital firms, banks, pension funds about responsible investment. And this is a new jargon for me as a lawyer trying to learn about how to help them discuss their own risk portfolio, understanding how to translate concepts from international humanitarian law into why they should care where they put their money. The same goes for our engagement with industry, talking with Palantirs, OpenAIs, Microsofts, big tech companies of the world. They have a different concern and they want to build a business case for their products. So how do you translate why they should care about these legal and ethical guardrails and how that helps them build a business case? We also engage with operators to talk about building in friction points. The idea is that these systems speed up operational tempo so quickly that ostensibly there’s time left over, right? Used to take humans so much longer and now the AI does it so much faster. But then the real crucial question is what are we doing with that extra time that we’re winning? Are we filling it up, which is what I’m afraid, seeing how this is playing out in Gaza and in Iran? Are we filling that up with just more lethality or more targeting? Or can we use it in a way that can introduce more friction points where human operators need to critically engage with it? So I think that I’m looking at accountability. I’m trying to upstream it so that we don’t see anybody in the dock because no, we cannot put a computer in the dock. We cannot hold machines responsible for this stuff. Responsibility from a legal perspective is first and foremost about state responsibility or individual criminal responsibility. And that is way down the line. And I’m hoping that we can prevent this stuff from happening to a larger extent than we currently are.

Janet 16:21 You say we, engaging with companies, we talking to, we trying to understand, who’s we?

Jessica Dorsey 16:21 We, the royal we, no. In work that I’m doing, certainly together with Elke, the Responsible by Design initiative is one of the initiatives that have sprung up. This is sort of a follow on from some work that we did within the Global Commission on REAIM. I was an expert member of that. Some of the colleagues, academics mostly, we’ve banded together to try to continue the work that started through the REAIM process. The idea is what is responsible use? Those are sort of abstract terms that happen at 10,000 feet and we’re trying to operationalize them. What does that look like at the ground level, right? So understanding, teaming up with academics like Elke, myself, we write together. We also, Elke and I have had meetings with industry. We’ve talked with them about their approach to some of these questions. Oftentimes when we do engage with these stakeholders, they don’t know what they don’t know. And the idea is they didn’t know they were supposed to be thinking about ethics in a certain way or IHL. And somewhere you can forgive that. At the same time, now we’re trying to put them on notice that they do need to be thinking about this stuff. So yes, the we is a royal we. It’s the academic work we’re doing, also with the Realities of Algorithmic Warfare project at Utrecht University. And Elke is involved in a number of other amazing things.

Steph 17:36 When we talk, one of the things that you both picked out as one of the problems with AI in military use is the proportionality. And can we dig a little deeper into that? Some of the media reports I’ve seen is of these models churning out an enormous amount of targets. And then what is the problem with that legally? Why would that create a problem if you had to explain it to me?

Elke Schwarz 18:08 Well, maybe we’ll roll back a little bit and think about how these systems actually work, right? So AI targeting systems basically take a large number of data troves. They collect data or use data that is collected from a range of sensors and other data sources to then statistically assess what constitutes a potential target. And that is based on a number of parameters that have been set either beforehand, or targeting systems could potentially also be used to discover targets, again based on a number of parameters that were set beforehand. And these parameters can be as wide as anybody who’s a man, or as narrow as this very specific named person with these exact features in that location, and so on and so forth. This computation of the various data sources then produces an output, a percentage output, say an 80% confidence that a suggested target is what is deemed to be an appropriate target or a legitimate target. And the data comes from a range of sources, you know, cell phone data, demographic data, radio data, all kinds of data sources. But it needs a large volume of relevant data, up-to-date data, appropriate data, to be able to function just at a basic level. The idea is that an AI system can crunch this data faster and spit out, if you will, appropriate and legitimate targets based on a certain percentage much faster than human teams could possibly be. That is the key point. The idea is always to speed up and scale up the process of lethal action. In this resides a particular logic which makes the question of accountability, responsibility very challenging, as Jessica had already highlighted. And I think what we can’t get away from is that, in principle, a displacement of human control or oversight or agency is precisely the point of systems that operate at this speed, that operate at machine speed with cognitive processing capacities that exceed those of a human, wherever the human is on the loop. And so I think that then raises the question of how reliable are these systems, right? So we already have a good sense that especially for very dynamic contexts where you may not have sufficient data or appropriate data or up-to-date data, the margin of error, the possibility for errors being made, is quite high. We also know that AI in large language models, or large language models themselves, present an additional challenge. So computationally they have a tendency to confabulate, to put things together that do not belong together, or hallucinate is sometimes the other term that is used. All of that means that there is always a margin of error that the human cannot necessarily identify being in whatever loop they are on or in at machine speed. So there is an inherent tension and this necessitates these points of frictions that Jessica raised. But for me as an ethicist, this raises some other questions also. In this machine configuration, human-machine configuration approach, what are the nodes at which the human can then take responsibility when the action is sped up and scaled up to the point where it becomes about not running out of targets? So what kind of cognitive insights can the human have in this kind of loop? What kind of knowledge does the human have about the accuracy of the data sources, the training data that was used, whether it was culturally sensitive data, and so on? What kind of ability does the human have to intervene rather than succumb to the urgency of action in this loop? So all of these challenges in this human-machine teaming and the logic of the system complicate questions of proportionality and distinction, ethically speaking but certainly also then legally speaking. And there’s also always a plausible deniability that is at work with systems that are not visible.

Steph 22:27 I wanted to talk about that because one of the things that we often hear with algorithms that are used by governments, for example, there’s a big scandal in the Netherlands about benefits fraud, where they use an algorithm to identify possible fraudsters, and it turns out that it predominantly spit out people with migration backgrounds as potentially big fraudsters. We always, when you talk about these large language models, they don’t often let you look under the hood to see what is kind of the special sauce that makes up this algorithm. So a very philosophical question, maybe, can you essentially know what this machine is using to come to the determination that it is? Because in my understanding, these companies that make these AI models don’t share what they put in. Am I correct in that? Or am I fear-mongering, Jessica?

Jessica Dorsey 23:21 You’re not fear-mongering. It’s correct. Ten out of ten, no notes here. Like transparency is one of the main issues. And without transparency, accountability in all of its shapes and forms is very difficult. And these are proprietary companies with, you know, IP patents on top of the technologies. So they’re not open with the kinds of source codes, etc., that underpin these. I think that makes it problematic, not only from an academic outsider accountability, a public accountability perspective, but also for militaries themselves who are using these things. If they don’t know how to trace back what technology happened in a particular operation based on AI recommendations, etc., etc., they’re not able then, legally speaking, to, you know, assess what might happen again in the same kind of context, right? And these are legal requirements, right? To understand what the effects of your attacks were in order to inform your subsequent attacks. This is whatever weapon you’re using, whatever weapon system or whatever technology you’re using, you need to be able to do that. And this kind of knocks back, I want to pick up on that point you raised about proportionality. I think we really need to, and I’m going to take the opportunity because I try to in every particular setting I’m in, to push back on this idea that proportionality is something we should preface. We should actually roll that back. Proportionality assessments in warfare, from a legal perspective, are a signal that you have actually failed to take the precautions you are legally obligated to take, and that civilian harm still remains. So the idea is that within IHL, we have the duty of constant care to minimize and avoid civilian harm. And you have to do a number of steps to do that. And Elke just spelled out a very helpful way what kind of data parameters are set, etc. Those are things you have to ensure if a target is generated, that that person or object is in fact a legitimate military target. You have to verify, you have to take actual steps to ensure that that individual that you based on, I don’t know, metadata triangulation, is in fact a legitimate military target. At speed and scale, that’s very difficult, if not impossible. So that’s one thing about the duty of constant care, but then also precautions in attack. All of that comes before we even can start thinking about proportionality. Proportionality is an exercise to weigh the anticipated military advantage against the incidental civilian harm expected. The idea is that you’re already expecting civilian harm. That means you’ve failed to avoid or perhaps to minimize it. So I think that getting the order of this is very important. And I think at machine speed, it makes it extraordinarily complicated. And the idea, as Elke highlighted so brilliantly, the logics that are inherent to this, the quantification logics, the idea that we can pin a number to an acceptable amount of civilian harm. We’ve seen that, as she also highlighted, in a longer trajectory with the drone warfare, the global war on terror movement, pinning the idea to a non-combatant casualty cutoff value, the idea that there’s a number that we would be okay to accept if we could just get that high value target. Fast forward to the most recent war in Gaza, the idea that IDF started pinning that number at 20 or 50 or 100 civilians that were acceptable to kill. What that says to me is that they are skipping over entirely their obligations to take those precautionary steps to verify the target. And they’re just moving toward this idea of maximum lethality. And I think that that’s a really dangerous shift we’re seeing.

Janet 27:01 Thanks for bringing in some of the current conflicts. As you’ve already mentioned, we’re in the week where extraordinary threat against Iran and now maybe a ceasefire. And by the time we put this out, who knows what will have happened. But just at the beginning of the bombing of Iran, there was this attack on the girls’ school, which left very large numbers of civilians, young girls, dead. And I found it really interesting that the immediate response was, of course, this must have been a problem with AI. But I mean, it looks, there still isn’t a full investigation. It looks like this was a problem. I’m just doing inverted commas around that with faulty basic information that the Pentagon had. And I was wondering what that tells us about AI. The way that we’re regarding AI and the way that we’re kind of almost automatically seeing it as the automatic enemy. I mean, it must be AI is going to be the problem for everything. We’re going to shove all of the responsibility for everything that goes wrong in a war over to AI.

Elke Schwarz 28:08 It’s interesting because without actually having the results of this investigation and without the investigation being actually thorough and being able to really tell us to what degree any kind of digital system or algorithm or AI was involved and to what degree it was just really poor, outdated information that was gathered by humans, it’s very difficult to ascertain that, right? Any wrong committed through technological recommendations or a decision to ratify that is going to be an uphill struggle. For the reasons Jessica highlighted, right? There’s proprietary technology at stake. And unless the companies are willing to allow people to look exactly under the hood and what has happened within the system, what kind of data sources were used to train the system or to provide the system with the data sources it needed, we don’t know. The two are not necessarily easily disentangled because it could be human failure in providing a data source that was then used for an AI model to do whatever computations are necessary, then work with other systems in tandem. And what you have is a really horrifically, tragically wrong, erroneous decision in that set of thousand targets that was targeted in the first 24 hours. But the logic really is one of speed and scale that informs the tactics of the military operations in the first place. And in some ways, speed and scale, perhaps taking less care, less time to interrogate the data sources to make sure you have current and up-to-date data. So there’s a bit of a logical transference from the technological context onto the human context. And we see this also in the treatment of ethics, for example. Very often, ideas of ethical reasoning are transferred into a kind of like a technological, technical realm or domain where ideas of utility and efficiency dominate, where humans are primarily captured as data points or numbered. And other more fluid categories or considerations kind of take a backseat or are bracketed out entirely in favor of a computational kind of approach to warfare, whether an AI system was in the mix or not.

Jessica Dorsey 30:47 We also mustn’t forget that there’s always some human somewhere in or on the decision loop, even if that is just the decision to use an AI system for something that it shouldn’t be used for. I mean, this is me banging on the same drum, but I think it’s important here because this particular incident that you raised about the girls’ school, this was on the first day of campaign. So ostensibly, this was part of the target package that they had prepared for the longest amount of time, or they had the longest amount of lead time. This school and the understanding of the school as no longer part of the military compound was able to be reconstructed through open source investigation techniques by journalists, by NRC, by The Guardian, by Al Jazeera and others. Within two days, they knew that this was a school and it was very clear on the satellite images for at least ten years. So that says to me, the U.S. knew or should have known. So this translates, legally speaking, to the fact that, again, it is a failure, an abject failure to take precautions. And this is being done by one of, or both of, the most advanced technologically advanced militaries in the world. So my reading with respect to precautions, to do an academic flex as one is wont to do, which we’ll link to in the show notes, of course, but for whatever it means, it’s about a careful reading of what IHL requires. Precautions, you have to take the precautions that are feasible. And that means practicable or practically possible. This is how this has evolved over time. That doesn’t mean you have to do everything possible and exhaust all remedies. But the idea is that if you have a technologically advanced system, if you have ISR, so intelligence, surveillance, and reconnaissance that allows for you to have a heightened amount of information, and I would argue that Israel and the United States have that, you also have a parallel heightened responsibility to take precautions to prevent this from happening. So it’s absolutely unacceptable that we just write this off as another mistaken war in quotation marks and move on, because this is not how law was built to function. And that kind of knocks on to the political responsibility of other states to respect and ensure respect for IHL. And that is also tragically missing in this particular case.

Steph 33:09 And we’ve only just scratched the surface, I think, of a lot of these issues, but we have a limited time on the podcast. So we wanted to kind of wrap up with how you both propose to go forward. And Jessica’s already mentioned some things. Maybe you want, is there more that you think is really needed in terms of discussion and regulation?

Jessica Dorsey 33:36 From a legal perspective, I think it’s really important, the ICRC is leading the charge on this about IHL, but understanding the object and purpose behind why we have the Geneva Conventions, what they mean, and that then is an obligation on states to give a good faith reading to it, to act in the letter and the spirit of the law. So yes, there is some gray area. There is open room for interpretation, but interpretation not to the least common denominator, especially when your interpretation fed into AI and advanced technologies means you will exacerbate whatever existing problems there are at speed and scale. So we need to make different choices and prioritize civilian harm mitigation, avoiding them, prioritizing precautions, over-proportionality logics. So I think that those are ways that we can make inroads also in this kind of multi-stakeholder dialogue. We have to do better work, I think, as lawyers to translate information why IHL is important to these multi-stakeholders, to explain to investors why it matters to their portfolio, or to explain to industry why they can build a better case if they can demonstrate compliance prior to procurement. But that requires work and it requires practice and we require more people to come into the mix. So I would put this out as an open call that all are welcome if you want to help us do things responsibly by design.

Elke Schwarz 34:54 So just to add to that, I think the specificities really require much more political will to understand that without this political will, this becomes kind of a runaway train. And in this runaway train, civilian harm is all but guaranteed, or an expansion of civilian harm is all but guaranteed. I think, you know, there’s crucial work being done in this project that’s responsible by design, crucial work being done thinking about what responsible artificial intelligence in the military domain can mean, and what kind of limitations we have to put onto AI models or the use of certain AI models in certain contexts. So that is already underway and has been underway for quite a while, but somehow there’s waning political will, which always butts against a crisis narrative or a sense of urgency and a mandate of acting now. But I would want to just in closing, perhaps zoom out and make sure that we understand that the ethics of this is really crucial and make sure that we understand that we as humans have an interactive kind of relationship with the technologies that we create. AI-enabled systems will not bring an end to human involvement in warfare, but they will very likely further work towards a distortion of our human relationship to violence. It will very likely lead to a technification of practices and processes, and that brings with it a certain dehumanization. So bringing it back to the ethos of IHL, bringing it back to the ethos of civilian protection, foregrounding the reduction of civilian harm is really crucial because what we’re seeing at the moment is the opposite. What we’re seeing at the moment is an erosion of civilian harm practices, reduction or moral restraint in the use of force in favor of a workflow management process and targeting.

Janet 36:48 I was wondering when Jessica said that we had to have Elke on as well, why on earth we had an ethicist joining the podcast, but I think I got my answer in the end there. So thanks Elke for that summing up the disaster that we are heading towards unless we all grow up in some form. We’ve only got the chance for our normal regular Asymmetrical Haircuts question right at the end, which is to both of you, what have you been reading, watching or listening to recently that you would like to recommend? It can be something in the field or something out of the field. I can make a stab at what Jessica is going to flash on the screen in front of us. What would you like to tell us about, Jessica? I wonder.

Jessica Dorsey 37:36 Well, I think Elke’s comments were just so lovely in just plugging my own work. This is what I’ve been thinking a lot about lately. I just defended my PhD not long ago. This is keeping our humanity. So understanding that the work that we’re doing here is trying to foreground what Elke mentioned to keep the human with the upper hand. So that was what I was working on for a long time. But something I would recommend, I’ve been also trying to do deeper dives in tech-related podcasts. So understanding this technology and one of the greatest ways to do that, I think, is with a friend of mine and a colleague that we also work with, Zena Assaad, has a great series of podcasts called Responsible Bites. And she is a system safety engineer. So it’s a great perspective to have from somebody with actual deep technological experience. But she does a nice series with interdisciplinary experts. So I would recommend that to your listeners as well.

Elke Schwarz 38:32 My recommendation, I mean, you know, my book is quite old at this point, but it’s still very relevant. And it is called Death Machines: The Ethics of Violent Technologies, which traces some of the logics that I have been speaking about in the encroaching technification of thinking about ethics in the context of warfare and the consequences that that brings with it. But what I’m currently reading is actually the history, or a history, of prediction systems. The book is by Jill Lepore. It is called If Then, just from 2020. And it is a really interesting excursion into the predecessors, if you will, of Maven’s smart system, kind of prediction as a political system, which comes out of the behavioral sciences in the 60s and 70s. Always very interesting to connect what we’re talking about today to historical origins. Nothing happens out of nothing. Everything is situated somewhere. So I recommend that book.

Steph 39:33 Thank you so much for your recommendations. I’ll check out the tech podcast. I’ve been listening to Hard Fork, the New York Times podcast, which is also very much about AI, but it’s more about the business side, just to also get an idea, because working for a large media company that also likes to speed up news progression, you have to also reconcile yourself to dealing with certain aspects of AI in your work. And so I’m trying to school myself to understand what this stuff is about, because it’s coming in every kind of aspect of our work, not only the stuff that I have to write and how I have to write it, but also now, of course, with the warfare accelerating in that. So thank you for these recommendations. And thank you very much for coming on the show. I really enjoyed the double act. And I think I’m afraid that if things are progressing, keep continue to progress the way they are progressing, we will have to have you back on a lot more and pick out some more of these details because there’s so many interesting things. I could talk for hours about this. Probably you could both also. So let’s pencil in that we’re probably going to need another session, maybe this year even still. Thank you so much for taking the time to talk to us.

Jessica Dorsey 40:52 Thank you. It was a pleasure.

Elke Schwarz 40:54 Thank you very much.

[OUTRO MUSIC]

This was asymmetrical haircuts, your international justice podcast, created and presented by Janet Anderson and Stephanie van den Berg. This episode was created in partnership with Justiceinfo.net,  an independent site covering justice efforts for mass violence, and with the Hague Humanity Hub. You can find show notes and everything about the podcast on asymmetricalhaircuts.com. This show is available on every major podcast service, so please subscribe, give us a rating and spread the word.