Well, on any journey, to work out the best way of getting somewhere its often useful to know where you’ve come from!
1 Like
A very important issue, and one which manufacturers of self-driving cars are currently grappling with - in particular when accidents occur caused by failures in the car’s self driving ‘intelligence’.
Who is accountable? The car maufacturer, or the software programmer(s) who designed/programmed the AI, or the ‘driver’ who didn’t immediately take control of the self driving vehicle when they realised an accident was imminent, or …?
There is a current, and ongoing, legal case in the US of a self driving car driving straight into a lorry crossing its path at a junction. Why? Well the lorry belonged to a farming company and its side had been painted with sky and a road in the middle, with cows in fields either side. The self driving car’s AI mistook the picture of an ongoing road for a real road and just ploughed on straight ahead.
So, who should be held accountable in this case?
As ever, technology is outpacing the formulation of new laws and regulations to protect people and to make sure the right people (whomever these may be) are held accountable for such catastrophic and potentially life threatening failures.
1 Like
EAIA says that the user is responsible for the use of AI. If a Foundation LLM lets bad [things] happen (either by bad prompting, bad training DB, hallucination, whatever circumstances) the user is the one who trusted it and didn’t prevent [things] himself. The user is the only person who can be held responsible (and who can be sued), so it is unlikely to have human-free AI use in near future.
And I support that.
No machine should be allowed to make choices that heavily influence a human (though tech-bro CEOs seem to think unethically different).
Cheers
2 Likes
I’m not familiar with the ‘EAIA’ (i.e. I’ve never heard of it), but from your description do I assume its some kind of AI industry group, or maybe a lobbying group of AI companies, who’d rather somebody else (i.e. the user) accept liability for issues when AI models behave adversely?
And I am also guessing that what they are issuing are ‘guidelines’ - because only nation states can pass laws covering the regulation of AI.
An interesting point of view, but also a problematic one I think. What if the user has no opportunity or capability to prevent the AI from making mistakes or preventing bad things from happening? What if they don’t even know that they are using an ‘AI powered’ product?
Going back to the example of the AI powered self driving car, a number of countries have now licensed self driving taxis (‘Robotaxis’), with many others, including the UK, currently trialling them with a view to issuing licences in 2026/27.
Your view would suggest that if a robotaxi collided with, and killed, a pedestrian (say because of some fault in the AI’s reasoning model such that it didn’t recognise/register a pedestrian was in its path) then the robotaxi’s passenger (i.e. its user) should then be held responsible.
Furthermore it would be that person, the user of the robotaxi, who would be sued (in this case prosecuted, because killing a person without intent is manslaughter, which is a criminal offence, not a civil one), yes? The penalty for manslaughter is, in pretty much every jurisdiction, a prison sentence.
Now obviously that fault in the robotaxi’s AI model remains, so if another pedestrian got killed presumably another, different, robotaxi user would then then get prosecuted for manslaughter and sent to prison.
And so on …
A robotaxi user might be a mother with a baby, or a frail 80 year old woman, or a couple of lads drunk from a night out who sensibly decided to take a taxi home rather than drive themselves. In other words none of them are likely to be able to prevent such an ‘accident’ from occurring. I’m not even sure that robotaxis allow passengers (users) to take control of the vehicle to prevent an accident - and even if they did, what would happen if the user did not have a driving license?
And all the time this happens the people, or company, who designed/developed this AI model would bear no responsibility or liability for these deaths (other than perhaps in the court of public opinion), only the AI’s user would be legally responsible.
In reality, under current laws (and I accept that these laws were not written with AI in mind, nor might they be the final position on the regulation of AI) if an AI model ‘misbehaved’ and, as a consequence, somebody died, it is likely that the company responsible for the AI model (e.g. Waymo) would be charged with corporate manslaughter, not the end user.
Incorporated organisations, like a limited liability company for example, are treated as a ‘legal person’ in the same way that an individual is a ‘legal person’. This means that such organisations can be sued, prosecuted, etc in their own right without having to establish exactly who in the organisation actually made the ‘error’. This is because (a) organisations can be large and have many 1,000s of employees, so it may be difficult or even impossible to identify responsible individuals beyond reasonable doubt, and (b) often ‘errors’ of this nature are not down to an individual, but to a chain of individuals (i.e. in AI those who design, then develop, then test, the management who decide, etc).
In cases involving a corporate prosecution you also have the doctrine of ‘vicarious responsibility’. This is where the employer (usually embodied by the directors of the company) is held legally responsible for the actions of its employees in a work context, even if they did not personally make the error themselves. So in a successful prosecution of an AI company for corporate manslaughter, it will likely be the directors of that company who end up in prison.
At the moment that’s probably the right way of dealing with this in my view, and it will certainly focus the company on ensuring they do all they can to prevent such errors and to correct them as soon as they are found.
Whether it will still be the best way when AI develops further I don’t know.
1 Like
Yes, I agree, especially if there is a slowdown and a significant shortage of the most requested and used applications. Its advantage is the speed of development, knowing that the secret to the success and spread of any operating system in general is the abundance and diversity of applications that meet the needs of its users from all categories.
For clarification:
“EAIA” means the “European AI Act”, valid since 01.01.2026, in effect from 01.09.2026 on - a law passed by the EU for EU member states.
Cheers
Further edit: I was confronted with that kinda new law only last week so I do not have enough information about its depth, regulations and application/executive measures.
For the Robotaxi issue, my guess is that not the passenger but the holder (the company that offers taxi rides with those) is the jur. person who is deemed responsible, but that is only a guess.
We Germans love laws, and I think there will be a strict regulation or ruleset for these things somewhere, and if not, it will be made soon ^^
Another edit: take that info with a grain of salt. Combing the web for that law only shows a first small step from 2024, while my talking points were based on an AI certification course I had to take last week. Let’s see if/when further valid information is provided.
Ah, thanks. I shall have to read up on that one. I suppose not valid here in the UK though since we left the EU.
However, I do find it a remarkable and fundamentally flawed decision (if this is indeed what the EU has decided and enacted in this statute) to place the legal liability for any adverse consequences of a AI model’s operation on its end user, who will likely have little or no control over the way it operates and likely no opportunity to prevent or change the model’s operation.
It would also appear to give carte blanche to AI companies to do what they like, knowing full well they would have no legal liability for things going wrong.
2 Likes
But even in this case a ‘taxi license holding company’ would not be able to prevent accidents caused by AI developed and controlled by another company - they are more removed from the situation than the user in the taxi. Yet they would still be legally responsible for them.
This sounds like madness!
Idk how AI can be fully tested, audited etc. yet. As long as a developer is unable to control its creation, it can be seen as an unsafe product, right? There’s a nice way of developing where the AI not only tells you what it did to the code but also lets itself tell by the programmer what they understood of that report (making clear if both AI and developer think and work together in synchro or if the developer lost it somewhere and just hopes that AI doesn’t put smelly stuff into the code), but I don’t know if that can be implemented easily (and the product gets a nice “fair programming” sticker).
Cheers
Edit: typo correction
From the citizen’s point of view, yes complete and utter madness. This is what lets “self-driving” software bail out a fraction of a second before a crash, handing over the controls to a person, who probably isn’t all that ready to take over. (After all, what’s the point of self-driving cars if you still need to be as vigilant as when teaching a teenager to drive?)
We don’t allow that when it comes to other safety-related systems of a car or medical devices.
I get the impression we’re so afraid to be left behind in the AI race, that any barrier has to be removed. Make bulk transfers of PII (including medical data etc) to AI businesses, bury reports on the environmental footprint of data centers in secrecy, shift risks to end users, remove guard rails.
2 Likes
The driver is always responsible until Level 4 is reached. In level 3 under certain circumstances the driver is allowed to do other things. Level 2, which most cars have, the driver is fully responsible to take control in seconds…in my opinion this is useless. If i have to be able to take conttol in seconds, i would drive by myself.
2 Likes
I am voting for using AI with development work, when AI user has taught needed skills for LLM for doing exactly what has been asked. Human is still responsible for these codes. Good tests to repository and reviews, that’s good start for using AI as helper tool. Autonomous agents are not yet good solution but will take care of maintenance of our sw code projects near future. And we are responsible for AI agent engineering.