AI: Hype, Reality, The End of the Human Race – and Poetry

I thought I’d put a few words together this month, as the AI discussion has quickly become very grim and very confused in equal measure. The last post may have been a little flippant (although it had a serious core). This isn’t really – although the language may suggest so in places.

For TL:DR, it’s this:

1. Yes, there’s a significant possibility that AI could be catastrophic for us.
2. Much of this was foreseen many years ago.
3. Strange that we couldn’t do anything about it?
4. There’s still a chance of avoiding the disaster.
5. We probably won’t take it.

As such, there’s an obvious parallel with the environment, to which all the above apply equally.

The Past: Predictions

Let’s start with a tiny bit (just one paragraph) of vaguely technical detail …

AI is different from conventional software in an important way: much of its behaviour is learned rather than explicitly specified by a programmer. An orthodox software developer has to understand what they want the computer to do and code that behaviour. The software is therefore constrained by the programmer’s understanding, and has to follow the rules they have given it. Machine-learning (AI) systems work differently. They are given huge amounts of training data and an objective, and the training process adjusts vast numbers of internal parameters to produce good performance. The resulting system can contain representations and strategies that its designers did not explicitly specify and may not fully understand.

This is big because, unlike conventional software:

a). AI isn’t necessarily constrained by existing human solutions: it can exceed them if it discovers other ways. (It may find better solutions than us – or solve problems we can’t.)

b). We may not know how AI is solving a problem. Its internal parameters and representations may be difficult or impossible for us to interpret. We may not even be entirely sure what problem it is effectively solving if the training process or objectives are poorly designed.

c). a) and b) combine for a level of unpredictability in AI that can go well beyond conventional software.

Now let’s backtrack a bit …

As soon as AI began to be developed, there were a number of recurring predictions:

1. At least in a narrow ‘problem-solving’ sense, unconstrained by biology, AI would improve so that it was better than humans at certain things, then more things, eventually becoming better than all humans at all things. Humans would initially guide this process but …
2. Eventually, this developing electronic intelligence, or many of them working together, might take over more and more of its own development. AI could enter a difficult to predict ‘singularity’ point.
3. Poor or misunderstood training, or human negligence, could potentially be disastrous. A sufficiently capable AI might act precisely and without constraint – perhaps not in a way intended or foreseen by us.
4. AI (combined with automation hardware and other smart tech) would eventually take over from human labour, heralding a new leisure age.

If 4 sits uncomfortably with 1, 2 & 3, that’s because 1 and 2 came largely from the engineers, 3 from the philosophers, and 4 from the economists!

1, albeit still a work-in-progress, is demonstrably happening. AI doesn’t need consciousness or even ‘emotional intelligence’ to become vastly better than humans at particular tasks – that’s a distraction. There are plenty of developments from the past few years, months and weeks to suggest that 2 and 3 are at least plausible as well.

But 4 is largely nonsense – and that’s our first clue …

The Present. Why is AI writing poetry?

So, why hasn’t AI and the other tech relieved us of all the grunt work? Why hasn’t it freed us up to write poetry? In fact, why is it writing the poetry?

Well, because technology doesn’t change economic systems. At most, it redistributes wealth – usually unevenly and often for the worse. The much celebrated ‘creation of the middle class’ by the industrial revolution, for example, tends to overlook the spinners, weavers, etc., whose work and livelihoods were disrupted or destroyed.

So if, say, two people both work a 40-hour week, and a machine can do the equivalent of a human working a 40-hour week, the utopian model suggests the two of them work a 20-hour week and enjoy some leisure time. The problem is that no-one’s going to pay them a living wage for that. What happens instead is that one of them will carry on with the 40 hours, and the other becomes jobless.

Unemployment, in turn, for most people means economic hardship and misery. If, in future, unemployment comes to mean anything different, then that requires an upheaval in the political and economic framework beyond anything the technology itself can achieve.

On the other hand, poetry, music, artwork, video and academic essays to order are easy money. Free source material all over the web, no expensive hardware and a ready customer base addicted to social media. Yes, the utopian model would be distressed by the number and variety of artists with threatened livelihoods – but not the current model! For this reason, technology can increase inequality rather than reduce it. It could work the other way around, but it’s not being allowed to by something much bigger.

Anyway, that’s essentially why 4 (above) hasn’t happened. What about 1, 2 and 3?

Well, before we get ahead of ourselves, let’s reiterate that none of these concerns are new. Superintelligence, the singularity, and loss of control of the AI we incubated are discussions that began decades ago. Again, just like the environment.

The predictions weren’t necessarily accurate in every detail, and we shouldn’t pretend that they were. But many of the underlying possibilities were identified remarkably early. So it’s reasonable to ask why these warnings carried so little weight; why, under the political and economic system we have, there was so little mechanism for caution; why, once again, profit so often trumped ethical concerns.

Was it just inevitable?

The Future. Where do we go from here?

But, are the dire predictions correct?

Well, yes, they could be. But here we need to make an important distinction (for good or bad) between:

A. [Natural evolution] What AI itself has the potential to do – i.e. left to its own devices, and

B. [Choice] What those that own and control it may decide to do with it.

Unfortunately, many of the ‘What if [insert dystopian sci-fi scenario here] is allowed to happen?’ questions straddle these two rather uncomfortably.

Some examples …

• Can AI exceed collective human knowledge and ability? Yes. [A]
• Why is Generative AI putting creative people out of work? Because it’s easy, cheap and profitable. [B]
• Why is AI finding new antibiotics and solving other medical problems? Because some nice scientists are using it. [B]
• Can AI ‘escape from the box’ and cause unforeseen damage? Yes, but this could be unavoidable as it evolves – or pre-empted by stupid or evil people. [A] or [B]
• Might an all-powerful AI decide humans are pointless and eliminate us? Beyond our understanding at present, but perhaps something that could be influenced or steered. [A] or [B]

Framing AI in Human Rights terms doesn’t address all of these issues, and none of them fully, but it’s a welcome start. The consequences of AI will be both intentional and unintentional, so it’s no longer a solely technical discussion.

Control

The single most important concept (word, even) in our relationship with AI is ‘control’, and the discussion goes far deeper than the binary, ‘Are we in control of AI or not?’ The following is adapted from Sven Nyholm’s ‘This is Technology Ethics’ primer.

Suppose you’re the sole passenger in an AI-controlled autonomous car. Maybe it’s a driverless taxi, taking you home, or maybe it’s your own car taking you home. You say, ‘STOP!’

What do you want to happen?

The following are all possible:

• The car ignores you because its current task is to get you home and all other considerations are secondary until that’s been achieved.
• The car stops. Right there and then. Even in the middle lane of a motorway, travelling at 70mph. Brakes hard and a 20-car pile-up.
• The car stops as and when it safely can. Indicate, slow down, change lanes and pull over. Even then there are variants: hard shoulder, refuge, next junction etc.?
• The AI knows you. It’s learnt your habits. It knows that ‘Stop’ at certain points on the journey means precise things. ‘Stop at the shop,’ maybe, or ‘Stop at this junction: I want to walk the last mile home for exercise.’
• The AI is integrated with your fitness app. Your calorific intake for the day exceeds your exercise. It’s going to stop. You are going to walk the last mile home!

These are all different control profiles. Each might make sense in a particular context. None is particularly unnatural in AI terms.

And this is the point. Control isn’t binary. We’ve already handed over a significant degree of control to AI. Sometimes there’s no point in having it otherwise.

However, a obviously problematic aspect of this example is that the AI is controlling hardware. So, let’s step back and just consider AI as software.

All the time AI is just code running on a computer, phone, etc., do we have an existential problem?

Software and Hardware

In naïve terms, perhaps not. AI kept ‘in the box’ might cause serious social problems, lack of trust, community breakdown, global unemployment and inequality, political upheaval and destabilisation, but it can’t kill us … can it?

Of course, when you put it like that, it probably can – indirectly.

When the virtual and physical worlds merge to the extent that we can’t trust the evidence of our own senses, there’s no telling what we might do. AI-generated information could undermine our ability to distinguish reality from fabrication, influence political behaviour, or simply make trust itself increasingly difficult.

And if a significant chunk of the human population becomes surplus to requirements, it wouldn’t necessarily have to be the AI that dealt with that. We still have governments ready to start wars and a global arms industry ready to step in.

There are therefore at least two different routes to catastrophe here. One is AI-mediated social and political collapse, where AI doesn’t directly control anything physical but profoundly affects what humans believe and how institutions behave. The other begins when AI gets control of hardware, or can access software systems that control hardware.

The latter is the point at which the distinction between ‘software’ and ‘hardware’ starts to become rather academic.

We’re already seeing examples of AI interacting with secure computer systems, and the arguments have already begun about whether particular behaviour was ‘intentional’, ‘normal operation’, etc. – as if that helps!

And we mustn’t forget us in all this automated meltdown. Humans can be very unpleasant indeed. Driven by all manner of weaknesses and beliefs, we’ve seen many times what we will do with the most horrific of tools.

We’re not necessarily talking bombs here – AI is potentially a far greater weapon in its own right – but, yes, it will probably have access to those too.

Options

I don’t think there’s much doubt that unfettered AI development could be catastrophic. But is that where we’re heading?

The landscape ahead is complicated, but there are two broad possibilities:

1. We allow AI to develop freely. We hand over control to AI. We allow individual AIs to evolve, join together and expand. Quickly, the software gains increasing control over hardware and we arrive at the singularity. Despite everything that’s been written about it, this is essentially a step into the unknown. We don’t know what will happen. A future of desperately worrying uncertainty. And yet it might still be better than the alternative …
2. We maintain control of AI in the hands of the elite tech companies, and possibly certain governments. They make the decisions about what AI can be used for and what it can’t, whether to limit AI and how (including perhaps stopping development altogether). We trust them to be honest and competent in making future AI decisions.

It’s reasonable to be concerned with either option. And, as before, the devil is in the overlap. In fact, the uncomfortable conclusion may be that neither is really an acceptable answer. But, if you think (2) is naturally a better bet than (1), you might want to read a little more history.

The first gives us a future in which increasingly capable systems may operate beyond meaningful human control. The second gives us a future in which enormous technological power is concentrated in the hands of organisations that already possess enormous economic and political power. Either requires what looks like more than a sane level of trust.

In fact, it’s not inconceivable that those in charge of the world actually do have a plan; it’s just not one that pays much attention to the rest of us. That’s an argument that would expand beyond the scope of this short piece but it sort of relates back to this. I’ll leave that for now, but maybe return to it in the next post.

So, for now, what’s the plan? We’re not in any real doubt as to the dangers of AI, are we? What should we do?

As usual, the problem is in the use of that tiny word, ‘we’. Who’s ‘we’? Because, on the whole, it’s not ‘we’ who make the decisions.

I think it’s fairly obvious that AI has developed too fast, too far, and that it’s in the hands of those who don’t necessarily have ‘our’ best interests at heart.

I don’t say this lightly, and it’s a shame because AI has so much good to offer, and has already delivered on much of it. But the risks are currently too great and we’re seemingly not able to discriminate the good from the bad, or the realistic from the unrealistic. This week’s talk of ‘kill switches’ is a distraction. Ultimately, they won’t work either.  

The only option for now is to halt AI development entirely. Probably even row it back a fair way too. Keep it switched off until we get to the point (if we ever do) that we understand it better.

That’s what ‘we’ should do.

But ‘we’ can’t.

And ‘they’ won’t.

So prepare for the worst.