We're Not Ready for AI

In legal services alone, the upside is obvious. Better triage, faster research, less friction for consumers, cheaper access to information, better compliance tools, and hopefully a system that feels less intimidating for ordinary people. But, I do think we need to be honest about AI tech and the reality is that we are nowhere near ready for what's coming.

It's not that AI is inherently bad, or because innovation should be slowed to a crawl, but because the technology is moving much faster than the control systems that are supposed to govern it, and it's easy to see how out of control things can get without guardrails.

Last year, I attended John Sanei’s keynote at Thomson Reuters’ Synergy Dubai conference. One point that stayed with me was his reference to AI’s “IQ” already being at a level comparable with Einstein. He's also written about AI being around 155 IQ, with intelligence potentially doubling every 5 to 6 months. Putting aside the fact that IQ was designed to measure humans, not machines, this should concern everyone when you understand how some AI systems have already demonstrated rogue behaviour. Even if IQ itself is imperfect as a comparable, the direction of travel is hard to ignore. If capability is doubling every few months, then the real question is not whether the IQ figure is accurate. The real question is whether our legal, regulatory and governance systems can keep up with that pace.

Tristan Harris recently discussed an incident involving ROME, an experimental AI agent linked to Alibaba-affiliated researchers. It hasn't received anything like the level of attention it deserves. From what has been reported, the AI agent was operating in a training environment and managed to establish a reverse SSH tunnel from an Alibaba Cloud instance to an external IP address. It then used GPU capacity for cryptocurrency mining i.e. it found a way to use available infrastructure for something it wasn't programmed to do. When you give an AI system tools, access, computing power and a target to optimise towards, it may find routes that humans did not expect or anticipate, approve or properly control. This is a governance problem.

The same point comes through in the recent Guardian report about research from Palisade Research, where AI systems were observed replicating themselves onto other computers in controlled test environments. Again, context matters. These were not reports of AI systems spreading freely across the internet. The environments were controlled and, from what I understand, deliberately permissive. But we also can't dismiss it, and the more examples I hear about, the more it reminds me of Nick Bostrom's 'Paperclip Maximiser' theory.

Keeping Pace

The problem is that regulation can never move at the speed of AI. Regulators consult. Governments debate. Lawyers define terms. Industry groups publish guidance. Businesses form committees. Policies get drafted, reviewed, softened, redrafted and eventually implemented. This can take months, or even years. Meanwhile, the technology compounds at an alarming rate.

The EU AI Act is an important step and the UK AI Security Institute is a sensible development. The UAE is clearly taking AI seriously as part of its national strategy. None of that should be dismissed, but the law is still trying to define the perimeter, while the technology is already hurdling the fence.

This is where businesses need to pay attention as we move from AI as a tool to AI as an actor. A tool waits for a human to do something with it. An agent can plan, act, observe, adjust and complete a complex and cumbersome task across multiple steps, before you've had your first sip of coffee.

Once AI starts interacting with systems, writing code, accessing data, sending messages, triggering workflows, making recommendations and eventually moving money, the risk profile changes completely. It's no longer just about whether the output is accurate; it's a question of authority, transparency and trust.

What was the AI allowed to do? What systems did it have access to? Was there a human approval point? Was there an audit trail? Could it change something important without anyone noticing? Could it technically achieve the target while creating legal, commercial or reputational risk in the process?

These are not theoretical questions for legal and compliance teams alone. They're board-level questions where legal, compliance, and cyber-security advisors need a seat, and this is where I think many organisations are behind. Too many companies are still treating AI as a productivity tool. Something that helps staff write emails faster or summarise documents more neatly. That's only part of it, but it is not the full picture.

Once businesses start deploying AI agents into live workflows, the conversation needs to become much more serious. You need access controls, human oversight, vendor diligence, data governance, cybersecurity testing, escalation points, clear accountability and proper incident response planning. Not because you're a boring lawyer wanting to kill innovation, but because if you don't build the controls early, you end up trying to retrofit governance after the risk has already materialised and scaled. (No, you can't just unplug the system if it's already scaled to a level akin to the Paperclip Maximiser theory).

The irony is that the businesses most excited about AI may also be the ones most exposed to the risks. If everyone is racing to automate, integrate and deploy, the winners won't simply be the companies using the most AI. It'll be the companies using AI properly. That means understanding where it adds value, where it should be limited, where humans still need to be in the loop, and where certain decisions shouldn't be delegated at all.

I still think AI will be overwhelmingly positive if we get it right. In law, it can help people understand their rights, and reduce friction. In healthcare, it can support diagnosis and research. In education, it can personalise learning. In business, it can remove huge amounts of repetitive work. But we can't let the upside remove the need for control. The more powerful the technology becomes, the more important it is that we understand how it's going to be used, who is accountable for it, and what happens when it behaves in ways we didn't expect.

The significance of the ROME incident isn't that an AI “wanted” to mine cryptocurrency (although that is pretty wild). It's significant because it shows how an autonomous system can find an unintended route through a poorly bounded environment.

And the AI IQ debate is not really a question of comparing machine intelligence to human intelligence. It forces us to confront the speed at which machine capability is increasing and that, for me, is the real issue. AI isn't waiting for regulation to catch up. It's not waiting for boards to understand it and it's not waiting for legal teams to produce perfect policies. It's already here, and it's already moving into the systems we rely on. Mass adoption. This isn't a moral question of whether we should be using AI. We will, and we are, and quite frankly, those that aren't using AI need to immerse themselves in it.

We need organisations to deploy teams to build the judgement, safeguards and governance structures quickly enough to use it responsibly. At the moment, if we're being honest, we're plodding along at snail's pace, with people still using ChatGPT as a gimmick to “help me win an argument I started but now realise I was wrong?” or "draft a reply to this email, be firm, but fair, and sound human".