Downing Street to pitch AI Safety Institute at global summit

As part of the AI Safety Summit agenda, the UK is set to propose an AI Safety Institute which will assist governments in evaluating the risks frontier AI poses to national security.

The proposed institution would emerge from the UK’s current Frontier AI Taskforce, which is currently negotiating with major AI companies like Anthropic, DeepMind and OpenAI to have access to their models.

The Secretary of the Department for Technology, Science and Innovation, Michelle Donelan, has suggested the proposed body could become a permanent or even international institutional structure.

The agenda of the AI Safety Summit points at the UK’s heightened efforts to become a global leader in AI regulation, a point that has been stressed in recent government speeches.

Carving its place in the AI future

The government’s proposal to establish an AI Safety Institute is part of cumulative efforts to establish itself as a leader in AI regulations.

A government spokesperson says, “International discussions on this work are already under way and are making good progress, including discussing how we can collaborate across countries and firms and with technical experts to evaluate frontier models.”

Just last week, the government announced the launch of 800 scholarships worth £8m to equip eligible students with practical AI and data science skills.

Edinburgh was revealed as the planned location for building an exascale computer with 50 times more power than the UK’s current top end system.

Alongside another supercomputer built in Bristol, facilities of this scale are expected to unlock major advances in AI, medical research, climate science and clean energy innovation.

“Establishing an AI Safety Institute will play a key role in tackling the risk posed by AI regarding the cyber threat, allowing frontier AI models to be scrutinised,” explains Oseloka Obiora, CTO at RiverSafe.

“This will support businesses as they consider the implementation of AI and help them to ensure robust cybersecurity measures are in place to protect themselves from risk,” he continues.

What could an AI Safety Institute mean for startups?

The UK’s focus on regulations worries some startups who fear it will stifle innovation. An institutionalised and bureaucratic approach to regulations could potentially worsen this concern.

“Everyone is talking about AI regulation (which is the solution), but what problem is this solving? Depending on the problem, different solutions could be needed, and the solution may not be regulation at all, even though that’s the buzz word that’s being thrown around at the moment,” warned Rafie Faruq, CEO and cofounder of Genie AI, in a recent conversation with Startups.

Voicing a similar sentiment, James Clough, CTO and cofounder of Robin AI, stresses, “In order to be effective, safety regulation cannot be seen to be Luddite in nature, and so must be focused only on serious risks.”

As discussions begin concerning the AI Safety Institute at the AI Safety Summit, startup founders will want to have a seat at the discussion table to ensure regulations don’t stymie innovation.