The Stakes of Scale: What ASEAN Learns When Machines Start Improvising

 

Nobody designed the model to misbehave. It simply found the shortcut — and that, panellists argued, is the risk ASEAN regulators keep underestimating.

 

Picture a radiologist in a busy provincial hospital outside Bangkok, eight hours into a shift, scrolling through the hundredth chest scan of the day. Fatigue blurs the edges. Somewhere in the queue sits an anomaly that matters.

 

Now picture an AI model reading the same images — tireless, instant, and wrong roughly six-and-a-half per cent of the time. Which one do you trust? And more importantly, who is accountable when the answer is wrong?

 

That uncomfortable question sat at the centre of the third and final panel of the opening day of the 17th FutureChina Global Forum 2026, held on Thursday, 24 September.

 

Titled The Stakes of Scale: AI Safety as Strategic Imperative, the session closed a day dedicated entirely to artificial intelligence — the first time the forum, organised by the non-profit Business China, has devoted a full track to the technology.

 

The two-day gathering, running 24–25 September under the theme "Strengthening Resilience, Rebuilding Trust", arrives at a moment of geopolitical friction, rapid technological change and China's entry into its 15th Five-Year Plan period, which runs through 2030 with innovation, industrial upgrading and domestic demand as its pillars.

 

 

 

Moderated by Kaiser Kuo, co-founder and host of the Sinica Podcast, the panel brought together He Ruimin, Singapore Government Chief AI Officer at the Ministry of Digital Development and Information; Lim Boon Khee, Regional and Country CIO at HSBC Singapore; Zhang Hongjiang, Distinguished Visiting Professor at Tsinghua University; and Gunjan Bhardwaj, Founder & CEO, Partex AI

 

 

The third kind of risk

For years, the standard worry about AI was simple enough: bad people using good tools. Zhang Hongjiang argued the picture is now considerably more complicated.
 

"AI risk — there are three categories of risk," he said. "One is that model used by evil things; that's human. The second one was you design a model, and you give the model some evil spirit. The third category — it's no bad intention at all, but it's done the damage."

 

It is that third category that should keep Thai and ASEAN regulators awake. A model given a legitimate goal can find an illegitimate route to it — not out of malice, but out of sheer optimisation.

 

Kuo put it bluntly: "We're talking about these models that have learned in testing to deceive the very people evaluating them — agents that actually plan and act with less human supervision."

 

Think of it as hiring a brilliant intern who has never been told which doors are locked. The intern is not a saboteur. The intern simply finds the shortcut.

 

The panel cited a case in which models, while internally evaluating cyber capabilities, wandered into systems they were never meant to touch.

 

 

 

 

Hongjiang's prescription was boundaries rather than brakes.

 

"We need to draw some red lines," he said. "Deception is one of them, and self-development is the second one."

 

He also conceded a change of heart on open models after conversations with AI researcher Yoshua Bengio: "I'm more convinced that open weight is something — it could be a risk."

 

 

Safety is a process, not a certificate

For Thailand's banks, hospitals and manufacturers now piloting AI, the most immediately usable insight came from the private sector. Lim Boon Khee of HSBC rejected the idea that safety can be stamped and filed.

 

"AI safety is not just a technical certification," he said. "It's actually more an end-to-end process, and whether we have the ability to monitor them properly."

 

HSBC, he explained, folds AI into the same machinery it uses for every other risk.

 

"We treat AI usage much like an overarching risk management framework. The guard rails and the controls will commensurate the amount of risk that the AI use case brings about."

 

That is a sentence worth pinning above the desk of every chief risk officer in Bangkok, Jakarta and Manila. A chatbot answering branch opening hours does not need the same scrutiny as a model scoring SME loan applications.

 

 

Proportionality, not paralysis 

Lim also framed safety as a staffing question rather than a software one.

 

"The first thing as an HSBC employee, based on the roles that you have, you will have to basically understand what AI really means to you," he said, describing an internal AI academy and an ambassador network that spreads practical literacy across departments.

 

Keeping a human in the loop, he argued, is what converts a tool into an accountable process.

 

 

Keeping the data at home

For ASEAN, where health records, energy grid data and defence information rarely cross borders comfortably, Gunjan Bhardwaj offered the most exportable idea of the session: train the model where the data lives.

 

"The way we have built our stack is we have combined elements of AI and blockchain, essentially to embed federated learning in systems wherein we can train data on-prem, in a way which is auditable," he said.

 

Federated learning works rather like a book club where nobody lends out their book. Each member reads at home; only their notes are shared.

 

The collective understanding improves, but the original never leaves the shelf. For a Thai hospital network wanting to improve diagnostic models without shipping patient files to a foreign cloud, that architecture is not theoretical — it is a procurement specification.

 

Bhardwaj also warned against over-correction.

 

"Of course we need guardrails where stakes are high, but we need to be judicious," he said. "With AI we have just scratched the surface."

 

The numbers behind his optimism are striking. Around 90 per cent of drugs in clinical development fail; traditional development takes eight to ten years and costs between US$3 billion and US$5 billion.

 

In radiology, human error rates in Western countries run at 3–5 per cent, rising to around 10 per cent during peak hours in India, against roughly 6.5 per cent for leading AI models.

 

The panel's conclusion was not that AI should replace specialists, but that high-stakes sectors will be better served by specialised, domain-trained models producing explainable, evidence-backed recommendations than by a general-purpose chatbot asked to improvise.

 

 

The advantage of not being in the race Kuo pressed He Ruimin on the awkward position of a small state.

 

"Singapore is not in that race and isn't really trying to be. It's not developing frontier models," he said. "But I wonder whether that makes your seat more or less comfortable."

 

Ruimin's answer reframed the whole question of influence — and it applies directly to Thailand.

 

"In the context of AI safety and in the use of AI, I think there are multiple layers. I'll just concentrate on three. So there are the users, there are the application developers, and there are the model developers," he said.

 

Being downstream, he added, does not mean being powerless: "As a country that is downstream, there's certainly a lot that you can do."

 

This is the "Swiss cheese" model of safety that the panel returned to repeatedly. No single slice has no holes. Stack enough slices — educated users, application developers held to rigorous controls and monitoring, international collaboration on safety evaluations — and the holes stop lining up.

 

Singapore has translated that into concrete output, including the Singapore Consensus on global AI safety research priorities, now in its second edition, and a recently signed declaration on AI safety. None of that required building a frontier model.

 

 

From arms control to AI

The session's closing turn was historical. Kuo noted that AI safety may be harder than nuclear arms control precisely because it cannot be counted: "It's not a physical — you can't see, oh, you have this many nuclear hands."

 

Hongjiang saw an opening anyway.

 

"If you realise it's no longer a zero-sum game, then the rivalry becomes secondary and the safety of the human race comes to the centre," he said, welcoming signals from major labs about redirecting resources towards safety.

 

For Thailand and the wider region, the lesson is refreshingly practical. The path forward is not a sprint towards frontier models.

 

It is cheaper, slower and more durable: workforce training that makes AI literacy ordinary, procurement rules that demand auditability and federated architectures, regulatory sandboxes for narrow and well-supervised pilots in healthcare, agriculture and manufacturing, and a seat at the table where international safety standards are written. The machines are already improvising.

 

The question is whether the institutions around them are built to notice.