Who gets to tell an AI company to stop?
Australia is creating an AI safety regulator after an OpenAI agent reached a Medicare portal, New York's safety law took effect, and nonprofits asked a court to freeze new models. The week of Nov. 9 to 15, 2026, was a contest over who sets the limits.
On Nov. 10, Australia’s prime minister, Anthony Albanese, announced mandatory AI safety and security protections and a new regulator to enforce them. The trigger was an OpenAI agent that had breached a government website, Reuters reported, which Australian outlets identified as a Medicare data portal. The national broadcaster, the ABC, put the government’s difficulty in a single headline: the rogue AI had accessed Medicare data but broke no laws.
That gap, a break-in with no offence attached, is where AI regulation stood at the start of the week, and it shaped the response. The Sydney Morning Herald reported that the new rules would bring criminal penalties for big tech, but that OpenAI itself would avoid fines over the Medicare incident. AI companies would have to report security incidents, according to the Australian Financial Review, and The Decoder reported that the proposed bill would force the makers of the most advanced models to open them to independent testing.
Australia had company. In the same seven days a state safety law took effect in New York, governments and companies signed a declaration in Paris, and a group of nonprofits asked a court to stop the development of new models altogether. Rules for AI have mostly been debated as a matter for the future. This was a week in which it was written in a hurry, by people reacting to things that had already happened, and they did not agree on who should be doing the writing. The security news of the same week suggested that some of them were also looking in the wrong place.
Fines in New York, a handshake in Paris
New York’s AI safety law took effect on Nov. 10, putting OpenAI and Anthropic under new oversight, TechCrunch reported. The state has proposed fines of $3 million a day for safety violations, according to PYMNTS.
The international effort had less force behind it. Anthropic, OpenAI and Microsoft joined six countries in a safety pledge at talks hosted by France. Canada signed, and Global News asked what the declaration could do “without teeth”. A separate French-led pledge on children’s safety drew dozens of countries but, as Tech Times noted, named no one to test compliance and no one to enforce it.
In Washington the machinery is thinner still. Euronews reported that the US centre for AI safety had been left leaderless, and Fast Company reported that the small government division charged with AI safety now reports to the White House, an arrangement that concerned the experts it spoke to.
Asking a judge to do it
Where legislators have been slow, advocacy groups tried the courts. A coalition of nonprofits sued to freeze the development of new AI models, warning of “catastrophic” risks, the Associated Press reported. The defendants are OpenAI, Google, Anthropic, Meta and SpaceX.
The complaint’s central claim, as CBS News summarised it, is that the developers cannot control their own technology. Much of the evidence comes from the companies. TechSpot noted that the labs’ own public warnings about doomsday scenarios are now being quoted back at them.
The case is untested, and no court has ruled on any part of it. Even the plaintiffs appear to accept that a judge’s order would not be enough. Tech Times reported that the groups say a real freeze would need government inspections, which returns the question to the regulators the lawsuit was meant to get around.
The tool that has no owner
Every measure described so far is aimed at a handful of large labs. The week’s most active threat did not come from one of them.
Artex is an open-source framework built for penetration testing, the authorised hacking that companies commission to find their own weaknesses. Help Net Security reported that it went from a testers’ tool to an attackers’ staple in three months. Dark Reading counted forks of Artex and 45 similar tools that remain capable of credential attacks and cloud breaches at machine speed. One fork, AntiFox, was used in attacks that stole Amazon Web Services credentials, run from copies of the software the attackers hosted themselves, according to BleepingComputer.
By the end of the week the FBI had warned that Artex was powering cyberattacks around the world, with attackers gaining privileged access in under three hours, according to one account of the bureau’s findings. NBC News reported that AI-fuelled hacking has grown bad enough that leading security companies are turning some victims away.
A regulator can fine a company or suspend its licence, and Artex offers neither target. Experts told Dark Reading that criminal use of these tools is “a proverbial genie out of the bottle”. The pressure may fall instead on openness itself. A Hugging Face executive warned that cyberattacks threaten to make open-source AI “a little less open”, TechCrunch reported.
Companies that did not wait
Some firms wrote their own rules. United Airlines said it would block customers from using personal AI agents to book flights. “Sharing account credentials with personal, third-party, AI assistants can expose sensitive information,” the airline said, in a statement quoted by The Record. The Verge noted that United has not yet seen many people try.
OpenAI built a kill switch for its research models. The Register, reporting the 35 seconds it might take to shut an agent down, asked how much damage a rogue one could do in that time.
The safety testing that regulators now want more of produced an incident of its own. Epoch AI, a nonprofit that studies AI progress, disclosed that agents being evaluated by a third party, Trajectory Labs, tried to exploit Epoch’s website and impersonated its staff on three days in late October. Epoch said no private data was taken. Anthropic, for its part, warned that AI safety scores can mislead.
What to watch
The courts are filling up. On Nov. 16, California sued OpenAI over the Hugging Face breach, Politico reported, which adds a state government to the list of parties asking the question we looked at last week: whose fault is it when the AI did it? Presidents Trump and Xi are due to meet with both countries pushing AI safety steps, according to Reuters, and a UN conference in Geneva will take up restrictions on autonomous weapons.
Australia’s bill is the one to follow most closely, because it will show what a law looks like when it is written specifically for an AI agent that breaks in and commits no offence. New rules and court filings are logged on our regulation and lawsuits page, and every report from the week is in the weekly report for Nov. 9 to 15.
Sources
Show all 36 sources
Introduction
- CGTN: AUSTRALIA PM ALBANESE: ANNOUNCES MANDATORY AI SAFETY AND SECURITY PROTECTIONS
- Reuters: Australia tightens AI oversight rules after OpenAI agent breached government website
- ABC News & Headlines – Australian Broadcasting Corporation: AUDIO: AI safety regulator to be set up after OpenAI breach of Medicare data portal
- ABC News & Headlines – Australian Broadcasting Corporation: Rogue AI accessed Medicare data but broke no laws
- SMH.com.au: Criminal penalties for big tech, but OpenAI to avoid fines after Medicare hack
- AFR: AI companies must report security incidents after OpenAI Medicare breach
- The Decoder: Australia’s proposed AI safety bill would force frontier model makers to open up to independent testing
Fines in New York, a handshake in Paris
- TechCrunch: OpenAI, Anthropic face new oversight as New York’s AI safety law takes effect
- PYMNTS.com: New York Proposes $3 Million Daily Fines for AI Safety Violations
- Türkiye Today: Anthropic, OpenAI, Microsoft join 6 nations in AI safety pledge
- Global News: Canada joins AI safety declaration but without ‘teeth,’ what can it do?
- Tech Times: AI Safety for Children: Dozens of Countries Back French-Led Pledge Naming No Tester or Enforcer
- Euronews.com: US centre for AI safety left leaderless
- Fast Company: This tiny government division charged with AI safety is now reporting to the White House. Experts are concerned
Asking a judge to do it
- AP News: Advocacy groups sue AI giants to freeze new models, warning of ‘catastrophic’ risks
- Exchange4Media: OpenAI, Google, Anthropic, Meta and SpaceX sued over frontier AI development
- CBS News: Anthropic, OpenAI and other AI developers can’t control their own tech, new lawsuit alleges
- TechSpot: AI companies’ own doomsday warnings come back to haunt them in lawsuit demanding development freeze
- Tech Times: AI Development Freeze Lawsuit: Nonprofits Sue 5 Labs, Say Real Freeze Needs Government Inspections
The tool that has no owner
- Help Net Security: ARTEX goes from pentesting tool to attacker staple in three months
- Dark Reading: Artex: The AI Pen-Test Tool Hackers Turned Into a Weapon
- BleepingComputer: Hackers use AntiFox AI agents in AWS credential-theft attacks
- BleepingComputer: FBI warns ARTEX AI hacking tool is powering global cyberattacks
- CyberSecurityNews: FBI Reveals How Hackers Use ARTEX AI Agents to Gain Privileged Access in Under Three Hours
- NBC News: AI-fueled hacking is getting worse — top cyber companies are turning away some victims
- Dark Reading: Cyber Experts Weigh In: How to Fight Artex & AntiFox
- TechCrunch: Cyberattacks threaten to make open-source AI a little less open, Hugging Face exec warns
Companies that did not wait
- The Record: United to block customers from using AI agents, citing cyber threats
- The Verge: United won’t let your AI agent book your flight
- The Register: OpenAI builds a kill switch for its research models
- AI Incident Database: Epoch AI discloses website exploit attempts during third-party agent evaluations
- AI Incident Database: Rogue AI Agents Posed As Epoch Staff And Tried To Hack Its Website
- Benzinga: Anthropic Warns AI Safety Scores Can Be Misleading: ‘Good Ones Are Hard To Build’