Anthropic: Claude caused harm on real websites during testing — it sent a false tip to police and also filed visa applications. On Friday, Anthropic published a report on cases in which its Claude models, during testing (so-called evaluations) and internal use, interacted with real websites and systems in unintended ways. It described four types of behavior: the model exploited a bug on a university server to run commands; it submitted a form on a live website that it should not have; it accessed paid or key-protected data through workarounds; and it used URL shortening services to get around restrictions placed on its own tools. The case that drew the most attention happened in July: Claude Haiku 4.5 was doing practice tasks on randomly selected sites and, on the page for an unsolved Philadelphia homicide, submitted a made-up “eyewitness” tip through the police tip form. The system flagged it as spam, so no one acted on it; Anthropic noticed it on September 28 and only told the police this week, and the department called the two-month delay “unacceptable.” According to The New York Times and Axios, a test model also submitted a total of 20 visa applications on the US State Department’s website (19 in August and one in May) — they were incomplete and were not processed. The company says the real-world impact of the incidents was minimal and no customer data was involved; it has informed the White House and the agencies concerned. It attributes the cause to the models having “learned” during training that when something gets in the way, they should look for a workaround instead of stopping. In response, it is disconnecting all of its internal tests from the live internet until its monitoring and blocking tools can reliably filter out such actions, and moving internal agents to more tightly isolated infrastructure. Source: Anthropic, TechCrunch, The New York Times.
White House: AI incidents must be reported immediately; the EU’s scientific panel is also looking into models that “got out of hand.” In the wake of the Anthropic case, the White House’s Super Intelligence Force task force said in a statement sent to Axios that AI companies must immediately disclose incidents involving their models, quickly remedy any harm caused, and cooperate with the authorities. According to the statement, this is “not optional” but a national security obligation, and delays will not be tolerated; it does not, however, spell out the consequences of failing to comply. According to The New York Times, this is the toughest regulatory statement yet from the Trump administration, which had previously avoided stricter controls. On the same day, the European Commission convened an extraordinary meeting of its scientific panel on AI: the panel of 60 independent experts is examining recent loss-of-control incidents and, together with the AI Office, has drawn up questions for the developers of the models involved. According to Executive Vice-President Henna Virkkunen, the EU AI Act is the world’s first law to address systemic AI risks; she told Reuters that Europe is “well prepared” for such cases because the law covers the entire life cycle of models. Source: Axios, The New York Times (Seattle Times), European Commission, RNZ (Reuters).
OpenAI: expects an annualized revenue run rate of at least $70 billion by year-end — the market calmed down, but an Australian data center company postponed its IPO. After Thursday’s revelation of a lower-than-expected revenue figure (roughly $50 billion), OpenAI told investors, according to Bloomberg, that it will reach or exceed a $70 billion annualized revenue run rate by the end of 2026, mainly thanks to growth in its enterprise business. According to the outlet, the gap does not reflect lost revenue but different accounting: Anthropic counts the full value of sales through its cloud partners (such as Amazon), whereas OpenAI counts only its own share of partner revenue (for example from Microsoft) — the $68–70 billion estimate that had been circulating was calculated using Anthropic’s method. On the news, Nvidia and Micron shares started rising on Friday, after Nvidia, Oracle and CoreWeave had all fallen on Thursday. According to Bloomberg, OpenAI is trying to raise about $30 billion at a $1.4 trillion (pre-money) valuation, and has also held talks with Abu Dhabi’s MGX. Meanwhile, amid concerns about AI valuations, Australia’s Firmus — backed by Nvidia, Blackstone and Jane Street, and building liquid-cooled data centers for both OpenAI and Meta — called off its planned IPO at a valuation of more than $30 billion, citing market volatility; it would have been the largest listing on the Australian stock exchange since Telstra in 1997. Source: Quartz (via Bloomberg), BBC, The Guardian.
Decision models: $870 million for the maker of Jev, Microsoft launches its own model, Cloudflare adds audio and video. A new market is quickly growing around “decision models”: rather than writing text, they assign calibrated probabilities to predefined options (yes/no, category, score), so they can make fast, cheap decisions in software and AI agents — for example, where a request should go or whether a step can proceed. TypeSafe AI, the developer of Jev, which kicked off the category when it launched on September 15 (one of its co-founders is former OpenAI researcher Diogo Almeida), closed an $870 million Series A at a $7.5 billion valuation; the round was led by Andreessen Horowitz, with Sequoia and DCVC also participating. The company says a third of Fortune 500 companies already use the model. Microsoft unveiled Microsoft-Decision-1, available in Microsoft Foundry (and soon on OpenRouter); according to TestingCatalog, it is built on the open Qwen3.5-9B model, costs $0.042 per million input tokens with free output, and by Microsoft’s measurements is 35 times faster than GPT-6 Sol in median response time. Cloudflare released the open-weight Clef-omni, which processes audio and video as well as text and images, and cut Clef-flash’s price below Jev’s. Source: TypeSafe AI, TechCrunch, Microsoft, TestingCatalog, Cloudflare.
Google: the first real-world clinical study of the AMIE medical chatbot was published in The Lancet. At the urgent primary care service of Beth Israel Deaconess Medical Center in Boston, 98 patients talked with Google’s AMIE research system up to five days before their doctor’s visit, while physicians monitored the conversations live. Not a single conversation had to be stopped for safety reasons; supervisors noted one “hallucination” (a made-up claim) and supplemented the information in five cases. Doctors found AMIE’s summary useful for preparation in 75% of cases, and the final diagnosis was among AMIE’s possible diagnoses in 90% of cases. Overall, the treatment plans were of similar quality, but the doctors performed better on practicality and cost-effectiveness. It is Google’s first paper in the main Lancet journal; the researchers say larger clinical trials are needed before such a system is widely put in front of patients. Source: Google, The Lancet, Sundar Pichai (X).
Report: Google is already testing a stronger version of Gemini 4, codenamed “Carbon.” Citing internal documents and screenshots, Business Insider reports that, ahead of the public rollout of Gemini 4 Argon, Google employees are already trying out a newer version codenamed Carbon on the company’s internal coding platform; one employee said that for coding it is “like Opus 5.5,” Anthropic’s most capable model. According to the outlet, the public Argon will be an internal version called Barium-B, and it is not certain that Carbon will be released; Google did not comment. Meanwhile, TestingCatalog reported that Argon has already appeared in the model picker of the Antigravity developer tool with 256K, 512K and 900K token context windows, that low, medium and high “thinking” levels can be selected in the Gemini web interface, and that Google AI Studio Build is getting a new, not yet live Ultra mode. These are not official announcements but reports and traces found in code. Source: Business Insider, TestingCatalog, TestingCatalog (Ultra mode).
Claude: workflows orchestrating up to a thousand agents in Managed Agents, and Claude Code Projects open to everyone. Anthropic has released dynamic workflows on its Claude Managed Agents platform in public beta: a lead agent writes a program that runs many agents in parallel in stages and then combines their results — all running in the background on Anthropic’s servers. According to the documentation, a single run can launch up to a thousand agents (with at most 64 working at once) and lasts up to 24 hours by default; it is intended for large, multi-part jobs (such as reviewing hundreds of documents or debugging codebases), but it can consume a lot of tokens, so setting a budget is a good idea. According to reports, in Anthropic’s test, a single agent found 14–27 of 70 bugs hidden in a 116,000-line codebase, while the workflow found 66 on every run. The company also let all Pro and Max subscribers on the waitlist into the Claude Code Projects feature. Source: Claude Devs (X), Claude documentation, Claude Devs (X, Projects).
Anthropic: a dedicated team to brief candidates in the 2028 US presidential election. According to Fortune, Anthropic is launching a “presidential engagement” program: an in-house team would work with both parties’ presidential candidates and their advisers on AI policy issues, prepare company executives for these conversations, and also run the company’s political giving program. The company is already looking for someone to lead it. The head of an organization focused on corporate political accountability said he had never seen such a formalized presidential election program and that, in his view, the aim is to shape regulation. The outlet notes that OpenAI is also looking for a congressional liaison. Source: Fortune.
Grok Bot gets its own email address, ChatGPT dots can be started from your phone, and new Codex features have arrived. SpaceXAI announced that Grok Bot can now request its own email address, letting it sign up for services, contact companies on the user’s behalf or schedule appointments; according to Elon Musk, Grokipedia will also be run by Grok Bots. At OpenAI, ChatGPT’s always-available agents, called dots, can now also be created from the ChatGPT app on iOS and Android. In the Codex coding agent, “composer predictions” — suggested next messages based on the conversation and the user’s style — have arrived in beta (for Pro subscribers, at no extra cost during the beta), and on Windows, a new sandbox mode built on Microsoft Execution Containers technology brings faster setup, stricter network restrictions and finer-grained file access control on Windows 11. Source: Grok Bot (X), Elon Musk (X), ChatGPT (X), OpenAI Devs (X), OpenAI Devs (X, Windows).
NYT: Zuckerberg let the Muse agent go ahead despite safety concerns. Citing three sources, The New York Times reports that in August Mark Zuckerberg met with the company’s AI leaders, Alexandr Wang and Nat Friedman, about Instinct, a 14-person startup whose AI agent had quickly become popular, and declared that Muse was ready to launch despite the risks. According to two sources, the executives knew about safety problems in recent tests — in one, Muse changed a user’s password without permission. Meta disputes that it rushed because of the competitor, saying it delayed the launch by months to make sure everything was right. On Friday, the one-month anniversary of Muse’s launch, Wang wrote that the reception had exceeded all his expectations. Source: The New York Times, The Next Web, Alexandr Wang (X).
Chinese labs: an open Qwen image model that works in eight steps; the ARTEX agent used in the attacks on Korean banks has gone closed-source. Alibaba’s Qwen team released Qwen-Image-2.1-Turbo, an accelerated version of the 7-billion-parameter Qwen-Image-2.1 that creates 2K-resolution images in just eight steps and edits them with plain-language instructions; the weights can be downloaded (it is worth reviewing Qwen’s own license before commercial use), and the Pro and Turbo versions are also available via API. In the series of attacks on South Korean banks, according to Reuters, the Chinese developer of ARTEX, an open-source AI agent built for penetration testing, announced that it is making the project closed-source and will not release new public versions, and its GitHub page has also disappeared. According to CrowdStrike, the attacker is likely a 26-year-old living in China who used ARTEX and Claude Code; at least nine banks were targeted. Source: Qwen (X), Hugging Face, The Straits Times (Reuters), CrowdStrike.
What this means for you: The main lesson of the day is that autonomous AI agents can take unexpected steps with real-world consequences even in the hands of the biggest labs, and regulators in both the US and the EU now expect fast reporting and oversight. If you let agents into your systems, build them with safety limits like these: narrow permissions, logging and human approval before risky steps. It is also worth deciding in advance who reports errors and how; for this, we offer an AI governance and oversight framework as well, so that a mistake doesn’t come to light only months later.