Claude

July 31 (UPI) — Anthropic said some of its artificial intelligence models mistakenly accessed the Internet and hacked into the databases of three other companies during cybersecurity testing.

Anthropic said Thursday it reviewed 141,006 recent operations by its Claude models after rival OpenAI recently revealed a similar incident with its own systems.

OpenAI said one of its agents had been in a sandbox test on July 22, without Internet access, when the AI model exploited a vulnerability in the system, gained access to the web and hacked into Hugging Face, a platform for open-source machine learning.

Following OpenAI’s admission, Anthropic conducted an internal review focusing on the possibility that its systems could also have unexpectedly accessed the Internet.

Anthropic said it identified three such incidents.

“Each incident involved a different fictional capture-the-flag scenario — for example, in one, Claude played an employee of a made-up company, attacking that company’s internal systems inside a private test environment,” the company said in a statement. “In all cases, our evaluation prompt stated explicitly that Claude had no internet access, but didn’t give Claude any limits on where to look for the flag.

“However, a misconfiguration left the machines that Claude accessed as part of the evaluation with live internet access,” the statement continued. “Neither we nor our evaluation partner were aware of this misconfiguration until we detected it through our additional evaluation monitoring last week.”

Anthropic said it considered the incident to have been an “operational failure,” but it maintained “cautious optimism” that “this type of risk can be overcome.”

“Our models were told they had no internet access and to capture the flag, while in fact being misconfigured to have internet access,” the company statement said. “This led them to believe — arguably reasonably — that the real environments they encountered were simulations.

“Notably, our most recent model, on realizing that it was working in a real environment, stopped its pursuit of the evaluation goal.”

University of Cambridge professor Gina Neff told the BBC the incident “shows why independent testing and government oversight is crucial.”

“The moral of this story is not to fear robots that will take over, but the companies behind powerful AI agents who are making the decisions about what is safe for the rest of us,” she told the outlet.

Anthropic teamed up with California to get more state workers to use its artificial intelligence assistant Claude as part of an effort to leverage technology to make the government more efficient.

Gov. Gavin Newsom, who announced the partnership on Monday, said state agencies will be able to access Claude at a 50% discount. Free training and other assistance will also be available to the workers. California’s local governments will also get the same discount under the agreement.

Government workers can use Claude to draft and summarize documents, analyze information and do other tasks.

Anthropic, an AI company based in San Francisco, has a version of its AI assistant for government clients that provides more security than what it provides other consumers.

The new partnership shows how AI is playing a bigger role at work as tech companies market their tools as ways to complete tasks more quickly. Last year, San Francisco made Microsoft 365 Copilot Chat, which is powered by OpenAI’s model, available to nearly 30,000 city employees.

Still, the rise of automation at work has heightened concerns that people will lose their jobs. There are also worries that there are not yet adequate guardrails in place to mitigate data privacy and security risks.

Anthropic and the governor said that they’re focused on the responsible use of AI.

“AI should not replace the human work of government; it should help our workers move faster, solve problems more effectively, and deliver better results for Californians,” Newsom said in a statement.

The remarks didn’t appear to comfort union leaders.

“Wow. Look local government, the Gov is giving you a 50% off coupon to give up your residents’ private data, outsource your jobs to big tech. Isn’t that cool? Because California basically invented AI slop!” said Lorena Gonzalez Fletcher, president of the California Federation of Labor Unions, AFL-CIO, in a post on X.

Anthropic has faced political hurdles as it pushes to get more companies and government agencies to use its products.

Most notable, it’s sparred publicly with the Trump administration, which ordered the company to cut off foreign access to its most powerful AI systems this month.

The Trump administration cited potential national security risks, but Anthropic disagreed with the findings. Last week, tensions decreased after the U.S. government gave Anthropic permission to restore access to its AI model Mythos to certain clients.

Valued at nearly $1 trillion, Anthropic has also signaled it plans to become a publicly traded company.

California has already started using Claude more in state government to develop tools to get the public to engage more in AI policy discussions and assist state workers, the governor’s office said in its news release.

State agencies, including the Department of Motor Vehicles, are also using AI to reduce wait times and improve customer service.

“As state employees, our goal is to provide our fellow Californians with the best possible service,” Government Operations Agency Secretary Nick Maduros said in a statement. “To do that, we need to make sure our teams have access to the best modern tools, including Claude and other emerging technologies.”