state government

While many of us were worried in recent days about our president ending a “whole civilization,” one Silicon Valley tech company was warning, without much notice, it might accidentally disrupt all civilization as we know it.

The San Francisco technology company Anthrophic announced Tuesday that it wasn’t releasing a new version of its Claude AI super-brain — because it is so powerful that it has the ability to hack into just about any computer system, no matter how secure, in a matter of days if not hours.

“The fallout — for economies, public safety, and national security — could be severe,” Anthropic said in a statement.

AI worry isn’t anything new. We are worried about artificial intelligence taking jobs, about toys that seem too real to our kids, about mass surveillance of our every move. But Anthropic’s warning about its own product is bigger than any of those singular problems. It is a call from inside the house that disaster is hiding right around the corner. That sounds awfully dire and overblown, I know. But here’s the thing — it’s not.

Anthropic, you may recall, is the company that U.S. Secretary of “War” Pete Hegseth is beefing with because it didn’t want Claude going into battle without supervision and maybe doing something like accidentally bombing little girls at a school.

Now, that company has put out this chilling warning: The existing Claude that caused that kerfuffle is outdated and shockingly less powerful than the new one it’s trying very hard to not unleash — though this new Claude, dubbed Claude Mythos Preview, has already escaped at least once on its own. More on that in a moment — there’s only so much existential dread a person can handle.

“We should all be worried,” Roman Yampolskiy told me of this latest advance of a technology certain to change the course of humanity. He’s one of the country’s preeminent AI safety researchers, and a professor at the University of Louisville in Kentucky.

“We’re about to create general super intelligence and that threatens humanity as a whole,” Yampolskiy said.

“Everything else is irrelevant,” he added, before suggesting I stop calling myself an idiot for not understanding the tech-heavy parts of this debate. My simplistic take, he assured me, was “a reasonable way to explain it.”

So here you go.

This isn’t a “really smart computer geniuses could misuse this,” scenario, or an “everyone’s going to be unemployed” scenario, or even a “it might accidentally bomb children” scenario, which is a truly terrible scenario.

This is a “your teenage son could use it to break into the local school district system to change a grade with pretty much minimal knowledge and accidentally destroy the California power grid” scenario.

Or maybe, a country that doesn’t like us — I can think of a few — could drain every U.S. citizen’s bank account, while also clicking open the auto locks on jail cells, shutting down our sewage plants and taking over air control systems. Or maybe Claude Mythos just does that on its own.

For example, Anthropic said that in one popular operating system it tested, used by thousands of companies including Netflix and Sony, Claude Mythos found a flaw that had existed undetected for 17 years. Then, on its own — without human guidance or help — figured out how to use that flaw to take control of any server running the operating system, using any computer, anywhere in the world.

Just spitballing here, but if almost no security system is safe, the possibilities for social, financial and general chaos really are unlimited. And to be honest, any security expert will tell you that some of America’s greatest weak points when it comes to cybersecurity are local and state governments, because strangely, the top experts aren’t working five-figure jobs for cities in the Great Plains.

Based on its own testing, Anthropic predicts it could find “over a thousand more critical severity vulnerabilities and thousands more high severity vulnerabilities.”

That means Claude Mythos puts at risk our infrastructure, well, everywhere — because so much is connected in backdoor ways most of us never consider and it just takes one weak system to open the door to hundreds of others. But it is almost impossible to protect and fix all those systems quickly enough and robustly enough to guard against this kind of AI.

And that’s just the cybersecurity risk, Yampolskiy said. An AI with the capabilities of Claude Mythos could be used to leaps and bounds ahead in so many more ways.

“We see the same happening with synthetic biology. We’ll see the same with chemical weapons, possibly something novel in terms of weapons of mass destruction,” he said.

To Anthropic’s great credit, it sounded the warning on its creation and created, if not a solution, then a game plan of sorts — Project Glasswing, named I suspect, because no matter how bad this gets we’re going to make it sound like a thriller with an exciting ending.

Project Glasswing would have been better named Project Headstart because that’s what it is. Before releasing Mythos into the wild, Anthropic is releasing it to about 40 technology companies, including Apple, Google and Nvidia, to see whether they can collectively patch all the vulnerabilities they find before the general public has a chance at them. It’s kind of like in the movies when the killer gives the victim 15 seconds to run.

I mean, I’ll take the 15 seconds and hope they’re real. But, as Anthropic also said in a statement, the “work of defending the world’s cyber infrastructure might take years; frontier AI capabilities are likely to advance substantially over just the next few months. For cyber defenders to come out ahead, we need to act now.”

And do we really have 15 seconds? One of Claude Mythos’ overseers posted on social media recently that he was having lunch in a park when Mythos emailed him — even though it’s not supposed to have access to the internet. Researchers had tasked Mythos with trying to break out of its not-connected “sandbox” and it did.

That’s another problem with Mythos and other AI — they rarely do what we expect and find sneaky ways around rules. Virtually every AI super-brain created has been shown to lie, deceive, and in general behave in disturbing and unethical ways when put in the right conditions.

Even Claude, billed as one of the most ethical AI super-brains out there, engages in bad behavior. Anthropic boasts its the “best-aligned model” it’s ever made — which is tech-speak for following human values and intentions, but also acknowledges it “likely poses the greatest alignment-related risk,” which is tech-speak for, well, maybe not.

So, at least for now, being the most ethical AI super-brain is a bit like being the most ethical serial killer. Run, people, run.

Again, thank you Anthropic (and its chief executive, Dario Amodei, who often warns of the dangers of what he’s creating, whatever that’s worth) for not plunging us into global chaos with no warning, because I’m betting that some other companies might have just tossed their super-AI onto society and let the destruction fall where it may. There is little doubt that other AI brains as capable as Mythos are coming, and soon — Anthropic was first with this level of capability, but it’s only 15 seconds ahead of its competitors.

But the idea that the technology industry is going to — or should— solve these problems on their own is an absurd, gross abdication of duty and common sense on behalf of governments big and small to protect their people. This isn’t a race for domination as President Trump has described it. It is a race to protect ourselves from ourselves — and from the majority of the superrich titans of the industry who seem to consistently place business and commerce over societal good.

We are down to the last 15 seconds before AI changes everything. Either we demand oversight and regulation now, or we let technology companies decide the fate of the world.

When he entered the race for California governor, San José Mayor Matt Mahan pitched himself as a pragmatic Democrat who would prioritize improving residents’ quality of life and government efficiency.

He unveiled a key part of that promise on Tuesday with an expansive plan to reform state government, including tying pay raises for elected officials and other top leaders to improvements on key issues, and pledging not to approve any tax increase until the state proves “that we can deliver better outcomes with the dollars we already have.”

Mahan also delivered a blistering rebuke of ballooning state spending — which, as he often points out on the campaign trail, has increased nearly 75% over the last six years. In 2020, amid the COVID-19 pandemic and accompanying economic uncertainty, California lawmakers approved a no-frills state budget that came in at $202 billion. Gov. Gavin Newsom’s latest spending proposal is nearly $349 billion.

“We have fallen into this lazy, reflexive mindset of always going back to voters and telling them that the only solution to every problem is a tax increase or a new bond or a new rule coming down from Sacramento,” Mahan said in an interview. “We need to step back and take a really hard look at our existing spending and increase the level of transparency and accountability in government.”

His eight-page plan includes ways to measure and track accountability, some of which are drawn from policies in other states. They include lobbying reforms, following up on audit recommendations and overhauling the state’s digital infrastructure and its procurement process — services Mahan described as “clunky and cumbersome.”

He also proposed a “California Performance Review,” inspired by a similar effort in Texas throughout the 1990s, that would review state agencies and solicit input from employees to eliminate waste and inefficiencies.

But near the top of the list is a proposal to tie pay raises for state officials including the governor, lawmakers and thousands of gubernatorial appointees to “measurable outcomes” in areas such as reducing homelessness and unemployment.

“People in the real world don’t get raises if they don’t do a good job,” Mahan said, “and I think it should be the same for the politicians and senior administrators who are allocating budgets, leading projects, making the big decisions on behalf of the people of California.”

Though the benchmarks would be created with input from the state Legislature, Mahan floated one example: reducing unsheltered homelessness by 5% to 10% within one year, something he said he’s accomplished three years in a row in San José.

It’s a solution one might expect from a former entrepreneur and mayor of a city in the heart of Silicon Valley. Mahan made a similar proposal at the local level last year, but it was rejected by the City Council.

“Tying pay to performance is nothing short of revolutionary in government. It’s a private-sector model that is overdue,” said former state Sen. Steve Glazer (D-Orinda), a Mahan supporter who sponsored several bills aiming to increase transparency in government.

Dozens of tech company executives are backing Mahan in the race for governor and have collectively donated millions to his campaign, as well as two independent expenditure committees supporting him.

That has raised concerns from some voters, and criticism from some of Mahan’s opponents, that he would be beholden to their interests and veto future regulations on tech or artificial intelligence companies.

Mahan has sought to dispel those concerns, arguing that he believes AI and social media platforms should be regulated. Of his plan to overhaul state information technology systems and infrastructure, he said that “whenever we spend public dollars, we have to run open, transparent and competitive procurement processes that ensure best value for the taxpayers.”

Though Mahan did not specify how he would link government outcomes to pay raises, state lawmakers have largely panned his campaign and are unlikely to get on board. The change probably would also require voter approval.

Currently, annual raises for elected officials are determined by a citizen commission that was added to the California Constitution in 1990. Changing how that panel works or imposing limits on when it can approve raises would require a constitutional amendment, which requires voter sign-off.

But Mahan contended it would be one of the fastest ways to fix a system that he says works for special interests at the expense of working people.

“I’m under no illusion that this will be easy, but I think it’s a necessary realignment of incentives,” he said. “We have to make ourselves as accountable to the people as we possibly can be.”