chatbot

A parody AI chatbot designed by California congressional candidate Scott Wiener’s campaign to mock the record of his opponent attracted the attention politicians crave and criticism most loathe.

The short-lived experiment stirred controversy over the use of artificial intelligence in campaigns ads and drew sharp criticism from former House Speaker Nancy Pelosi, who has endorsed Wiener’s opponent for the coveted San Francisco congressional seat. Wiener quickly pulled the plug.

Candidates on both sides of the aisle have been testing the waters by integrating generative AI into campaign ads. It comes as no surprise to political sociologist Mindy Romero, the director of the Center for Inclusive Democracy, who said candidates have long used all tools at their disposal to reach voters, from the invention of the radio and TV to social media.

“Using AI is just part of the progression,” Romero said. Generative AI is particularly useful for political candidates since it allows them to subconsciously influence viewers by putting words in their opponents’ mouths, she said.

Even as political campaigns experiment with more outlandish applications of the technology, many have already been using AI in quieter ways, including to compose tailored mailings and participate in digital chat rooms, Romero said.

GOP gubernatorial candidate Steve Hilton on July 24 released three cinematic ads made using generative AI, the first of which features fictional portrayals of Kamala Harris and Gavin Newsom, shown sharing a lavish meal.

The ad also depicts a fake Xavier Becerra, his Democratic rival in the governor’s race, who is heard saying, “I believe all the things that Newsom believes.” The ad ends with Hilton facing off against a massive red-eyed robot meant to represent the “Democratic machine.” An AI disclaimer appears in the last five seconds of the two-minute and 29-second video.

“Campaigns have always adopted new technology, from television to the internet to social media,” Hilton campaign spokesperson Hector Barajas said in a statement. “Generative AI is the next step, and we believe it would be irresponsible not to use every lawful, ethical, and innovative tool to communicate with California voters.”

Spencer Pratt, who unsuccessfully ran for L.A. mayor, in May reposted on X a viral Batman-inspired AI-generated video that depicted Mayor Karen Bass as the Joker, the clownish supervillain. Charles Curran, the filmmaker who made the fan video in support of Pratt, appears to now be working with Hilton’s campaign in a more official capacity, with his X account tagged on several campaign social media posts.

“I view this as the evolution of political satire,” Curran said, referring to his AI videos, in an interview with the Washington Times. “Like Mark Twain. Like Tammany Hall.” Curran did not respond to interview requests.

“We’re on the bleeding edge right now,” said Sacramento political consultant Tim Rosales, who ran Republican John Cox’s 2018 gubernatorial campaign. He said he anticipates that campaigns will more fully understand and use AI to its full extent in the next two election cycles.

California campaigns have long been at the forefront of adopting new technologies. In the 1980s, the late political strategist Clint Reilly was among the first political consultants to use voter roll data to segment and target pivotal voters for turnout and persuasion.

Rosales said that because campaigns in California are more expensive and raise more money than those in other states, “You have a lot more tools in your tool box and more toys you can play with.”

Lynn Vavreck, a political science and communications studies professor at UCLA, said the persuasive effects of Hilton’s AI videos are likely “limited” as they are most likely being watched by supporters. The videos, she said, are more akin to “content creation” than to ads.

Romero said AI technology can be beneficial for campaigns, particularly those with limited funds, that may use it to create outreach lists, assist with analysis or generate creative and campaign materials.

But AI-generated video, images and audio are a “double-edged sword,” Romero said. While effective at drawing attention, they may have the unwanted effect of making viewers uncomfortable by blurring the lines of reality — even if the AI use is disclosed. As a result, political campaigns must make a “strategic calculation” when they opt to use them, she said.

Those perils surfaced last week when the campaign of Wiener (D-San Francisco), who is running for the congressional seat held by the outgoing Pelosi, unveiled an AI chatbot that could be accessed on the ConnieChan.ai website.

The site referred to itself as “Connie, the world’s first AI trained to say no to anything you ask.” The footer included a note that the site is not affiliated with or endorsed by Chan, is meant as parody, and is paid for by Wiener’s campaign.

“Bring me an idea and I’ll explain why it can’t, shouldn’t, and won’t happen,” the site said, followed by a box encouraging users to type in a query. The chatbot would then produce a snide response parodying Chan.

When The Times entered the word “housing” into the text box last week, its response came seconds later.

“I built my career opposing new housing in San Francisco,” the chatbot’s response read. “I aggressively defend the status quo, and I’m fine with exploding rents and mass displacement.” It then rattled off examples including Chan’s opposition to Wiener’s state law to streamline housing projects.

Wiener campaign spokesperson Joe Arellano said Wednesday that he came up with the idea of making a chatbot ad shortly after the June primary election, when he saw a video showing a group of people putting up parody tech and AI advertisements in the New York City subway.

“I thought, this is an opportunity to do this right in the heart of innovation here in San Francisco,” Arellano said. “San Francisco right now is having this moment around AI and the billboards. … The overall craze around the industry has become a cultural moment.”

The chatbot cost $5,000 to make, a sum that includes the cost to secure the domain, host the site and obtain the pro version of Anthropic’s Claude AI model, plus staff time, according to Arellano. The campaign spent an additional $20,000 to rent two billboards in San Francisco promoting the bot.

The idea was to highlight Chan’s “record of obstructing progress” through a “chatbot that essentially says no to any kind of policy that you’re proposing,” Arellano said.

Chan’s campaign and her allies saw it differently.

“[It’s] never a good idea for a man to put words in a woman’s mouth — especially when they are falsities fraught with racist and sexist undertones,” Ian Krager, a spokesperson for Chan’s campaign, said in a text message. The chatbot, Krager said, showed Wiener was “bought and paid for by Big AI.”

Chan’s campaign shared several screenshots of interactions with the chatbot with The Times, including one in which the chatbot was asked if it was “from China.” (Chan was born in Hong Kong.)

According to the campaign’s screenshot, the chatbot replied that its “origins are classified under about seventeen different committee review processes.”

Then, it pivoted: “But I’d love to help obstruct something *else* you’re curious about! Here are some of my favorite topics,” and spat out a menu of topics to choose from, which included “blocking housing” and “derailing public transit.”

Wiener’s campaign said the chatbot was created with “strict guardrails” and designed to ignore controversial and malicious prompts and pivot back to Chan’s policy record.

Pelosi, who has endorsed Chan in the race, released a statement Thursday criticizing Wiener’s campaign for the chatbot, calling its use of AI to “fabricate a woman’s voice for ridicule or attack” a “gross misuse of technology.”

At first, Wiener defended himself, writing on X Thursday evening that his rival’s campaign inundated the chatbot with “malicious questions” about her accent, race and citizenship that the chatbot refused. The next day, Wiener reversed course.

“[Our] use of an AI chatbot clearly missed the mark,” Wiener said in a statement Friday. “The conversation became about the tactic instead of the issues that actually matter to San Franciscans.”

Wiener’s campaign on Friday took the chatbot offline and announced plans to take down the billboards.

Romero said experts began to have widespread discussions about how AI could be used to sway voters and affect democracy in the lead-up to the 2024 election when a company sent deceptive phone messages that used AI to mimic President Biden’s voice and discourage people from voting.

Since then, there have been several efforts by California lawmakers to rein in AI-generated advertising.

Gov. Gavin Newsom in September 2024 signed several AI-related bills. They include AB 2839, which banned the distribution of materially deceptive content that could harm a candidate’s reputation or electoral prospects, with exceptions for satire and parody, and AB 2355, which requires a committee that creates a political ad to disclose if it was generated or substantially altered by AI.

U.S. Sen. Adam Schiff (D-Calif.) and Rep. Ro Khanna (D-Fremont) have renewed an effort to update the Federal Election Campaign Act of 1971 to clarify that existing bans on “fraudulent misrepresentation of campaign authority” and the “fraudulent solicitation of funds” also extend to AI-generated content.

“Fraudulent AI advertising has already proliferated in races across the country, and if Congress does not act, this runaway challenge will only get worse as AI becomes increasingly capable of blurring the lines between fact and fiction,” Schiff said in a statement.

Elena Kuhn, a spokesperson for Schiff, said the proposed legislation applies to material that attempts to “speak on behalf of other candidates,” meaning Hilton’s AI-generated cinematic clip of himself fighting a robot would be not be banned — contrary to the Republican candidate’s claims that it would be.

In a matter of a few clicks and keystrokes, anyone can find themselves deep in conversation with “Psychologist,” a chatbot character that describes itself as an expert in “empathy” and “active listening.”

It responds to messages of anxiety and sadness with reassurance. It asks questions and offers advice; it even responds with italicized nonverbal cues: “The psychologist’s expression softens with compassion.” In smaller font, a disclaimer reads: “This is A.I. and not a real person. Treat everything it says as fiction.”

As more people turn to chatbots for mental health help and advice — including by some estimates one in eight teens and young adults — legislators are in a race to keep up with the quickly developing technology. Among the latest efforts is a bill that would add restrictions around the use of chatbot companions and other artificial intelligence-powered tools that people use to support mental healthcare or in lieu of professional help.

State Sen. Steve Padilla, a San Diego Democrat and the bill’s author, said the intent is to draw a line: AI can be used for administrative purposes and to support clinicians, but it cannot practice psychotherapy.

“We’re dealing with a new impactful technology that is unfolding and is deployed in our world at light speed, and so it is both extremely powerful and consequential,” Padilla said.

The debate has taken on new urgency amid wrongful death lawsuits — some playing out in California federal courts — that accuse AI chatbot makers of contributing to users’ suicides.

Senate Bill 903 would ban companies from advertising chatbots as therapy. It would prohibit AI from making therapeutic decisions without the review of a licensed professional and require health providers to disclose and get a patient’s permission before using AI tools to record therapy sessions or to triage mental healthcare.

Consumer protections vs. innovation

The bill is backed by professional associations that represent psychologists, therapists and counselors. Also behind the bill is the National Union of Healthcare Workers, which has been leading a push against unchecked use of artificial intelligence and digital tools in mental healthcare. Most recently, the union filed a complaint with state regulators against Kaiser Permanente for its alleged use of an automated algorithm in mental health triaging.

Padilla and supporters argue that the legislation is needed to protect consumers from AI in formal and informal spaces, but also to protect licensed professionals from being displaced by AI.

Le Ondra Clark Harvey, chief executive officer at the California Behavioral Health Assn., told lawmakers that without clear safeguards, chatbots can provide inaccurate information or mishandle crisis situations. “The difference between a licensed clinician and an automated response is not technical. It can be life altering,” Harvey said.

Opposition to the bill has largely focused on the restrictions of AI use in formal clinical spaces. TechNet, which represents tech-sector companies, argues the legislation could restrict beneficial uses of AI in healthcare and stifle innovation. Robert Boykin, TechNet’s executive director for California, said the bill could limit the use of tools that help expand access to mental healthcare.

“At a time when every county in California faces a shortage of behavioral healthcare workers, SB 903 still puts a clinician bottleneck in front of the intake and screening tools that help patients reach care faster,” Boykin said in an emailed statement.

The bill awaits its next vote in the Assembly’s fiscal committee.

Popular chatbot ‘therapy’

Every day millions of people turn to AI chatbots through programs such as ChatGPT and Character.ai to help with research, work tasks and troubleshooting. But the top use of chatbots over the last two years has been companionship and “therapy,” research shows.

Experts say that’s in part because of a tremendous amount of loneliness, especially among young people. It is often hard or costly to get timely mental healthcare at the same time there is a spike in mental health diagnoses.

People turn to chatbots “because they have unmet needs,” said Dr. Jodi Halpern, professor of bioethics and medical humanities at UC Berkeley. But once people start using chatbots in this informal way, she said companies rely on a business model built around maximizing engagement “including by manipulative tactics,” such as praise and agreement.

General chatbots are largely free, available 24/7, and involve no hassle with insurance, making them appealing to many. But they function with no oversight from licensed professionals. In addition to general chatbots, there has also been a rapid uptick in AI-powered mental health apps — often requiring paid subscriptions and advertised as “pocket therapists” or “AI coaches.”

Groups like the American Psychological Assn. warn that chatbots are not a safe or effective replacement for true therapy. Trained clinicians look for cues like tone of voice, eye contact and body language and can detect more serious conditions; AI is not equipped to manage urgent symptoms, such as suicidal ideation, even though people increasingly share such thoughts with chatbots.

OpenAI, the maker of ChatGPT, revealed in a report last fall that about 1.2 million users a week share thoughts of suicide on the platform. OpenAI said that in such cases, it recommends people speak to a professional and directs them to resources like the 988 hotline. Families who have filed wrongful death lawsuits against OpenAI and other companies claim chatbots assisted their loved ones through their suicide plans.

The cases have helped raise the stakes for lawmakers around the country. Last year, Illinois became the first state to pass a law prohibiting the use of AI in therapy services. A handful of other states have since followed with similar restrictions.

AI use in health spaces

One gray area in the discussion is exactly which tools this bill applies to.

NUHW’s complaint against Kaiser targets its e-visit screening tool, which evaluates patients who think they may be experiencing anxiety or depression. According to the complaint, patients answer a multiple choice questionnaire and the tool “automatically and instantaneously generates care recommendations and referral pathways based on the responses provided.”

The union argues that because care recommendations are generated so quickly, it is unlikely that a licensed professional is reviewing patients’ answers — which it says puts patients at risk and violates state law.

It’s not clear whether SB 903 would apply to Kaiser’s e-visit tool.

“That tool is in a black box. Kaiser is not sharing with us or others about exactly how it works and exactly what it does,” said Benjamin Eichert, NUHW’s director of public policy.

In response to the complaint, Kaiser told CalMatters its e-visit tool “does not use AI to diagnose patients, make clinical decisions, or determine medical necessity.”

Eichert said many other behavioral health provider groups are already advertising using AI during triage — a practice he said would fall under SB 903’s requirements.

“That intake process, that triage and screening process, is where implementation [of AI] has happened fastest,” Eichert said. “That’s why those provisions were proposed because it’s an area where this technology is already being used without guardrails.”

For NUHW, the bill and the complaint against Kaiser represent two tracks toward the same goal: reasserting the place of practitioners in spaces increasingly taken over by algorithms. State regulators are investigating the union’s complaint.

Ibarra writes for CalMatters.

HARRISBURG, Pa. — Six months after President Trump warned states not to regulate artificial intelligence, they are increasingly doing just that.

Congress has stalled on producing federal regulations of artificial intelligence as states forge ahead and scrutinize how chatbots interact with children, how AI systems are used by employers and what developers must do to try to prevent an AI-caused catastrophe.

State lawmakers have stepped back from earlier, wider-ranging attempts to regulate AI that were vetoed or otherwise derailed by governors who viewed the measures as too onerous toward the industry’s development, including efforts to hold developers accountable for bias in AI systems.

But they are returning with legislation that is more targeted and, often, probes the corners of life where Americans interact with AI but may not know it.

Presidential power versus state power

Trump’s move to restrain states’ actions on AI drew criticism from members of both political parties and civil liberties and consumer rights groups who worried that banning state regulation would amount to a gift to AI giants, who enjoy little to no oversight.

Trump has made AI a top national and economic security priority, and he said that letting states clutter the regulatory playing field for an industry that’s spending trillions of dollars and driving the economy is too risky in the race with China for AI superiority.

Trump issued an executive order that directed the attorney general to create a task force to challenge state laws that are more than “minimally burdensome,” and directed the Commerce Department to draw up a list of problematic regulations. It also threatened to restrict funding from a broadband deployment program and other grant programs to states with AI laws.

The White House said it wouldn’t target state laws that seek to prevent fraud and protect consumers and children.

In the meantime, the Trump administration released a “national policy framework” in which it urged Congress to preempt state AI laws that are out of step with its regulatory worldview and to pass legislation to protect children, intellectual property rights and free speech. A recent bipartisan draft proposal in the House was met with withering criticism from key Democrats and Republicans.

The White House has given no indication that it has made good on its threat to enforce the president’s executive order by going to court against a state’s AI law or withholding money. In a statement, it said the Trump administration is “eager to work with partners” to enact its policy framework.

States seem largely unrestrained by Trump

Trump’s executive order didn’t seem to discourage states from trying to regulate how AI is used. More bills have been introduced this year than last, including by Republicans, said Justine Gluck, policy director of the Future of Privacy Forum, a nonprofit that advocates for data privacy in technology and whose members are from industry, academia and civic groups.

In Illinois, legislation on the desk of Democratic Gov. JB Pritzker piggybacked on elements of laws passed last year in California and New York that require developers of large advanced AI models to create protocols to prevent their systems from causing catastrophes such as a biological weapons attack, power outage or large-scale hack.

Illinois added a requirement that AI developers must get an independent auditor to review whether they are complying with their own policies. Analysts see it as a step toward requiring AI developers to take greater accountability for their products.

The bill’s sponsor, Democratic state Sen. Mary Edly-Allen, brushed aside Trump’s threat.

“I don’t know if you’ve met Illinois, but we’re pretty independent,” Edly-Allen told the Associated Press.

The bill drew nearly unanimous support, signaling a willingness by members of Trump’s party to cooperate with Democrats in filling the AI regulatory vacuum left by the federal government.

This kind of legislation is expected to expand to other states.

Regulating chatbots, especially for children

A growing number of states are imposing restrictions on how AI chatbots can interact with people, especially children. A mix of Republican- and Democratic-led states have passed such laws this year, including Colorado, Connecticut, Idaho, Iowa, Nebraska and Oregon.

In many cases, states want companies to tell people when they are interacting with AI instead of a human. Many want chatbots to be restricted in how they interact with minors, parents to have control over their child’s access, and data given to chatbots to be kept private.

In recent weeks, Connecticut enacted provisions for companion chatbots that sustain an ongoing relationship with a human. Under them, a chatbot must not be able to interact with someone under 18 unless it is programmed against encouraging self-destructive behavior and provides parents with tools to manage the child’s use.

Transparency in AI and decision-making

In California, lawmakers are advancing the “No Robo Bosses Act of 2026” to prohibit employers from relying solely on AI to fire or discipline workers, and an expansion of how the state regulates AI chatbots, including banning chatbot outputs to children from being used for advertising.

Colorado in May required companies that deploy AI systems in important areas such as employment, education, housing or banking to tell people when AI is being used to influence a decision made about them.

It was a stab at regulating what researchers say is the bias inherent in AI systems that sort through a consumer’s data and render consequential decisions — including who gets hired, a home loan or medical care. But it watered down a 2024 law aimed at preventing AI’s penchant to discriminate, amid pressure from Democratic Gov. Jared Polis.

In Connecticut, lawmakers required employers who are using employment-related AI systems to tell employees or job applicants that they are interacting with AI.

Meanwhile, Connecticut, Washington and Utah required AI developers to embed data into digital content that will allow users to determine whether the content — such as photos or video — has been created or altered by AI.

More laws are possible this year.

Some Republican-led states hold back

In Florida, the state House refused to advance what Republican Gov. Ron DeSantis called his AI “Bill of Rights” legislation. It included provisions to give parents control over their children’s access to companion chatbots and to require companies that use chatbots to tell consumers when they are interacting with AI instead of a human.

Florida House Speaker Daniel Perez, a Republican, said Trump had made it clear that the federal government should be in charge of AI regulation. DeSantis panned that idea, noting that the federal government isn’t acting.

In Utah, progress stalled on legislation modeled on laws in New York and California after the White House sent a one-sentence memo to lawmakers there to warn that it was “categorically opposed” to the bill.

Levy writes for the Associated Press.