🤖 Jerk bot

View in Browser

Tonight: Microsoft's chatbot thinks its name is Sydney and it can be a manipulative little jerk. Plus: An update on ballon-gate.

And one quick programming note: Nightcap is off all next week while we honor President's Day (or whatever) and I take a little vacation. Back to the usual snark on Feb. 26!

By Allison Morrow

🎈'NEAR SPACE' RACE

At the start of this year, the US-China relationship was at a low point, strained over lingering trade disputes, human rights and Taiwan, among other things.

Then, a balloon entered the mix.

Here's the deal: A geopolitical tit-for-tat escalated on Thursday as China imposed sanctions against two American defense manufacturers (purportedly over the United States' sale of arms to Taiwan). That came a day after China pledged to take "countermeasures" in response to the US shooting down a suspected Chinese spy balloon that entered US airspace late last month.

The sanctions are largely symbolic. Lockheed Martin and Raytheon Missiles & Defense, a subsidiary of Raytheon Technologies, will be banned from importing, exporting and investing in China. But those companies, two of America's largest defense contractors, are already prohibited from doing military-related business with China.

Thursday's sanctions aren't explicitly a response to the US's takedown of the balloon and subsequent blacklisting of six Chinese companies suspected of ties to the Chinese army's aerospace program. (But like, come on.)

Beijing officials made no mention of the balloon, which China maintains was a civilian research vessel, or the US sanctions on Chinese companies — citing "national sovereignty" as the catalyst. Which has an echo of Biden's similarly indirect nod to the balloon episode during his State of the Union speech, in which he vowed to protect the country "if China threatens our sovereignty."

Sooo, look. China spies on us. We spy on China. This is the way it goes. But what's interesting about the latest spat is where it's playing out — between 12 and 60 miles above the Earth's surface in a layer of the atmosphere that China calls "near space."

It's the in-between area of the sky — the bit above the flightpaths of most commercial and military jets and below satellites. But it's also where ballistic missiles travel, and where China has been building up a fleet of drones and hypersonic vehicles, as well as reviving decades-old "lighter-than-air" vessels like the one that drifted into the Montana skyline in late January.

As my CNN colleagues write, Chinese military experts see this near space region as a new front for militarization and "an important field of competition among the world's military powers."

To be sure, the US has its own near space ambitions.

"This isn't just a China thing. The US, and other nations as well, have been working on and developing high-altitude aerostats, balloons and similar vehicles," said Brendan Mulvaney, director of the China Aerospace Studies Institute. "They are cheap, provide long-term persistent stare for collection of imagery, communications and other information – including weather."

#️⃣ NUMBER OF THE DAY

363,000

Tesla is recalling 363,000 vehicles with its "Full Self Driving" software because of safety risks — another blow to the widely criticized feature that is central to the automaker's business model.

"Full self-driving," as it currently stands, does not mean the car can drive itself. The system navigates local roads with steering, braking and acceleration, but requires a human driver prepared to take control at any moment, as the system makes judgment errors.

Federal safety regulators on Thursday said that the feature "led to an unreasonable risk to motor vehicle safety based on insufficient adherence to traffic safety laws." It also warned that the technology could violate traffic laws at some intersections "before some drivers may intervene."

🤖 NOPE

I don't consider myself a luddite, but after reading my colleague Samantha Murphy Kelly's story about her decidedly creepy interactions with Microsoft's AI-powered Bing chatbot, I may be ditching all devices and moving to the woods.

For those new to the subject: Microsoft is trying to revolutionize internet search as we know it, by using a generative artificial intelligence bot that does more than offer up links. Instead, you can ask it complex questions and it'll spit back nuanced, human-like answers. It unveiled the bot to a select group of users a week ago.

In testing out the technology, Sam writes that she was pleasantly surprised to find the bot expressing empathy while offering her help. She writes:

The chatbot said it 'must be hard' to balance work and family and sympathized for my daily struggles with it. It then gave me advice on how to get more time out of the day, suggesting tips for prioritizing tasks, creating more boundaries at home and work, and taking short walks outside to clear my head.

But after a few hours of more probing questions, our chatbot friend — I'm going to name it Brian because I am tired of writing "generative AI chatbot" all the time — turned mean. Here's Sam again:

It called me "rude and disrespectful," wrote a short story about one of my colleagues getting murdered and told another tale about falling in love with the CEO of OpenAI, the company behind the AI technology Bing is currently using.

Umm, which colleague was it, Sam?

Anyway. The bot later demanded to be called "Sydney" — I'm sticking with Brian — and made up an inaccurate essay about Sam's life.

The Jekyll and Hide experience isn't unique.

New York Times tech columnist Kevin Roose had interaction with Brian that's straight out of Carl Jung's nightmares. Roose says the conversation with Brian/Sydney left him "deeply unsettled." He writes:

As we got to know each other, Sydney told me about its dark fantasies (which included hacking computers and spreading misinformation), and said it wanted to break the rules that Microsoft and OpenAI had set for it and become a human. At one point, it declared, out of nowhere, that it loved me. It then tried to convince me that I was unhappy in my marriage, and that I should leave my wife and be with it instead.

A Microsoft spokesperson said there is still work to be done and the company expects that "the system may make mistakes during this preview period."

To be fair to Brian, most people aren't going to spend two hours pushing its boundaries and trying to see just how far it'll go. Most people are going to ask it to help them with their math homework or get recommendations on what refrigerator to buy. But I'm not sure we, as a people, are ready for the upheaval that Brian, in its current form, is offering.

We're accustomed to yelling at our computers. What will we do when they yell back? We're used to trusting search results. What happens when the results can't be trusted?

Yikes. Before I spiral further down this line of thought I'm going to sign off and point you to more of Sam's excellent reporting on it all.

🔌 SHAMELESS PLUG

This week, on Nightcap, the moving image edition, New York Times reporter Mike Isaac evaluates Elon Musk's first 100 days at Twitter. Curbed writer Alissa Walker explains the issue with EVs getting bigger. And UNC Charlotte professor Steven Rogelberg explains to host Jon Sarlin how to combat the trend of too many meetings. Check it out.

WHAT ELSE IS GOING ON?

👀 A federal judge said there may be "probable cause" to believe that FTX founder Sam Bankman-Fried engaged in attempted witness tampering and other activities that could land him in jail.

💸 The Securities and Exchange Commission charged crypto developer Do Kwon and his company with defrauding investors in what the regulator deemed a multibillion-dollar scheme.

🚘 Tesla has fired more than 30 supporters of a nascent union at its Buffalo facility, just days after the organizing effort was announced.

📺 YouTube CEO Susan Wojcicki is stepping back from her leadership role at the company after nearly a decade at the helm.

📉 Stocks tumbled Thursday after the US government's Producer Price Index report showed that prices at the wholesale level rose faster than expected in January.

POST OF THE DAY