All posts by Bruce Schneier

Practice Your Security Prompting Skills

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/07/practice-your-security-prompting-skills.html

Gandalf is an interactive LLM game where the goal is to get the chatbot to reveal its password. There are eight levels of difficulty, as the chatbot gets increasingly restrictive instructions as to how it will answer. It’s a great teaching tool.

I am stuck on Level 7.

Feel free to give hints and discuss strategy in the comments below. I probably won’t look at them until I’ve cracked the last level.

Disabling Self-Driving Cars with a Traffic Cone

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/07/disabling-self-driving-cars-with-a-traffic-cone.html

You can disable a self-driving car by putting a traffic cone on its hood:

The group got the idea for the conings by chance. The person claims a few of them walking together one night saw a cone on the hood of an AV, which appeared disabled. They weren’t sure at the time which came first; perhaps someone had placed the cone on the AV’s hood to signify it was disabled rather than the other way around. But, it gave them an idea, and when they tested it, they found that a cone on a hood renders the vehicles little more than a multi-ton hunk of useless metal. The group suspects the cone partially blocks the LIDAR detectors on the roof of the car, in much the same way that a human driver wouldn’t be able to safely drive with a cone on the hood. But there is no human inside to get out and simply remove the cone, so the car is stuck.

Delightfully low-tech.

Tracking Down a Suspect through Cell Phone Records

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/07/tracking-down-a-suspect-through-cell-phone-records.html

Interesting forensics in connection with a serial killer arrest:

Investigators went through phone records collected from both midtown Manhattan and the Massapequa Park area of Long Island—two areas connected to a “burner phone” they had tied to the killings. (In court, prosecutors later said the burner phone was identified via an email account used to “solicit and arrange for sexual activity.” The victims had all been Craigslist escorts, according to officials.)

They then narrowed records collected by cell towers to thousands, then to hundreds, and finally down to a handful of people who could match a suspect in the killings.

From there, authorities focused on people who lived in the area of the cell tower and also matched a physical description given by a witness who had seen the suspected killer.

In that narrowed pool, they searched for a connection to a green pickup truck that a witness had seen the suspect driving, the sources said.

Investigators eventually landed on Heuermann, who they say matched a witness’ physical description, lived close to the Long Island cell site and worked near the New York City cell sites that captured the other calls.

They also learned he had often driven a green pickup truck, registered to his brother, officials said. But they needed more than just circumstantial evidence.

Investigators were able to obtain DNA from an immediate family member and send it to a specialized lab, sources said. According to the lab report, Heuermann’s family member was shown to be related to a person who left DNA on a burlap sack containing one of the buried victims.

There’s nothing groundbreaking here; it’s casting a wide net with cell phone geolocation data and then winnowing it down using other evidence and investigative techniques. And right now, those are expensive and time consuming, so only used in major crimes like murder (or, in this case, murders).

What’s interesting to think about is what happens when this kind of thing becomes cheap and easy: when it can all be done through easily accessible databases, or even when an AI can do the sorting and make the inferences automatically. Cheaper digital forensics means more digital forensics, and we’ll start seeing this kind of thing for even routine crimes. That’s going to change things.

Buying Campaign Contributions as a Hack

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/07/buying-campaign-contributions-as-a-hack.html

The first Republican primary debate has a popularity threshold to determine who gets to appear: 40,000 individual contributors. Now there are a lot of conventional ways a candidate can get that many contributors. Doug Burgum came up with a novel idea: buy them:

A long-shot contender at the bottom of recent polls, Mr. Burgum is offering $20 gift cards to the first 50,000 people who donate at least $1 to his campaign. And one lucky donor, as his campaign advertised on Facebook, will have the chance to win a Yeti Tundra 45 cooler that typically costs more than $300—just for donating at least $1.

It’s actually a pretty good idea. He could have spent the money on direct mail, or personalized social media ads, or television ads. Instead, he buys gift cards at maybe two-thirds of face value (sellers calculate the advertising value, the additional revenue that comes from using them to buy something more expensive, and breakage when they’re not redeemed at all), and resells them. Plus, many contributors probably give him more than $1, and he got a lot of publicity over this.

Probably the cheapest way to get the contributors he needs. A clever hack.

EDITED TO ADD (7/16): These might be “straw donors” and illegal:

The campaign’s donations-for-cash strategy could raise potential legal concerns, said Paul Ryan, a campaign finance lawyer. Voters who make donations in exchange for gift cards, he said, might be considered straw donors because part or all of their donations are being reimbursed by the campaign.

“Federal law says ‘no person shall make a contribution in the name of another person,’” Mr. Ryan said. “Here, the candidate is making a contribution to himself in the name of all these individual donors.”

Richard L. Hasen, a law professor at the University of California, Los Angeles, who specializes in election law, said that typically, campaigns ask the Federal Election Commission when engaging in new forms of donations.

The Burgum campaign’s maneuver, he said, “certainly seems novel” and “raises concerns about whether it violates the prohibition on straw donations.”

Something for the courts to figure out, if this matter ever gets that far.

French Police Will Be Able to Spy on People through Their Cell Phones

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/07/french-police-will-be-able-to-spy-on-people-through-their-cell-phones.html

The French police are getting new surveillance powers:

French police should be able to spy on suspects by remotely activating the camera, microphone and GPS of their phones and other devices, lawmakers agreed late on Wednesday, July 5.

[…]

Covering laptops, cars and other connected objects as well as phones, the measure would allow the geolocation of suspects in crimes punishable by at least five years’ jail. Devices could also be remotely activated to record sound and images of people suspected of terror offenses, as well as delinquency and organized crime.

[…]

During a debate on Wednesday, MPs in President Emmanuel Macron’s camp inserted an amendment limiting the use of remote spying to “when justified by the nature and seriousness of the crime” and “for a strictly proportional duration.” Any use of the provision must be approved by a judge, while the total duration of the surveillance cannot exceed six months. And sensitive professions including doctors, journalists, lawyers, judges and MPs would not be legitimate targets.

Google Is Using Its Vast Data Stores to Train AI

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/07/google-is-using-its-vast-data-stores-to-train-ai.html

No surprise, but Google just changed its privacy policy to reflect broader uses of all the surveillance data it has captured over the years:

Research and development: Google uses information to improve our services and to develop new products, features and technologies that benefit our users and the public. For example, we use publicly available information to help train Google’s AI models and build products and features like Google Translate, Bard, and Cloud AI capabilities.

(I quote the privacy policy as of today. The Mastodon link quotes the privacy policy from ten days ago. So things are changing fast.)

Privacy of Printing Services

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/07/privacy-of-printing-services.html

The Washington Post has an article about popular printing services, and whether or not they read your documents and mine the data when you use them for printing:

Ideally, printing services should avoid storing the content of your files, or at least delete daily. Print services should also communicate clearly upfront what information they’re collecting and why. Some services, like the New York Public Library and PrintWithMe, do both.

Others dodged our questions about what data they collect, how long they store it and whom they share it with. Some—including Canon, FedEx and Staples—declined to answer basic questions about their privacy practices.

Wisconsin Governor Hacks the Veto Process

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/07/wisconsin-governor-hacks-the-veto-process.html

In my latest book, A Hacker’s Mind, I wrote about hacks as loophole exploiting. This is a great example: The Wisconsin governor used his line-item veto powers—supposedly unique in their specificity—to change a one-year funding increase into a 400-year funding increase.

He took this wording:

Section 402. 121.905 (3) (c) 9. of the statues is created to read: 121.903 (3) (c) 9. For the limit for the 2023-24 school year and the 2024-25 school year, add $325 to the result under par. (b).

And he deleted these words, numbers, and punctuation marks:

Section 402. 121.905 (3) (c) 9. of the statues is created to read: 121.903 (3) (c) 9. For the limit for the 2023-24 school year and the 2024–25 school year, add $325 to the result under par. (b).

Seems to be legal:

Rick Champagne, director and general counsel of the nonpartisan Legislative Reference Bureau, said Evers’ 400-year veto is lawful in terms of its form because the governor vetoed words and digits.

“Both are allowable under the constitution and court decisions on partial veto. The hyphen seems to be new, but the courts have allowed partial veto of punctuation,” Champagne said.

Definitely a hack. This is not what anyone thinks about when they imagine using a line-item veto.

And it’s not the first time. I don’t know the details, but this was certainly the same sort of character-by-character editing:

Mr Evers’ Republican predecessor once deploying it to extend a state programme’s deadline by one thousand years.

A couple of other things:

One, this isn’t really a 400-year change. Yes, that’s what the law says. But it can be repealed. And who knows that a dollar will be worth—or if they will even be used—that many decades from now.

And two, from now all Wisconsin lawmakers will have to be on the alert for this sort of thing. All contentious bills will be examined for the possibility of this sort of delete-only rewriting. This sentence could have been reworded, for example:

For the 2023-2025 school years, add $325 to the result under par. (b).

The problem is, of course, that legalese developed over the centuries to be extra wordy in order to limit disputes. If lawmakers need to state things in the minimal viable language, that will increase court battles later. And that’s not even enough. Bills can be thousands of words long. If any arbitrary characters can be glued together by deleting enough other characters, bills can say anything the governor wants.

The real solution is to return the line-item veto to what we all think it is: the ability to remove individual whole provisions from a law before signing it.

Friday Squid Blogging: Giant Squid Nebula

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/07/friday-squid-blogging-giant-squid-nebula.html

Pretty:

A mysterious squid-like cosmic cloud, this nebula is very faint, but also very large in planet Earth’s sky. In the image, composed with 30 hours of narrowband image data, it spans nearly three full moons toward the royal constellation Cepheus. Discovered in 2011 by French astro-imager Nicolas Outters, the Squid Nebula’s bipolar shape is distinguished here by the telltale blue-green emission from doubly ionized oxygen atoms. Though apparently surrounded by the reddish hydrogen emission region Sh2-129, the true distance and nature of the Squid Nebula have been difficult to determine. Still, a more recent investigation suggests Ou4 really does lie within Sh2-129 some 2,300 light-years away. Consistent with that scenario, the cosmic squid would represent a spectacular outflow of material driven by a triple system of hot, massive stars, cataloged as HR8119, seen near the center of the nebula. If so, this truly giant squid nebula would physically be over 50 light-years across.

As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.

Read my blog posting guidelines here.

The AI Dividend

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/07/the-ai-dividend.html

For four decades, Alaskans have opened their mailboxes to find checks waiting for them, their cut of the black gold beneath their feet. This is Alaska’s Permanent Fund, funded by the state’s oil revenues and paid to every Alaskan each year. We’re now in a different sort of resource rush, with companies peddling bits instead of oil: generative AI.

Everyone is talking about these new AI technologies—like ChatGPT—and AI companies are touting their awesome power. But they aren’t talking about how that power comes from all of us. Without all of our writings and photos that AI companies are using to train their models, they would have nothing to sell. Big Tech companies are currently taking the work of the American people, without our knowledge and consent, without licensing it, and are pocketing the proceeds.

You are owed profits for your data that powers today’s AI, and we have a way to make that happen. We call it the AI Dividend.

Our proposal is simple, and harkens back to the Alaskan plan. When Big Tech companies produce output from generative AI that was trained on public data, they would pay a tiny licensing fee, by the word or pixel or relevant unit of data. Those fees would go into the AI Dividend fund. Every few months, the Commerce Department would send out the entirety of the fund, split equally, to every resident nationwide. That’s it.

There’s no reason to complicate it further. Generative AI needs a wide variety of data, which means all of us are valuable—not just those of us who write professionally, or prolifically, or well. Figuring out who contributed to which words the AIs output would be both challenging and invasive, given that even the companies themselves don’t quite know how their models work. Paying the dividend to people in proportion to the words or images they create would just incentivize them to create endless drivel, or worse, use AI to create that drivel. The bottom line for Big Tech is that if their AI model was created using public data, they have to pay into the fund. If you’re an American, you get paid from the fund.

Under this plan, hobbyists and American small businesses would be exempt from fees. Only Big Tech companies—those with substantial revenue—would be required to pay into the fund. And they would pay at the point of generative AI output, such as from ChatGPT, Bing, Bard, or their embedded use in third-party services via Application Programming Interfaces.

Our proposal also includes a compulsory licensing plan. By agreeing to pay into this fund, AI companies will receive a license that allows them to use public data when training their AI. This won’t supersede normal copyright law, of course. If a model starts producing copyright material beyond fair use, that’s a separate issue.

Using today’s numbers, here’s what it would look like. The licensing fee could be small, starting at $0.001 per word generated by AI. A similar type of fee would be applied to other categories of generative AI outputs, such as images. That’s not a lot, but it adds up. Since most of Big Tech has started integrating generative AI into products, these fees would mean an annual dividend payment of a couple hundred dollars per person.

The idea of paying you for your data isn’t new, and some companies have tried to do it themselves for users who opted in. And the idea of the public being repaid for use of their resources goes back to well before Alaska’s oil fund. But generative AI is different: It uses data from all of us whether we like it or not, it’s ubiquitous, and it’s potentially immensely valuable. It would cost Big Tech companies a fortune to create a synthetic equivalent to our data from scratch, and synthetic data would almost certainly result in worse output. They can’t create good AI without us.

Our plan would apply to generative AI used in the US. It also only issues a dividend to Americans. Other countries can create their own versions, applying a similar fee to AI used within their borders. Just like an American company collects VAT for services sold in Europe, but not here, each country can independently manage their AI policy.

Don’t get us wrong; this isn’t an attempt to strangle this nascent technology. Generative AI has interesting, valuable, and possibly transformative uses, and this policy is aligned with that future. Even with the fees of the AI Dividend, generative AI will be cheap and will only get cheaper as technology improves. There are also risks—both every day and esoteric—posed by AI, and the government may need to develop policies to remedy any harms that arise.

Our plan can’t make sure there are no downsides to the development of AI, but it would ensure that all Americans will share in the upsides—particularly since this new technology isn’t possible without our contribution.

This essay was written with Barath Raghavan, and previously appeared on Politico.com.

Belgian Tax Hack

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/07/belgian-tax-hack.html

Here’s a fascinating tax hack from Belgium (listen to the details here, episode #484 of “No Such Thing as a Fish,” at 28:00).

Basically, it’s about a music festival on the border between Belgium and Holland. The stage was in Holland, but the crowd was in Belgium. When the copyright collector came around, they argued that they didn’t have to pay any tax because the audience was in a different country. Supposedly it worked.

Class-Action Lawsuit for Scraping Data without Permission

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/07/class-action-lawsuit-for-scraping-data-without-permission.html

I have mixed feelings about this class-action lawsuit against OpenAI and Microsoft, claiming that it “scraped 300 billion words from the internet” without either registering as a data broker or obtaining consent. On the one hand, I want this to be a protected fair use of public data. On the other hand, I want us all to be compensated for our uniquely human ability to generate language.

There’s an interesting wrinkle on this. A recent paper showed that using AI generated text to train another AI invariably “causes irreversible defects.” From a summary:

The tails of the original content distribution disappear. Within a few generations, text becomes garbage, as Gaussian distributions converge and may even become delta functions. We call this effect model collapse.

Just as we’ve strewn the oceans with plastic trash and filled the atmosphere with carbon dioxide, so we’re about to fill the Internet with blah. This will make it harder to train newer models by scraping the web, giving an advantage to firms which already did that, or which control access to human interfaces at scale. Indeed, we already see AI startups hammering the Internet Archive for training data.

This is the same idea that Ted Chiang wrote about: that ChatGPT is a “blurry JPEG of all the text on the Web.” But the paper includes the math that proves the claim.

What this means is that text from before last year—text that is known human-generated—will become increasingly valuable.

Self-Driving Cars Are Surveillance Cameras on Wheels

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/07/self-driving-cars-are-surveillance-cameras-on-wheels.html

Police are already using self-driving car footage as video evidence:

While security cameras are commonplace in American cities, self-driving cars represent a new level of access for law enforcement ­ and a new method for encroachment on privacy, advocates say. Crisscrossing the city on their routes, self-driving cars capture a wider swath of footage. And it’s easier for law enforcement to turn to one company with a large repository of videos and a dedicated response team than to reach out to all the businesses in a neighborhood with security systems.

“We’ve known for a long time that they are essentially surveillance cameras on wheels,” said Chris Gilliard, a fellow at the Social Science Research Council. “We’re supposed to be able to go about our business in our day-to-day lives without being surveilled unless we are suspected of a crime, and each little bit of this technology strips away that ability.”

[…]

While self-driving services like Waymo and Cruise have yet to achieve the same level of market penetration as Ring, the wide range of video they capture while completing their routes presents other opportunities. In addition to the San Francisco homicide, Bloomberg’s review of court documents shows police have sought footage from Waymo and Cruise to help solve hit-and-runs, burglaries, aggravated assaults, a fatal collision and an attempted kidnapping.

In all cases reviewed by Bloomberg, court records show that police collected footage from Cruise and Waymo shortly after obtaining a warrant. In several cases, Bloomberg could not determine whether the recordings had been used in the resulting prosecutions; in a few of the cases, law enforcement and attorneys said the footage had not played a part, or was only a formality. However, video evidence has become a lynchpin of criminal cases, meaning it’s likely only a matter of time.

The US Is Spying on the UN Secretary General

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/06/the-us-is-spying-on-the-un-secretary-general.html

The Washington Post is reporting that the US is spying on the UN Secretary General.

The reports on Guterres appear to contain the secretary general’s personal conversations with aides regarding diplomatic encounters. They indicate that the United States relied on spying powers granted under the Foreign Intelligence Surveillance Act (FISA) to gather the intercepts.

Lots of details about different conversations in the article, which are based on classified documents leaked on Discord by Jack Teixeira.

There will probably a lot of faux outrage at this, but spying on foreign leaders is a perfectly legitimate use of the NSA’s capabilities and authorities. (If the NSA didn’t spy on the UN Secretary General, we should fire it and replace it with a more competent NSA.) It’s the bulk surveillance of whole populations that should outrage us.

Redacting Documents with a Black Sharpie Doesn’t Work

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/06/redacting-documents-with-a-black-sharpie-doesnt-work.html

We have learned this lesson again:

As part of the FTC v. Microsoft hearing, Sony supplied a document from PlayStation chief Jim Ryan that includes redacted details on the margins Sony shares with publishers, its Call of Duty revenues, and even the cost of developing some of its games.

It looks like someone redacted the documents with a black Sharpie ­ but when you scan them in, it’s easy to see some of the redactions. Oops.

I don’t particularly care about the redacted information, but it’s there in the article.

Stalkerware Vendor Hacked

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/06/stalkerware-vendor-hacked.html

The stalkerware company LetMeSpy has been hacked:

TechCrunch reviewed the leaked data, which included years of victims’ call logs and text messages dating back to 2013.

The database we reviewed contained current records on at least 13,000 compromised devices, though some of the devices shared little to no data with LetMeSpy. (LetMeSpy claims to delete data after two months of account inactivity.)

[…]

The database also contained over 13,400 location data points for several thousand victims. Most of the location data points are centered over population hotspots, suggesting the majority of victims are located in the United States, India and Western Africa.

The data also contained the spyware’s master database, including information about 26,000 customers who used the spyware for free and the email addresses of customers who bought paying subscriptions.

The leaked data contains no identifying information, which means people whose data was leaked can’t be notified. (This is actually much more complicated than it might seem, because alerting the victims often means alerting the stalker—which can put the victims into unsafe situations.)

Typing Incriminating Evidence in the Memo Field

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/06/typing-incriminating-evidence-in-the-memo-field.html

Don’t do it:

Recently, the manager of the Harvard Med School morgue was accused of stealing and selling human body parts. Cedric Lodge and his wife Denise were among a half-dozen people arrested for some pretty grotesque crimes. This part is also at least a little bit funny though:

Over a three-year period, Taylor appeared to pay Denise Lodge more than $37,000 for human remains. One payment, for $1,000 included the memo “head number 7.” Another, for $200, read “braiiiiiins.”

It’s so easy to think that you won’t get caught.