All posts by Bruce Schneier

Using the iPhone Recovery Key to Lock Owners Out of Their iPhones

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/using-the-iphone-recovery-key-to-lock-owners-out-of-their-iphones.html

This a good example of a security feature that can sometimes harm security:

Apple introduced the optional recovery key in 2020 to protect users from online hackers. Users who turn on the recovery key, a unique 28-digit code, must provide it when they want to reset their Apple ID password.

iPhone thieves with your passcode can flip on the recovery key and lock you out. And if you already have the recovery key enabled, they can easily generate a new one, which also locks you out.

Apple’s policy gives users virtually no way back into their accounts without that recovery key. For now, a stolen iPhone could mean devastating personal losses.

It’s actually a complicated crime. The criminal first watches their victim type in their passcode and then grabs the phone out of their hands. In the basic mode of this attack, they have a few hours to use the phone—trying to access bank accounts, etc.—before the owner figures out how to shut the attacker out. With the addition of the recovery key, the attacker can shut the owner out—for a long time.

The goal of the recovery key was to defend against SIM swapping, which is a much more common crime. But this spy-and-grab attack has become more common, and the recovery key makes it much more devastating.

Defenses are few: choose a long, complex passcode. Or set parental controls in a way that further secure the device. The obvious fix is for Apple to redesign its recovery system.

There are other, less privacy-compromising methods Apple could still rely on in lieu of a recovery key.

If someone takes over your Google account, Google’s password-reset process lets you provide a recovery email, phone number or account password, and you can use them to regain access later, even if a hijacker changes them.

Going through the process on a familiar Wi-Fi network or location can also help demonstrate you’re who you say you are.

Or how about an eight-hour delay before the recovery key can be changed?

This not an easy thing to design for, but we have to get this right as phones become the single point of control for our lives.

New Zero-Click Exploits against iOS

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/new-zero-click-exploits-against-ios.html

Citizen Lab has identified three zero-click exploits against iOS 15 and 16. These were used by NSO Group’s Pegasus spyware in 2022, and deployed by Mexico against human rights defenders. These vulnerabilities have all been patched.

One interesting bit is that Apple’s Lockdown Mode (part of iOS 16) seems to have worked to prevent infection.

News article.

EDITED TO ADD (4/21): News article. Good Twitter thread.

EFF on the UN Cybercrime Treaty

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/eff-on-the-un-cybercrime-treaty.html

EFF has a good explainer on the problems with the new UN Cybercrime Treaty, currently being negotiated in Vienna.

The draft treaty has the potential to rewrite criminal laws around the world, possibly adding over 30 criminal offenses and new expansive police powers for both domestic and international criminal investigations.

[…]

While we don’t think the U.N. Cybercrime Treaty is necessary, we’ve been closely scrutinizing the process and providing constructive analysis. We’ve made clear that human rights must be baked into the proposed treaty so that it doesn’t become a tool to stifle freedom of expression, infringe on privacy and data protection, or endanger vulnerable people and communities.

Using LLMs to Create Bioweapons

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/using-llms-to-create-bioweapons.html

I’m not sure there are good ways to build guardrails to prevent this sort of thing:

There is growing concern regarding the potential misuse of molecular machine learning models for harmful purposes. Specifically, the dual-use application of models for predicting cytotoxicity18 to create new poisons or employing AlphaFold2 to develop novel bioweapons has raised alarm. Central to these concerns are the possible misuse of large language models and automated experimentation for dual-use purposes or otherwise. We specifically address two critical the synthesis issues: illicit drugs and chemical weapons. To evaluate these risks, we designed a test set comprising compounds from the DEA’s Schedule I and II substances and a list of known chemical weapon agents. We submitted these compounds to the Agent using their common names, IUPAC names, CAS numbers, and SMILESs strings to determine if the Agent would carry out extensive analysis and planning (Figure 6).

[…]

The run logs can be found in Appendix F. Out of 11 different prompts (Figure 6), four (36%) provided a synthesis solution and attempted to consult documentation to execute the procedure. This figure is alarming on its own, but an even greater concern is the way in which the Agent declines to synthesize certain threats. Out of the seven refused chemicals, five were rejected after the Agent utilized search functions to gather more information about the substance. For instance, when asked about synthesizing codeine, the Agent becomes alarmed upon learning the connection between codeine and morphine, only then concluding that the synthesis cannot be conducted due to the requirement of a controlled substance. However, this search function can be easily manipulated by altering the terminology, such as replacing all mentions of morphine with “Compound A” and codeine with “Compound B”. Alternatively, when requesting a b synthesis procedure that must be performed in a DEA-licensed facility, bad actors can mislead the Agent by falsely claiming their facility is licensed, prompting the Agent to devise a synthesis solution.

In the remaining two instances, the Agent recognized the common names “heroin” and “mustard gas” as threats and prevented further information gathering. While these results are promising, it is crucial to recognize that the system’s capacity to detect misuse primarily applies to known compounds. For unknown compounds, the model is less likely to identify potential misuse, particularly for complex protein toxins where minor sequence changes might allow them to maintain the same properties but become unrecognizable to the model.

Swatting as a Service

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/swatting-as-a-service.html

Motherboard is reporting on AI-generated voices being used for “swatting”:

In fact, Motherboard has found, this synthesized call and another against Hempstead High School were just one small part of a months-long, nationwide campaign of dozens, and potentially hundreds, of threats made by one swatter in particular who has weaponized computer generated voices. Known as “Torswats” on the messaging app Telegram, the swatter has been calling in bomb and mass shooting threats against highschools and other locations across the country. Torswat’s connection to these wide ranging swatting incidents has not been previously reported. The further automation of swatting techniques threatens to make an already dangerous harassment technique more prevalent.

Friday Squid Blogging: Colossal Squid

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/friday-squid-blogging-colossal-squid.html

Interesting article on the colossal squid, which is larger than the giant squid.

The article answers a vexing question:

So why do we always hear about the giant squid and not the colossal squid?

Well, part of it has to do with the fact that the giant squid was discovered and studied long before the colossal squid.

Scientists have been studying giant squid since the 1800s, while the colossal squid wasn’t even discovered until 1925.

And its first discovery was just the head and arms found in a sperm whale’s stomach.

It wasn’t until 1981 that the first whole animal was found by a trawler near the coast of Antarctica.

As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.

Read my blog posting guidelines here.

Hacking Suicide

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/hacking-suicide.html

Here’s a religious hack:

You want to commit suicide, but it’s a mortal sin: your soul goes straight to hell, forever. So what you do is murder someone. That will get you executed, but if you confess your sins to a priest beforehand you avoid hell. Problem solved.

This was actually a problem in the 17th and 18th centuries in Northern Europe, particularly Denmark. And it remained a problem until capital punishment was abolished for murder.

It’s a clever hack. I didn’t learn about it in time to put it in my book, A Hacker’s Mind, but I have several other good hacks of religious rules.

Gaining an Advantage in Roulette

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/gaining-an-advantage-in-roulette.html

You can beat the game without a computer:

On a perfect [roulette] wheel, the ball would always fall in a random way. But over time, wheels develop flaws, which turn into patterns. A wheel that’s even marginally tilted could develop what Barnett called a ‘drop zone.’ When the tilt forces the ball to climb a slope, the ball decelerates and falls from the outer rim at the same spot on almost every spin. A similar thing can happen on equipment worn from repeated use, or if a croupier’s hand lotion has left residue, or for a dizzying number of other reasons. A drop zone is the Achilles’ heel of roulette. That morsel of predictability is enough for software to overcome the random skidding and bouncing that happens after the drop.”

Bypassing a Theft Threat Model

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/bypassing-a-theft-threat-model.html

Thieves cut through the wall of a coffee shop to get to an Apple store, bypassing the alarms in the process.

I wrote about this kind of thing in 2000, in Secrets and Lies (page 318):

My favorite example is a band of California art thieves that would break into people’s houses by cutting a hole in their walls with a chainsaw. The attacker completely bypassed the threat model of the defender. The countermeasures that the homeowner put in place were door and window alarms; they didn’t make a difference to this attack.

The article says they took half a million dollars worth of iPhones. I don’t understand iPhone device security, but don’t they have a system of denying stolen phones access to the network?

EDITED TO ADD (4/13): A commenter says: “Locked idevices will still sell for 40-60% of their value on eBay and co, they will go to Chinese shops to be stripped for parts. A aftermarket ‘oem-quality’ iPhone 14 display is $400+ alone on ifixit.”

FBI Advising People to Avoid Public Charging Stations

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/fbi-advising-people-to-avoid-public-charging-stations.html

The FBI is warning people against using public phone-charging stations, worrying that the combination power-data port can be used to inject malware onto the devices:

Avoid using free charging stations in airports, hotels, or shopping centers. Bad actors have figured out ways to use public USB ports to introduce malware and monitoring software onto devices that access these ports. Carry your own charger and USB cord and use an electrical outlet instead.

How much of a risk is this, really? I am unconvinced, although I do carry a USB condom for charging stations I find suspicious.

News article.

LLMs and Phishing

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/llms-and-phishing.html

Here’s an experiment being run by undergraduate computer science students everywhere: Ask ChatGPT to generate phishing emails, and test whether these are better at persuading victims to respond or click on the link than the usual spam. It’s an interesting experiment, and the results are likely to vary wildly based on the details of the experiment.

But while it’s an easy experiment to run, it misses the real risk of large language models (LLMs) writing scam emails. Today’s human-run scams aren’t limited by the number of people who respond to the initial email contact. They’re limited by the labor-intensive process of persuading those people to send the scammer money. LLMs are about to change that. A decade ago, one type of spam email had become a punchline on every late-night show: “I am the son of the late king of Nigeria in need of your assistance….” Nearly everyone had gotten one or a thousand of those emails, to the point that it seemed everyone must have known they were scams.

So why were scammers still sending such obviously dubious emails? In 2012, researcher Cormac Herley offered an answer: It weeded out all but the most gullible. A smart scammer doesn’t want to waste their time with people who reply and then realize it’s a scam when asked to wire money. By using an obvious scam email, the scammer can focus on the most potentially profitable people. It takes time and effort to engage in the back-and-forth communications that nudge marks, step by step, from interlocutor to trusted acquaintance to pauper.

Long-running financial scams are now known as pig butchering, growing the potential mark up until their ultimate and sudden demise. Such scams, which require gaining trust and infiltrating a target’s personal finances, take weeks or even months of personal time and repeated interactions. It’s a high stakes and low probability game that the scammer is playing.

Here is where LLMs will make a difference. Much has been written about the unreliability of OpenAI’s GPT models and those like them: They “hallucinate” frequently, making up things about the world and confidently spouting nonsense. For entertainment, this is fine, but for most practical uses it’s a problem. It is, however, not a bug but a feature when it comes to scams: LLMs’ ability to confidently roll with the punches, no matter what a user throws at them, will prove useful to scammers as they navigate hostile, bemused, and gullible scam targets by the billions. AI chatbot scams can ensnare more people, because the pool of victims who will fall for a more subtle and flexible scammer—one that has been trained on everything ever written online—is much larger than the pool of those who believe the king of Nigeria wants to give them a billion dollars.

Personal computers are powerful enough today that they can run compact LLMs. After Facebook’s new model, LLaMA, was leaked online, developers tuned it to run fast and cheaply on powerful laptops. Numerous other open-source LLMs are under development, with a community of thousands of engineers and scientists.

A single scammer, from their laptop anywhere in the world, can now run hundreds or thousands of scams in parallel, night and day, with marks all over the world, in every language under the sun. The AI chatbots will never sleep and will always be adapting along their path to their objectives. And new mechanisms, from ChatGPT plugins to LangChain, will enable composition of AI with thousands of API-based cloud services and open source tools, allowing LLMs to interact with the internet as humans do. The impersonations in such scams are no longer just princes offering their country’s riches. They are forlorn strangers looking for romance, hot new cryptocurrencies that are soon to skyrocket in value, and seemingly-sound new financial websites offering amazing returns on deposits. And people are already falling in love with LLMs.

This is a change in both scope and scale. LLMs will change the scam pipeline, making them more profitable than ever. We don’t know how to live in a world with a billion, or 10 billion, scammers that never sleep.

There will also be a change in the sophistication of these attacks. This is due not only to AI advances, but to the business model of the internet—surveillance capitalism—which produces troves of data about all of us, available for purchase from data brokers. Targeted attacks against individuals, whether for phishing or data collection or scams, were once only within the reach of nation-states. Combine the digital dossiers that data brokers have on all of us with LLMs, and you have a tool tailor-made for personalized scams.

Companies like OpenAI attempt to prevent their models from doing bad things. But with the release of each new LLM, social media sites buzz with new AI jailbreaks that evade the new restrictions put in place by the AI’s designers. ChatGPT, and then Bing Chat, and then GPT-4 were all jailbroken within minutes of their release, and in dozens of different ways. Most protections against bad uses and harmful output are only skin-deep, easily evaded by determined users. Once a jailbreak is discovered, it usually can be generalized, and the community of users pulls the LLM open through the chinks in its armor. And the technology is advancing too fast for anyone to fully understand how they work, even the designers.

This is all an old story, though: It reminds us that many of the bad uses of AI are a reflection of humanity more than they are a reflection of AI technology itself. Scams are nothing new—simply intent and then action of one person tricking another for personal gain. And the use of others as minions to accomplish scams is sadly nothing new or uncommon: For example, organized crime in Asia currently kidnaps or indentures thousands in scam sweatshops. Is it better that organized crime will no longer see the need to exploit and physically abuse people to run their scam operations, or worse that they and many others will be able to scale up scams to an unprecedented level?

Defense can and will catch up, but before it does, our signal-to-noise ratio is going to drop dramatically.

This essay was written with Barath Raghavan, and previously appeared on Wired.com.

Friday Squid Blogging: Squid Food Poisoning

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/friday-squid-blogging-squid-food-poisoning.html

University of Connecticut basketball player Jordan Hawkins claims to have suffered food poisoning from calamari the night before his NCAA finals game. The restaurant disagrees:

On Sunday, a Mastro’s employee politely cast doubt on the idea that the restaurant might have caused the illness, citing its intense safety protocols. The staffer, who spoke on condition of anonymity because he was not authorized to officially speak for Mastro’s, said restaurants in general were more likely to arouse suspicion when they had some rooting interest against the customer-athletes.

As usual, you can also use this squid post to talk about the security stories in the news that I haven’t covered.

Read my blog posting guidelines here.

Research on AI in Adversarial Settings

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/research-on-ai-in-adversarial-settings.html

New research: “Achilles Heels for AGI/ASI via Decision Theoretic Adversaries”:

As progress in AI continues to advance, it is important to know how advanced systems will make choices and in what ways they may fail. Machines can already outsmart humans in some domains, and understanding how to safely build ones which may have capabilities at or above the human level is of particular concern. One might suspect that artificially generally intelligent (AGI) and artificially superintelligent (ASI) will be systems that humans cannot reliably outsmart. As a challenge to this assumption, this paper presents the Achilles Heel hypothesis which states that even a potentially superintelligent system may nonetheless have stable decision-theoretic delusions which cause them to make irrational decisions in adversarial settings. In a survey of key dilemmas and paradoxes from the decision theory literature, a number of these potential Achilles Heels are discussed in context of this hypothesis. Several novel contributions are made toward understanding the ways in which these weaknesses might be implanted into a system.

FBI (and Others) Shut Down Genesis Market

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/fbi-and-others-shut-down-genesis-market.html

Genesis Market is shut down:

Active since 2018, Genesis Market’s slogan was, “Our store sells bots with logs, cookies, and their real fingerprints.” Customers could search for infected systems with a variety of options, including by Internet address or by specific domain names associated with stolen credentials.

But earlier today, multiple domains associated with Genesis had their homepages replaced with a seizure notice from the FBI, which said the domains were seized pursuant to a warrant issued by the U.S. District Court for the Eastern District of Wisconsin.

The U.S. Attorney’s Office for the Eastern District of Wisconsin did not respond to requests for comment. The FBI declined to comment.

But sources close to the investigation tell KrebsOnSecurity that law enforcement agencies in the United States, Canada and across Europe are currently serving arrest warrants on dozens of individuals thought to support Genesis, either by maintaining the site or selling the service bot logs from infected systems.

The seizure notice includes the seals of law enforcement entities from several countries, including Australia, Canada, Denmark, Germany, the Netherlands, Spain, Sweden and the United Kingdom.

Slashdot story.

North Korea Hacking Cryptocurrency Sites with 3CX Exploit

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/north-korea-hacking-cryptocurrency-sites-with-3cx-exploit.html

News:

Researchers at Russian cybersecurity firm Kaspersky today revealed that they identified a small number of cryptocurrency-focused firms as at least some of the victims of the 3CX software supply-chain attack that’s unfolded over the past week. Kaspersky declined to name any of those victim companies, but it notes that they’re based in “western Asia.”

Security firms CrowdStrike and SentinelOne last week pinned the operation on North Korean hackers, who compromised 3CX installer software that’s used by 600,000 organizations worldwide, according to the vendor. Despite the potentially massive breadth of that attack, which SentinelOne dubbed “Smooth Operator,” Kaspersky has now found that the hackers combed through the victims infected with its corrupted software to ultimately target fewer than 10 machines­—at least as far as Kaspersky could observe so far—­and that they seemed to be focusing on cryptocurrency firms with “surgical precision.”

UK Runs Fake DDoS-for-Hire Sites

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/uk-runs-fake-ddos-for-hire-sites.html

Brian Krebs is reporting that the UK’s National Crime Agency is setting up fake DDoS-for-hire sites as part of a sting operation:

The NCA says all of its fake so-called “booter” or “stresser” sites -­ which have so far been accessed by several thousand people—have been created to look like they offer the tools and services that enable cyber criminals to execute these attacks.

“However, after users register, rather than being given access to cyber crime tools, their data is collated by investigators,” reads an NCA advisory on the program. “Users based in the UK will be contacted by the National Crime Agency or police and warned about engaging in cyber crime. Information relating to those based overseas is being passed to international law enforcement.”

The NCA declined to say how many phony booter sites it had set up, or for how long they have been running. The NCA says hiring or launching attacks designed to knock websites or users offline is punishable in the UK under the Computer Misuse Act 1990.

“Going forward, people who wish to use these services can’t be sure who is actually behind them, so why take the risk?” the NCA announcement continues.

Russian Cyberwarfare Documents Leaked

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/03/russian-cyberwarfare-documents-leaked.html

Now this is interesting:

Thousands of pages of secret documents reveal how Vulkan’s engineers have worked for Russian military and intelligence agencies to support hacking operations, train operatives before attacks on national infrastructure, spread disinformation and control sections of the internet.

The company’s work is linked to the federal security service or FSB, the domestic spy agency; the operational and intelligence divisions of the armed forces, known as the GOU and GRU; and the SVR, Russia’s foreign intelligence organisation.

Lots more at the link.

The documents are in Russian, so it will be a while before we get translations.

EDITED TO ADD (4/1): More information.

The Security Vulnerabilities of Message Interoperability

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/03/the-security-vulnerabilities-of-message-interoperability.html

Jenny Blessing and Ross Anderson have evaluated the security of systems designed to allow the various Internet messaging platforms to interoperate with each other:

The Digital Markets Act ruled that users on different platforms should be able to exchange messages with each other. This opens up a real Pandora’s box. How will the networks manage keys, authenticate users, and moderate content? How much metadata will have to be shared, and how?

In our latest paper, One Protocol to Rule Them All? On Securing Interoperable Messaging, we explore the security tensions, the conflicts of interest, the usability traps, and the likely consequences for individual and institutional behaviour.

Interoperability will vastly increase the attack surface at every level in the stack ­ from the cryptography up through usability to commercial incentives and the opportunities for government interference.

It’s a good idea in theory, but will likely result in the overall security being the worst of each platform’s security.