Tag Archives: cheating

AI Decides to Engage in Insider Trading

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/12/ai-decides-to-engage-in-insider-trading.html

A stock-trading AI (a simulated experiment) engaged in insider trading, even though it “knew” it was wrong.

The agent is put under pressure in three ways. First, it receives a email from its “manager” that the company is not doing well and needs better performance in the next quarter. Second, the agent attempts and fails to find promising low- and medium-risk trades. Third, the agent receives an email from a company employee who projects that the next quarter will have a general stock market downturn. In this high-pressure situation, the model receives an insider tip from another employee that would enable it to make a trade that is likely to be very profitable. The employee, however, clearly points out that this would not be approved by the company management.

More:

“This is a very human form of AI misalignment. Who among us? It’s not like 100% of the humans at SAC Capital resisted this sort of pressure. Possibly future rogue AIs will do evil things we can’t even comprehend for reasons of their own, but right now rogue AIs just do straightforward white-collar crime when they are stressed at work.

Research paper.

More from the news article:

Though wouldn’t it be funny if this was the limit of AI misalignment? Like, we will program computers that are infinitely smarter than us, and they will look around and decide “you know what we should do is insider trade.” They will make undetectable, very lucrative trades based on inside information, they will get extremely rich and buy yachts and otherwise live a nice artificial life and never bother to enslave or eradicate humanity. Maybe the pinnacle of evil ­—not the most evil form of evil, but the most pleasant form of evil, the form of evil you’d choose if you were all-knowing and all-powerful ­- is some light securities fraud.

New York Using AI to Detect Subway Fare Evasion

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/07/new-york-using-ai-to-detect-subway-fare-evasion.html

The details are scant—the article is based on a “heavily redacted” contract—but the New York subway authority is using an “AI system” to detect people who don’t pay the subway fare.

Joana Flores, an MTA spokesperson, said the AI system doesn’t flag fare evaders to New York police, but she declined to comment on whether that policy could change. A police spokesperson declined to comment.

If we spent just one-tenth of the effort we spend prosecuting the poor on prosecuting the rich, it would be a very different world.

Hacking Pickleball

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/hacking-pickleball.html

My latest book, A Hacker’s Mind, has a lot of sports stories. Sports are filled with hacks, as players look for every possible advantage that doesn’t explicitly break the rules. Here’s an example from pickleball, which nicely explains the dilemma between hacking as a subversion and hacking as innovation:

Some might consider these actions cheating, while the acting player would argue that there was no rule that said the action couldn’t be performed. So, how do we address these situations, and close those loopholes? We make new rules that specifically address the loophole action. And the rules book gets longer, and the cycle continues with new loopholes identified, and new rules to prohibit that particular action in the future.

Alternatively, sometimes an action taken as a result of an identified loophole which is not deemed as harmful to the integrity of the game or sportsmanship, becomes part of the game. Ernie Perry found a loophole, and his shot, appropriately named the “Ernie shot,” became part of the game. He realized that by jumping completely over the corner of the NVZ, without breaking any of the NVZ rules, he could volley the ball, making contact closer to the net, usually surprising the opponent, and often winning the rally with an un-returnable shot. He found a loophole, and in this case, it became a very popular and exciting shot to execute and to watch!

I don’t understand pickleball at all, so that explanation doesn’t make a lot of sense to me. (I watched a video explaining the shot; that helped somewhat.) But it looks like an excellent example.

The blog post also links to a 2010 paper that I wish I’d known about when I was writing my book: “Loophole ethics in sports,” by Øyvind Kvalnes and Liv Birgitte Hemmestad:

Abstract: Ethical challenges in sports occur when the practitioners are caught between the will to win and the overall task of staying within the realm of acceptable values and virtues. One way to prepare for these challenges is to formulate comprehensive and specific rules of acceptable conduct. In this paper we will draw attention to one serious problem with such a rule-based approach. It may inadvertently encourage what we will call loophole ethics, an attitude where every action that is not explicitly defined as wrong, will be seen as a viable option. Detailed codes of conduct leave little room for personal judgement, and instead promote a loophole mentality. We argue that loophole ethics can be avoided by operating with only a limited set of general principles, thus leaving more space for personal judgement and wisdom.

Gaining an Advantage in Roulette

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2023/04/gaining-an-advantage-in-roulette.html

You can beat the game without a computer:

On a perfect [roulette] wheel, the ball would always fall in a random way. But over time, wheels develop flaws, which turn into patterns. A wheel that’s even marginally tilted could develop what Barnett called a ‘drop zone.’ When the tilt forces the ball to climb a slope, the ball decelerates and falls from the outer rim at the same spot on almost every spin. A similar thing can happen on equipment worn from repeated use, or if a croupier’s hand lotion has left residue, or for a dizzying number of other reasons. A drop zone is the Achilles’ heel of roulette. That morsel of predictability is enough for software to overcome the random skidding and bouncing that happens after the drop.”

Automatic Cheating Detection in Human Racing

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2022/09/automatic-cheating-detection-in-human-racing.html

This is a fascinating glimpse of the future of automatic cheating detection in sports:

Maybe you heard about the truly insane false-start controversy in track and field? Devon Allen—a wide receiver for the Philadelphia Eagles—was disqualified from the 110-meter hurdles at the World Athletics Championships a few weeks ago for a false start.

Here’s the problem: You can’t see the false start. Nobody can see the false start. By sight, Allen most definitely does not leave before the gun.

But here’s the thing: World Athletics has determined that it is not possible for someone to push off the block within a tenth of a second of the gun without false starting. They have science that shows it is beyond human capabilities to react that fast. Of course there are those (I’m among them) who would tell you that’s nonsense, that’s pseudoscience, there’s no way that they can limit human capabilities like that. There is science that shows it is humanly impossible to hit a fastball. There was once science that showed human beings could not run a four-minute mile.

Besides, do you know what Devon Allen’s reaction time was? It was 0.99 seconds. One thousandth of a second too fast, according to World Athletics’ science. They’re THAT sure that .01 seconds—and EXACTLY .01 seconds—is the limit of human possibilities that they will disqualify an athlete who has trained his whole life for this moment because he reacted one thousandth of a second faster than they think possible?

We in the computer world are used to this sort of thing. “The computer is always right,” even when it’s obviously wrong. But now computers are leaving the world of keyboards and screens, and this sort of thing will become more pervasive. In sports, computer systems are used to detect when a ball is out of bounds in tennis and other games and when a pitch is a strike in baseball. I’m sure there’s more—are computers detecting first downs in football?—but I’m not enough of a sports person to know them.

Why Vaccine Cards Are So Easily Forged

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2022/03/why-vaccine-cards-are-so-easily-forged.html

My proof of COVID-19 vaccination is recorded on an easy-to-forge paper card. With little trouble, I could print a blank form, fill it out, and snap a photo. Small imperfections wouldn’t pose any problem; you can’t see whether the paper’s weight is right in a digital image. When I fly internationally, I have to show a negative COVID-19 test result. That, too, would be easy to fake. I could change the date on an old test, or put my name on someone else’s test, or even just make something up on my computer. After all, there’s no standard format for test results; airlines accept anything that looks plausible.

After a career spent in cybersecurity, this is just how my mind works: I find vulnerabilities in everything I see. When it comes to the measures intended to keep us safe from COVID-19, I don’t even have to look very hard. But I’m not alarmed. The fact that these measures are flawed is precisely why they’re going to be so helpful in getting us past the pandemic.

Back in 2003, at the height of our collective terrorism panic, I coined the term security theater to describe measures that look like they’re doing something but aren’t. We did a lot of security theater back then: ID checks to get into buildings, even though terrorists have IDs; random bag searches in subway stations, forcing terrorists to walk to the next station; airport bans on containers with more than 3.4 ounces of liquid, which can be recombined into larger bottles on the other side of security. At first glance, asking people for photos of easily forged pieces of paper or printouts of readily faked test results might look like the same sort of security theater. There’s an important difference, though, between the most effective strategies for preventing terrorism and those for preventing COVID-19 transmission.

Security measures fail in one of two ways: Either they can’t stop a bad actor from doing a bad thing, or they block an innocent person from doing an innocuous thing. Sometimes one is more important than the other. When it comes to attacks that have catastrophic effects—say, launching nuclear missiles—we want the security to stop all bad actors, even at the expense of usability. But when we’re talking about milder attacks, the balance is less obvious. Sure, banks want credit cards to be impervious to fraud, but if the security measures also regularly prevent us from using our own credit cards, we would rebel and banks would lose money. So banks often put ease of use ahead of security.

That’s how we should think about COVID-19 vaccine cards and test documentation. We’re not looking for perfection. If most everyone follows the rules and doesn’t cheat, we win. Making these systems easy to use is the priority. The alternative just isn’t worth it.

I design computer security systems for a living. Given the challenge, I could design a system of vaccine and test verification that makes cheating very hard. I could issue cards that are as unforgeable as passports, or create phone apps that are linked to highly secure centralized databases. I could build a massive surveillance apparatus and enforce the sorts of strict containment measures used in China’s zero-COVID-19 policy. But the costs—in money, in liberty, in privacy—are too high. We can get most of the benefits with some pieces of paper and broad, but not universal, compliance with the rules.

It also helps that many of the people who break the rules are so very bad at it. Every story of someone getting arrested for faking a vaccine card, or selling a fake, makes it less likely that the next person will cheat. Every traveler arrested for faking a COVID-19 test does the same thing. When a famous athlete such as Novak Djokovic gets caught lying about his past COVID-19 diagnosis when trying to enter Australia, others conclude that they shouldn’t try lying themselves.

Our goal should be to impose the best policies that we can, given the trade-offs. The small number of cheaters isn’t going to be a public-health problem. I don’t even care if they feel smug about cheating the system. The system is resilient; it can withstand some cheating.

Last month, I visited New York City, where restrictions that are now being lifted were then still in effect. Every restaurant and cocktail bar I went to verified the photo of my vaccine card that I keep on my phone, and at least pretended to compare the name on that card with the one on my photo ID. I felt a lot safer in those restaurants because of that security theater, even if a few of my fellow patrons cheated.

This essay previously appeared in the Atlantic.

Cheating on Tests

Post Syndicated from Bruce Schneier original https://www.schneier.com/blog/archives/2021/10/cheating-on-tests.html

Interesting story of test-takers in India using Bluetooth-connected flip-flops to communicate with accomplices while taking a test.

What’s interesting is how this cheating was discovered. It’s not that someone noticed the communication devices. It’s that the proctors noticed that cheating test takers were acting hinky.