All posts by Robert Graham

Programming languages infosec professionals should learn

Post Syndicated from Robert Graham original https://blog.erratasec.com/2019/04/programming-languages-infosec.html

Code is an essential skill of the infosec professional, but there are so many languages to choose from. What language should you learn? As a heavy coder, I thought I’d answer that question, or at least give some perspective.

The tl;dr is JavaScript. Whatever other language you learn, you’ll also need to learn JavaScript. It’s the language of browsers, Word macros, JSON, NodeJS server side, scripting on the command-line, and Electron apps. You’ll also need to a bit of bash and/or PowerShell scripting skills, SQL for database queries, and regex for extracting data from text files. Other languages are important as well, Python is very popular for example. Actively avoid C++ and PHP as they are obsolete.

Also tl;dr: whatever language you decide to learn, also learn how to use an IDE with visual debugging, rather than just a text editor. That probably means Visual Code from Microsoft. Also, whatever language you learn, stash your code at GitHub.

Let’s talk in general terms. Here are some types of languages.

  • Unavoidable. As mentioned above, familiarity with JavaScript, bash/Powershell, and SQL are unavoidable. If you are avoiding them, you are doing something wrong.
  • Small scripts. You need to learn at least one language for writing quick-and-dirty command-line scripts to automate tasks or process data. As a tool using animal, this is your basic tool. You are a monkey, this is the stick you use to knock down the banana. Good choices are JavaScript, Python, and Ruby. Some domain-specific languages can also work, like PHP and Lua. Those skilled in bash/PowerShell can do a surprising amount of “programming” tasks in those languages. Old timers use things like PERL or TCL. Sometimes the choice of which language to learn depends upon the vast libraries that come with the languages, especially Python and JavaScript libraries.
  • Development languages.  Those scripting languages have grown up into real programming languages, but for the most part, “software development” means languages designed for that task like C, C++, Java, C#, Rust, Go, or Swift.
  • Domain-specific languages. The language Lua is built into nmap, snortWireshark, and many games. Ruby is the language of Metasploit. Further afield, you may end up learning languages like R or Matlab. PHP is incredibly important for web development. Mobile apps may need Java, C#, Kotlin, Swift, or Objective-C.

As an experienced developer, here are my comments on the various languages, sorted in alphabetic order.


bash (and other Unix shells)

You have to learn some bash for dealing with the command-line. But it’s also a fairly completely programming language. Perusing the scripts in an average Linux distribution, especially some of the older ones, and you’ll find that bash makes up a substantial amount of what we think of as the Linux operating system. Actually, it’s called bash/Linux.

In the Unix world, there are lots of other related shells that aren’t bash, which have slightly different syntax. A good example is BusyBox which has “ash”. I mention this because my bash skills are rather poor partly because I originally learned “csh” and get my syntax variants confused.

As a hard-core developer, I end up just programming in JavaScript or even C rather than trying to create complex bash scripts. But you shouldn’t look down on complex bash scripts, because they can do great things. In particular, if you are a pentester, the shell is often the only language you’ll get when hacking into a system, sod good bash language skills are a must.

C

This is the development language I use the most, simply because I’m an old-time “systems” developer. What “systems programming” means is simply that you have manual control over memory, which gives you about 4x performance and better “scalability” (performance doesn’t degrade as much as problems get bigger). It’s the language of the operating system kernel, as well as many libraries within an operating system.

But if you don’t want manual control over memory, then you don’t want to use it. It’s lack of memory protection leading to security problems makes it almost obsolete.

C++

None of the benefits of modern languages like Rust, Java, and C#, but all of the problems of C. It’s an obsolete, legacy language to be avoided.

C#

This is Microsoft’s personal variant of Java designed to be better than Java. It’s an excellent development language, for command-line utilities, back-end services, applications on the desktop (even Linux), and mobile apps. If you are working in a Windows environment at all, it’s an excellent choice. If you can at all use C# instead of C++, do so. Also, in the Microsoft world, there is still a lot of VisualBasic. OMG avoid that like the plague that it is, burn in a fire burn burn burn, and use C# instead.

Go

Once a corporation reaches a certain size, it develops its own programming language. For Google, their most important language is Go.

Go is a fine language in general, but it’s main purpose is scalable network programs using goroutines. This is does asynchronous user-mode programming in a way that’s most convenient for the programmer. Since Google is all about scalable network services, Go is a perfect fit for them.

I do a lot of scalable network stuff in C, because I’m an oldtimer. If that’s something you’re interested in, you should probably choose Go over C.

Java

This gets a bad reputation because it was once designed for browsers, but has so many security flaws that it can’t be used in browsers. You still find in-browser apps that use Java, even in infosec products (like consoles), but it’s horrible for that. If you do this, you are bad and should feel bad.

But browsers aside, it’s a great development language for command-line utilities, back-end services, apps on desktops, and apps on phones. If you want to write an app that runs on macOS, Windows, and on a Raspberry Pi running Linux, then this is an excellent choice.

JavaScript

As mentioned above, you don’t have a choice but to learn this language. One of your basic skills is learning how to open Chrome developer tools and manipulate JavaScript on a web page.

So the question is whether you learn just enough familiarity with the language in order to hack around with it, or whether you spend the effort to really learn the language to do development or write scripts. I suggest that you should. For one thing, you’ll often encounter weird usages of JavaScript that you are unfamiliar with unless you seriously learn the language, such as JQuery style constructions that look nothing like what you might’ve originally learned the language for.

JavaScript has actually become a serious app development language with NodeJS and frameworks like Electron. If there is one language in the world that can do everything, from writing back end services (NodeJS), desktop applications (Electron), mobile apps (numerous frameworks), quick-and-dirty scripts (NodeJS again), and browser apps — it’s JavaScript. It’s the lingua franca of the world.

In addition, remember that your choice of scripting language will often be based on the underlying libraries available. For example, if writing TensorFlow machine-learning programs, you need those libraries available to the language. That’s why JavaScript is popular in the machine-learning field, because there’s so many libraries available for it.

BTW, “JSON” is also a language, or at least a data format, in its own right. So you have to learn that, too.

Lua

Lua is a language similar to JavaScript in many respects, with the big difference that arrays start with 1 instead of 0. The reason its exists is that it’s extremely easy to embed in other programs as their scripting language, is lightweight in terms of memory/CPU, and is ultra-portable almost everywhere.

Thus, you find it embedded in security tools like nmap, snort, and Wireshark. You also see it as the scripting language in popular games. Like Go, it has extremely efficient coroutines, so you see it in the nginx web server, “OpenResty”, for backend scripting of applications.

PERL

PERL was a popular scripting language in the early days of infosec (1990s), but has fallen behind the other languages in modern times. In terms of language design, it’s a somewhat better language than shell languages like bash, yet not quite as robust as real programming languages like JavaScript, Python, and Ruby.

In addition, it was the primary web scripting language for building apps on servers in the 1990s before PHP came along.

Thus, it’s a popular legacy language, but not a lot of new stuff is done in this language.


PHP

Surprisingly, PHP is a complete programming language. You can use it on the command-line to write scripts just like Python or JavaScript. You may have to learn it, because it’s still the most popular language for creating webapps, but learning it well means being able to write backend scripts in it as well.

However, for writing web apps, it’s obsolete. There are so many unavoidable security problems that you should avoid using it to create new apps. Also, scalability is still difficult. Use NodeJS, OpenResty/Lua, or Ruby instead.

PowerShell

The same comments above that apply to bash also apply to PowerShell, except that PowerShell is Windows.

Windows has two command-lines, the older CMD/BAT command-line, and the newer PowerShell. Anything complex uses PowerShell these days. For pentesting, there are lots of fairly complete tools for doing interesting things from the command-line written in the PowerShell programming language.

Thus, if Windows is in your field, and it almost certainly is, then PowerShell needs to be part of your toolkit.

Python

This has become one of the most popular languages, driven by universities which use it heavily as the teaching language for programming concepts. Anything academic, like machine learning, will have great libraries for Python.

A lot of hacker command-line tools are written in Python. Since such tools are often buggy and poorly documented, you’ll end up having to reading the code a lot to figure out what is going wrong. Learning to program in Python means being able to contribute to those tools.

I personally hate the language because of the schism between v2/v3, and having to constantly struggle with that. Every language has a problem with evolution and backwards compatibility, but this v2 vs v3 issue with Python seems particularly troublesome.

Also, Python is slow. That shouldn’t matter in this age of JITs everywhere and things like Webassembly, but somehow whenever you have an annoyingly slow tool, it’s Python that’s at fault.

Note that whenever I read reviews of programming languages, I see praise for Python’s syntax. This is nonsense. After a short while, the syntax of all programming languages becomes quirky and weird. Most languages these days are multi-paradigm, a combination of imperative, object-oriented, and functional. Most all are JITted. “Syntax” is the least reason to choose a language. Instead, it’s the choice of support/libraries (which are great for Python), or specific features like tight “systems” memory control (like Rust) or scalable coroutines (like Go). Seriously, stop praising the “elegant” and “simple” syntax of languages.

Regex

Like SQL for database queries, regular expressions aren’t a programming language as such, but still a language you need to learn. They are patterns that match data. For example, if you want to find all social security numbers in a text file, you looked for that pattern of digits and dashes. Such pattern matching is so common that it’s built into most tools, and is a feature of most scripting languages.

One thing to remember from an infosec point of view is that they are highly insecure. Hackers craft content to incorrectly match patterns, evade patterns, or cause “algorithmic complexity” attacks that cause simple regexes to exploded with excessive computation.

You have learn regexes enough to be familiar with the basics, but the syntax can get unreasonably complex, so few master the full regex syntax.

Ruby

Ruby is a great language for writing web apps that makes security easier than with PHP, though like all web apps it still has some issues.

In infosec, the major reason to learn Ruby is Metasploit.

Like Python and JavaScript, it’s also a great command-line scripting language with lots of libraries available. You’ll find it often used in this roll.

Rust

Rust is Mozilla’s replacement language for C and especially C++. It’s supports tight control over memory structures for “systems” programming, but is memory safe so doesn’t have all those vulnerabilities. One of these days I’ll stop programming in C and use Rust instead.

The problem with Rust is that it doesn’t have quite the support that other languages have, like Java or C# for apps, and isn’t as tightly focused on network apps as Go. But as a language, it’s wonderful. In a perfect world, we’d all use JavaScript for scripting tasks and Rust for the backend work. But in the real world, other languages have better support.

SQL

SQL, “structure query language”, isn’t a programming language as such, but it’s still a language of some sort. It’s something that you unavoidably have to learn.

One of the reasons to learn a programming language is to process data. You can do that within a programming language, but an alternative is to shove the data into a database then write queries off that database. I have a server at home just for that purpose, with large disks and multicore processors. Instead of storing things as files, and writing scripts to process those files, I stick it in tables, and write SQL queries off those tables.


Swift

Back in the day, when computers were new, before C++ become the “object oriented” language standard, there was a competing object-oriented version of C known as “Objective C”. Because, as everyone knew, object-oriented was the future, NeXT adopted this as their application programming language. Apple bought NeXT, and thus it became Apple’s programming language.

But Objective C lost the object-oriented war to C++ and became an orphaned language. Also, it was really stupid, essentially two separate language syntaxes fighting for control of your code.

Therefore, a few years ago, Apple created a replacement called Swift, which is largely based on a variant of Rust. Like Rust, it’s an excellent “systems” programming language that has more manual control over memory allocation, but without all the buffer-overflows and memory leaks you see in C.

It’s an excellent language, and great when programming in an Apple environment. However, when choosing a “language” that’s not particularly Apple focused, just choose Rust instead.

Conclusion

As I mentioned above, familiarity with JavaScript, bash/PowerShell, and SQL is unavoidable. So start with those. JavaScript in particular has become a lingua franca, able to do, and do well, almost anything you need a language to do these days, so it’s worth getting into the finder details JavaScript.

However, there’s no One Language to Rule them all. There’s good reasons to learn most languages in this list. For some tasks, the support for a certain language is so good it’s just best to learn that language to solve that task. With the academic focus on Python, you’ll find well-written libraries that solve important tasks for you. If you want to work with a language that other people know, that you can ask questions about, then Python is a great choice.

The exceptions to this are C++ and PHP. They are so obsolete that you should avoid learning them, unless you plan on dealing with legacy.

Was it a Chinese spy or confused tourist?

Post Syndicated from Robert Graham original https://blog.erratasec.com/2019/04/was-it-chinese-spy-or-confused-tourist.html

Politico has an article from a former spy analyzing whether the “spy” they caught at Mar-a-lago (Trump’s Florida vacation spot) was actually a “spy”. I thought I’d add to it from a technical perspective about her malware, USB drives, phones, cash, and so on.

The part that has gotten the most press is that she had a USB drive with evil malware. We’ve belittled the Secret Service agents who infected themselves, and we’ve used this as the most important reason to suspect she was a spy.

But it’s nonsense.

It could be something significant, but we can’t know that based on the details that have been reported. What the Secret Service reported was that it “started installing software”. That’s a symptom of a USB device installing drivers, not malware. Common USB devices, such as WiFi adapters, Bluetooth adapters, microSD readers, and 2FA keys look identical to flash drives, and when inserted into a computer, cause Windows to install drivers.

Visual “installing files” is not a symptom of malware. When malware does its job right, there are no symptoms. It installs invisibly in the background. Thats the entire point of malware, that you don’t know it’s there. It’s not to say there would be no visible evidence. A popular way of hacking desktops with USB drives is by emulating a keyboard/mouse that quickly types commands, which will cause some visual artifacts on the screen. It’s just that “installing files” does not lend itself to malware as being the most likely explanation.

That it was “malware” instead of something normal is just the standard trope that anything unexplained is proof of hackers/viruses. We have no evidence it was actually malware, and the evidence we do have suggests something other than malware.

Lots of travelers carry wads of cash. I carry ten $100 bills with me, hidden in my luggage, for emergencies. I’ve been caught before when the credit card company fraud detection triggers in a foreign country leaving me with nothing. It’s very distressing, hence cash.

The Politico story mentioned the “spy” also has a U.S. bank account, and thus cash wasn’t needed. Well, I carry that cash, too, for domestic travel. It’s just not for international travel. In any case, the U.S. may have been just one stop on a multi-country itinerary. I’ve taken several “round the world” trips where I’ve just flown one direction, such as east, before getting back home. $8k is in the range of cash that such travelers carry.

The same is true of phones and SIMs. Different countries have different frequencies and technologies. In the past, I’ve traveled with as many as three phones (US, Japan, Europe). It’s gotten better with modern 4G phones, where my iPhone Xs should work everywhere. (Though it’s likely going to diverge again with 5G, as the U.S. goes on a different path from the rest of the world.)

The same is true with SIMs. In the past, you pretty much needed a different SIM for each country. Arrival in the airport meant going to the kiosk to get a SIM for $10. At the end of a long itinerary, I’d arrive home with several SIMs. These days, however, with so many “MVNOs”, such as Google Fi, this is radically less necessary. However, the fact that the latest high-end phones all support dual-SIMs proves it’s still an issue.

Thus, the evidence so far is that of a normal traveler. If these SIMs/phones are indeed because of spying, we would need additional evidence. A quick analysis of the accounts associated with the SIMs and the of the contents of the phones should tells us if she’s a traveler or spy.

Normal travelers may be concerned about hidden cameras. There’s this story from about Korean hotels filming guests, and this other one about AirBNB problems.

Again we are missing salient details. In the old days, such detectors were analog devices, because secret spy cameras were analog. These days, new equipment is almost always WiFi based. You’d detect more running software on your laptop looking for MAC addresses of camera makers than you would with those older analog devices. Or, there are tricks that look for glinting light off lenses.

Thus, the “hidden camera detector” sounds to me more like a paranoid traveler than a spy.

One of the frequently discussed things is her English language skills. As the Politico story above, her “constant lies” can be explained by difficulties speaking English. In other stories, the agents claim that she both understood and spoke English well.

Both can be true. The ability to speak foreign languages isn’t binary, on or off. I speak French and German in this middle skill level. In some cases, I can hold a conversation with apparent fluency, while in other cases I’m at a complete loss.

One issue is how understanding different speakers varies wildly. I can understand French news broadcasts with little difficulty, with nearly 100% comprehension. On the other hand, watching non-news French TV, like sitcoms, my comprehension goes to near 0%. The same is true of individuals, I many understand nearly everything one person says while understanding nearly nothing another person says.

99% comprehension is still far from 100%. I frequently understand large sections except for one essential key word. Like listening to French news, I understand everything the news story about some event that happened in that country, but I missed the country’s name at the start. Yes, I know there were storms, mudslides, floods, 100,000 without power, 300 deaths — I just haven’t a clue where in the world that happened.

Diplomats around the world recognize this. They often speak English well, use English daily, and yet in formal functions they still use translators, because there’s always a little bit they won’t understand.

Thus, we know any claim by the Secret Service that her language skills were adequate are false.

So in conclusion, we don’t see evidence pointing to a spy. Instead, we see a careful curation of evidence by the secret service and reporters to push the spying story. We haven’t seen any reporter question what other USB devices can cause software to load other than malware. She may be a spy, of course, but so far, there’s no evidence of anything other than a confused/crazy tourist.

Assange indicted for breaking a password

Post Syndicated from Robert Graham original https://blog.erratasec.com/2019/04/assange-indicted-for-breaking-password.html

In today’s news, after 9 years holed up in the Ecuadorian embassy, Julian Assange has finally been arrested. The US DoJ accuses Assange for trying to break a password. I thought I’d write up a technical explainer what this means.

According to the US DoJ’s press release:

Julian P. Assange, 47, the founder of WikiLeaks, was arrested today in the United Kingdom pursuant to the U.S./UK Extradition Treaty, in connection with a federal charge of conspiracy to commit computer intrusion for agreeing to break a password to a classified U.S. government computer.

The full indictment is here.

It seems the indictment is based on already public information that came out during Manning’s trial, namely this log of chats between Assange and Manning, specifically this section where Assange appears to agree to break a password:

What this says is that Manning hacked a DoD computer and found the hash “80c11049faebf441d524fb3c4cd5351c” and asked Assange to crack it. Assange appears to agree.

So what is a “hash”, what can Assange do with it, and how did Manning grab it?

Computers store passwords in an encrypted (sic) form called a “one way hash”. Since it’s “one way”, it can never be decrypted. However, each time you log into a computer, it again performs the one way hash on what you typed in, and compares it with the stored version to see if they match. Thus, a computer can verify you’ve entered the right password, without knowing the password itself, or storing it in a form hackers can easily grab. Hackers can only steal the encrypted form, the hash.

When they get the hash, while it can’t be decrypted, hackers can keep guessing passwords, performing the one way algorithm on them, and see if they match. With an average desktop computer, they can test a billion guesses per second. This may seem like a lot, but if you’ve chosen a sufficiently long and complex password (more than 12 characters with letters, numbers, and punctuation), then hackers can’t guess them.

It’s unclear what format this password is in, whether “NT” or “NTLM”. Using my notebook computer, I could attempt to crack the NT format using the hashcat password crack with the following command:

hashcat -m 3000 -a 3 80c11049faebf441d524fb3c4cd5351c ?a?a?a?a?a?a?a

As this image shows, it’ll take about 22 hours on my laptop to crack this. However, this doesn’t succeed, so it seems that this isn’t in the NT format. Unlike other password formats, the “NT” format can only be 7 characters in length, so we can completely crack it.

Instead of brute-force trying all possible combinations of characters each time we have a new password, we could do the huge calculation just once and save all the “password -> hash” combinations to a disk drive. Then, each time we get a new hash from hacking a computer, we can just do a simple lookup. However, this won’t work in practice, because the number of combinations is just too large — even if we used all the disk drives in the world to store the results, it still wouldn’t be enough.

But there’s a neat trick called “Rainbow Tables” that does a little bit of both, using both storage and computation. If cracking a password would be of 64 bits of difficulty, you can instead use 32 bits of difficulty for storage (storing 4 billion data points) and do 32 bits worth of computation (doing 4 billion password hashes). In other words, while doing 64 bits of difficulty is prohibitively difficult, 32 bits of both storage and computation means it’ll take up a few gigabytes of space and require only a few seconds of computation — an easy problem to solve.

That’s what Assange promises, that they have the Rainbow Tables and expertise needed to crack the password.

However, even then, the Rainbow Tables aren’t complete. While the “NT” algorithm has a limit of 7 characters, the “NTLM” has no real limit. Building the tables in the first place takes a lot of work. As far as I know, we don’t have NTLM Rainbow Tables for passwords larger than 9 complex characters (upper, lower, digits, punctuation, etc.).

I don’t know the password requirements that were in effect back then 2010, but there’s a good chance it was on the order of 12 characters including digits and punctuation. Therefore, Rainbow Cracking wouldn’t have been possible.

If we can’t brute-force all combinations of a 12 character password, or use Rainbow Tables, how can we crack it? The answer would be “dictionary attacks”. Over the years, we’ve acquired real-world examples of over a billion passwords people have used in real accounts. We can simply try all those, regardless of length. We can also “mutate” this dictionary, such as adding numbers on the end. This requires testing trillions of combinations, but with hardware that can try a billion combinations per second, it’s not too onerous.

But there’s still a limit to how effective we can be at password cracking. As I explain in other posts, the problem is exponential. Each additional character increases the difficult by around 100 times. In other words, if you can brute-force all combinations of a password of a certain length in a week, then adding one character to the length means you’ll take now 100 weeks, or two years. That’s why even nation state spies, like the NSA, with billions of dollars of hardware, may not be able to crack this password.

LinkedIn passwords, how long it takes a laptop or nation state to crack

Now let’s tackle the question of how Manning got the hash in the first place. It appears the issue is that Manning wanted to logon as a different user, hiding her tracks. She therefore wanted to grab the other person’s password hash, crack the password, then use it to logon, with all her nefarious activities now associated with the wrong user.

She can’t simply access the other user account. That’s what operating systems do, prevent you from accessing other parts of the disk that don’t belong to you.

To get around this, she booted the computer with a different operating system from a CD drive, with some sort of Linux distro. From that operating system, she had full access to the drive. As the chatlog reveals, she did the standard thing that all hackers do, copy over the SAM file, then dump the hashes from it. Here is an explanation from 2010 that roughly describes exactly what she did.

The term “Linux” was trending today on Twitter by people upset by the way the indictment seemed to disparage it as some sort of evil cybercrime tool, but I don’t read it that way. The evil cybercrime act the indictment refers to use is booting another operating system from a CD. It no more disparages Linux than it disparages CDs. It’s the booting an alternate operating system and stealing the SAM file that demonstrates criminality, not CDs or Linux.

Note that stealing another account’s password apparently wasn’t about being able to steal more documents. This can become an important factor later on when appealing the case.

The documents weren’t on the computer, but on the network. Thus, while booting Linux from a CD would allow full access to all the documents on the local desktop computer, it still wouldn’t allow access to the server.

Apparently, it was just another analyst’s account Manning was trying to hijack, who had no more permissions on the network than she did. Thus, she wouldn’t have been accessing any files she wasn’t already authorized to access.

Therefore, as CFAA/4thA expert Orin Kerr tweets, there may not have been a CFAA violation:

I don’t know his argument, he hasn’t posted his promised blogpost yet, but I assume it’s something similar to the Nosal case: just because somebody later does something unauthorized with the documents, like hand them to a third party, if they were authorized to access the documents in the first place, then it’s (possibly/probably) not a CFAA violation of “unauthorized” access.
Thus, from this interpretation, while Assange appears here to go beyond journalistic practices and help hack, it still may fall short of a CFAA violation.
Many are worried that Assange is being prosecuted for journalism. I’m not sure the evidence shows one way or the other. The indictment isn’t for acts of journalism, such as publishing the stolen documents, but for acts of hacking, which every journalistic organization calls unethical. Ethically, while you can publish the fruits of a crime, you can’t participate in the crime. If a journalist breaks into your home to steal documents, it’s still a crime — they can’t hide behind the shield “but I’m a journalist”.
On the other hand, the hacking Assange is being accused of here seems to be pretty weaksauce. It seems like a thin justification to prosecute him. In other words, it’s like how they prosecuted Al Capone for tax evasion rather than what they really wanted to prosecute him for. I was taught that was a clever use of the system in order to go after Capone, but here it seems like a corruption of the system in order to go after Assange.
One question I have for legal experts is whether this prosecution is politically motivated. We do a lot in this country to shield prosecutors from political influence from the top, which means prosecutors shouldn’t be influenced by what Obama or Trump want. On the other hand, they are human, and Assange is a famous target, and I can imagine a prosecutor wanting to make a name for themselves by going after such a famous target. Prosecutors have a lot of discretion, which means they may not indict some 18 year old nobody for helping crack a password, even though it may technically be illegal, but it also means they also have the discretion to go after such a big target as Assange for the same crime.
Note that booting from a separate CD meant Manning could easily hack the desktop computer, installing viruses/malware, changing account settings, and so on. Presumably, though, this would leave traces, so she didn’t want to go that far. Merely booting an alternate system to steal passwords, then use those passwords, wouldn’t leave any significant traces.

Conclusion

Many legal experts claim this is just the first indictment, that will probably see a superseding indictment later on with more changes. For the moment, however, this blogpsot discusses the technical details behind what’s currently being charged. As more charges of hacking appear, I’ll try to write them up here as well.
 

Some notes on the Raspberry Pi

Post Syndicated from Robert Graham original https://blog.erratasec.com/2019/03/some-notes-on-raspberry-pi.html

I keep seeing this article in my timeline today about the Raspberry Pi. I thought I’d write up some notes about it.

The Raspberry Pi costs $35 for the board, but to achieve a fully functional system, you’ll need to add a power supply, storage, and heatsink, which ends up costing around $70 for the full system. At that price range, there are lots of alternatives. For example, you can get a fully function $99 Windows x86 PC, that’s just as small and consumes less electrical power.

There are a ton of Raspberry Pi competitors, often cheaper with better hardware, such as a Odroid-C2, Rock64, Nano Pi, Orange Pi, and so on. There are also a bunch of “Android TV boxes” running roughly the same hardware for cheaper prices, that you can wipe and reinstall Linux on. You can also acquire Android phones for $40.

However, while “better” technically, the alternatives all suffer from the fact that the Raspberry Pi is better supported — vastly better supported. The ecosystem of ARM products focuses on getting Android to work, and does poorly at getting generic Linux working. The Raspberry Pi has the worst, most out-of-date hardware, of any of its competitors, but I’m not sure I can wholly recommend any competitor, as they simply don’t have the level of support the Raspberry Pi does.

The defining feature of the Raspberry Pi isn’t that it’s a small/cheap computer, but that it’s a computer with a bunch of GPIO pins. When you look at the board, it doesn’t just have the recognizable HDMI, Ethernet, and USB connectors, but also has 40 raw pins strung out across the top of the board. There’s also a couple extra connectors for cameras.

The concept wasn’t simply that of a generic computer, but a maker device, for robot servos, temperature and weather measurements, cameras for a telescope, controlling christmas light displays, and so on.

I think this is underemphasized in the above story. The reason it finds use in the factories is because they have the same sorts of needs for controlling things that maker kids do. A lot of industrial needs can be satisfied by a teenager buying $50 of hardware off Adafruit and writing a few Python scripts.

On the other hand, support for industrial uses is nearly non-existant. The reason commercial products cost $1000 is because somebody will answer your phone call, unlike the teenager whose currently out at the movies with their friends. However, with more and more people having experience with the Raspberry Pi, presumably you’ll be able to hire generic consultants soon that can maintain these juryrigged solutions.

One thing that’s interesting is how much that 40 pin GPIO interface has become a standard. There are a ton of competing devices that support that same standard, even with Intel x86 Windows computers. The Raspberry Pi foundation has three boards that support this standard, the RPi Zero, the Model A, and the Model B. Competitors have both smaller, more efficient boards to choose from, as well as larger, more powerful boards. But as I said, nothing is as well supported as Raspberry Pi boards themselves.

Raspberry Pi class machines are overpowered for a lot of maker projects. There are competing systems, like the Arduino, ESP32, and Micro:Bit. As a hacker, I love the ESP32 class devices. They come with a full WiFi stack and can be placed anywhere.

If you are buying a Raspberry Pi, I recommend Adafruit. Not only do they have the devices cheap ($35), they’ll have a lot of support for maker hardware that you may want to add to the device.

After buying the board, you have to choose the accessories to get it working.

Your first choice will be a power supply. You’ll be tempted to use the USB chargers and cables you have lying around the house, and it’ll appear to work at first, but will cause CPU throttling problems and file corruption. You need to get either the $8 “official” power supply, or one of those fast charging devices, like those from Anker. Remember that it’s not just a matter of the power supply providing enough current/amps, but also cables with 20 AWG wires that can handle the current.

Your next choice will be the flash drive for booting the computer. One choice is micro SD cards. You should choose cards with the “A1” rating, which are faster at random file access. Most other microSD cards are optimized for large sequential transfers, and are painfully slow at random accesses. If you write a lot of data to the device, you may need to get a card rated for “endurance” instead — micro SD cards wear out quickly.

Or, you may consider a real SSD connected to the USB port. You can get a $20 120-gig SSD and a $8 USB-to-SATA adapter. This will perform much faster, and not have the data corruption issues that micro SD cards have. You need an independent power supply for the drive, as it can’t be powered wholly from the USB port.

Your next decision will be a heatsink. The Raspberry Pi generates a lot of heat at full load. People assume ARM is efficient, but it’s not, and the Broadcom ARM CPU used by the RPi is very bad. Unless you have a heatsink, instead of running at 1.4-GHz, it’ll spend most of it’s time throttled back to 600-MHz. Because of their size, your choice of heatsink and fan depends upon your choice of case. There are some nice aluminum cases that act as a heatsink. You can also get combo kits on Amazon.com for $15 that include the case, heatsink, and fan together.

If looking at a competing device (e.g. Odroid-C2, Rock64), get one that supports eMMC. It’s much faster and more reliable than micro SD cards. For home server applications, its worth getting a lesser supported platform in order to get eMMC. It makes a huge difference. I stopped using Raspberry Pi’s for home server applications and went with Odroid-C2 machines instead, mostly because of the eMMC, but also because they have more RAM and faster Ethernet. I may switch to the Rock64 device in the future because of its support for USB 3.0. I have one on-order, but it’s taking (so far) more than a month to arrive.

As for the ARM ecosystem, there seems to be a lot of misunderstanding about “power efficiency”. People keep claiming they are more efficient. They aren’t. They consume less power by being slower. Scaled to the same performance, ARM CPUs use the same amount of power as Intel CPUs. Now that ARM has more powerful CPUs close to Intel in speed, and Intel now has their low speed “Atom” processors, we see that indeed they have roughly the same efficiency. The Raspberry Pi’s Broadcom CPU is extremely inefficient. It uses the decade old 40nm manufacturing process, which means it consumes a lot of power. Intel’s latest Atom processors built on 22nm or 14nm technology consume a lot less power. There are things that impact efficiency, but the least important of which is whether it’s ARM or Intel x86, or RISC vs. CISC.

For hackers, there’s a lot you can do with a Raspberry Pi (or competitor). We are surrounded by things that we can hack. For example, you can use it to hack the CEC feature of HDMI to control your TV. You can attach a cheap RTL-SDR device and monitor radio frequencies. You can connect it to the CAN bus of your car. You can connect it to your ZigBee devices in your home and control your lights. If there’s a wire or radio wave around you, it’s something you can start hacking with the RPi.

Conclusion

A feel the above article does the subject a disservice. It’s less “industrial IoT” and more “crossover between maker culture and industry”.

Every geek should get a Raspberry Pi and play with it, even if it’s only as simple as a Owncloud/Nextcloud backup server sitting in a closet. Don’t skimp on the power supply, as people who do get frustrated, you need a charger rated for at least 2.4 amps and a charging cable with thicker 20 AWG wires. If going the micro SD route, choose “A1” or “endurance” rated cards. Consider going a USB SSD route instead.

A quick lesson in confirmation bias

Post Syndicated from Robert Graham original https://blog.erratasec.com/2019/03/a-quick-lesson-in-confirmation-bias.html

In my experience, hacking investigations are driven by ignorance and confirmation bias. We regularly see things we cannot explain. We respond by coming up with a story where our pet theory explains it. Since there is no alternative explanation, this then becomes evidence of our theory, where this otherwise inexplicable thing becomes proof.

For example, take that “Trump-AlfaBank” theory. One of the oddities noted by researchers is lookups for “trump-email.com.moscow.alfaintra.net“. One of the conspiracy theorists explains has proof of human error, somebody “fat fingered” the wrong name when typing it in, thus proving humans were involved in trying to communicate between the two entities, as opposed to simple automated systems.

But that’s because this “expert” doesn’t know how DNS works. Your computer is configured to automatically put local suffices on the end of names, so that you only have to lookup “2ndfloorprinter” instead of a full name like “2ndfloorprinter.engineering.example.com”.

When looking up a DNS name, your computer may try to lookup the name both with and without the suffix. Thus, sometimes your computer looks up “www.google.com.engineering.exmaple.com” when it wants simply “www.google.com”.

Apparently, Alfabank configures its Moscow computers to have a suffix “moscow.alfaintra.net”. That means any DNS name that gets resolved will sometimes get this appended, so we’ll sometimes see “www.google.com.moscow.alfaintra.net”.

Since we already know there were lookups from that organization for “trump-email.com”, the fact that we also see “trump-email.com.moscow.alfaintra.net” tells us nothing new.

In other words, the conspiracy theorists didn’t understand it, so came up with their own explanation, and this confirmed their biases. In fact, there is a simpler explanation that neither confirms nor refutes anything.

The reason for the DNS lookups for “trump-email.com” are still unexplained. Maybe they are because of something nefarious. The Trump organizations had all sorts of questionable relationships with Russian banks, so such a relationship wouldn’t be surprising. But here’s the thing: just because we can’t come up with a simpler explanation doesn’t make them proof of a Trump-Alfabank conspiracy. Until we know why those lookups where generated, they are an “unknown” and not “evidence”.

The reason I write this post is because of this story about a student expelled due to “grade hacking”. It sounds like this sort of situation, where the IT department saw anomalies it couldn’t explain, so the anomalies became proof of the theory they’d created to explain them.

Unexplained phenomena are unexplained. They are not evidence confirming your theory that explains them.

A basic question about TCP

Post Syndicated from Robert Graham original https://blog.erratasec.com/2019/02/a-basic-question-about-tcp.html

So on Twitter, somebody asked this question:

I have a very basic computer networking question: when sending a TCP packet, is the packet ACK’ed at every node in the route between the sender and the recipient, or just by the final recipient?

This isn’t just a basic question, it is the basic question, the defining aspect of TCP/IP that makes the Internet different from the telephone network that predated it.

Remember that the telephone network was already a cyberspace before the Internet came around. It allowed anybody to create a connection to anybody else. Most circuits/connections were 56-kilobits-per-secondl using the “T” system, these could be aggregated into faster circuits/connections. The “T1” line consisting of 1.544-mbps was an important standard back in the day.

In the phone system, when a connection is established, resources must be allocated in every switch along the path between the source and destination. When the phone system is overloaded, such as when you call loved ones when there’s been an earthquake/tornado in their area, you’ll sometimes get a message “No circuits are available”. Due to congestion, it can’t reserve the necessary resources in one of the switches along the route, so the call can’t be established.

“Congestion” is important. Keep that in mind. We’ll get to it a bit further down.

The idea that each router needs to ACK a TCP packet means that the router needs to know about the TCP connection, that it needs to reserve resources to it.

This was actually the original design of the the OSI Network Layer.

Let’s rewind a bit and discuss “OSI”. Back in the 1970s, the major computer companies of the time all had their own proprietary network stacks. IBM computers couldn’t talk to DEC computers, and neither could talk to Xerox computers. They all worked differently. The need for a standard protocol stack was obvious.

To do this, the “Open Systems Interconnect” or “OSI” group was established under the auspices of the ISO, the international standards organization.

The first thing the OSI did was create a model for how protocol stacks would work. That’s because different parts of the stack need to be independent from each other.

For example, consider the local/physical link between two nodes, such as between your computer and the local router, or your router to the next router. You use Ethernet or WiFi to talk to your router. You may use 802.11n WiFi in the 2.4GHz band, or 802.11ac in the 5GHz band. However you do this, it doesn’t matter as far as the TCP/IP packets are concerned. This is just between you and your router, and all the information is stripped out of the packets before they are forwarded to across the Internet.

Likewise, your ISP may use cable modems (DOCSIS) to connect your router to their routers, or they may use xDSL. This information is likewise is stripped off before packets go further into the Internet. When your packets reach the other end, like at Google’s servers, they contain no traces of this.

There are 7 layers to the OSI model. The one we are most interested in is layer 3, the “Network Layer”. This is the layer at which IPv4 and IPv6 operate. TCP will be layer 4, the “Transport Layer”.

The original idea for the network layer was that it would be connection oriented, modeled after the phone system. The phone system was already offering such a service, called X.25, which the OSI model was built around. X.25 was important in the pre-Internet era for creating long-distance computer connections, allowing cheaper connections than renting a full T1 circuit from the phone company. Normal telephone circuits are designed for a continuous flow of data, whereas computer communication is bursty. X.25 was especially popular for terminals, because it only needed to send packets from the terminal when users were typing.

Layer 3 also included the possibility of a connectionless network protocol, like IPv4 and IPv6, but it was assumed that connection oriented protocols would be more popular, because that’s how the phone system worked, which meant that was just how things were done.

The designers of the early Internet, like Bob Kahn (pbuh) and Vint Cerf (pbuh), debated this. They looked at Cyclades, a French network, which had a philosophical point of view called the end-to-end principle, by which I mean the End-To-End Principle. This principle distinguishes the Internet from the older phone system. The Internet is an independent network from the phone system, rather than an extension of the phone system like X.25.

The phone system was defined as a smart network with dumb terminals. Your home phone was a simple circuit with a few resisters, speaker, and microphone. It had no intelligence. All the intelligence was within the network. Unix was developed in the 1970s to run on phone switches, because it was the switches inside the network that were intelligent, not the terminals on the end. That you are now using Unix in your iPhone is the opposite of what they intended.

Even mainframe computing was designed this way. Terminals were dumb devices with just enough power to display text. All the smart processing of databases happened in huge rooms containing the mainframe.

The end-to-end principle changes this. It instead puts all the intelligence on the ends of the network, with smart terminals and smart phones. It dumbs down the switches/routers to their minimum functionality, which is to route packets individually with no knowledge about what connection they might be a part of. A router receives a packet on a link, looks at it’s destination IP address, and forwards it out the appropriate link in necessary direction. Whether it eventually reaches its destination is of no concern to the router.

In the view of the telephone network, new applications meant upgrading the telephone switches, and providing the user a dumb terminal. Movies of the time, like 2001: A Space Odyssey and Blade Runner would show video phone calls, offered by AT&T, with the Bell logo. That’s because such applications where always something that the phone company would provide in the future.

With the end-to-end principle the phone company simply routes the packets, and the apps are something the user chooses separately. You make video phones calls today, but you use FaceTime, Skype, WhatsApp, Signal, and so on. My wireless carrier is AT&T, but it’s absurd thinking I would ever make a video phone call using an app provided to me by AT&T, as I was shown in the sci-fi movies of my youth.

So now let’s talk about congestion or other errors that cause packets to be lost.

It seems obvious that the best way to deal with lost packets is at the point where it happens, to retransmit packets locally instead of all the way from the remote ends of the network.

This turns out not to be the case. Consider streaming video from Netflix when congestion happens. When that happens, it wants to change the encoding of the video to a lower bit rate. You see this when watching Netflix during prime time (6pm to 11pm), where videos are of poorer quality than during other times of the day. It’s streaming them at a lower bit rate due to their system being overloaded.

If routers try to handle dropped packets locally, then they give limited feedback about the congestion. It would require some sort of complex signaling back to the ends of the network informing them about congestion in the middle.

With the end-to-end principle, when congestion happens, when a router can’t forward a packet, it silently drops it, performing no other processing or signaling about the event. It’s up to the ends to notice this. The sender doesn’t receive an ACK, and after a certain period of time, resends the data. This in turn allows the app to discover congestion is happening, and to change its behavior accordingly, such as lowering the bitrate at which its sending video.

Consider what happens with a large file download, such as your latest iOS update, which can be a gigabyte in size. How fast can the download happen?

Well, with TCP, it uses what’s known as the slow start algorithm. It starts downloading the file slowly, but keeps increasing the speed of transmission until a packet is dropped, at which point it backs off.

You can see this behavior when visiting a website like speedtest.net. You see it slowly increase the speed until it reaches its maximum level. This isn’t a property of the SpeedTest app, but a property of how TCP works.

TCP also tracks the round trip time (RTT), the time it takes for a packet to be acknowledged. If the two ends are close, RTT should be small, and the amount of time waiting to resend a lost packet should be shorter, which means it can respond to congestion faster, and more carefully tune the proper transmit rate.

This is why buffer bloat is a problem. When a router gets overloaded, instead of dropping a packet immediately, it can instead decide to buffer the packet for a while. If the congestion is transitory, then it’ll be able to send the packet a tiny bit later. Only if the congestion endures, and the buffer fills up, will it start dropping packets.

This sounds like a good idea, to improve reliability, but it messes up TCP’s end-to-end behavior. It can no longer reliably reliably measure RTT, and it can no longer detect congestion quickly and backoff on how fast it’s transmitting, causing congestion problems to be worse. It means that buffering in the router doesn’t work, because when congestion happens, instead of backing off quickly, TCP stacks on the ends will continue to transmit at the wrong speed, filling the buffer. In many situations, buffering increases dropped packets instead of decreasing them.

Thus, the idea of trying to fix congestion in routers by adding buffers is a bad idea.

Routers will still do a little bit of buffering. Even on lightly loaded networks, two packets will arrive at precisely the same time, so one needs to be sent before the other. It’s insane to drop the other at this point when there’s plenty of bandwidth available, so routers will buffer a few packets. The solution is to reduce buffer to the minimum, but not below the minimum.

Consider  Google’s HTTP/3 protocol, how they are moving to UDP instead of TCP. There are various reasons for doing this, which I won’t go into here. Notice how if routers insisted on being involved in the transport layer, of retransmitting TCP packets locally, how the HTTP/3 upgrade on the ends within the browser wouldn’t work. HTTP/3 takes into consideration information that is encrypted within the the protocol, something routers don’t have access to.

This end-to-end decision was made back in the early 1970s as the Internet has wildly evolved. Our experience 45 years later is that this decision was a good one.

Now let’s discuss IPv6 and NAT.

As you know, IPv4 uses 32-bit network addresses, which have only 4-billion combinations, allowing only 4-billion devices on the Internet. However, there are more than 10-billion devices on the network currently, more than 20-billion by some estimates.

The way this is handled is network address translation or NAT. Your home router has one public IPv4 address, like 50.73.69.230. Then, internal to your home or business, you get a local private IPv4 address, likely in the range 10.x.x.x or 192.168.x.x. When you transmit packets, your local router changes the source address from the private one to the public one, and on incoming packets, changes the public address back to your private address.

It does this by tracking the TCP connection, tracking the source and destination TCP port numbers. It’s really a TCP/IP translator rather than just an IP translator.

This violates the end-to-end principle, but only a little bit. While the NAT is translating addresses, it’s still not doing things like acknowledging TCP packets. That’s still the job of the ends.

As we all know, IPv6 was created in order to expand the size of addresses, from 32-bits to 128-bits, making a gazillion addresses available. It’s often described in terms of the Internet running out addresses needing more, but that’s not the case. With NAT, the IPv4 Internet will never run out of addresses.

Instead, what IPv6 does is preserve the end-to-end principle, by keeping routers dumb.

I mention this because I find discussions of IPv6 a bit tedious. The standard litany is that we need IPv6 so that we can have more than 4-billion devices on the Internet, and people keep repeating this despite there being more than 10-billion devices on the IPv4 Internet.

Conclusion

As I stated above, this isn’t just a basic question, but the basic question. It’s at the center of a whole web of interlocking decisions that define the nature of cyberspace itself.

From the time the phone system was created in the 1800s up until the 2007 release of the iPhone, phone companies wanted to control the applications that users ran on their network. The OSI Model that you learn as the basis of networking isn’t what you think it is: it was designed with the AT&T phone network and IBM mainframes being in control over your applications.

The creation of TCP/IP and the Internet changed this, putting all the power in the hands of the ends of the network. The version of the OSI Model you end up learning is a retconned model, with all the original important stuff stripped out, and only the bits that apply to TCP/IP left remaining.

Of course, now we live in a world monopolized by the Google, the Amazon, and the Facebook, so we live in some sort of dystopic future. But it’s not a future dominated by AT&T.

Haywood Floyd phones home in 2001: A Space Oddesy

How Bezo’s dick pics might’ve been exposed

Post Syndicated from Robert Graham original https://blog.erratasec.com/2019/02/how-bezos-dick-pics-mightve-been-exposed.html

In the news, the National Enquirer has extorted Amazon CEO Jeff Bezos by threatening to publish the sext-messages/dick-pics he sent to his mistress. How did the National Enquirer get them? There are rumors that maybe Trump’s government agents or the “deep state” were involved in this sordid mess. The more likely explanation is that it was a simple hack. Teenage hackers regularly do such hacks — they aren’t hard.

This post is a description of how such hacks might’ve been done.

To start with, from which end were they stolen? As a billionaire, I’m guessing Bezos himself has pretty good security, so I’m going to assume it was the recipient, his girlfriend, who was hacked.

The hack starts by finding the email address she uses. People use the same email address for both public and private purposes. There are lots of “people finder” services on the Internet that you can use to track this information down. These services are partly scams, using “dark patterns” to get you to spend tons of money on them without realizing it, so be careful.

Using one of these sites, I quickly found a couple of a email accounts she’s used, one at HotMail, another at GMail. I’ve blocked out her address. I want to describe how easy the process is, I’m not trying to doxx her.

Next, I enter those email addresses into the website http://haveibeenpwned.com to see if hackers have ever stolen her account password. When hackers break into websites, they steal the account passwords, and then exchange them on the dark web with other hackers. The above website tracks this, helping you discover if one of your accounts has been so compromised. You should take this opportunity to enter your email address in this site to see if it’s been so “pwned”.

I find that her email addresses have been included in that recent dump of 770 million accounts called “Collection#1”.

The http://haveibeenpwned.com won’t disclose the passwords, only the fact they’ve been pwned. However, I have a copy of that huge Collection#1 dump, so I can search it myself to get her password. As this output shows, I get a few hits, all with the same password.

At this point, I have a password, but not necessarily the password to access any useful accounts. For all I know, this was the password she chose for http://UnderwaterBasketWeaving.com, which wouldn’t be terribly useful to me.

But most people choose the same password across all their websites. Therefore, chances are good this password is the one she uses for email, for her Apple iPhone, for Facebook, and for Twitter.

I can’t know this, because even testing this password on those sites (though without accessing the information in her accounts) may be violation of the law. I say “may be” because nobody knows, and I’m not willing to be the first test case to go to trial and find out.

But the National Enquirer is (evidently) a bunch of sleazeballs, so I’m assuming they grabbed a copy of Collection#1 and researched all the accounts of people that interest them to find out precisely this sort of information, and extort them with it. It’s real easy, as this post demonstrates. Or, if they didn’t do it themselves, they are wildly known in the United States as one of the few media outlets who would pay for such information if an independent hacker were to obtain it.

So which accounts did the sexting images come from? Were they SMS/iMessage messages? Were they sent via Twitter/Facebook private messages, like with the Anthony Wiener scandal? Were they sent via email? Or was some encrypted app like Signal used?

If it’s Twitter or Facebook, then knowing the email address and passwords are enough. A hacker knowing this information can simply log in and view the old messages without the owner of the account knowing.

They do offer something called “two-factor authentication”, such as sending a numeric code to your phone that must be entered along with the password, but most people haven’t enabled this. Furthermore, using the phone as a second-factor has it’s own hacks that skilled hackers can bypass. Phone numbers that belong to her are also on that “people finder” report I paid for:

If the sexy images were sent via email, then likewise simply knowing her email password would grant somebody access to them. GMail makes it really easy to access old emails that you don’t care about anymore. You can likewise enable “two-factor authentication” to protect your email account, with a better factor that just text messages to your phone.

If she has an iPhone, and the pics were sent as normal text messages, then hacking her Apple account might reveal them. By default, iPhone’s back these up to the cloud (“iCloud”). But not so fast. Apple has strong-armed their customers to enable “two-factor authentication”, so the hacker would need to intercept the message.

But Apple text messages don’t always go across the phone system. When it’s two iPhones involved, or Apple-to-iPhone, such messages go across their end-to-end encrypted iMessage service, which even state actors like the NSA and FBI have trouble penetrating. Apple does a better job than anybody protecting their phones, such that even if I knew the password to your account, I’m not sure I could steal your sexting images.

Lastly, maybe an encrypted messaging service like Signal was used. This is generally pretty secure, though hey have a number of holes. For example, when receiving a sexting message, the user can simply take a screenshot. At that point, we are back into the “cloud backup” situation we were before.

Maybe it wasn’t her phone/accounts that were hacked. Maybe she shared them with her siblings, friends, or agent. Diligent hackers go after those accounts as well. Famous celebrity hackers often get nude pics via this route, rather than hacking the celebrity directly. That “people finder” report includes a list of her close relatives, and enough information I can track down her other associates.

So here’s how you can avoid getting into the same situation:

  • Setup different email accounts, ones you use for personal reasons that can easily be discovered, and ones you use in other situations that cannot be tied to your name.
  • Don’t reuse passwords, as was done in the case, where all the accounts I found have the same password. At least one site where you’ve used that password will get hacked and have that password shared in the underground. Use unique password for major sites. Knowing your GMail password should not give me access to your iPhone account because that’s a different password. Write these passwords down on paper and store them in a safe place. For unimportant accounts you don’t care about, sure, go ahead and use the same password, or common password pattern, for all of them. They’ll get hacked but you don’t care.
  • Check https://haveibeenpwned.com to see how many of your accounts have pwned in hacker attacks against websites. Obviously, the passwords you used for those websites should never be used again.
  • If you send sexy messages and you are a celebrity, there are large parts of the hacker underground who specialize in trying to steal them.
This post describes one path to hack the accounts, from password reuse. Another vector for teenage hackers is phishing, such as in the DNC hack. This leaves traces behind, so presumably if this were the vector, they would’ve discovered this by now. The point is that there’s little reason to assume nation state actors or corrupt law enforcement officials in the pay of Donald Trump.
To summarize this post:
  • No, I didn’t hack her accounts. However, her email addresses and some passwords are public on the Internet for hackers who look for them.
  • Some passwords are public. That doesn’t mean the important passwords that would gain access to real accounts are public. I didn’t try them to find out.
  • Even though I didn’t fully test this, people get their sensitive information (like nude pics) stolen this way all the time.
  • Getting celebrity nude pics is fairly simple, such as through password reuse and phishing, so there is no reason to consider conspiracy theories at this time.

Passwords in a file

Post Syndicated from Robert Graham original https://blog.erratasec.com/2019/01/passwords-in-file.html

My dad is on some sort of committee for his local home owners association. He asked about saving all the passwords in a file stored on Microsoft’s cloud OneDrive, along with policy/procedures for the association. I assumed he called because I’m an internationally recognized cyberexpert. Or maybe he just wanted to chat with me*. Anyway, I thought I’d write up a response.

The most important rule of cybersecurity is that it depends upon the risks/costs. That means if what you want to do is write down the procedures for operating a garden pump, including the passwords, then that’s fine. This is because there’s not much danger of hackers exploiting this. On the other hand, if the question is passwords for the association’s bank account, then DON’T DO THIS. Such passwords should never be online. Instead, write them down and store the pieces of paper in a secure place.

OneDrive is secure, as much as anything is. The problem is that people aren’t secure. There’s probably one member of the home owner’s association who is constantly infecting themselves with viruses or falling victim to scams. This is the person who you are giving OneDrive access to. This is fine for the meaningless passwords, but very much not fine for bank accounts.

OneDrive also has some useful backup features. Thus, when one of your members infects themselves with ransomware, which will encrypt all the OneDrive’s contents, you can retrieve the old versions of the documents. I highly recommend groups like the home owner’s association use OneDrive. I use it as part of my Office 365 subscription for $99/year.

Just don’t do this for banking passwords. In fact, not only should you not store such a password online, you should strongly consider getting “two factor authentication” setup for the account. This is a system where you need an additional hardware device/token in addition to a password (in some cases, your phone can be used as the additional device). This may not work if multiple people need to access a common account, but then, you should have multiple passwords, for each individual, in such cases. Your bank should have descriptions of how to set this up. If your bank doesn’t offer two factor authentication for its websites, then you really need to switch banks.

For individuals, write your passwords down on paper. For elderly parents, write down a copy and give it to your kids. It should go without saying: store that paper in a safe place, ideally a safe, not a post-it note glued to your monitor. Again, this is for your important passwords, like for bank accounts and e-mail. For your Spotify or Pandora accounts (music services), then security really doesn’t matter.

Lastly, the way hackers most often break into things like bank accounts is because people use the same password everywhere. When one site gets hacked, those passwords are then used to hack accounts on other websites. Thus, for important accounts, don’t reuse passwords, make them unique for just that account. Since you can’t remember unique passwords for every account, write them down.

You can check if your password has been hacked this way by checking http://haveibeenpwned.com and entering your email address. Entering my dad’s email address, I find that his accounts at Adobe, LinkedIn, and Disqus has been discovered by hackers (due to hacks of those websites) and published. I sure hope whatever these passwords were that they are not the same or similar to his passwords for GMail or his bank account.


* the lame joke at the top was my dad’s, so don’t blame me 🙂

Notes on Build Hardening

Post Syndicated from Robert Graham original https://blog.erratasec.com/2018/12/notes-on-build-hardening.html

I thought I’d comment on a paper about “build safety” in consumer products, describing how software is built to harden it against hackers trying to exploit bugs.

What is build safety?

Modern languages (Java, C#, Go, Rust, JavaScript, Python, etc.) are inherently “safe”, meaning they don’t have “buffer-overflows” or related problems.

However, C/C++ is “unsafe”, and is the most popular language for building stuff that interacts with the network. In other cases, while the language itself may be safe, it’ll use underlying infrastructure (“libraries“) written in C/C++. When we are talking about hardening builds, making them safe or security, we are talking about C/C++.

In the last two decades, we’ve improved both hardware and operating-systems around C/C++ in order to impose safety on it from the outside. We do this with  options when the software is built (compiled and linked), and then when the software is run.
That’s what the paper above looks at: how consumer devices are built using these options, and thereby, measuring the security of these devices.
In particular, we are talking about the Linux operating system here and the GNU compiler gcc. Consumer products almost always use Linux these days, though a few also use embedded Windows or QNX. They are almost always built using gcc, though some are built using a clone known as clang (or llvm).

How software is built

Software is first compiled then linked. Compiling means translating the human-readable source code into machine code. Linking means combining multiple compiled files into a single executable.
Consider a program hello.c. We might compile it using the following command:
gcc -o hello hello.c
This command takes the file, hello.c, compiles it, then outputs -o an executable with the name hello.
We can set additional compilation options on the command-line here. For example, to enable stack guards, we’d compile with a command that looks like the following:
gcc -o hello -fstack-protector hello.c
In the following sections, we are going to look at specific options and what they do.

Stack guards

A running program has various kinds of memory, optimized for different use cases. One chunk of memory is known as the stack. This is the scratch pad for functions. When a function in the code is called, the stack grows with additional scratchpad needs of that functions, then shrinks back when the function exits. As functions call other functions, which call other functions, the stack keeps growing larger and larger. When they return, it then shrinks back again.
The scratch pad for each function is known as the stack frame. Among the things stored in the stack frame is the return address, where the function was called from so that when it exits, the caller of the function can continue executing where it left off.

The way stack guards work is to stick a carefully constructed value in between each stack frame, known as a canary. Right before the function exits, it’ll check this canary in order to validate it hasn’t been corrupted. If corruption is detected, the program exits, or crashes, to prevent worse things from happening.

This solves the most common exploited vulnerability in C/C++ code, the stack buffer-overflow. This is the bug described in that famous paper Smashing the Stack for Fun and Profit from the 1990s.
To enable this stack protection, code is compiled with the option -fstack-protector. This looks for functions that have typical buffers, inserting the guard/canary on the stack when the function is entered, and then verifying the value hasn’t been overwritten before exit.
Not all functions are instrumented this way, for performance reasons, only those that appear to have character buffers. Only those with buffers of 8 bytes or more are instrumented by default. You can change this by adding the option –param ssp-buffer-size=n, where n is the number of bytes. You can include other arrays to check using -fstack-protector-strong instead. You can instrument all functions with -fstack-protector-all.

Since this feature was added, many vulnerabilities have been found that evade the default settings. Recently, -fstack-protector-strong has been added to gcc that significantly increases the number of protected functions. The setting -fstack-protector-all is still avoided due to performance cost, as even trivial functions which can’t possibly overflow are still instrumented.

Heap guards

The other major dynamic memory structure is known as the heap (or the malloc region). When a function returns, everything in its scratchpad memory on the stack will lost. If something needs to stay around longer than this, then it must be allocated from the heap rather than the stck.

Just as there are stack buffer-overflows, there can be heap overflows, and the same solution of using canaries can guard against them.
The heap has two additional problems. The first is use-after-free, when memory on the heap is freed (marked as no longer in use), an then used anyway. The other is double-free, where the code attempts to free the memory twice. These problems don’t exist on the stack, because things are either added to or removed from the top of the stack, as in a stack of dishes. The heap looks more like the game Jenga, where things can be removed from the middle.

Whereas stack guards change the code generated by the compiler, heap guards don’t. Instead, the heap exists in library functions.

The most common library added by a linker is known as glibc, the standard GNU C library. However, this library is about 1.8-megabytes in size. Many of the home devices in the paper above may only have 4-megabytes total flash drive space, so this is too large. Instead, most of these home devices use an alternate library, something like uClibc or musl, which is only 0.6-megabytes in size. In addition, regardless of the standard library used for other features, a program my still replace the heap implementation with a custom one, such as jemalloc.

Even if using a library that does heap guards, it may not be enabled in the software. If using glibc, a program can still turn off checking internally (using mallopt), or it can be disabled externally, before running a program, by setting the environment variable MALLOC_CHECK_.

The above paper didn’t evaluate heap guards. I assume this is because it can be so hard to check.

ASLR

When a buffer-overflow is exploited, a hacker will overwrite values pointing to specific locations in memory. That’s because locations, the layout of memory, are predictable. It’s a detail that programmers don’t know when they write the code, but something hackers can reverse-engineer when trying to figure how to exploit code.

Obviously a useful mitigation step would be to randomize the layout of memory, so nothing is in a predictable location. This is known as address space layout randomization or ASLR.

The word layout comes from the fact that the when a program runs, it’ll consist of several segments of memory. The basic list of segments are:

  • the executable code
  • static values (like strings)
  • global variables
  • libraries
  • heap (growable)
  • stack (growable)
  • mmap()/VirtualAlloc() (random location)

Historically, the first few segments are laid out sequentially, starting from address zero. Remember that user-mode programs have virtual memory, so what’s located starting at 0 for one program is different from another.

As mentioned above, the heap and the stack need to be able to grow as functions are called data allocated from the heap. The way this is done is to place the heap after all the fixed-sized segments, so that it can grow upwards. Then, the stack is placed at the top of memory, and grows downward (as functions are called, the stack frames are added at the bottom).

Sometimes a program may request memory chunks outside the heap/stack directly from the operating system, such as using the mmap() system call on Linux, or the VirtualAlloc() system call on Windows. This will usually be placed somewhere in the middle between the heap and stack.

With ASLR, all these locations are randomized, and can appear anywhere in memory. Instead of growing contiguously, the heap has to sometimes jump around things already allocated in its way, which is a fairly easy problem to solve, since the heap isn’t really contiguous anyway (as chunks are allocated and freed from the middle). However, the stack has a problem. It must grow contiguously, and if there is something in its way, the program has little choice but to exit (i.e. crash). Usually, that’s not a problem, because the stack rarely grows very large. If it does grow too big, it’s usually because of a bug that requires the program to crash anyway.

ASLR for code

The problem for executable code is that for ASLR to work, it must be made position independent. Historically, when code would call a function, it would jump to the fixed location in memory where that function was know to be located, thus it was dependent on the position in memory.

To fix this, code can be changed to jump to relative positions instead, where the code jumps at an offset from wherever it was jumping from.

To  enable this on the compiler, the flag -fPIE (position independent executable) is used. Or, if building just a library and not a full executable program, the flag -fPIC (position independent code) is used.

Then, when linking a program composed of compiled files and libraries, the flag -pie is used. In other words, use -pie -fPIE when compiling executables, and -fPIC when compiling for libraries.

When compiled this way, exploits will no longer be able to jump directly into known locations for code.

ASLR for libraries

The above paper glossed over details about ASLR, probably just looking at whether an executable program was compiled to be position independent. However, code links to shared libraries that may or may not likewise be position independent, regardless of the settings for the main executable.

I’m not sure it matters for the current paper, as most programs had position independence disabled, but in the future, a comprehensive study will need to look at libraries as a separate case.

ASLR for other segments

The above paper equated ASLR with randomized location for code, but ASLR also applies to the heap and stack. The randomization status of these programs is independent of whatever was configured for the main executable.

As far as I can tell, modern Linux systems will randomize these locations, regardless of build settings. Thus, for build settings, it just code randomization that needs to be worried about. But when running the software, care must be taken that the operating system will behave correctly. A lot of devices, especially old ones, use older versions of Linux that may not have this randomization enabled, or be using custom kernels where it has been turned off.

RELRO

Modern systems have dynamic/shared libraries. Most of the code of a typical program consists of standard libraries. As mentioned above, the GNU standard library glibc is 8-megabytes in size. Linking that into every one of hundreds of programs means gigabytes of disk space may be needed to store all the executables. It’s better to have a single file on the disk, libglibc.so, that all programs can share it.
The problem is that every program will load libraries into random locations. Therefore, code cannot jump to functions in the library, either with a fixed or relative address. To solve this, what position independent code does is jump to an entry in a table, relative to its own position. That table will then have the fixed location of the real function that it’ll jump to. When a library is loaded, that table is filled in with the correct values.
The problem is that hacker exploits can also write to that table. Therefore, what you need to do is make that table read-only after it’s been filled in. That’s done with the “relro” flag, meaning “relocation read-only”. An additional flag, “now”, must be set to force this behavior at program startup, rather than waiting until later.
When passed to the linker, these flags would be “-z relro -z now“. However, we usually call the linker directly from the compiler, and pass the flags through. This is done in gcc by doing “-Wl,-z,relro -Wl,-z,now“.

Non-executable stack

Exploiting a stack buffer overflow has three steps:
  • figure out where the stack is located (mitigated by ASLR)
  • overwrite the stack frame control structure (mitigated by stack guards)
  • execute code in the  buffer
We can mitigate the third step by preventing code from executing from stack buffers. The stack contains data, not code, so this shouldn’t be a problem. In much the same way that we can mark memory regions as read-only, we can mark them no-execute. This should be the default, of course, but as the paper above points out, there are some cases where code is actually placed on the stack.

This open can be set with -Wl,-z,noexecstack, when compiling both the executable and the libraries. This is the default, so you shouldn’t need to do anything special. However, as the paper points out, there are things that get in the way of this if you aren’t careful. The setting is more what you’d call “guidelines” than actual “rules”. Despite setting this flag, building software may result in an executable stack.

So, you may want to verify it after building software, such as using the “readelf -l [programname]”. This will tell you what the stack has been configured to be.

Non-executable heap

The above paper focused on executable stack, but there is also the question of an executable heap. It likewise contains data and not code, so should be marked no-execute. Like for heap guards mentioned above, this isn’t a build setting but a feature of the library. The default library for Linux, glibc, marks the heap no-execute. However, it appears the other standard libraries or alternative heaps mark the stack as executable.

FORTIFY_SOURCE

The paper above doesn’t discuss this hardening step, but it’s an important one.
One reason for so many buffer-overflow bugs is that the standard functions that copy buffers have no ability to verify whether they’ve gone past the end of a buffer. A common recommendation for code is to replace those inherently unsafe functions with safer alternatives that include length checks. For example the notoriously unsafe function strcpy() can be replaced with strlcpy(), which adds a length check.
Instead of editing the code, the GNU compiler can do this automatically. This is done with the build option -O2 -D_FORTIFY_SOURCE=2.
This is only a partial solution. The compiler can’t always figure out the size of the buffer being copied into, and thus will leave the code untouched. Only when the compiler can figure things out does it make the change.
It’s fairly easy to detect if code has been compiled with this flag, but the above paper didn’t look much into it. That’s probably for the same reason it didn’t look into heap checks: it requires the huge glibc library. These devices use the smaller libraries, which don’t support this feature.

Format string bugs

Because this post is about build hardening, I want to mention format-string bugs. This is a common  bug in old code that can be caught by adding warnings for it in the build options, namely:

 -Wformat -Wformat-security -Werror=format-security

It’s hard to check if code has been built with these options, however. Instead of simple programs like readelf that can verify many of the issues above, this would take static analysis tools that read the executable code and reverse engineer what’s going on.

Warnings and static analysis

When building code, the compiler will generate warnings about confusion, possible bugs, or bad style. The compiler has a default set of warnings. Robust code is compiled with the -Wall, meaning “all” warnings, though it actually doesn’t enable all of them. Paranoid code uses the -Wextra warnings to include those not included with -Wall. There is also the -pedantic or -Wpedantic flag, which warns on C compatibility issues.
All of these warnings can be converted into errors, which prevents the software from building, using the -Werror flag. As shown above, this can also be used with individual error names to make only some warnings into errors.

Optimization level

Compilers can optimize code, looking for common patterns, to make it go faster. You can set your desired optimization level.
Some things, namely the FORTIFY_SOURCE feature, don’t work without optimization enabled. That’s why in the above example, -O2 is specified, to set optimization level 2.
Higher levels aren’t recommended. For one thing, this doesn’t make code faster on modern, complex processors. The advanced processors you have in your desktop/mobile-phone themselves do extensive optimization. What they want is small code size that fits within cache. The -O3 level make code bigger, which is good for older, simpler processors, but is bad for modern, advanced processors.
In addition, the aggressive settings of -O3 have lead to security problems over “undefined behavior”. Even -O2 is a little suspect, with some guides suggesting the proper optimization is -O1. However, some optimizations are actually more secure than no optimizations, so for the security paranoid, -O1 should be considered the minimum.

What about sanitizers?

You can compile code with address sanitizers that do more comprehensive buffer-overflow checks, as well as undefined behavior sanitizers for things like integer overflows.
While these are good for testing and debugging, they’ve proven so far too difficult to get working in production code. These may become viable in the future.

Summary

If  you are building code using gcc on Linux, here are the options/flags you should use:

-Wall -Wformat -Wformat-security -Werror=format-security -fstack-protector -pie -fPIE -D_FORTIFY_SOURCE=2 -O2 -Wl,-z,relro -Wl,-z,now -Wl,-z,noexecstack

If you are more paranoid, these options would be:

-Wall -Wformat -Wformat-security -Wstack-protector -Werror -pedantic -fstack-protector-all –param ssp-buffer-size=1 -pie -fPIE -D_FORTIFY_SOURCE=2 -O1 -Wl,-z,relro -Wl,-z,now -Wl,-z,noexecstack

Notes about hacking with drop tools

Post Syndicated from Robert Graham original https://blog.erratasec.com/2018/12/notes-about-hacking-with-drop-tools.html

In this report, Kasperky found Eastern European banks hacked with Raspberry Pis and “Bash Bunnies” (DarkVishnya). I thought I’d write up some more detailed notes on this.

Drop tools

A common hacking/pen-testing technique is to drop a box physically on the local network. On this blog, there are articles going back 10 years discussing this. In the old days, this was done with $200 “netbook” (cheap notebook computers). These days, it can be done with $50 “Raspberry Pi” computers, or even $25 consumer devices reflashed with Linux.

A “Raspberry Pi” is a $35 single board computer, for which you’ll need to add about another $15 worth of stuff to get it running (power supply, flash drive, and cables). These are extremely popular hobbyist computers that are used everywhere from home servers, robotics, and hacking. They have spawned a large number of clones, like the ODROID, Orange Pi, NanoPi, and so on. With a quad-core, 1.4 GHz, single-issue processor, 2 gigs of RAM, and typically at least 8 gigs of flash, these are pretty powerful computers.

Typically what you’d do is install Kali Linux. This is a Linux “distro” that contains all the tools hackers want to use.

You then drop this box physically on the victim’s network. We often called these “dropboxes” in the past, but now that there’s a cloud service called “Dropbox”, this becomes confusing, so I guess we can call them “drop tools”. The advantage of using something like a Raspberry Pi is that it’s cheap: once dropped on a victim’s network, you probably won’t ever get it back again.

Gaining physical access to even secure banks isn’t that hard. Sure, getting to the money is tightly controlled, but other parts of the bank aren’t not nearly as secure. One good trick is to pretend to be a banking inspector. At least in the United States, they’ll quickly bend over an spread them if they think you are a regulator. Or, you can pretend to be maintenance worker there to fix the plumbing. All it takes is a uniform with a logo and what appears to be a valid work order. If questioned, whip out the clipboard and ask them to sign off on the work. Or, if all else fails, just walk in brazenly as if you belong.

Once inside the physical network, you need to find a place to plug something in. Ethernet and power plugs are often underneath/behind furniture, so that’s not hard. You might find access to a wiring closet somewhere, as Aaron Swartz famously did. You’ll usually have to connect via Ethernet, as it requires no authentication/authorization. If you could connect via WiFi, you could probably do it outside the building using directional antennas without going through all this.

Now that you’ve got your evil box installed, there is the question of how you remotely access it. It’s almost certainly firewalled, preventing any inbound connection.

One choice is to configure it for outbound connections. When doing pentests, I configure reverse SSH command-prompts to a command-and-control server. Another alternative is to create a SSH Tor hidden service. There are a myriad of other ways you might do this. They all suffer the problem that anybody looking at the organization’s outbound traffic can notice these connections.

Another alternative is to use the WiFi. This allows you to physically sit outside in the parking lot and connect to the box. This can sometimes be detected using WiFi intrusion prevention systems, though it’s not hard to get around that. The downside is that it puts you in some physical jeopardy, because you have to be physically near the building. However, you can mitigate this in some cases, such as sticking a second Raspberry Pi in a nearby bar that is close enough to connection, and then use the bar’s Internet connection to hop-scotch on in.

The third alternative, which appears to be the one used in the article above, is to use a 3G/4G modem. You can get such modems for another $15 to $30. You can get “data only” plans, especially through MVNOs, for around $1 to $5 a month, especially prepaid plans that require no identification. These are “low bandwidth” plans designed for IoT command-and-control where only a few megabytes are transferred per month, which is perfect for command-line access to these drop tools.

With all this, you are looking at around $75 for the hardware, software, and 3G/4G plan for a year to remotely connect to a box on the target network.

As an alternative, you might instead use a cheap consumer router reflashed with the OpenWRT Linux distro. A good example would be a Gl.INET device for $19. This a cheap Chinese manufacturer that makes cheap consumer routers designed specifically for us hackers who want to do creative things with them.

The benefit of such devices is that they look like the sorts of consumer devices that one might find on a local network. Raspberry Pi devices stand out as something suspicious, should they ever be discovered, but a reflashed consumer device looks trustworthy.

The problem with these devices is that they are significantly less powerful than a Raspberry Pi. The typical processor is usually single core around 500 MHz, and the typical memory is only around 32 to 128 megabytes. Moreover, while many hacker tools come precompiled for OpenWRT, you’ll end up having to build most of the tools yourself, which can be difficult and frustrating.

Hacking techniques

Once you’ve got your drop tool plugged into the network, then what do you do?

One question is how noisy you want to be, and how good you think the defenders are. The classic thing to do is run a port scanner like nmap or masscan to map out the network. This is extremely noisy and even clueless companies will investigate.

This can be partly mitigated by spoofing your MAC and IP addresses. However, a properly run network will still be able to track back the addresses to the proper port switch. Therefore, you might want to play with a bunch of layer 2 things. For example, passively watch for devices that get turned off a night, then spoof their MAC address during your night time scans, so that when they come back in the morning, they’ll trace it back to the wrong device causing the problem.

An easier thing is to passively watch what’s going on. In purely passive mode, they really can’t detect that you exist at all on the network, other than the fact that the switch port reports something connected. By passively looking at ARP packets, you can get a list of all the devices on your local segment. By passively looking at Windows broadcasts, you can map out large parts of what’s going on with Windows. You can also find MacBooks, NAT routers, SIP phones, and so on.

This allows you to then target individual machines rather than causing a lot of noise on the network, and therefore go undetected.

If you’ve got a target machine, the typical procedure is to port scan it with nmap, find the versions of software running that may have known vulnerabilities, then use metasploit to exploit those vulnerabilities. If it’s a web server, then you might use something like burpsuite in order to find things like SQL injection. If it’s a Windows desktop/server, then you’ll start by looking for unauthenticated file shares, man-in-the-middle connections, or exploit it with something like EternalBlue.

The sorts of things you can do is endless, just read any guide on how to use Kali Linux, and follow those examples.

Note that your command-line connection may be a low-bandwidth 3G/4G connection, but when it’s time to exfiltrate data, you’ll probably use the corporate Internet connection to transfer gigabytes of data.

USB hacking tools

The above paper described not only drop tools attached to the network, but also tools attached view USB. This is a wholly separate form of hacking.

According to the description, the hackers used BashBunny, a $100 USB device. It’s a computer than can emulate things like a keyboard.

However, a cheaper alternative is the Raspberry Pi Zero W for $15, with Kali Linux installed, especially a Kali derivative like this one that has USB attack tools built in and configured.

One set of attacks is through a virtual keyboard and mouse. It can keep causing mouse/keyboard activity invisibly in the background to avoid the automatic lockout, then presumably at night, run commands that will download and run evil scripts. A good example is the “fileless PowerShell” scripts mentioned in the article above.

This may be combined with emulation of a flash drive. In the old days, hostile flash drives could directly infect a Windows computer once plugged in. These days, that won’t happen without interaction by the user — interaction using a keyboard/mouse, which the device can also emulate.

Another set of attacks is pretending to be a USB Ethernet connection. This allows network attacks, such as those mentioned above, to travel across the USB port, without being detectable on the real network. It also allows additional tricks. For example, it can configure itself to be the default route for Internet (rather than local) access, redirecting all web access to a hostile device on the Internet. In other words, the device will usually be limited in that it doesn’t itself have access to the Internet, but it can confuse the network configuration of the Windows device to cause other bad effects.

Another creative use is to emulate a serial port. This works for a lot of consumer devices and things running Linux. This will get you a shell directly on the device, or a login that accepts a default or well-known backdoor password. This is a widespread vulnerability because it’s so unexpected.

In theory, any USB device could be emulated. Today’s Windows, Linux, and macOS machines have a lot of device drivers that are full of vulnerabilities that an be exploited. However, I don’t see any easy to use hacking toolkits that’ll make this easy for you, so this is still mostly just theoretical.

Defense

The purpose of this blogpost isn’t “how to hack” by “how to defend”. Understanding what attackers do is the first step in understanding how to stop them.
Companies need to understand the hardware on their network. They should be able to list all the hardware devices on all their switches and have a running log of any new device that connects. They need to be able to quickly find the physical location of any device, with well-documented cables and tracking which MAC address belongs to which switch port. Better yet, 802.11x should be used to require authentication on Ethernet just like you require authentication on WiFi.
The same should be done for USB. Whenever a new USB device is plugged into Windows, that should be logged somewhere. I would suggest policies banning USB devices, but they are so useful this can become very costly to do right.
Companies should have enough monitoring that they can be notified whenever somebody runs a scanner like nmap. Better yet, they should have honeypot devices and services spread throughout their network that will notify them if somebody is already inside their network.

Conclusion

Hacking a target like a bank consists of three main phrases: getting in from the outside, moving around inside the network to get to the juice bits, then stealing money/data (or causing harm). That first stage is usually the hardest, and can be bypassed with physical access, dropping some sort of computer on the network. A $50 device like a Raspberry Pi running Kali Linux is perfect for this. 

Every security professional should have experience with this. Whether it’s actually a Raspberry Pi or just a VM on a laptop running Kali, security professionals should have experience with this. They should run nmap on their network, they should run burpsuite on their intranet websites, and so on. Of course, this should only be done with knowledge and permission from their bosses, and ideally, boss’s bosses.

Some notes about HTTP/3

Post Syndicated from Robert Graham original https://blog.erratasec.com/2018/11/some-notes-about-http3.html

HTTP/3 is going to be standardized. As an old protocol guy, I thought I’d write up some comments.

Google (pbuh) has both the most popular web browser (Chrome) and the two most popular websites (#1 Google.com #2 Youtube.com). Therefore, they are in control of future web protocol development. Their first upgrade they called SPDY (pronounced “speedy”), which was eventually standardized as the second version of HTTP, or HTTP/2. Their second upgrade they called QUIC (pronounced “quick”), which is being standardized as HTTP/3.

SPDY (HTTP/2) is already supported by the major web browser (Chrome, Firefox, Edge, Safari) and major web servers (Apache, Nginx, IIS, CloudFlare). Many of the most popular websites support it (even non-Google ones), though you are unlikely to ever see it on the wire (sniffing with Wireshark or tcpdump), because it’s always encrypted with SSL. While the standard allows for HTTP/2 to run raw over TCP, all the implementations only use it over SSL.

There is a good lesson here about standards. Outside the Internet, standards are often de jure, run by government, driven by getting all major stakeholders in a room and hashing it out, then using rules to force people to adopt it. On the Internet, people implement things first, and then if others like it, they’ll start using it, too. Standards are often de facto, with RFCs being written for what is already working well on the Internet, documenting what people are already using. SPDY was adopted by browsers/servers not because it was standardized, but because the major players simply started adding it. The same is happening with QUIC: the fact that it’s being standardized as HTTP/3 is a reflection that it’s already being used, rather than some milestone that now that it’s standardized that people can start using it.

QUIC is really more of a new version of TCP (TCP/2???) than a new version of HTTP (HTTP/3). It doesn’t really change what HTTP/2 does so much as change how the transport works. Therefore, my comments below are focused on transport issues rather than HTTP issues.

The major headline feature is faster connection setup and latency. TCP requires a number of packets being sent back-and-forth before the connection is established. SSL again requires a number of packets sent back-and-forth before encryption is established. If there is a lot of network delay, such as when people use satellite Internet with half-second ping times, it can take quite a long time for a connection to be established. By reducing round-trips, connections get setup faster, so that when you click on a link, the linked resource pops up immediately

The next headline feature is bandwidth. There is always a bandwidth limitation between source and destination of a network connection, which is almost always due to congestion. Both sides need to discover this speed so that they can send packets at just the right rate. Sending packets too fast, so that they’ll get dropped, causes even more congestion for others without improving transfer rate. Sending packets too slowly means unoptimal use of the network.

How HTTP traditionally does this is bad. Using a single TCP connection didn’t work for HTTP because interactions with websites require multiple things to be transferred simultaneously, so browsers opened multiple connections to the web server (typically 6). However, this breaks the bandwidth estimation, because each of your TCP connections is trying to do it independently as if the other connections don’t exist. SPDY addressed this by its multiplexing feature that combined multiple interactions between browser/server with a single bandwidth calculation.

QUIC extends this multiplexing, making it even easier to handle multiple interactions between the browser/server, without any one interaction blocking another, but with a common bandwidth estimation. This will make interactions smoother from a user’s perspective, while at the same time reduce congestion that routers experience.

Now let’s talk user-mode stacks. The problem with TCP, especially on the server, is that TCP connections are handled by the operating system kernel, while the service itself runs in usermode. Moving things across the kernel/usermode boundary causes performance issues. Tracking a large number of TCP connections causes scalability issues. Some people have tried putting the services into the kernel, to avoid the transitions, which is a bad because it destabilizes the operating system. My own solution, with the BlackICE IPS and masscan, was to use a usermode driver for the hardware, getting packets from the network chip directly to the usermode process, bypassing the kernel (see PoC||GTFO #15), using my own custom TCP stack. This has become popular in recent years with the DPDK kit.

But moving from TCP to UDP can get you much the same performance without usermode drivers. Instead of calling the well-known recv() function to receive a single packet at a time, you can call recvmmsg() to receive a bunch of UDP packets at once. It’s still a kernel/usermode transition, but one amortized across a hundred packets received at once, rather a transition per packet.

In my own tests, you are limited to about 500,000 UDP packets/second using the typical recv() function, but with recvmmsg() and some other optimizations (multicore using RSS), you can get to 5,000,000 UDP packets/second on a low-end quad-core server. Since this scales well per core, moving to the beefy servers with 64 cores should improve things even further.

BTW, “RSS” is a feature of network hardware that splits incoming packets into multiple receive queues. The biggest problem with multi-core scalability is whenever two CPU cores need to read/modify the same thing at the same time, so sharing the same UDP queue of packets becomes the biggest bottleneck. Therefore, first Intel and then other Ethernet vendors added RSS giving each core it’s own non-shared packet queue. Linux and then other operating systems upgraded UDP to support multiple file descriptors for a single socket (SO_REUSEPORT) to handle the multiple queues. Now QUIC uses those advances allowing each core to manage it’s own stream of UDP packets without the scalability problems of sharing things with other CPU cores. I mention this because I personally had discussions with Intel hardware engineers about having multiple packet queues back in 2000. It’s a common problem and an obvious solution, and it’s been fun watching it progress over the last two decades until it appears on the top end as HTTP/3. Without RSS in the network hardware, it’s unlikely QUIC would become a standard.

Another cool solution in QUIC is mobile support. As you move around with your notebook computer to different WiFI networks, or move around with your mobile phone, your IP address can change. The operating system and protocols don’t gracefully close the old connections and open new ones. With QUIC, however, the identifier for a connection is not the traditional concept of a “socket” (the source/destination port/address protocol combination), but a 64-bit identifier assigned to the connection.

This means that as you move around, you can continue with a constant stream uninterrupted from YouTube even as your IP address changes, or continue with a video phone call without it being dropped. Internet engineers have been struggling with “mobile IP” for decades, trying to come up with a workable solution. They’ve focused on the end-to-end principle of somehow keeping a constant IP address as you moved around, which isn’t a practical solution. It’s fun to see QUIC/HTTP/3 finally solve this, with a working solution in the real world.

How can use use this new transport? For decades, the standard for network programing has been the transport layer API known as “sockets”. That where you call functions like recv() to receive packets in your code. With QUIC/HTTP/3, we no longer have an operating-system transport-layer API. Instead, it’s a higher layer feature that you use in something like the go programming language, or using Lua in the OpenResty nginx web server.

I mention this because one of the things that’s missing from your education about the OSI Model is that it originally envisioned everyone writing to application layer (7) APIs instead of transport layer (4) APIs. There was supposed to be things like application service elements that would handling things like file transfer and messaging in a standard way for different applications. I think people are increasingly moving to that model, especially driven by Google with go, QUIC, protobufs, and so on.

I mention this because of the contrast between Google and Microsoft. Microsoft owns a popular operating system, so it’s innovations are driven by what it can do within that operating system. Google’s innovations are driven by what it can put on top of the operating system. Then there is Facebook and Amazon themselves which must innovate on top of (or outside of) the stack that Google provides them. The top 5 corporations in the world are, in order, Apple-Google-Microsoft-Amazon-Facebook, so where each one drives innovation is important.

Conclusion

When TCP was created in the 1970s, it was sublime. It handled things, like congestion, vastly better than competing protocols. For all that people claim IPv4 didn’t anticipate things like having more than 4-billion addresses, it anticipated the modern Internet vastly better than competing designs throughout the 70s and 80s. The upgrade from IPv4 to IPv6 largely maintains what makes IP great. The upgrade from TCP to QUIC is similarly based on what makes TCP great, but extending it to modern needs. It’s actually surprising TCP has lasted this long, and this well, without an upgrade.

Brian Kemp is bad on cybersecurity

Post Syndicated from Robert Graham original https://blog.erratasec.com/2018/11/brian-kemp-is-bad-on-cybersecurity.html

I’d prefer a Republican governor, but as a cybersecurity expert, I have to point out how bad Brian Kemp (candidate for Georgia governor) is on cybersecurity. When notified about vulnerabilities in election systems, his response has been to shoot the messenger rather than fix the vulnerabilities. This was the premise behind the cybercrime bill earlier this year that was ultimately vetoed by the current governor after vocal opposition from cybersecurity companies. More recently, he just announced that he’s investigating the Georgia State Democratic Party for a “failed hacking attempt“.

According to news stories, state elections websites are full of common vulnerabilities, those documented by the OWASP Top 10, such as “direct object references” that would allow any election registration information to be read or changed, as allowing a hacker to cancel registrations of those of the other party.

Testing for such weaknesses is not a crime. Indeed, it’s desirable that people can test for security weaknesses. Systems that aren’t open to test are insecure. This concept is the basis for many policy initiatives at the federal level, to not only protect researchers probing for weaknesses from prosecution, but to even provide bounties encouraging them to do so. The DoD has a “Hack the Pentagon” initiative encouraging exactly this.

But the State of Georgia is stereotypically backwards and thuggish. Earlier this year, the legislature passed SB 315 that criminalized this activity of merely attempting to access a computer without permission, to probe for possibly vulnerabilities. To the ignorant and backwards person, this seems reasonable, of course this bad activity should be outlawed. But as we in the cybersecurity community have learned over the last decades, this only outlaws your friends from finding security vulnerabilities, and does nothing to discourage your enemies. Russian election meddling hackers are not deterred by such laws, only Georgia residents concerned whether their government websites are secure.

It’s your own users, and well-meaning security researchers, who are the primary source for improving security. Unless you live under a rock (like Brian Kemp, apparently), you’ll have noticed that every month you have your Windows desktop or iPhone nagging you about updating the software to fix security issues. If you look behind the scenes, you’ll find that most of these security fixes come from outsiders. They come from technical experts who accidentally come across vulnerabilities. They come from security researchers who specifically look for vulnerabilities.

It’s because of this “research” that systems are mostly secure today. A few days ago was the 30th anniversary of the “Morris Worm” that took down the nascent Internet in 1988. The net of that time was hostile to security research, with major companies ignoring vulnerabilities. Systems then were laughably insecure, but vendors tried to address the problem by suppressing research. The Morris Worm exploited several vulnerabilities that were well-known at the time, but ignored by the vendor (in this case, primarily Sun Microsystems).

Since then, with a culture of outsiders disclosing vulnerabilities, vendors have been pressured into fix them. This has led to vast improvements in security. I’m posting this from a public WiFi hotspot in a bar, for example, because computers are secure enough for this to be safe. 10 years ago, such activity wasn’t safe.

The Georgia Democrats obviously have concerns about the integrity of election systems. They have every reason to thoroughly probe an elections website looking for vulnerabilities. This sort of activity should be encouraged, not suppressed as Brian Kemp is doing.

To be fair, the issue isn’t so clear. The Democrats aren’t necessarily the good guys. They are probably going to lose by a slim margin, and will cry foul, pointing to every election irregularity as evidence they were cheated. It’s like how in several races where Republicans lost by slim numbers they claimed criminals and dead people voted, thus calling for voter ID laws. In this case, Democrats are going to point to any potential vulnerability, real or imagined, as disenfranchising their voters. There has already been hyping of potential election systems vulnerabilities out of proportion to their realistic threat for this reason.

But while not necessarily in completely good faith, such behavior isn’t criminal. If an election website has vulnerabilities, then the state should encourage the details to be made public — and fix them.

One of the principles we’ve learned since the Morris Worm is that of “full disclosure”. It’s not simply that we want such vulnerabilities found and fixed, we also want the complete details to be made public, even embarrassing details. Among the reasons for this is that it’s the only way that everyone can appreciate the consequence of vulnerabilities.

In this case, without having the details, we have only the spin from both sides to go by. One side is spinning the fact that the website was wide open. The other side, as in the above announcement, claims the website was completely secure. Obviously, one side is lying, and the only way for us to know is if the full details of the potential vulnerability are fully disclosed.

By the way, it’s common for researchers to falsely believe in the severity of the vulnerability. This is at least as common as the embarrassed side trying to cover it up. It’s impossible to say which side is at fault here, whether it’s a real vulnerability or false, without full disclosure. Again, the wrong backwards thinking is to believe that details of vulnerabilities should be controlled, to avoid exploitation by bad guys. In fact, they should be disclosed, even if it helps the bad guys.

But regardless if these vulnerabilities are real, we do know that criminal investigation and prosecution is the wrong way to deal with the situation. If the election site is secure, then the appropriate response is to document why.

With that said, there’s a good chance the Democrats are right and Brian Kemp’s office is wrong. In the very announcement declaring their websites are secure, Google Chrome indicates their website is not secure in the URL sos.ga.gov, because they don’t use encryption.

Using LetsEcnrypt to enable encryption on websites is such a standard security feature we have to ask ourselves what else they are getting wrong. Normally, I’d run scanners against their systems in order to figure this out, but I’m afraid to, because they are jackbooted thugs who’ll come after me, instead of honest people who care about what vulnerabilities I might find so they can fix them.

Conclusion

I’m Libertarian, so I’m going to hate a Democrat governor more than a Republican governor. However, I’m also a cybersecurity expert and somebody famous for scanning for vulnerabilities. As a Georgia resident, I’m personally threatened by this backwards thuggish behavior by Brian Kemp. He learned nothing from this year’s fight over SB 315, and unlike the clueful outgoing governor who vetoed that bill, Kemp is likely to sign something similar, or worse, into law.

The integrity of election systems is an especially important concern. The only way to guarantee them is to encourage research, the probing by outsiders for vulnerabilities, and fully disclosing the results. Even if Georgia had the most secure systems, embarrassing problems are going to be found. Companies like Intel, Microsoft, and Apple are the leaders in cybersecurity, and even they have had embarrassing vulnerabilities in the last few months. They have responded by paying bounties to the security researchers who found those problems, not by criminally investigating them.

Why no cyber 9/11 for 15 years?

Post Syndicated from Robert Graham original https://blog.erratasec.com/2018/11/why-no-cyber-911-for-15-years.html

This The Atlantic article asks why hasn’t there been a cyber-terrorist attack for the last 15 years, or as it phrases it:

National-security experts have been warning of terrorist cyberattacks for 15 years. Why hasn’t one happened yet?

As a pen-tester who has broken into power grids and found 0dayss in control center systems, I thought I’d write up some comments.

Instead of asking why one hasn’t happened yet, maybe we should instead ask why national-security experts keep warning about them.

One possible answer is that national-security experts are ignorant. I get the sense that “national” security experts have very little expertise in “cyber” security. That’s why I include a brief resume at the top of this article, I’ve actually broken into a power grid and found 0days in critical power grid products (specifically, the ABB implementation of ICCP on AIX — it’s rather an obvious buffer-overflow, *cough* ASN.1 *cough*, I don’t know if they ever fixed it).

Another possibility is that they are fear mongering in order to support their agenda. That’s the problem with “experts”, they get their expertise by being employed to achieve some goal. The ones who know most about an issue are simultaneously the ones most biased. They have every incentive to make people be afraid, and little incentive to tell the truth.

The most likely answer, though, is simply because they can. Anybody can warn of “digital 9/11” and be taken seriously, regardless of expertise. It’s always the Morally Right thing to say. You never have to back it up with evidence. Conversely, those who say the opposite don’t get the same level of press, and are frequently challenged to defend their abnormal stance.

Indeed, that’s how this article by The Atlantic works. It’s entire premise is that the national security experts are still “right” even though their predictions haven’t happened, and it’s reality that’s somehow “wrong”.

Now let’s consider the original question.

One good answer in the article is that terrorists want attacks that “cause certain types of fear and terror, that garner certain media attention, that galvanize followers”. Blowing something up causes more fear in the target population than deleting some data.

But something similar is true of the terrorists themselves, that they prefer violence. In other words, what motivates terrorists, the ends or the means? It is it the need to achieve a political goal? Or is it simply about looking for an excuse to commit violence?

I suspect that it’s the later issue. It’s not that terrorists are violent so much as violent people are attracted to terrorism. This can explain a lot, such as why they have such poor op-sec and encryption, as I’ve written about before. They enjoy learning how to shoot guns and trigger bombs, but they don’t enjoy learning how to use a computer correctly.

I’ve explored the cyber Islamic dark web and come to a couple conclusions about it. The primary motivation of these hackers is gay porn. A frequent initiation rite to gain access to these forums is to post pictures of your, well, equipment. Such things are repressed in their native countries and societies, so hacking becomes a necessary skill in order to get it.

It’s hard for us to understand their motivations. From our western perspective, we’d think gay young men would be on our side, motivated to fight against their own governments in defense of gay rights, in order to achieve marriage equality. None of them want that, as far as I can tell. Their goal is to get married and have children. Sure, they want gay sex and intimate relationships with men, but they also want a subservient wife who manages the household, and the deep family ties that come with spawning progeny. Thus, their motivation is still to defend the umma (the whole community of Muslims bound together by ties of religion) against the West, not pursue personal rights.

The point is, when asking why terrorists do and don’t do types of attacks, their own personal motivations are probably pretty darn important.

Another explanation in that article is simply because Islamic hackers aren’t good enough. This requires a more sophisticated discussion of what skills they need. As The Atlantic says in their article:

The most powerful likely barrier, though, is also the simplest. For all the Islamic State’s much-vaunted technical sophistication, the skills needed to tweet and edit videos are a far cry from those needed to hack.

It’s indeed not just “editing videos”. Most hacker attacks you read about use un-sophisticated means like phishing. They are only believed to be sophisticated because people get confused by the results they achieve with the means with which they do it. For example, much of the DNC hack which had important consequences for our election was done simply by phishing the password from people like John Podesta.

A convincing cyber terrorism attack, such as causing a power black out, would take different skills — much rarer skills. I refer to my pentests above. The techniques used were all painfully simple, such as SQL injection from the Internet, but at the same time, it’s a much rarer skill. No matter how simple we think SQL injection is, it takes a different skillset than phishing. It takes people more interested in things like math. By the time such skills are acquired, they get gainfully employed at a technical job and no longer have free time to pursue the Struggle. Phishing skills won’t land you a high paying job, but web programming (which you need for SQL injection) will.

Lastly, I want to address the complexity of the problem. The Atlantic quotes Robert M. Lee of Dragos, a well-respected technical expert in this area, but I don’t think they get the quote right. He points out the complexity of the power grid. What he means is not complex as in hard but complex as in diverse. There’s 10,000 different companies involved in power production, long haul, distribution to homes, and so forth. Every state is different, every city is different, and even within cities there may be multiple small companies involved.

What this means is that while hacking any one of these entities would be easy, it’d only cause a small-scale effect. To cause big-scale effects would require a much larger hacking campaign, of a lot of targets, over a long period of time. Chances are high that before you hacked enough for a convincing terror effect, they’d catch on to you, and take moves to stop you. Thus while any individual target is easy, the campaign as a whole is complex.

In the end, if your goal is to cause major power blackouts, your best bet is to bomb power lines and distribution centers, rather than hack them.

Conclusion

I’m not sure if I have any better answers, just more complex perspectives.

I think there are lots of warning from so-called “experts” who aren’t qualified to make such warnings, that the press errs on the side of giving such warnings credibility instead of challenging them.

I think mostly the reason why cyberterrorism doesn’t happen is that which motivates violent people is different than what which motivates technical people, pulling apart the groups who would want to commit cyberterrorism from those who can.

At least for power grid blackouts, while small attacks would be easy, the ones large enough to grab people’s attention would be difficult, due to our power grid’s diversity.

Masscan and massive address lists

Post Syndicated from Robert Graham original https://blog.erratasec.com/2018/11/masscan-and-massive-address-lists.html

I saw this go by on my Twitter feed. I thought I’d blog on how masscan solves the same problem.

Both nmap and masscan are port scanners. The differences is that nmap does an intensive scan on a limited range of addresses, whereas masscan does a light scan on a massive range of addresses, including the range of 0.0.0.0 – 255.255.255.255 (all addresses). If you’ve got a 10-gbps link to the Internet, it can scan the entire thing in under 10 minutes, from a single desktop-class computer.

How massan deals with exclude ranges is probably its defining feature. That seems kinda strange, since it’s a little used feature in nmap. But when you scan the entire list, people will complain, with nasty emails, so you are going to build up a list of hundreds, if not thousands, of addresses to exclude from your scans.
Therefore, the first design choice is to combine the two lists, the list of targets to include and the list of targets to exclude. Other port scanners don’t do this because they typically work from a large include list and a short exclude list, so they optimize for the larger thing. In mass scanning the Internet, the exclude list is the largest thing, so that’s what we optimize for. It makes sense to just combine the two lists.
So the performance now isn’t how to lookup an address in an exclude list efficiently, it’s how to quickly choose a random address from a large include target list.
Moreover, the decision is how to do it with as little state as possible. That’s the trick for sending massive numbers of packets at rates of 10 million packets-per-second, it’s not keeping any bookkeeping of what was scanned. I’m not sure exactly how nmap randomizes it’s addresses, but the documentation implies that it does a block of a addresses at a time, and randomizes that block, keeping state on which addresses it’s scanned and which ones it hasn’t.
The way masscan is not to randomly pick an IP address so much as to randomize the index.
To start with, we created a sorted list of IP address ranges, the targets. The total number of IP addresses in all the ranges is target_count (not the number of ranges but the number of all IP addresses). We then define a function pick() that returns one of those IP addresses given the index:
    ip = pick(targets, index);
Where index is in the range [0..target_count].
This function is just a binary search. After the ranges have been sorted, a start_index value is added to each range, which is the total number of IP addresses up to that point. Thus, given a random index, we search the list of start_index values to find which range we’ve chosen, and then which IP address address within that range. The function is here, though reading it, I realize I need to refactor it to make it clearer. (I read the comments telling me to refactor it, and I realize I haven’t gotten around to that yet :-).
Given this system, we can now do an in-order (not randomized) port scan by doing the following:
    for (index=0; index<target_count; index++) {
        ip = pick(targets, index);
        scan(ip);
    }
Now, to scan in random order, we simply need to randomize the index variable.
    for (index=0; index<target_count; index++) {
        xXx = shuffle(index);
        ip = pick(targets, xXx);
        scan(ip);
    }

The clever bit is in that shuffle function (here). It has to take an integer in that range [0..target_count] and return another pseudo-random integer in the same range. It has to be a function that does a one-to-one mapping. Again, we are stateless. We can’t create a table of all addresses, then randomize the order of the table, and then enumerate that table. We instead have to do it with an algorithm.
The basis of that algorithm, by the way, is DES, the Data Encryption Standard. That’s how a block cipher works. It takes 64-bit number (the blocksize for DES) and outputs another 64-bit block in a one-to-one mapping. In ECB mode, every block is encrypted to a unique other block. Two input blocks can’t encrypt into the same output block, or you couldn’t decrypt it.
The only problem is the range isn’t neat 64-bit blocks, or any number of bits. It’s an inconveniently sized number. A cryptographer Phillip Rogaway wrote a paper how to change DES to support integer ranges instead. The upshot is that it uses integer division instead of shifts, which makes it more expensive.
So how we randomize that input variable is that we encrypt it, where the encrypted number is still in the same range.
Thus, the source of masscan‘s speed is the way it randomizes the IP addresses in a wholly stateless manner. It:
  • doesn’t use any state, just enumerates an index from [0..target_count]
  • has a fast function given an index, retrieve the indexed IP address from a large list of ranges
  • has a fast function to randomize that index using the Power of Crypto
Given this as the base, there’s lots of additional features we can add. For one thing, we are randomizing not only IP addresses to scan, but also ports. I think nmap picks the IP address first, then runs through a list of ports on that address. Masscan combines them altogether, so when scanning many ports on an address, they won’t come as a burst in the middle of the scan, but be spread evenly throughout the scan. It allows you to do things like:
    masscan 0.0.0.0/0 -p0-65535

For this to work, we make the following change to the inner loop:
    range = port_count * target_count;
    for (index=0; index<range; index++) {
        xXx = shuffle(index);
        ip = pick(targets, xXx % target_count);
        port = pick(targets, xXx / target_count);
        scan(ip, port);
    }

By the way, the compile optimizes both the modulus and division operations into a single IDIV opcode on Intel x86, since that’s how that instruction works, returning both results at once. Which is cool.
Another change we can make is sharding, spreading the scan across several CPUs or several servers. Let’s say this is server #3 out of 7 servers sharing the load of the scan:
    for (index=shard; index<range; index += shard_count) {
        …
    }
Again, notice how we don’t keep track of any state here, it’s just a minor tweak to the loop, and now *poof* the sharding feature appears out of nowhere. It takes vastly more instructions to parse the configuration parameter (masscan –shard 3/7 …) than it takes to actually do it.
Let’s say that we want to pause and resume the scan. What state information do we need to save? The answer is just the index variable. Well, we also need the list of IP addresses that we are scanning. A limitation of this approach is that we cannot easily pause a scan and change the list of IP addresses.
Conclusion

The upshot here is that we’ve twisted the nature of the problem. By using a crypto function to algorithmically create a one-to-one mapping for the index variable, we can just linearly enumerate a scan — but magically in random order. This avoids keeping state. It avoids having to lookup addresses in an exclude list. And we get other features that naturally fall out of the process.

What about IPv6?
You’ll notice I talking only about IPv4, and masscan supports only IPv4. The maximum sized scan right now is 48 bits (16-bit port number plus 32-bit IPv4 address). Won’t larger scans mean using 256 bit integers?

When I get around to adding IPv6, I’ll still keep a 64-bit index. The index variable is the number of things you are going to probe, and you can’t scan 64-bit space right now. You won’t scan the entire IPv6 128-bit address space, but a lot of smaller address spaces that add up to less than 64-bits. So when I get around to adding IPv6, the concept will still work.

Systemd is bad parsing and should feel bad

Post Syndicated from Robert Graham original https://blog.erratasec.com/2018/10/systemd-is-bad-parsing-and-should-feel.html

Systemd has a remotely exploitable bug in its DHCPv6 client. That means anybody on the local network can send you a packet and take control of your computer. The flaw is a typical buffer-overflow. Several news stories have pointed out that this client was rewritten from scratch, as if that were the moral failing, instead of reusing existing code. That’s not the problem.

The problem is that it was rewritten from scratch without taking advantage of the lessons of the past. It makes the same mistakes all over again.

In the late 1990s and early 2000s, we learned that parsing input is a problem. The traditional ad hoc approach you were taught in school is wrong. It’s wrong from an abstract theoretical point of view. It’s wrong from the practical point of view, error prone and leading to spaghetti code.
The first thing you need to unlearn is byte-swapping. I know that this was some sort epiphany you had when you learned network programming but byte-swapping is wrong. If you find yourself using a macro to swap bytes, like the be16toh() macro used in this code, then you are doing it wrong.
But, you say, the network byte-order is big-endian, while today’s Intel and ARM processors are little-endian. So you have to swap bytes, don’t you?
No. As proof of the matter I point to every other language other than C/C++. They don’t don’t swap bytes. Their internal integer format is undefined. Indeed, something like JavaScript may be storing numbers as a floating points. You can’t muck around with the internal format of their integers even if you wanted to.
An example of byte swapping in the code is something like this:

In this code, it’s taking a buffer of raw bytes from the DHCPv6 packet and “casting” it as a C internal structure. The packet contains a two-byte big-endian length field, “option->len“, which the code must byte-swap in order to use.

Among the errors here is casting an internal structure over external data. From an abstract theory point of view, this is wrong. Internal structures are undefined. Just because you can sort of know the definition in C/C++ doesn’t change the fact that they are still undefined.
From a practical point of view, this leads to confusion, as the programmer is never quite clear as to the boundary between external and internal data. You are supposed to rigorously verify external data, because the hacker controls it. You don’t keep double-checking and second-guessing internal data, because that would be stupid. When you blur the lines between internal and external data, then your checks get muddled up.
Yes you can, in C/C++, cast an internal structure over external data. But just because you can doesn’t mean you should. What you should do instead is parse data the same way as if you were writing code in JavaScript. For example, to grab the DHCP6 option length field, you should write something like:

The thing about this code is that you don’t know whether it’s JavaScript or C, because it’s both, and it does the same thing for both.

Byte “swapping” isn’t happening. We aren’t extracting an integer from a packet, then changing it’s internal format. Instead, we are extracting two bytes and combining them. This description may seem like needless pedantry, but it’s really really important that you grok this. For example, there is no conditional macro here that does one operation for a little-endian CPU, and another operation for a big-endian CPU — it does the same thing for both CPUs. Whatever words you want to use to describe the difference, it’s still profound and important.
The other thing that you shouldn’t do, even though C/C++ allows it, is pointer arithmetic. Again, it’s one of those epiphany things C programmers remember from their early days. It’s something they just couldn’t grasp until one day they did, and then they fell in love with it. Except it’s bad. The reason you struggled to grok it is because it’s stupid and you shouldn’t be using it. No other language has it, because it’s bad.
I mean, back in the day, it was a useful performance optimization. Iterating through an array can be faster adding to pointer than incrementing an index. But that’s virtually never the case today, and it just leads to needless spaghetti code. It leads to convoluted constructions like the following at the heart of this bug where you have to both do arithmetic on the pointer as well as on the length which you are checking against. This nonsense leads to confusion and ultimately, buffer overflows.

In a heckofalot of buffer overflows over the years, there’s a construct just like this lurking near the bug. If you are going to do a rewrite of code, then this is a construct you need to avoid. Just say no to pointer arithmetic.

In my code, you see a lot of constructs where it’s buf, offset, and length. The buf variable points to the start of the buffer and is never incremented. The length variable is the max length of the buffer and likewise never changes. It’s the offset variable that is incremented throughout.
Because of simplicity, buffer overflow checks become obvious, as it’s always “offset + x < length“, and easy to verify. In contrast, here is the fix for the DHCPv6 buffer overflow. That this is checking for an external buffer overflow is less obvious:

Now let’s look at that error code. That’s not what ENOBUFS really means. That’s an operating system error code that has specific meaning about kernel buffers. Overloading it for userland code is inappropriate.

That argument is a bit pedantic I grant you, but that’s only the start. The bigger issue is that it’s identifying the symptom not the problem. The ultimate problem is that the code failed to sanitize the original input, allowing the hacker to drive computation this deep in the system. The error isn’t that the buffer is too small to hold the output, the original error is that the input was too big. Imagine if this gets logged and the sysadmin reviewing dmesg asks themselves how they can allocate bigger buffers to the DHCP6 daemon. That is entirely the wrong idea.
Again, we go back to lessons of 20 years that this code ignored, the necessity of sanitizing input.
Now let’s look at assert(). This is a feature in C that we use to double-check things in order to catch programming mistakes early. An example is the code below, which is checking for programming mistakes where the caller of the function may have used NULL-pointers:

This is pretty normal, but now consider this other use of assert().

This isn’t checking errors by the programmer here, but is instead validating input. That’s not what you are supposed to use asserts for. This are very different things. It’s a coding horror that makes you shriek and run away when you see it. In my fact, that’s my Halloween costume this year, using asserts to validate network input.

This reflects a naive misunderstanding by programmers who don’t understand the difference between out-of-band checks validating the code, and what the code is supposed to be doing validating input. Like the buffer overflow check above, EINVAL because a programmer made a mistake is a vastly different error than EINVAL because a hacker tried to inject bad input. These aren’t the same things, they aren’t even in the same realm.
Conclusion

Rewriting old code is a good thing — as long as you are fixing the problems of the past and not repeating them. We have 20 years of experience with what goes wrong in network code. We have academic disciplines like langsec that study the problem. We have lots of good examples of network parsing done well. There is really no excuse for code that is of this low quality.
This code has no redeeming features. It must be thrown away and rewritten yet again. This time by an experienced programmer who know what error codes mean, how to use asserts properly, and most of all, who has experience at network programming.

Masscan as a lesson in TCP/IP

Post Syndicated from Robert Graham original https://blog.erratasec.com/2018/10/masscan-as-lesson-in-tcpip.html

When learning TCP/IP it may be helpful to look at the masscan port scanning program, because it contains its own network stack. This concept, “contains its own network stack“, is so unusual that it’ll help resolve some confusion you might have about networking. It’ll help challenge some (incorrect) assumptions you may have developed about how networks work.

For example, here is a screenshot of running masscan to scan a single target from my laptop computer. My machine has an IP address of 10.255.28.209, but masscan runs with an address of 10.255.28.250. This works fine, with the program contacting the target computer and downloading information — even though it has the ‘wrong’ IP address. That’s because it isn’t using the network stack of the notebook computer, and hence, not using the notebook’s IP address. Instead, it has its own network stack and its own IP address.
At this point, it might be useful to describe what masscan is doing here. It’s a “port scanner”, a tool that connects to many computers and many ports to figure out which ones are open. In some cases, it can probe further: once it connects to a port, it can grab banners and version information.
In the above example, the parameters to masscan used here are:
  • -p80 : probe for port “80”, which is the well-known port assigned for web-services using the HTTP protocol
  • –banners : do a “banner check”, grabbing simple information from the target depending on the protocol. In this case, it grabs the “title” field from the HTML from the server, and also grabs the HTTP headers. It does different banners for other protocols.
  • –source-ip 10.255.28.250 : this configures the IP address that masscan will use
  • 172.217.197.113 : the target to be scanned. This happens to be a Google server, by the way, though that’s not really important.
Now let’s change the IP address that masscan is using to something completely different, like 1.2.3.4. The difference from the above screenshot is that we no longer get any data in response. Why is that?
The answer is that the routers don’t know how to send back the response. It doesn’t go to me, it goes to whoever owns the real address 1.2.3.4. If you visualize the Internet, the subnetworks are on the edges. The routers in between examine the destination address of each packet and route it in the proper direction. You can send packets from 1.2.3.4 from anywhere in the network, but responses will always go back to the proper owner of that address.
Thus, masscan can spoof any address it wants, but if it’s an address that isn’t on the local subnetwork, then it’s never going to see the response — the response is going to go back to the real owner of the address. By the way, I’ve made this mistake before. When doing massive scans of the Internet, generating billions of packets, I’ve accidentally typed the wrong source address. That meant I saw none of the responses — but the hapless owner of that address was inundated with replies. Oops.
So let’s consider what masscan does when you use –source-ip to set its address. It does only three things:
  • Uses that as the source address in the packets it sends.
  • Filters incoming packets to make sure they match that address.
  • Responds to ARP packets for that address.
Remember that on the local network, communication isn’t between IP addresses but between Ethernet/WiFi addresses. IP addresses are for remote ends of the network, MAC addresses are how packets travel across the local network. It’s like when you send your kid to grab the mail from the mailbox: the kid is Ethernet/WiFi, the address on the envelope is the IP address.
In this case, when masscan transmits packets to the local router, it needs to first use ARP to find the router’s MAC address. Likewise, when the router receives a response from the Internet destined for masscan, it must first use ARP to discover the MAC address masscan is using.
As you can see in the picture at the top of this post, the MAC address of the notebook computer’s WiFi interface is 14:63:a3:11:2d:d4. Therefore, when masscan see’s an ARP request for 10.255.28.250, it must respond back with that MAC address.
These three steps should impress upon you that there’s not actually a lot that any operating system does with the IP address assigned to it. We imagine there is a lot of complicated code involved. In truth, there isn’t — there’s only a few simple things the operating system does with the address.
Moreover, this should impress upon you that the IP address is a property of the network not of the operating system. It’s what the network uses to route packets to you, and the operating system has very little control over which addresses will work and which ones don’t. The IP address isn’t the name or identity of the operating system. It’s like how your postal mailing address isn’t you, isn’t your identity, it’s simply where you live, how people can reach you.
Another thing to notice is the difference between phone numbers and addresses. Your IP address depends upon your location. If you move your laptop computer to a different location, you need a different IP address that’s meaningful for that location. In contrast, the phone has the same phone number wherever you travel in the world, even if you travel overseas. There have been decades of work in “mobile IP” to change this, but frankly, the Internet’s design is better, though that’s beyond the scope of this document.
That you can set any source address in masscan means you can play tricks on people. Spoof the source address of some friend you don’t like, and they’ll get all the responses. Moreover, angry people who don’t like getting scanned may complain to their ISP and get them kicked off for “abuse”.
To stop this sort of nonsense, a lot of ISPs do “egress filtering”. Normally, a router only examines the destination address of a packet in order to figure out the direction to route it. With egress filtering, it also looks at the source address, and makes sure it can route responses back to it. If not, it’ll drop the packet. I tested this by sending such spoofed addresses from 1.2.3.4 to a server of mine on the Internet, and found that I did not receive them. (I used the famous tcpdump program to filter incoming traffic looking for those packets).
By the way, masscan also has to ARP the local router. in order to find it’s MAC address before it can start sending packets to it. The first thing it does when it starts up is ARP the local router. It’s the reason there’s a short delay when starting the program. You can bypass this ARP by setting the router’s MAC address manually.
First of all, you have to figure out what the local router’s MAC address is. There are many ways of doing this, but the easiest is to run the arp command from the command-line, asking the operating system for this information. It, too, must ARP the router’s MAC address, and it keeps this information in a table.
Then, I can run masscan using this MAC address:
    masscan –interface en0 –router-mac ac:86:74:78:28:b2 –source-ip ….
In the above examples, while masscan has it’s own stack, it still requests information about the operating system’s configuration, to find things like the local router. Instead of doing this, we can run masscan completely indepedently from the operating system, specifying everything on the command line.
To do this, we have to configure all the following properties of a packet:
  • the network interface of my MacBook computer that I’m using
  • the destination MAC address of the local router
  • the source hardware address my MacBook computer 
  • the destination IP address of the target I’m scanning
  • the source IP address where the target can respond to
  • the destination port number of the port I am scanning
  • the source port number of the connection
An example is shown below. When I generated these screenshots I’m located on a different network, so the local addresses have changed from the examples above. Here is a screenshot of running masscan:
And here is a screenshot from Wireshark, a packet sniffer, that captures the packets involved:
As you can see from Wireshark, the very first packet is sent without any preliminaries, based directly on the command-line parameters. There is no other configuration of the computer or network involved.
When the response packet comes back in packet #4, the local router has to figure out the MAC address of where to send it, so it sends an ARP in packet #2, to which masscan responds in packet #3, after which that incoming packet can successfully be forwarded in packet #4.
After this, the TCP connection proceeds as normal, with a three way handshake, an HTTP request, an HTTP response, and so forth, with a couple extra ACK packets (noted in red) that happen because masscan is actually a bit delayed in responding to things.
What I’m trying to show here is again that what happens on the network, the packets that are sent, and how things deal with them, is a straightforward function of the initial starting conditions.
One thing about this example is that I had to set the source MAC address the same as my laptop computer. That’s because I’m using WiFi. There’s actually a bit of invisible setup here where my laptop must connect to the access-point. The access-point only knows the MAC address of the laptop, so that’s the MAC address masscan must use. Had this been Ethernet instead of WiFi, this invisible step wouldn’t be necessary, and I would be able to spoof any MAC address. In theory, I could also add a full WiFi stack to masscan so that it could create it’s own independent association with the WiFi access-point, but that’d be a lot of work.
Lastly, masscan supports a feature where you can specify a range of IP addresses. This is useful for a lot of reasons, such as stress-testing networks. An example:
    masscan –source-ip 10.1.10.100-10.1.10.64 ….
For every probe, it’ll choose a random IP address from that range. If you really don’t like somebody, you can use masscan and flood them with source addresses in the range 0.0.0.0-255.255.255.255. It’s one of the many “stupid pet tricks” you can do with masscan that have no purpose, but which comes from a straightforward applications of the principles of manually configuring things.
Likewise, masscan can be used in DDoS amplification attacks. Like addresses, you can configure payloads. Thus, you can set the –source-ip to that of your victim, a list of destination addresses consisting of amplifiers, and a payload that triggers the amplification. The victim will then be flooded with responses. It’s not something the program is specifically designed for, but usage that I can’t prevent, as again, it’s a straightforward application of the basic principles involved.
Conclusion

Learning about TCP/IP networking leads to confusion about the boundaries between what the operating-system does, and what the network does. Playing with masscan, which has it’s own network stack, helps clarify this.
You can download masscan source code at:

Some notes for journalists about cybersecurity

Post Syndicated from Robert Graham original https://blog.erratasec.com/2018/10/some-notes-for-journalists-about.html

The recent Bloomberg article about Chinese hacking motherboards is a great opportunity to talk about problems with journalism.

Journalism is about telling the truth, not a close approximation of the truth,  but the true truth. They don’t do a good job at this in cybersecurity.

Take, for example, a recent incident where the Associated Press fired a reporter for photoshopping his shadow out of a photo. The AP took a scorched-earth approach, not simply firing the photographer, but removing all his photographs from their library.

That’s because there is a difference between truth and near truth.

Now consider Bloomberg’s story, such as a photograph of a tiny chip. Is that a photograph of the actual chip the Chinese inserted into the motherboard? Or is it another chip, representing the size of the real chip? Is it truth or near truth?

Or consider the technical details in Bloomberg’s story. They are garbled, as this discussion shows. Something like what Bloomberg describes is certainly plausible, something exactly what Bloomberg describes is impossible. Again there is the question of truth vs. near truth.

There are other near truths involved. For example, we know that supply chains often replace high-quality expensive components with cheaper, lower-quality knockoffs. It’s perfectly plausible that some of the incidents Bloomberg describes is that known issue, which they are then hyping as being hacker chips. This demonstrates how truth and near truth can be quite far apart, telling very different stories.

Another example is a NYTimes story about a terrorist’s use of encryption. As I’ve discussed before, the story has numerous “near truth” errors. The NYTimes story is based upon a transcript of an interrogation of the hacker. The French newspaper Le Monde published excerpts from that interrogation, with details that differ slightly from the NYTimes article.

One the justifications journalists use is that near truth is easier for their readers to understand. First of all, that’s not justification for false hoods. If the words mean something else, then it’s false. It doesn’t matter if its simpler. Secondly, I’m not sure they actually are easier to understand. It’s still techy gobbledygook. In the Bloomberg article, if I as an expert can’t figure out what actually happened, then I know that the average reader can’t, either, no matter how much you’ve “simplified” the language.

Stories can solve this by both giving the actual technical terms that experts can understand, then explain them. Yes, it eats up space, but if you care about the truth, it’s necessary.

In groundbreaking stories like Bloomberg’s, the length is already enough that the average reader won’t slog through it. Instead, it becomes a seed for lots of other coverage that explains the story. In such cases, you want to get the techy details, the actual truth, correct, so that we experts can stand behind the story and explain it. Otherwise, going for the simpler near truth means that all us experts simply question the veracity of the story.

The companies mentioned in the Bloomberg story have called it an outright lie. However, the techniques roughly are plausible. I have no trouble believing something like that has happened at some point, that an intelligence organization subverted chips to hack BMC controllers in servers destined for specific customers. I’m sure our own government has done this at least once, as Snowden leaked documents imply. However, that doesn’t mean the Bloomberg story is truthful. We know they have smudged details. We know they’ve hyped details, like the smallness of the chips involved.

This is why I trust the high-tech industry press so much more than the mainstream press. Despite priding itself as the “newspaper of record”, on these technical issues the NYTimes is anything but. It’s the techy sites like Ars Technica and sometimes Wired that become the “paper of record” on things cyber. I mention this because David Sanger gets all the credit for Stuxnet reporting when he’s done a horrible job, while numerous techy sites have done the actual work figuring out what went on.

TCP/IP, Sockets, and SIGPIPE

Post Syndicated from Robert Graham original https://blog.erratasec.com/2018/10/tcpip-sockets-and-sigpipe.html

There is a spectre haunting the Internet — the spectre of SIGPIPE errors. It’s a bug in the original design of Unix networking from 1981 that is perpetuated by college textbooks, which teach students to ignore it. As a consequence, sometimes software unexpectedly crashes. This is particularly acute on industrial and medical networks, where security professionals can’t run port/security scans for fear of crashing critical devices.

An example of why this bug persists is the well-known college textbook “Unix Network Programming” by Richard Stevens. In section 5.13, he correctly describes the problem.

When a process writes to a socket that has received an RST, the SIGPIPE signal is sent to the process. The default action of this signal is to terminate the process, so the process must catch the signal to avoid being involuntarily terminated.

This description is accurate. The “Sockets” network APIs was based on the “pipes” interprocess communication when TCP/IP was first added to the Unix operating system back in 1981. This made it straightforward and comprehensible to the programmers at the time. This SIGPIPE behavior made sense when piping the output of one program to another program on the command-line, as is typical under Unix: if the receiver of the data crashes, then you want the sender of the data to also stop running. But it’s not the behavior you want for networking. Server processes need to continue running even if a client crashes.
But Steven’s description is insufficient. It portrays this problem as optional, that only exists if the other side of the connection is misbehaving. He never mentions the problem outside this section, and none of his example code handles the problem. Thus, if you base your code on Steven’s, it’ll inherit this problem and sometimes crash.
The simplest solution is to configure the program to ignore the signal, such as putting the following line of code in your main() function:
   signal(SIGPIPE, SIG_IGN);
If you search popular projects, you’ll find that this there solution most of the time, such as openssl.
But there is a problem with this approach, as OpenSSL demonstrates: it’s both a command-line program and a library. The command-line program handles this error, but the library doesn’t. This means that using the SSL_write() function to send encrypted data may encounter this error. Nowhere in the OpenSSL documentation does it mention that the user of the library needs to handle this.
Ideally, library writers would like to deal with the problem internally. There are platform-specific ways to deal with this. On Linux, an additional parameter MSG_NOSIGNAL can be added to the send() function. On BSD (including macOS), setsockopt(SO_NOSIGPIPE) can be configured for the socket when it’s created (after socket() or after accept()). On Windows and some other operating systems, the SIGPIPE isn’t even generated, so nothing needs to be done for those platforms.
But it’s difficult. Browsing through cross platform projects like curl, which tries this library technique, I see the following bit:
#ifdef __SYMBIAN32__
/* This isn’t actually supported under Symbian OS */
#undef SO_NOSIGPIPE
#endif
Later in the code, it will check whether SO_NOSIGPIPE is defined, and if it is, to use it. But that fails with Symbian because while it defines the constant in the source code, it doesn’t actually support it, so it then must be undefined.
So as you can see, solving this issue is hard. My recommendation for your code is to use all three techniques: signal(SIGPIPE), setsocktopt(SO_NOSIGPIPE), and send(MSG_NOSIGNAL), surrounded by the appropriate #ifdefs. It’s an annoying set of things you have to do, but it’s a non-optional thing you need to handle correctly, that must survive later programmers who may not understand this issue.
Now let’s talk abstract theory, because it’s important to understanding why Stevens’ description of SIGPIPE is wrong. The #1 most important theoretical concept in network programming is this:

Hackers control input.

What that means is that if input can go wrong, then it will — because eventually a hacker will discover your trusting of input and create the necessary input to cause something bad to happen, such as crashing your program, or taking remote control of it.
The way Steven’s presents this SIGPIPE problem is as if it’s a bug in the other side of the connection. A correctly written program on the other side won’t generate this problem, so as long as you have only well-written peers to deal with, then you’ll never see this. In other words, Stevens trusts input isn’t created by hackers.
And that’s indeed what happens in industrial control networks (factories, power plants, hospitals, etc.). These are tightly controlled networks where the other side of the connection is by the same manufacturer. Nothing else is allowed on the network, so bugs like this never happen.
Except that networks are never truly isolated like this. Once a hacker breaks into the network, they’ll cause havoc.
Worse yet, other people may have interest in the network. Security professionals, who want to stop hackers, will run port/vuln scanners on the network. These will cause unexpected input, causing these devices to crash.
Thus we see how this #1 principle gets corrupted, from Stevens on down. Stevens’ textbook teaches it’s the peer’s problem, a bug in the software on the other side of the connection. This then leads to industrial networks being based on this principle, as the programmers were taught in the university. This leads to persistent, intractable vulnerabilities to hackers in these networks. Not only are they vulnerable now, they can’t be fixed, because we can’t scan for vulnerabilities in order to fix them.
In this day and age of “continuous integration”, programmers are interested not only in solving this in their code, but solving this in their unit/regression test suites. In the modern perspective, until you can create a test that exercises this bug, it’s not truly fixed.
I’m not sure how to write code that adequately does this. It’s not straightforward generating RSTs from the Sockets API, especially at the exact point you need them. There’s also timing issues, where you may need to do something a million times repeatedly to just to get the timing right.
For example, I have a sample program that calls send() as fast as it can until it hit the limit on how much this side can buffer, and then closes the socket, causing a reset to be sent. For my simple “echo” server trying to echo back everything it receives, this will cause a SIGPIPE condition.
However, when testing a webserver, this may not work. A typical web server sends a short amount of data, so the send() has returned before you can get a RST packet sent. The web server software you are testing needs to be sending a large enough response that it’ll keep sending until it hits this condition. You may need to run the client program trying to generate this error a ton of times until just the right conditions are met.
Conclusion

I don’t know of any network programming textbook I like. They all tend to perpetuate outdated and incomplete information. That SIGPIPE is ignored so completely is a major cause of problems on the Internet.
To summarize: your code must deal with this. The most appropriate solution is signal(SIGPIPE) at the top of your program. If that doesn’t work for you, then you may be able to use pthread_sigprocmask() for just the particular threads doing network traffic. Otherwise, you need the more platform specific methods for BSD and Linux that deal with this at the socket or function call level.
It persists because it’s a bug in the original Sockets definition from 1981, it’s not properly described by textbook, it escapes testing, and there is a persistent belief that if a program receives bad input, it’s the sender’s responsibility to fix, rather than the receiver’s responsibility.

Election interference from Uber and Lyft

Post Syndicated from Robert Graham original https://blog.erratasec.com/2018/10/election-interference-from-uber-and-lyft.html

Almost nothing can escape the taint of election interference. A good example is the announcements by Uber and Lyft that they’ll provide free rides to the polls on election day. This well-meaning gesture nonetheless calls into question how this might influence the election.

“Free rides” to the polls is a common thing. Taxi companies have long offered such services for people in general. Political groups have long offered such services for their constituencies in particular. Political groups target retirement communities to get them to the polls, black churches have long had their “Souls to the Polls” program across the 37 states that allow early voting on Sundays.

But with Uber and Lyft getting into this we now have concerns about “big data”, “algorithms”, and “hacking”.

As the various Facebook controversies have taught us, these companies have a lot of data on us that can reliably predict how we are going to vote. If their leaders wanted to, these companies could use this information in order to get those on one side of an issue to the polls. On hotly contested elections, it wouldn’t take much to swing the result to one side.

Even if they don’t do this consciously, their various algorithms (often based on machine learning and AI) may do so accidentally. As is frequently demonstrated, unconscious biases can lead to real world consequences, like facial recognition systems being unable to read Asian faces.

Lastly, it makes these companies prime targets for Russian hackers, who may take all these into account when trying to muck with elections. Or indeed, to simply claim that they did in order to call the results into question. Though to be fair, Russian hackers have so many other targets of opportunity. Messing with the traffic lights of a few cities would be enough to swing a presidential election, specifically targeting areas with certain voters with traffic jams making it difficult for them to get to the polls.

Even if it’s not “hackers” as such, many will want to game the system. For example, politically motivated drivers may choose to loiter in neighborhoods strongly on one side or the other, helping the right sorts of people vote at the expense of not helping the wrong people. Likewise, drivers might skew the numbers by deliberately hailing rides out of opposing neighborhoods and taking them them out of town, or to the right sorts of neighborhoods.

I’m trying to figure out which Party this benefits the most. Let’s take a look at rider demographics to start with, such as this post. It appears that income levels and gender are roughly evenly distributed.

Ridership is skewed urban, with riders being 46% urban, 48% suburban, and 6% rural. In contrast, US population is 31% urban, 55% suburban, and 15% rural. Giving the increasing polarization among rural and urban voters, this strongly skews results in favor of Democrats.

Likewise, the above numbers show that Uber ridership is strongly skewed to the younger generation, with 55% of the riders 34 and younger. This again strongly skews “free rides” by Uber and Lyft toward the Democrats. Though to be fair, the “over 65” crowd has long had an advantage as the parties have fallen over themselves to bus people from retirement communities to the polls (and that older people can get free time on weekdays to vote).

Even if you are on the side that appears to benefit, this should still concern you. Our allegiance should first be to a robust and fair voting system, and to our Party second. I mention this because the increased polarization of our politics seems to favor those (from both sides) who don’t mind watching the entire system burn as long as their Party wins.

Right now, such programs are probably going to have a small impact. But the future is trending toward fewer individually owned cars and more dependence on services like Uber and Lyft. In a few years, big-data, machine learning algorithms, and hackers may be able to strongly influence elections.

Notes on the UK IoT cybersec "Code of Practice"

Post Syndicated from Robert Graham original https://blog.erratasec.com/2018/10/notes-on-uk-iot-cybersec-code-of.html

The British government has released a voluntary “Code of Practice” for securing IoT devices. I thought I’d write some notes on it.

First, the good parts

Before I criticize the individual points, I want to praise if for having a clue. So many of these sorts of things are written by the clueless, those who want to be involved in telling people what to do, but who don’t really understand the problem.
The first part of the clue is restricting the scope. Consumer IoT is so vastly different from things like cars, medical devices, industrial control systems, or mobile phones that they should never really be talked about in the same guide.
The next part of the clue is understanding the players. It’s not just the device that’s a problem, but also the cloud and mobile app part that relates to the device. Though they do go too far and include the “retailer”, which is a bit nonsensical.
Lastly, while I’m critical of most all the points on the list and how they are described, it’s probably a complete list. There’s not much missing, and the same time, it includes little that isn’t necessary. In contrast, a lot of other IoT security guides lack important things, or take the “kitchen sink” approach and try to include everything conceivable.

1) No default passwords

Since the Mirai botnet of 2016 famously exploited default passwords, this has been at the top of everyone’s list. It’s the most prominent feature of the recent California IoT law. It’s the major feature of federal proposals.
But this is only a superficial understanding of what really happened. The issue wasn’t default passwords so much as Internet-exposed Telnet.
IoT devices are generally based on Linux which maintains operating-system passwords in the /etc/passwd file. However, devices almost never use that. Instead, the web-based management interface maintains its own password database. The underlying Linux system is vestigial like an appendix and not really used.
But these devices exposed Telnet, providing a path to this otherwise unused functionality. I bought several of the Mirai-vulnerable devices, and none of them used /etc/passwd for anything other than Telnet.
Another way default passwords get exposed in IoT devices is through debugging interfaces. Manufacturers configure the system one way for easy development, and then ship a separate “release” version. Sometimes they make a mistake and ship the development backdoors as well. Programmers often insert secret backdoor accounts into products for development purposes without realizing how easy it is for hackers to discover those passwords.
The point is that this focus on backdoor passwords is misunderstanding the problem. Device makers can easily believe they are compliant with this directive while still having backdoor passwords.
As for the web management interface, saying “no default passwords” is useless. Users have to be able to setup the device the first time, so there has to be some means to connect to the device without passwords initially. Device makers don’t know how to do this without default passwords. Instead of mindless guidance of what not to do, a document needs to be written that explains how devices can do this both securely as well as easy enough for users to use.
Humorously, the footnotes in this section do reference external documents that might explain this, but they are the wrong documents, appropriate for things like website password policies, but inappropriate for IoT web interfaces. This again demonstrates how they have only a superficial understanding of the problem.

2) Implement a vulnerability disclosure policy

This is a clueful item, and it should be the #1 item on every list.
Though they do add garbage on top of this, but demanding companies respond in a “timely manner”, but overall this isn’t a bad section.

3) Keep software updated

This is another superficial understanding of the problem.
Software patching works for desktop and mobile phones because they have interfaces the user interacts with, ones that can both notify the user of a patch as well as the functionality to apply it. IoT devices are usually stuck in a closet somewhere without such interfaces.
Software patching works for normal computers because they sell for hundreds of dollars and thus have sufficient memory and storage to reliably do updates. IoT devices sell for cut-throat margins and have barely enough storage to run. This either precludes updates altogether, or at least means the update isn’t reliable, that upon every update, a small percentage of customer devices will be “bricked”, rendered unusable. Adding $1 for flash memory to a $30 device is not a reasonable solution to the problem.
Software patching works for software because of its enormous margins and longevity. A software product is basically all profit. The same doesn’t apply to hardware, where devices are sold with slim margins. Device makers have a hard time selling them for more because there are always no-named makers of almost identical devices in Shenzen willing to undercut them. (Indeed, looking at Mirai, it appears that was the majority of infected devices, not major brands, but no-named knock-offs). 
The document says that device makers need to publish how long the device will be supported. This ignores the economics of this. Devices makers cannot know how long they will support a device. As long as they are selling new ones, they’ve got incentive and profits to keep supplying updates. After that, they don’t. There’s really no way for them to predict the long term market success of their devices.
Guarantees cost money. If they guarantee security fixes for 10 years, then that’s a liability they have to account for on their balance sheet. It’s a huge risk: if the product fails to sell lots of units, then they are on the hook for a large cost without the necessary income to match it.
Lastly, the entire thing is a canard. Users rarely update firmware for devices. Blaming vendors for not providing security patches/updates means nothing without blaming users for not applying them.

4) Securely store credentials and security-sensitive data

Like many guides, this section makes the superficial statement “Hard-coded credentials in device software are not acceptable”. The reason this is silly is because public-keys are a “credential”, and you indeed want “hard-coded” public-keys. Hard-coded public-key credentials is how you do other security functions, like encrypted and signature verification.
This section tells device makers to use the trusted-enclave features like those found on phones, but this is rather silly. For one thing, that’s a feature of only high-end CPUs, not the low-end CPUs found in such devices. For another thing, IoT devices don’t really contain anything that needs that level of protection.
Storing passwords in clear-text on the device is almost certain adequate security, and this section can be ignored.

5) Communicate securely

In other words, use SSL everywhere, such as on the web-based management interface.
But this is only a superficial understanding of how SSL works. You (generally) can’t use SSL for devices because there’s no secure certificate on the device. It forces users to bypass nasty warnings in the browser, which hurts the entire web ecosystem. Some IoT devices do indeed try to use SSL this way, and it’s bad, very bad.
On the other hand, IoT devices can and should use SSL when connecting outbound to the cloud.

6) Minimise exposed attack surfaces

This is certainly a good suggestion, but it’s a platitude rather than an action item. IoT devices already minimize as much as they can in order to reduce memory/storage requires. Where this is actionable requires subtler understanding. A lot of exposed attack services come from accidents. 
A lot of other exposed attack surfaces come about because device makers know no better way. Actual helpful, meaning advice would consist of telling them what to do in order to solve problems, rather than telling them what not to do.
The reason Mirai-devices exposed Telnet was for things like “remote factory reset”. Mirai infected mostly security cameras which don’t have factory reset buttons. That’s because they are located high up out of reach, or if they are in reach, they don’t want to allow the public to press the factory reset button. Thus, doing a factory reset meant doing it remotely. That appears to be the major reason for Telnet and “hardcoded passwords”, to allow remote factory reset. Instead of telling them not to expose Telnet, you need a guide explaining how to securely do remote factory resets.
This guide discussed “ports”, but the reality is that the attack surface in the web-based management interface on port 80 is usually more than all other ports put together. Focusing on “ports” reflects a superficial understanding of the problem.

7) Ensure software integrity

The guide says “Software on IoT devices should be verified using secure boot
mechanisms”. No, they shouldn’t be. In the name of security, they should do the opposite.
First of all, getting “secure boot” done right is extraordinarily difficult. Apple does it the best with their iPhone and still they get it wrong. For another thing, it’s expensive. Like trusted enclaves in processors, most of the cheap low-end processors used in IoT don’t support it.
But the biggest issue is that you don’t want it. “Secure boot” means the only operating system the device can boot comes from the vendor, which will eventually stop supporting the product, making it impossible to fix any security problem. Not having secure boot means that customers can still be able to patch bugs without the manufacturer’s help.
Instead of secure boot, device makers should do the opposite and make it easy for customers to build their own software. They are required to do so under the GNU Public License anyway. That doesn’t mean open-sourcing everything, they can still provide their private code as binaries. But they should allow users to fix any bug in open-source and repackage a new firmware update.

8) Ensure that personal data is protected

I suppose giving the GDPR, this section is required, but GDPR is a pox on the Internet.

9) Make systems resilient to outages

Given the recent story of Yale locks locking people out of their houses due to a system outage, this seems like an obviously good idea.
But it should be noted that this is hard. Obviously such a lock should be resilient if the network connection is down, or their servers have crashed. But what happens when such a lock can contact their servers, but some other component within their organization has crashed, such that the servers give unexpected responses, neither completely down, but neither completely up and running, either?
We saw that in the Mirai attacks against Dyn. It left a lot servers up and running, but took down on some other component that those servers relied upon, leaving things in an intermediate state that was neither unfunctional nor completely functional.
It’s easy to stand on a soapbox and proclaim devices need to be resilient, but this is unhelpful. What would instead be helpful is a catalog of failures that IoT will typically experience.

10) Monitor system telemetry data

Security telemetry is a desirable feature in general. When a hack happens, you want to review logfiles to see how it happened. This item reflects various efforts to come up with such useful information
But again we see something so devoid of technical details as to be useless. Worse, it’s going to be exploited by others, such as McAffee wanting you to have anti-virus on TV sets, which is an extraordinarily bad idea.

11) Make it easy for consumers to delete personal data

This is kinda silly in that the it’s simply a matter of doing a “factory reset”. Having methods to delete personal details other than factory resets is bad.
The useful bit of advise is that factory resets don’t always “wipe” information, they just “forget” it in a way that can be recovered. Thus, we get printers containing old documents and voting machines with old votes.
On the other hand, this is a guide for “consumer IoT”, so just the normal factory reset is probably sufficient, even if private details can be gleaned.

12) Make installation and maintenance of devices easy

Of course things should be easy, everyone agrees on this. The problem is they don’t know how. Companies like Microsoft and Apple spend billions on this problem and still haven’t cracked it.
My home network WiFi password uses quotes as punctuation to improve security. The Amazon Echo app uses Bluetooth to pair with the device and set which password to use for WiFi. This is well done from a security point of view.
However, their app uses an input field that changes quotes to curly-quotes making it impossible to type in the password. I instead had to go to browser, type the password in the URL field, copy it, then go back to the Alexa app and paste it into the field. Then I could get things to work.
Amazon is better at making devices easy and secure with Echo and they still get things spectacularly wrong.

13) Validate input data

Most security vulnerabilities are due to improper validation of input data. However, “validate input data” is stupid advice. It’s like how most phishing attacks come from strangers, but how telling people to not open emails from strangers is stupid advice. In both cases, it’s a superficial answer that doesn’t really understand how the problem came about.
Let’s take PHP and session cookies, for example. A lot of programmers think the session identifier in PHP is some internal feature of PHP. They therefore trust it, because it isn’t input. They don’t perceive how it’s not internal to PHP, but external, part of HTTP, and something totally hackable by hackers.
Or take the famous Jeep hacker where hackers were able to remotely take control of the car and do mischievous things like turn it off on the highway. The designers didn’t understand how the private connection to the phone network was in fact “input” coming from the Internet. And then there was data from the car’s internal network, which wasn’t seen as “input” from an external source.
Then there is the question of what “validation” means. A lot of programmers try to solve SQL injection by “blacklisting” known bad characters. Hackers are adept at bypassing this, using other bad characters, especially using Unicode. Whitelisting known good characters is a better solution. But even that is still problematic. The proper solution to SQL injection isn’t “input validation” at all, but using “parameterized queries” that don’t care about input.

Conclusion

Like virtually every other guide, this one is based upon platitudes and only a superficial understanding of the problem. It’s got more clue than most, but is still far from something that could actually be useful. The concept here is virtue signaling, declaring what would be virtuous and moral for an IoT device, rather than something that could be useful to device makers in practice.