Post Syndicated from Unknown original https://lcamtuf.blogspot.com/2014/09/bash-bug-apply-unofficial-patch-now.html

OK, rebuild bash and deploy Florian’s unofficial patch or its now-upstream version now. If you’re a distro maintainer, please consider doing the same.

My previous post has more information about the original vulnerability (CVE-2014-6271). It also explains Tavis’ and my original negative sentiment toward the original upstream patch. In short, the revised code did not stop bash from parsing the code seen in potentially attacker-controlled, remotely-originating environmental variables. Instead, the fix simply seeks to harden the parsing to prevent RCE. It relies on two risky assumptions:

• That spare for this one bug we’re fixing now, the process of parsing attacker-controlled functions is guaranteed to have no side effects on the subsequently executed trusted code.

• That the underlying parser, despite probably not being designed to deal with attacker-supplied inputs, is free from the usual range of C language bugs.

From the very early hours, we have argued on the oss-security mailing list that a more reasonable approach would be to shield the parser from remotely-originating strings. I proposed putting the function export functionality behind a runtime flag or using a separate, prefixed namespace for the exported functions – so that variables such as HTTP_COOKIE do not go through this code path at all. Unfortunately, we made no real progress on that early in the game.

Soon thereafter, people started to bump into additional problems in the parser code. The first assumption behind the patch – the one about the parsing process not having other side effects – was quickly proved wrong by Tavis, who came up with a code construct that would get the parser in an inconsistent state, causing bash to create a bogus file and mangle any subsequent code that /bin/sh is supposed to execute.

This was assigned CVE-2014-7169 and led to a round of high-profile press reports claiming that we’re still doomed, and people assigning the new bug CVSS scores all the way up to 11. The reality was a bit more nuanced: the glitch demonstrated by Tavis’ code is a bit less concerning, because it does not translate into a universally exploitable RCE – at least not as far as we could figure it out. Some uses of /bin/sh would be at risk, but most would just break in a probably-non-exploitable way. The maintainer followed with another patch that locked down this specific hole.

The second assumption started showing cracks, too. First came a report from Todd Sabin, who identified an static array overflow error when parsing more than ten stacked redirects. The bug, assigned CVE-2014-7186, would cause a crash, but given the nature of the underlying assignment, immediate exploitability seemed fairly unlikely. Another probably non-security off-by-one issue with line counting in loops cropped up shortly thereafter (CVE-2014-7187).

The two latter issues do not have an officially released upstream patch at that point, but they prompted Florian Weimer of Red Hat to develop an unofficial patch that takes a seemingly more durable approach that we argued for earlier on: putting function exports in a separate namespace. Florian’s fix effectively isolates the function parsing code from attacker-controlled strings in almost all the important use cases we can currently think of.

(One major outlier would be any solutions that rely on blacklisting environmental variables to run restricted shells or restricted commands as a privileged user – sudo-type stuff – but it’s a much smaller attack surface and a a very dubious security boundary to begin with.)

Well… so, to get to the point: I’ve been fuzzing the underlying function parser on the side – and yesterday, bumped into a new parsing issue (CVE-2014-6277) that is almost certainly remotely exploitable and made easier to leverage due to the fact that bash is seldom compiled with ASLR. I’ll share the technical details later on; for now, I sent the info to the maintainer of bash and to several key Linux distros. In general terms, it’s an attempt to access uninitialized memory leading to reads from, and then subsequent writes to, a pointer that is fully within attacker’s control. Here’s a pretty telling crash:

bash[3054]: segfault at 41414141 ip 00190d96 &#8230;<br />

Soon after posting this entry, I also bumped in the sixth and most severe issue so far, essentially permitting very simple and straightforward remote code execution (CVE-2014-6278) on the systems that are patched against the first bug. It’s a “put your commands here” type of a bug similar to the original report. I will post additional details in a couple of days to give people enough time to upgrade.

At this point, I very strongly recommend manually deploying Florian’s patch unless your distro is already shipping it. (Florian’s patch has been also finally included upstream shortly after I first posted this entry.)

From within the shell itself, the simplest way to check if you already have it installed would be:

_x='() { echo vulnerable; }&#8217; bash -c &#8216;_x 2>/dev/null || echo not vulnerable&#8217;<br />

If the command shows “vulnerable”, you don’t have the patch and you are still vulnerable to a (currently non-public) RCE, even if you applied the original one (or the subsequent upstream patch that addressed the issue found by Tavis).

Post Syndicated from Unknown original https://lcamtuf.blogspot.com/2014/09/quick-notes-about-bash-bug-its-impact.html

We spent a good chunk of the day investigating the now-famous bash bug (CVE-2014-6271), so I had no time to make too many jokes about it on Twitter – but I wanted to jot down several things that have been getting drowned out in the noise earlier in the day.

Let’s start with the nature of the bug. At its core, the problem caused by an obscure and little-known feature that allows bash programs to export function definitions from a parent shell to children shells, similarly to how you can export normal environmental variables. The functionality in action looks like this:

$ function foo { echo &#8220;hi mom&#8221;; }<br /> $ export -f foo<br /> $ bash -c &#8216;foo&#8217; # Spawn nested shell, call &#8216;foo&#8217;<br /> hi mom<br />

The behavior is implemented as a hack involving specially-formatted environmental variables: in essence, any variable starting with a literal “() {” will be dispatched to the parser just before executing the main program. You can see this in action here:

$ foo='() { echo &#8220;hi mom&#8221;; }&#8217; bash -c &#8216;foo&#8217;<br /> hi mom<br />

The concept of granting magical properties to certain values of environmental variables clashes with several ancient customs – most notably, with the tendency for web servers such as Apache to pass client-supplied strings in the environment to any subordinate binaries or scripts. Say, if I request a CGI or PHP script from your server, the env variables $HTTP_COOKIE and $HTTP_USER_AGENT will be probably initialized to the raw values seen in the original request. If the values happen to begin with “() {” and are ever seen by /bin/bash, events may end up taking an unusual turn.

And so, the bug we’re dealing with stems from the observation that trying to parse function-like strings received in HTTP_* variables could have some unintended side effects in that shell – namely, it could easily lead to your server executing arbitrary commands trivially supplied in a HTTP header by random people on the Internet.

With that out of the way, it is important to note that the today’s patch provided by the maintainer of bash does not stop the shell from trying to parse the code within headers that begin with “() {” – it merely tries to get rid of that particular RCE side effect, originally triggered by appending commands past the end of the actual function def. But even with all the current patches applied, you can still do this:

Cookie: () { echo &#8220;Hello world&#8221;; }<br />

…and witness a callable function dubbed HTTP_COOKIE() materialize in the context of subshells spawned by Apache; of course, the name will be always prefixed with HTTP_*, so it’s unlikely to clash with anything or be called by incident – but intuitively, it’s a pretty scary outcome.

In the same vein, doing this will also have an unexpected result:

Cookie: () { oops<br />

If specified on a request to a bash-based CGI script, you will see a scary bash syntax error message in your error log.

All in all, the fix hinges on two risky assumptions:

1. That the bash function parser invoked to deal with variable-originating function definitions is robust and does not suffer from the usual range of low-level C string parsing bugs that almost always haunt similar code – a topic that, when it comes to shells, hasn’t been studied in much detail before now. (In fact, I am aware of a privately made now disclosed report of such errors in the parser – CVE-2014-7186 and CVE-2014-7187.)

   Update (Sep 26): I also bumped into what seems to be a separate and probably exploitable use of an uninitialized pointer in the parser code; shared the details privately upstream.

2. That the parsing steps are guaranteed to have no global side effects within the child shell. As it happens, this assertion has been already proved wrong by Tavis (CVE-2014-7169); the side effect he found probably-maybe isn’t devastating in the general use case (at least until the next stroke of brilliance), but it’s certainly a good reason for concern.

   Update (Sep 26): Found a sixth and most severe issue that is essentially equivalent to the original RCE on all systems that only have the original, maintainer-provided patch.

Contrary to multiple high-profile reports, the original fix was not “broken” in the sense that there is no universal RCE exploit for it – but if I were a betting man, I would not bet on the patch holding up in the long haul (Update: as noted above, it did not hold up). A more reasonable solution would involve temporarily disabling function imports, putting them behind a runtime flag, or blacklisting some of the most dangerous variable patterns (e.g., HTTP_*); and later on, perhaps moving to a model where function exports use a distinct namespace while present in the environment.

What else? Oh, of course: the impact of this bug is an interesting story all in itself. At first sight, the potential for remote exploitation should be limited to CGI scripts that start with #!/bin/bash and to several other programs that explicitly request this particular shell. But there’s a catch: on a good majority of modern Linux systems, /bin/sh is actually a symlink to /bin/bash!

This means that web apps written in languages such as PHP, Python, C++, or Java, are likely to be vulnerable if they ever use libcalls such as popen() or system(), all of which are backed by calls to /bin/sh -c '...'. There is also some added web-level exposure through #!/bin/sh CGI scripts, <!--#exec cmd="..."> calls in SSI, and possibly more exotic vectors such as mod_ext_filter.

For the same reason, userland DHCP clients that invoke configuration scripts and use variables to pass down config details are at risk when exposed to rogue servers (e.g., on open wifi). A handful of MTAs, MUAs, or FTP server architectures may be also of concern – in particular, there are third-party reports of qmail installations being at risk. Finally, there is some exposure for environments that use restricted SSH shells (possibly including Git) or restricted sudo commands, but the security of such approaches is typically fairly modest to begin with.

Exposure on other fronts is possible, but probably won’t be as severe. The worries around PHP and other web scripting languages, along with the concern for userspace DHCP, are the most significant reasons to upgrade – and perhaps to roll out more paranoid patches, rather than relying solely on the two official ones. On the upside, you don’t have to worry about non-bash shells – and that covers a good chunk of embedded systems out there. In particular, contrary to several claims, Busybox should be fine.

Update (Sep 28): the previously-unofficial namespace isolation patch from Florian has eventually made it upstream. You should deploy that patch ASAP.

PS. As for the inevitable “why hasn’t this been noticed for 15 years” / “I bet the NSA knew about it” stuff – my take is that it’s a very unusual bug in a very obscure feature of a program that researchers don’t really look at, precisely because no reasonable person would expect it to fail this way. So, life goes on.

Post Syndicated from Unknown original https://lcamtuf.blogspot.com/2014/09/cve-2014-1564-uninitialized-memory-when.html

The recent release of Firefox 32 fixes another interesting image parsing issue found by american fuzzy lop: following a refactoring of memory management code, the past few versions of the browser ended up using uninitialized memory for certain types of truncated images, which is easily measurable with a simple <canvas> + toDataURL() harness that examines all the fuzzer-generated test cases.

In general, problems like that may leak secrets across web origins, or more prosaically, may help attackers bypass security measures such as ASLR. For a slightly more detailed discussion, check out this post.

Here’s a short proof-of-concept that should work if you haven’t updated to 32 yet:

• http://lcamtuf.coredump.cx/ffgif/

This is tracked as CVE-2014-1564, Mozilla bug 1045977. Several more should be coming soon.

Post Syndicated from Unknown original https://lcamtuf.blogspot.com/2014/08/binary-fuzzing-strategies-what-works.html

Successful fuzzers live and die by their fuzzing strategies. If the changes made to the input file are too conservative, the fuzzer will achieve very limited coverage. If the tweaks are too aggressive, they will cause most inputs to fail parsing at a very early stage, wasting CPU cycles and spewing out messy test cases that are difficult to investigate and troubleshoot.

Designing the mutation engine for a new fuzzer has more to do with art than science. But one of the interesting side effects of the design of american fuzzy lop is that it provides a rare feedback loop: you can carefully measure what types of changes to the input file actually result in the discovery of new branches in the code, and which ones just waste your time or money.

This data is particularly easy to read because the fuzzer also approaches every new input file by going through a series of progressively more complex, but exhaustive and deterministic fuzzing strategies – say, sequential bit flips and simple arithmetics – before diving into purely random behaviors. The reason for this is the desire to generate the simplest and most elegant test cases first; but the design also provides a very good way to quantify how much value each new strategy brings in to the table – and whether we need it at all.

The measurements of afl fuzzing efficiency for reasonably-sized test cases are remarkably consistent across a variety of real-world binary formats – anything ranging from image files (JPEG, PNG, GIF, WebP) to archives (gzip, xz, tar) – and because of this, I figured that sharing the data more broadly will be useful to folks who are working on fuzzers of their own. So, let’s dive in:

• Walking bit flips: the first and most rudimentary strategy employed by afl involves performing sequential, ordered bit flips. The stepover is always one bit; the number of bits flipped in a row varies from one to four. Across a large and diverse corpus of input files, the observed yields are:
  • Flipping a single bit: ~70 new paths per one million generated inputs,
  • Flipping two bits in a row: ~20 additional paths per million generated inputs,
  • Flipping four bits in a row: ~10 additional paths per million inputs.

  (Note that the counts for every subsequent pass include only the paths that could not have been discovered by the preceding strategy.)

  Of course, the strategy is relatively expensive, with each pass requiring eight execve() per every byte of the input file. With the returns are diminishing rapidly, afl stops after these three passes – and switches to a second, less expensive strategy past that point.

• Walking byte flips: a natural extension of walking bit flip approach, this method relies on 8-, 16-, or 32-bit wide bitflips with a constant stepover of one byte. This strategy discovers around ~30 additional paths per million inputs, on top of what could have been triggered with shorter bit flips.

  It should be fairly obvious that each pass takes approximately one execve() call per one byte of the input file, making it surprisingly cheap, but also limiting its potential yields in absolute terms.

• Simple arithmetics: to trigger more complex conditions in a deterministic fashion, the third stage employed by afl attempts to subtly increment or decrement existing integer values in the input file; this is done with a stepover of one byte. The experimentally chosen range for the operation is -35 to +35; past these bounds, fuzzing yields drop dramatically. In particular, the popular option of sequentially trying every single value for each byte (equivalent to arithmetics in the range of -128 to +127) helps very little and is skipped by afl.

  When it comes to the implementation, the stage consists of three separate operations. First, the fuzzer attempts to perform subtraction and addition on individual bytes. With this out of the way, the second pass involves looking at 16-bit values, using both endians – but incrementing or decrementing them only if the operation would have also affected the most significant byte (otherwise, the operation would simply duplicate the results of the 8-bit pass). The final stage follows the same logic, but for 32-bit integers.

  The yields for this method vary depending on the format – ranging from ~2 additional paths per million in JPEG to ~8 per million in xz. The cost is relatively high, averaging around 20 execve() calls per one byte of the input file – but can be significantly improved with only a modest impact on path coverage by sticking to +/- 16.

• Known integers: the last deterministic approach employed by afl relies on a hardcoded set of integers chosen for their demonstrably elevated likelihood of triggering edge conditions in typical code (e.g., -1, 256, 1024, MAX_INT-1, MAX_INT). The fuzzer uses a stepover of one byte to sequentially overwrite existing data in the input file with one of the approximately two dozen “interesting” values, using both endians (the writes are 8-, 16-, and 32-bit wide).

  The yields for this stage are between 2 and 5 additional paths per one million tries; the average cost is roughly 30 execve() calls per one byte of input file.

• Stacked tweaks: with deterministic strategies exhausted for a particular input file, the fuzzer continues with a never-ending loop of randomized operations that consist of a stacked sequence of:
  • Single-bit flips,
  • Attempts to set “interesting” bytes, words, or dwords (both endians),
  • Addition or subtraction of small integers to bytes, words, or dwords (both endians),
  • Completely random single-byte sets,
  • Block deletion,
  • Block duplication via overwrite or insertion,
  • Block memset.

  Based on a fair amount of testing, the optimal execution path yields appear to be achieved when the probability of each operation is roughly the same; the number of stacked operations is chosen as a power-of-two between 1 and 64; and the block size for block operations is capped at around 1 kB.

  The absolute yield for this stage is typically comparable or higher than the total number of execution paths discovered by all deterministic stages earlier on.

• Test case splicing: this is a last-resort strategy that involves taking two distinct input files from the queue that differ in at least two locations; and splicing them at a random location in the middle before sending this transient input file through a short run of the “stacked tweaks” algorithm. This strategy usually discovers around 20% additional execution paths that are unlikely to trigger using the previous operation alone.

  (Of course, this method requires a good, varied corpus of input files to begin with; afl generates one automatically, but for other tools, you may have to construct it manually.)

As you can see, deterministic block operations (duplication, splicing) are not attempted in an exhaustive fashion; this is because they generally require quadratic time (or worse) – so while their yields may be good for very short inputs, they degrade very quickly.

Well, that’s it! If you ever decide to try out afl, you can watch these and other cool stats on your screen in real time.

Post Syndicated from Unknown original https://lcamtuf.blogspot.com/2014/08/a-bit-more-about-american-fuzzy-lop.html

Fuzzing is one of the most powerful strategies for identifying security issues in real-world software. Unfortunately, it also offers fairly shallow coverage: it is impractical to exhaustively cycle through all possible inputs, so even something as simple as setting three separate bytes to a specific value to reach a chunk of unsafe code can be an insurmountable obstacle to a typical fuzzer.

There have been numerous attempts to solve this problem by augmenting the process with additional information about the behavior of the tested code. These techniques can be divided into three broad groups:

• Simple coverage maximization. This approach boils down to trying to isolate initial test cases that offer diverse code coverage in the targeted application – and them fuzzing them using conventional techniques.

• Control flow analysis. A more sophisticated technique that leverages instrumented binaries to focus the fuzzing efforts on mutations that generate distinctive sequences of conditional branches within the instrumented binary.

• Static analysis. An approach that attempts to reason about potentially interesting states within the tested program and then make educated guesses about the input values that could possibly trigger them.

The first technique is surprisingly powerful when used to pre-select initial test cases from a massive corpus of valid data – say, the result of a large-scale web crawl. Unfortunately, coverage measurements provide only a very simplistic view of the internal state of the program, making them less suited for creatively guiding the fuzzing process later on.

The latter two techniques are extremely promising in experimental settings. That said, in real-world applications, they are not only very slow, but frequently lead to irreducible complexity: most of the high-value targets will have a vast number of internal states and possible execution paths, and deciding which ones are interesting and substantially different from the rest is an extremely difficult challenge that, if not solved, usually causes the “smart” fuzzer to perform no better than a traditional one.

American fuzzy lop tries to find a reasonable middle ground between sophistication and practical utility. In essence, it’s a fuzzer that relies on a form of edge coverage measurements to detect subtle, local-scale changes to program control flow without having to perform complex global-scale comparisons between series of long and winding execution traces – a common failure point for similar tools.

In almost-plain English, the fuzzer does this by instrumenting every effective line of C or C++ code (or any other GCC-supported language) to record a tuple in the following format:

[ID of current code location], [ID of previously-executed code location]

The ordering information for tuples is discarded; the primary signal used by the fuzzer is the appearance of a previously-unseen tuple in the output dataset; this is also coupled with coarse magnitude count for tuple hit rate. This method combines the self-limiting nature of simple coverage measurements with the sensitivity of control flow analysis. It detects both explicit conditional branches, and indirect variations in the behavior of the tested app.

The output from this instrumentation is used as a part of a simple, vaguely “genetic” algorithm:

1. Load user-supplied initial test cases into the queue,

2. Take input file from the queue,

3. Repeatedly mutate the file using a balanced variety of traditional fuzzing strategies (see later),

4. If any of the generated mutations resulted in a new tuple being recorded by the instrumentation, add mutated output as a new entry in the queue.

5. Go to 2.

The discovered test cases are also periodically culled to eliminate ones that have been made obsolete by more inclusive finds discovered later in the fuzzing process. Because of this, the fuzzer is useful not only for identifying crashes, but is exceptionally effective at turning a single valid input file into a reasonably-sized corpus of interesting test cases that can be manually investigated for non-crashing problems, handed over to valgrind, or used to stress-test applications that are harder to instrument or too slow to fuzz efficiently. In particular, it can be extremely useful for generating small test sets that may be programatically or manually examined for anomalies in a browser environment.

(For a quick partial demo, click here.)

Of course, there are countless “smart” fuzzer designs that look good on paper, but fail in real-world applications. I tried to make sure that this is not the case here: for example, afl can easily tackle security-relevant and tough targets such as gzip, xz, lzo, libjpeg, libpng, giflib, libtiff, or webp – all with absolutely no fine-tuning and while running at blazing speeds. The control flow information is also extremely useful for accurately de-duping crashes, so the tool does that for you.

In fact, I spent some time running it on a single machine against libjpeg, giflib, and libpng – some of the most robust best-tested image parsing libraries out there. So far, the tool found:

• CVE-2013-6629: JPEG SOS component uninitialized memory disclosure in jpeg6b and libjpeg-turbo,

• CVE-2013-6630: JPEG DHT uninitialized memory disclosure in libjpeg-turbo,

• MSRC 0380191: A separate JPEG DHT uninitialized memory disclosure in Internet Explorer,

• CVE-2014-1564: Uninitialized memory disclosure via GIF images in Firefox,

• CVE-2014-1580: Uninitialized memory disclosure via <canvas> in Firefox,

• Chromium bug #398235, Mozilla bug #1050342: Probable library-related JPEG security issues in Chrome and Firefox (pending),

• PNG zlib API misuse bug in MSIE (DoS-only),

• Several browser-crashing images in WebKit browsers (DoS-only).

More is probably to come. In other words, you should probably try it out. The most significant limitation today is that the current fuzzing strategies are optimized for binary files; the fuzzer does:

• Walking bitflips – 1, 2, and 4 bits,

• Walking byte flips – 1, 2, and 4 bytes,

• Walking addition and subtraction of small integers – byte, word, dword (both endians),

• Walking insertion of interesting integers (-1, MAX_INT, etc) – byte, word, dword (both endians),

• Random stacked flips, arithmetics, block cloning, insertion, deletion, etc,

• Random splicing of synthetized test cases – pretty unique!

All these strategies have been specifically selected for an optimal balance between fuzzing cost and yields measured in terms of the number of discovered execution paths with binary formats; for highly-redundant text-based formats such as HTML or XML, syntax-aware strategies (template- or ABNF-based) will obviously yield better results. Plugging them into AFL would not be hard, but requires work.

Post Syndicated from Unknown original https://lcamtuf.blogspot.com/2014/06/boolean-algebra-with-css-when-you-can.html

Depending on how you look at it, CSS can be considered Turing-complete. But in one privacy-relevant setting – when styling :visited links – the set of CSS directives you can use is extremely limited, effectively letting you control not much more than the color of the text nested between <a href=…> and </a>. Can you perform any computations with that?

Well, as it turns out, you can – in a way. Check out this short write-up for a discussion on how to implement Boolean algebra by exploiting an interesting implementation-level artifact of CSS blending to steal your browsing history a bit more efficiently than before.

Vulnerability logo and vanity domain forthcoming – stay tuned.