Tag Archives: emulation

Wine 3.0 released

Post Syndicated from corbet original https://lwn.net/Articles/744741/rss

Version 3.0 of the
Wine Windows emulation layer has been released. “This release
represents a year of development effort and over 6,000 individual
changes.
” Most of the improvements seem to be around Direct3D
graphics, but it also now possible to package up Wine as an Android app;
see the release notes for
details.

Potential impact of the Intel ME vulnerability

Post Syndicated from Matthew Garrett original https://mjg59.dreamwidth.org/49611.html

(Note: this is my personal opinion based on public knowledge around this issue. I have no knowledge of any non-public details of these vulnerabilities, and this should not be interpreted as the position or opinion of my employer)

Intel’s Management Engine (ME) is a small coprocessor built into the majority of Intel CPUs[0]. Older versions were based on the ARC architecture[1] running an embedded realtime operating system, but from version 11 onwards they’ve been small x86 cores running Minix. The precise capabilities of the ME have not been publicly disclosed, but it is at minimum capable of interacting with the network[2], display[3], USB, input devices and system flash. In other words, software running on the ME is capable of doing a lot, without requiring any OS permission in the process.

Back in May, Intel announced a vulnerability in the Advanced Management Technology (AMT) that runs on the ME. AMT offers functionality like providing a remote console to the system (so IT support can connect to your system and interact with it as if they were physically present), remote disk support (so IT support can reinstall your machine over the network) and various other bits of system management. The vulnerability meant that it was possible to log into systems with enabled AMT with an empty authentication token, making it possible to log in without knowing the configured password.

This vulnerability was less serious than it could have been for a couple of reasons – the first is that “consumer”[4] systems don’t ship with AMT, and the second is that AMT is almost always disabled (Shodan found only a few thousand systems on the public internet with AMT enabled, out of many millions of laptops). I wrote more about it here at the time.

How does this compare to the newly announced vulnerabilities? Good question. Two of the announced vulnerabilities are in AMT. The previous AMT vulnerability allowed you to bypass authentication, but restricted you to doing what AMT was designed to let you do. While AMT gives an authenticated user a great deal of power, it’s also designed with some degree of privacy protection in mind – for instance, when the remote console is enabled, an animated warning border is drawn on the user’s screen to alert them.

This vulnerability is different in that it allows an authenticated attacker to execute arbitrary code within the AMT process. This means that the attacker shouldn’t have any capabilities that AMT doesn’t, but it’s unclear where various aspects of the privacy protection are implemented – for instance, if the warning border is implemented in AMT rather than in hardware, an attacker could duplicate that functionality without drawing the warning. If the USB storage emulation for remote booting is implemented as a generic USB passthrough, the attacker could pretend to be an arbitrary USB device and potentially exploit the operating system through bugs in USB device drivers. Unfortunately we don’t currently know.

Note that this exploit still requires two things – first, AMT has to be enabled, and second, the attacker has to be able to log into AMT. If the attacker has physical access to your system and you don’t have a BIOS password set, they will be able to enable it – however, if AMT isn’t enabled and the attacker isn’t physically present, you’re probably safe. But if AMT is enabled and you haven’t patched the previous vulnerability, the attacker will be able to access AMT over the network without a password and then proceed with the exploit. This is bad, so you should probably (1) ensure that you’ve updated your BIOS and (2) ensure that AMT is disabled unless you have a really good reason to use it.

The AMT vulnerability applies to a wide range of versions, everything from version 6 (which shipped around 2008) and later. The other vulnerability that Intel describe is restricted to version 11 of the ME, which only applies to much more recent systems. This vulnerability allows an attacker to execute arbitrary code on the ME, which means they can do literally anything the ME is able to do. This probably also means that they are able to interfere with any other code running on the ME. While AMT has been the most frequently discussed part of this, various other Intel technologies are tied to ME functionality.

Intel’s Platform Trust Technology (PTT) is a software implementation of a Trusted Platform Module (TPM) that runs on the ME. TPMs are intended to protect access to secrets and encryption keys and record the state of the system as it boots, making it possible to determine whether a system has had part of its boot process modified and denying access to the secrets as a result. The most common usage of TPMs is to protect disk encryption keys – Microsoft Bitlocker defaults to storing its encryption key in the TPM, automatically unlocking the drive if the boot process is unmodified. In addition, TPMs support something called Remote Attestation (I wrote about that here), which allows the TPM to provide a signed copy of information about what the system booted to a remote site. This can be used for various purposes, such as not allowing a compute node to join a cloud unless it’s booted the correct version of the OS and is running the latest firmware version. Remote Attestation depends on the TPM having a unique cryptographic identity that is tied to the TPM and inaccessible to the OS.

PTT allows manufacturers to simply license some additional code from Intel and run it on the ME rather than having to pay for an additional chip on the system motherboard. This seems great, but if an attacker is able to run code on the ME then they potentially have the ability to tamper with PTT, which means they can obtain access to disk encryption secrets and circumvent Bitlocker. It also means that they can tamper with Remote Attestation, “attesting” that the system booted a set of software that it didn’t or copying the keys to another system and allowing that to impersonate the first. This is, uh, bad.

Intel also recently announced Intel Online Connect, a mechanism for providing the functionality of security keys directly in the operating system. Components of this are run on the ME in order to avoid scenarios where a compromised OS could be used to steal the identity secrets – if the ME is compromised, this may make it possible for an attacker to obtain those secrets and duplicate the keys.

It’s also not entirely clear how much of Intel’s Secure Guard Extensions (SGX) functionality depends on the ME. The ME does appear to be required for SGX Remote Attestation (which allows an application using SGX to prove to a remote site that it’s the SGX app rather than something pretending to be it), and again if those secrets can be extracted from a compromised ME it may be possible to compromise some of the security assumptions around SGX. Again, it’s not clear how serious this is because it’s not publicly documented.

Various other things also run on the ME, including stuff like video DRM (ensuring that high resolution video streams can’t be intercepted by the OS). It may be possible to obtain encryption keys from a compromised ME that allow things like Netflix streams to be decoded and dumped. From a user privacy or security perspective, these things seem less serious.

The big problem at the moment is that we have no idea what the actual process of compromise is. Intel state that it requires local access, but don’t describe what kind. Local access in this case could simply require the ability to send commands to the ME (possible on any system that has the ME drivers installed), could require direct hardware access to the exposed ME (which would require either kernel access or the ability to install a custom driver) or even the ability to modify system flash (possible only if the attacker has physical access and enough time and skill to take the system apart and modify the flash contents with an SPI programmer). The other thing we don’t know is whether it’s possible for an attacker to modify the system such that the ME is persistently compromised or whether it needs to be re-compromised every time the ME reboots. Note that even the latter is more serious than you might think – the ME may only be rebooted if the system loses power completely, so even a “temporary” compromise could affect a system for a long period of time.

It’s also almost impossible to determine if a system is compromised. If the ME is compromised then it’s probably possible for it to roll back any firmware updates but still report that it’s been updated, giving admins a false sense of security. The only way to determine for sure would be to dump the system flash and compare it to a known good image. This is impractical to do at scale.

So, overall, given what we know right now it’s hard to say how serious this is in terms of real world impact. It’s unlikely that this is the kind of vulnerability that would be used to attack individual end users – anyone able to compromise a system like this could just backdoor your browser instead with much less effort, and that already gives them your banking details. The people who have the most to worry about here are potential targets of skilled attackers, which means activists, dissidents and companies with interesting personal or business data. It’s hard to make strong recommendations about what to do here without more insight into what the vulnerability actually is, and we may not know that until this presentation next month.

Summary: Worst case here is terrible, but unlikely to be relevant to the vast majority of users.

[0] Earlier versions of the ME were built into the motherboard chipset, but as portions of that were incorporated onto the CPU package the ME followed
[1] A descendent of the SuperFX chip used in Super Nintendo cartridges such as Starfox, because why not
[2] Without any OS involvement for wired ethernet and for wireless networks in the system firmware, but requires OS support for wireless access once the OS drivers have loaded
[3] Assuming you’re using integrated Intel graphics
[4] “Consumer” is a bit of a misnomer here – “enterprise” laptops like Thinkpads ship with AMT, but are often bought by consumers.

comment count unavailable comments

Christmas Special: The MagPi 52 is out now!

Post Syndicated from Lucy Hattersley original https://www.raspberrypi.org/blog/magpi-christmas-special/

The MagPi Christmas Special is out now.

For the festive season, the official magazine of the Raspberry Pi community is having a maker special. This edition is packed with fun festive projects!

The MagPi issue 52 cover

The MagPi issue 52

Click here to download the MagPi Christmas Special

Here are just some of the fun projects inside this festive issue:

  • Magazine tree: turn the special cover into a Christmas tree, using LED lights to create a shiny, blinky display
  • DIY decorations: bling out your tree with NeoPixels and code
  • Santa tracker: follow Santa Claus around the world with a Raspberry Pi
  • Christmas crackers: the best low-cost presents for makers and hackers
  • Yuletide game: build Sliders, a fab block-sliding game with a festive feel.

Sliders

A Christmas game from the MagPi No.52

Inside the MagPi Christmas special

If you’re a bit Grinchy when it comes to Christmas, there’s plenty of non-festive fun to be found too:

  • Learn to use VNC Viewer
  • Find out how to build a sunrise alarm clock
  • Read our in-depth guide to Amiga emulation
  • Discover the joys of parallel computing

There’s also a huge amount of community news this month. The MagPi has an exclusive feature on Pioneers, our new programme for 12- to 15-year-olds, and news about Astro Pi winning the Arthur Clarke Award.

The Pioneers

The MagPi outlines our new Pioneers programme in detail

After that, we see some of the most stylish projects ever. Inside is the beautiful Sisyphus table; that’s a moving work of art, a facial recognition door lock, and a working loom controlled by a Raspberry Pi.

The MagPi 52 Sisyphus Project Focus

The MagPi interviews the maker of this amazing Sisyphus table

If that wasn’t enough, we also have a big feature on adding sensors to your robots. These can be used to built a battle-bot ready for the upcoming Pi Wars challenge.

The MagPi team wishes you all a merry Christmas! You can grab The MagPi 52 in stores today: it’s in WHSmith, Tesco, Sainsbury’s, and Asda in the UK, and it will be in Micro Center and selected Barnes & Noble stores when it comes to the US. You can also buy the print edition online from our store, and it’s available digitally on our Android and iOS app.

Get a free Pi Zero
Want to make sure you never miss an issue? Subscribe today and get a Pi Zero bundle featuring the new, camera-enabled Pi Zero, and a cable bundle that includes the camera adapter.

If you subscribe to The MagPi before 7 December 2016, you will get a free Pi Zero in time for Christmas.

The post Christmas Special: The MagPi 52 is out now! appeared first on Raspberry Pi.

AWS Week in Review – September 5, 2016

Post Syndicated from Jeff Barr original https://aws.amazon.com/blogs/aws/aws-week-in-review-september-5-2016/

This is the third community-driven edition of the AWS Week in Review. Special thanks are due to the 15 internal and external contributors who helped to make this happen. If you would like to contribute, please take a look at the AWS Week in Review on GitHub.

Monday

September 5

Tuesday

September 6

Wednesday

September 7

Thursday

September 8

Friday

September 9

Saturday

September 10

Sunday

September 11

New & Notable Open Source

  • s3logs-cloudwatch is a Lambda function parsing S3 server access log files and putting extra bucket metrics in CloudWatch.
  • README.md is a curated list of AWS resources used to prepare for AWS certifications.
  • RedEye is a utility to monitor Redshift performance.
  • Dockerfile will build a Docker image, push it to the EC2 Container Registry, and deploy it to Elastic Beanstalk.
  • lambda-contact-form supports contact form posts from static websites hosted on S3/CloudFront.
  • dust is an SSH cluster shell for EC2.
  • aws-ssh-scp-connector is a utility to help connect to EC2 instances.
  • lambda-comments is a blog commenting system built with Lambda.

New SlideShare Presentations

New YouTube Videos

New Customer Stories

  • MYOB uses AWS to scale its infrastructure to support demand for new services and saves up to 30 percent by shutting down unused capacity and using Reserved Amazon EC2 Instances. MYOB provides business management software to about 1.2 million organizations in Australia and New Zealand. MYOB uses a wide range of AWS services, including Amazon Machine Learning to build smart applications incorporating predictive analytics and AWS CloudFormation scripts to create new AWS environments in the event of a disaster.
  • PATI Games needed IT solutions that would guarantee the stability and scalability of their game services for global market penetration, and AWS provided them with the most safe and cost-efficient solution. PATI Games is a Korean company primarily engaged in the development of games based on SNS platforms. AWS services including Amazon EC2, Amazon RDS (Aurora), and Amazon CloudFront enable PATI Games to maintain high reliability, decrease latency, and eventually boost customer satisfaction.
  • Rabbi Interactive scales to support a live-broadcast, second-screen app and voting system for hundreds of thousands of users, gives home television viewers real-time interactive capabilities, and reduces monthly operating costs by 60 percent by using AWS. Based in Israel, the company provides digital experiences such as second-screen apps used to interact with popular television shows such as “Rising Star” and “Big Brother.” Rabbi Interactive worked with AWS partner CloudZone to develop an interactive second-screen platform.

Upcoming Events

Help Wanted

Stay tuned for next week! In the meantime, follow me on Twitter and subscribe to the RSS feed.

Desktop Sense HAT emulator

Post Syndicated from David Honess original https://www.raspberrypi.org/blog/desktop-sense-hat-emulator/

If this post gives you a sense of déjà-vu it’s because, last month, we announced a web-based Sense HAT emulator in partnership with US-based startup Trinket.

Today, we’re announcing another Sense HAT emulator designed to run natively on your Raspberry Pi desktop, instead of inside a browser. Developed by Dave Jones, it’s intended for people who own a Raspberry Pi but not a Sense HAT. In the picture below, the sliders are used to change the values reported by the sensors while your code is running.

sense-emu

So, why do we need two versions?

  • For offline use, possibly the most common way Raspberry Pis are used in the classroom.
  • To accommodate the oldest 256 MB models of Raspberry Pi which cannot run the web version.
  • To allow you to integrate your Sense HAT program with any available Python modules, or other Raspberry Pi features such as the Camera Module.

The emulator will come pre-installed in the next Raspbian release but, for now, you can just install it by typing the commands below into a terminal window:

sudo apt-get update
sudo apt-get install python-sense-emu python3-sense-emu python-sense-emu-doc sense-emu-tools -y

You can then access it from the Desktop menu, under Programming.

The emulator closely simulates the Sense HAT hardware being attached to your Pi. You can read from the sensors or write to the LED matrix using multiple Python processes, for example.

sense-idle

Write your code in IDLE as before; there are also a number of examples that can be opened from the emulator’s built-in menu. If you then want to port your code to a physical Sense HAT, you just need to change

sense_emu

to

sense_hat

at the top of your program. Reverse this if you’re porting a physical Sense HAT program to the emulator, perhaps from one of our educational resources; this step isn’t required in the web version of the emulator.

sense-emu-prefs

There are a number of preferences that you can adjust to change the behaviour of the emulator, most notably sensor simulation, otherwise known as jitter. This costs some CPU time, and is disabled by default on the low-end Raspberry Pis, but it provides a realistic experience of how the hardware sensors would behave. You’ll see that the values being returned in your code drift according to the known error tolerances of the physical sensors used on the Sense HAT.

This emulator will allow more Raspberry Pi users to participate in future Astro Pi competitions without having to buy a Sense HAT: ideal for the classroom where 15 Sense HATs may be beyond the budget.

So, where do you start? If you’re new to the Sense HAT, you can just copy and paste many of the code examples from our educational resources, like this one. You can also check out our e-book Sense HAT Essentials. For a complete list of all the functions you can use, have a look at the Sense HAT API reference here.

You can even install this emulator on other types of Linux desktop, such as Ubuntu! For more information on how to do this, please visit the emulator documentation pages here.

The post Desktop Sense HAT emulator appeared first on Raspberry Pi.

Sense HAT emulator

Post Syndicated from David Honess original https://www.raspberrypi.org/blog/sense-hat-emulator/

Over the last few months, we’ve been working with US-based startup Trinket to develop a web-based emulator for the Sense HAT, the multipurpose add-on board for the Raspberry Pi which is also the core component of the Astro Pi units on the International Space Station. We wanted to provide a unique, free learning resource that brings the excitement of programming our space-qualified hardware to students, teachers, and others all over the world.

We’re delighted to announce its release today, and you can try it for yourself right now. Click the Run button below and see what happens!

trinket-logo

The emulator will allow more people to participate in future Astro Pi competitions – you’ll be able to join in without needing to own a Raspberry Pi computer or a Sense HAT.

British ESA Astronaut Tim Peake with an Astro Pi unit on the International Space Station

British ESA Astronaut Tim Peake with the Astro Pi. Image credit ESA

The new emulator builds on Trinket’s existing Python-in-browser platform, and provides the following features:

  • Virtual Sense HAT with environmental controls and joystick input
  • Full Python syntax highlighting
  • Contextual auto-complete
  • Intuitive error reporting and highlighting
  • Image upload
  • HTML page embedding
  • Social media integration
  • Project sharing via direct URL
  • Project download as zip (for moving to Raspberry Pi)
  • All major browsers supported

sense_hat_emu

The Sense HAT has temperature, pressure and humidity sensors, and can change its behaviour according to the values they report. The Sense HAT emulator has sliders you can move to change these values, so you can test how your code responds to environmental variables.

Part of a screenshot of the Astro Pi emulator, showing three silders with buttons that can be dragged to change the temperature, pressure and humidity that the virtual Sense HAT's sensors are reporting

You can move the sliders to change what the sensors are reporting

Code written in this emulator is directly portable to a physical Raspberry Pi with a Sense HAT without modification. This means any code you write can be run by the Astro Pi units on board the ISS! It is our hope that, within the next 12 months, code that has been written in the emulator will run in space. Look out for news on this, coming soon on the Astro Pi site!

We owe huge thanks to Trinket, who have been wonderful partners in this project. The development work has been completed in just over two months, and has been a huge collaborative effort from the start. The software relies heavily on open-source technology and a global community of developers who are committed to making the power of code more accessible to students.

A closed group of beta testers, made up of previous Astro Pi participants and Code Club champions, has been putting the emulator through its paces over recent weeks. We’re proud to say that we’ve just had a bug-free open beta over the weekend, and now we’re looking forward to seeing it used as widely as possible.

So, where do you start? If you’re new to the Sense HAT, you can just copy and paste a lot of the code examples from our educational resources like this one. You can also check out our e-book Sense HAT Essentials. For a complete list of all the functions you can use, have a look at the Sense HAT API reference here; please note that the IMU (movement-sensing) functions will be supported in a future update. Head over to the main Sense HAT emulator site to see loads of other cool examples of what’s possible. Flappy LED, anyone?

Don’t forget to share your projects!

The post Sense HAT emulator appeared first on Raspberry Pi.

Emulated atomic operations and real-time scheduling

Post Syndicated from Lennart Poettering original http://0pointer.net/blog/projects/atomic-rt.html

Unfortunately not all CPU architectures have native support for atomic
operations
, or only support a very limited subset. Most
prominently ARMv5 (and
older) hasn’t any support besides the most basic atomic swap
operation[1]. Now, more and more free code is starting to
use atomic operations and lock-free
algorithms
, one being my own project, PulseAudio. If you have ever done real-time
programming
you probably know that you cannot really do it without
support for atomic operations. One question remains however: what to
do on CPUs which support only the most basic atomic operations
natively?

On the kernel side atomic ops are very easy to emulate: just disable
interrupts temporarily, then do your operation non-atomically, and afterwards
enable them again. That’s relatively cheap and works fine (unless you are on SMP — which
fortunately you usually are not for those CPUs). The Linux
kernel does it this way and it is good. But what to do in user-space, where you cannot just go and disable interrupts?

Let’s see how the different userspace libraries/frameworks do it
for ARMv5, a very relevant architecture that only knows an atomic swap (exchange)
but no CAS
or even atomic arithmetics. Let’s start with an excerpt from glibc’s
atomic operations implementation for ARM
:

/* Atomic compare and exchange.  These sequences are not actually atomic;
   there is a race if *MEM != OLDVAL and we are preempted between the two
   swaps.  However, they are very close to atomic, and are the best that a
   pre-ARMv6 implementation can do without operating system support.
   LinuxThreads has been using these sequences for many years.  */

This comment says it all. Not good. The more you make use of atomic
operations the more likely you’re going to to hit this race. Let’s
hope glibc is not a heavy user of atomic operations. PulseAudio however is, and
PulseAudio happens to be my focus.

Let’s have a look on how Qt4
does it:

extern Q_CORE_EXPORT char q_atomic_lock;

inline char q_atomic_swp(volatile char *ptr, char newval)
{
    register int ret;
    asm volatile("swpb %0,%1,[%2]"
                 : "=&r"(ret)
                 : "r"(newval), "r"(ptr)
                 : "cc", "memory");
    return ret;
}

inline int q_atomic_test_and_set_int(volatile int *ptr, int expected, int newval)
{
    int ret = 0;
    while (q_atomic_swp(&q_atomic_lock, ~0) != 0);
    if (*ptr == expected) {
	*ptr = newval;
	ret = 1;
    }
    q_atomic_swp(&q_atomic_lock, 0);
    return ret;
}

So, what do we have here? A slightly better version. In standard
situations it actually works. But it sucks big time, too. Why? It
contains a spin lock: the variable q_atomic_lock is used for
locking the atomic operation. The code tries to set it to non-zero,
and if that fails it tries again, until it succeeds, in the hope that
the other thread — which currently holds the lock — gives it up. The
big problem here is: it might take a while until that happens, up to
1/HZ time on Linux. Usually you want to use atomic operations to
minimize the need for mutexes and thus speed things up. Now, here you
got a lock, and it’s the worst kind: the spinning lock. Not
good. Also, if used from a real-time thread the machine simply locks
up when we enter the loop in contended state, because preemption is
disabled for RT threads and thus the loop will spin forever. Evil. And
then, there’s another problem: it’s a big bottleneck, because all
atomic operations are synchronized via a single variable which is
q_atomic_lock. Not good either. And let’s not forget that
only code that has access to q_atomic_lock actually can
execute this code safely. If you want to use it for
lock-free IPC via shared memory this is going to break. And let’s not
forget that it is unusable from signal handlers (which probably
doesn’t matter much, though). So, in summary: this code sucks,
too.

Next try, let’s have a look on how glib
does it:

static volatile int atomic_spin = 0;

static int atomic_spin_trylock (void)
{
  int result;

  asm volatile (
    "swp %0, %1, [%2]\n"
    : "=&r,&r" (result)
    : "r,0" (1), "r,r" (&atomic_spin)
    : "memory");
  if (result == 0)
    return 0;
  else
    return -1;
}

static void atomic_spin_lock (void)
{
  while (atomic_spin_trylock())
    sched_yield();
}

static void atomic_spin_unlock (void)
{
  atomic_spin = 0;
}

gint
g_atomic_int_exchange_and_add (volatile gint *atomic,
			       gint           val)
{
  gint result;

  atomic_spin_lock();
  result = *atomic;
  *atomic += val;
  atomic_spin_unlock();

  return result;
}

Once again, a spin loop. However, this implementation makes use of
sched_yield() for asking the OS to reschedule. It’s a bit
better than the Qt version, since it doesn’t spin just burning CPU,
but instead tells the kernel to execute something else, increasing the
chance that the thread currently holding the lock is scheduled. It’s a
bit friendlier, but it’s not great either because this might still delay
execution quite a bit. It’s better then the Qt version. And probably
one of the very few ligitimate occasions where using
sched_yield() is OK. It still doesn’t work for RT — because
sched_yield() in most cases is a NOP on for RT threads, so
you still get a machine lockup. And it still has the
one-lock-to-rule-them-all bottleneck. And it still is not compatible
with shared memory.

Then, there’s libatomic_ops. It’s
the most complex code, so I’ll spare you to paste it here. Basically
it uses the same spin loop. With three differences however:

  1. 16 lock variables instead of a single one are used. The variable
    that is used is picked via simple hashing of the pointer to the atomic variable
    that shall be modified. This removes the one-lock-to-rule-them-all
    bottleneck.
  2. Instead of pthread_yield() it uses select() with
    a small timeval parameter to give the current holder of the lock some
    time to give it up. To make sure that the select() is not
    optimized away by the kernel and the thread thus never is preempted
    the sleep time is increased on every loop iteration.
  3. It explicitly disables signals before doing the atomic operation.

It’s certainly the best implementation of the ones discussed here:
It doesn’t suffer by the one-lock-to-rule-them-all bottleneck. It’s
(supposedly) signal handler safe (which however comes at the cost of
doing two syscalls on every atomic operation — probably a very high
price). It actually works on RT, due to sleeping for an explicit
time. However it still doesn’t deal with priority
inversion
problems — which is a big issue for real-time
programming. Also, the time slept in the select() call might
be relatively long, since at least on Linux the time passed to
select() is rounded up to 1/HZ — not good for RT either. And
then, it still doesn’t work for shared memory IPC.

So, what do we learn from this? At least one thing: better don’t do
real-time programming with ARMv5[2]. But more practically, how
could a good emulation for atomic ops, solely based on atomic swap
look like? Here are a few ideas:

  • Use an implementation inspired by libatomic_ops. Right
    now it’s the best available. It’s probably a good idea, though, to
    replace select() by a nanosleep(), since on recent
    kernels the latter doesn’t round up to 1/HZ anymore, at least when you
    have high-resolution timers[3] Then, if you can live
    without signal handler safety, drop the signal mask changing.
  • If you use something based on libatomic_ops and want to
    use it for shared memory IPC, then you have the option to move the
    lock variables into shared memory too. Note however, that this allows
    evil applications to lock up your process by taking the locks and
    never giving them up. (Which however is always a problem if not all
    atomic operations you need are available in hardware) So if you do
    this, make sure that only trusted processes can attach to your memory
    segment.
  • Alternatively, spend some time and investigate if it is possible
    to use futexes to sleep on the lock variables. This is not trivial
    though, since futexes right now expect the availability of an atomic
    increment operation. But it might be possible to emulate this good
    enough with the swap operation. There’s now even a FUTEX_LOCK_PI
    operation which would allow priority inheritance.
  • Alternatively, find a a way to allow user space disabling
    interrupts cheaply (requires kernel patching). Since enabling
    RT scheduling is a priviliged operation already (since you may easily
    lock up your machine with it), it might not be too problematic to
    extend the ability to disable interrupts to user space: it’s just yet
    another way to lock up your machine.
  • For the libatomic_ops based algorithm: if you’re lucky
    and defined a struct type for your atomic integer types, like the
    kernel does, or like I do in PulseAudio with pa_atomic_t,
    then you can stick the lock variable directly into your
    structure. This makes shared memory support transparent, and removes
    the one-lock-to-rule-them-all bottleneck completely. Of course, OTOH it
    increases the memory consumption a bit and increases cache pressure
    (though I’d assume that this is neglible).
  • For the libatomic_ops based algorithm: start sleeping for
    the time returned by clock_getres()
    (cache the result!). You cannot sleep shorter than that anyway.

Yepp, that’s as good as it gets. Unfortunately I cannot serve you
the optimal solution on a silver platter. I never actually did
development for ARMv5, this blog story just sums up my thoughts on all
the code I saw which emulates atomic ops on ARMv5. But maybe someone
who actually cares about atomic operations on ARM finds this
interesting and maybe invests some time to prepare patches for Qt,
glib, glibc — and PulseAudio.

Update: I added two more ideas to the list above.

Update 2: Andrew Haley just posted something like the optimal solution for the problem. It would be great if people would start using this.

Footnotes

[1] The Nokia 770 has an ARMv5 chip, N800 has ARMv6. The OpenMoko phone apparently uses ARMv5.

[2] And let’s not even think about CPUs which don’t even have an atomic swap!

[3] Which however you probably won’t, given that they’re only available on x86 on stable Linux kernels for now — but still, it’s cleaner.

Polypaudio 0.9.0 released

Post Syndicated from Lennart Poettering original http://0pointer.net/blog/projects/polypaudio-0.9.0.html

We are proud to announce Polypaudio
0.9.0
. This is a major step ahead since we decided to freeze the
current API. From now on we will maintain API compability (or at least
try to). To emphasize this starting with this release the shared
library sonames are properly versioned. While Polypaudio 0.9.0 is not
API/ABI compatible with 0.8 it is protocol compatible.

Other notable changes beyond bug fixing, bug fixing and bug fixing
are: a new Open Sound System /dev/dsp wrapper named
padsp and a module module-volume-restore have been
added.

padsp works more or less like that ESOUND tool known as
esddsp. However, it is much cleaner in design and thus works
with many more applications than the original tool. Proper locking is
implemented which allows it to work in multithreaded applications. In
addition to mere /dev/dsp emulation it wraps
/dev/sndstat and /dev/mixer. Proper synchronization
primitives are also available, which enables lip-sync movie playback
using padsp on mplayer. Other applications that are
known to work properly with padsp are aumix,
libao, XMMS, sox. There are some things
padsp doesn’t support (yet): that’s most notably recording,
and mmap() wrapping. Recording will be added in a later
version. mmap() support is available in esddsp but
not in padsp. I am reluctant to add support for this, because
it cannot work properly when it comes to playback latency
handling. However, latency handling this the primary reasoning for
using mmap(). In addition the hack that is included in
esddsp works only for Quake2 and Quake3, both being Free
Software now. It probably makes more sense to fix those two games than
implementing a really dirty hack in padsp. Remember that you
can always use the original esddsp tools since Polypaudio
offers full protocol compatibility with ESOUND.

module-volume-restore is a small module that stores the
volume of all playback streams and restores them when the applications
which created them creates a new stream. If this module is loaded,
Polypaudio will make sure that you Gaim sounds are always played at
low volume, while your XMMS music is always played at full volume.

Besides the new release of Polypaudio itself we released a bunch of
other packages to work with the new release:

  • gst-polyp
    0.9.0
    , a Polypaudio plugin for GStreamer 0.10. The
    plugin is quite sophisticated. In fact it is probably the only
    sink/source plugin for GStreamer that reaches the functionality of the
    ALSA plugin that is shipped with upstream. It implements the
    GstPropertyProbe and GstImplementsInterface
    interfaces, which allow gnome-volume-meter and other
    GStreamer tools to control the volume of a Polypaudio server. The sink
    element listens for GST_EVENT_TAG events, and can thus use
    ID3 tags and other meta data to name the playback stream in the
    Polypaudio server. This is useful to identify the stream in the Polypaudio
    Volume Control
    . In short: Polypaudio 0.9.0 now offers first class
    integration into GStreamer.
  • libao-polyp
    0.9.0
    , a simple plugin for libao, which is used for audio playback by tools like ogg123 and Gaim, besides others.
  • xmms-polyp
    0.9.0
    , an output plugin for XMMS. As special feature it uses the
    currently played song name for naming the audio stream in
    Polypaudio.
  • Polypaudio Manager 0.9.0, updated for Polypaudio 0.9.0
  • Polypaudio Volume Control 0.9.0, updated for Polypaudio 0.9.0
  • Polypaudio Volume Meter 0.9.0, updated for Polypaudio 0.9.0

A screenshot showing most of this in action:

Polypaudio Screenshot.

This screenshot shows: the Polypaudio Manager, the Polypaudio
Volume Control, the Polypaudio Volume Meter, the XMMS plugin, the
GStreamer plugin used by Rhythmbox and gstreamer-properties,
pacat playing some noise from /dev/urandom,
padsp used on MPlayer. (This screenshot actually shows some
post-0.9.0 work, like the icons used by the application windows)