Post Syndicated from Lennart Poettering original https://0pointer.net/blog/projects/canonical-contributions.html

#nocomments y

one small note: requiring copyright assignment from
contributors, and putting your code in exotic VCSes that only a minority of
potential contributors know or are willing to use is not helpful for attracting
contributions — right the contrary, it scares them away. Please fix that!

Post Syndicated from Lennart Poettering original https://0pointer.net/blog/projects/lpc-bluez-maemo-2009.html

Last week I’ve been at the Linux Plumbers Conference in
Portland. Like last year it kicked ass and proved again being one of the most
relevant Linux developer conferences (if not the most relevant one). I
ran the Audio MC at the conference which was very well attended. The slides
for our four talks in the
track are available online. (My own slides are probably a bit too terse
for most readers, the interesting stuff was in the talking, not the
reading…) Personally, for me the most interesting part was to see to which
degree Nokia actually adopted PulseAudio
in the N900. While I was aware that Nokia was using it, I wasn’t aware that
their use is as comprehensive as it turned out it is. And the industry
support from other companies is really impressive too. After the main track we
had a BoF session, which notes I’ll post a bit later. Many thanks to Paul,
Jyri, Pierre for their great talks. Unfortunately, Palm, the only manufacturer
who is actually already shipping a phone with PulseAudio didn’t send anyone to
the conference who wanted to talk about that. Let’s hope they’ll eventually
learn that just throwing code over the wall is not how Open Source works.
Maybe they’ll send someone to next year’s LPC in Boston, where I hope to be
able to do the Audio MC again.

Right now I am at the BlueZ Summit in Stuttgart. Among other things we have
been discussing how to improve Bluetooth Audio support in PulseAudio. I guess
one could say thet the Bluetooth support in PulseAudio is already one of its
highlights, in fact working better then the support on other OSes (yay, that’s
an area where Linux Audio really shines!). So up next is better support for
allowing PA to receive A2DP audio, i.e. making PA act as if it was a Headset or
your hifi. Use case: send music from from your mobile to your desktop’s hifi
speakers. (Actually this is already support in current BlueZ/PA versions, but
not easily accessible). Also Bluetooth headsets tend to support AC3 or MP3
decoding natively these days so we should support that in PA too. Codec
handling has been on the TODO list for PA for quite some time, for the SPDIF or
HDMI cases, and Bluetooth Audio is another reason why we really should have
that.

Next week I’ll be at the Maemo Summit in Amsterdam.
Nokia kindly invited me. Unfortunately I was a bit too late to get a proper
talk accepted. That said, I am sure if enough folks are interested we could do
a little ad-hoc BoF and find some place at the venue for it. If you have any
questions regarding PA just talk to me. The N900 uses PulseAudio for all things
audio so I am quite sure we’ll have a lot to talk about.

See you in Amsterdam!

One last thing: Check out Colin’s
work to improve integration of PulseAudio and KDE!

Post Syndicated from Lennart Poettering original https://0pointer.net/blog/projects/plumbers-audio-bof.html

Tomorrow, Thu 24th 10 am, there’s going to be an Audio BoF at LPC Portland, Salon E. Don’t miss it.

Post Syndicated from Lennart Poettering original https://0pointer.net/blog/projects/skype.html

A quick update on Skype: the next Skype version will include native
PulseAudio support. And not only that but they even tag their audio
streams properly. This enables PulseAudio to do fancy stuff like
automatically pausing your audio playback when you have a phone call. Good job!

In some ways they are now doing a better job with integration in to the modern
audio landscape than some Free Software telephony applications!

Unfortunately they didn’t fix the biggest bug though: it’s still not Free
Software!

Post Syndicated from Lennart Poettering original https://0pointer.net/blog/projects/mutrace2.html

Here’s a list of quick updates on my mutrace mutex profiler since
my initial announcement two weeks ago:

I added some special support for tracking down use of mutexes in realtime
threads. It’s a very simple extension that — if enabled — checks on each
mutex operation wheter it is executed by a realtime thread or not. (–track-rt) The
output of a test run of this you can find in this announcement on LAD.
Particularly interesting is that you can use this to track down which mutexes
are good candidates for priority inheritance.

The mutrace tarball now also includes a companion tool matrace
that can be used to track down memory allocation operations in realtime
threads. See the same lad announcement as above for example output of this
tool.

With help from Boudewijn Rempt I added some compatibility code for
profiling C++/Qt apps with mutrace, which he already used for some interesting
profiling results on krita.

Finally, after my comments on the locking hotspots in glib’s type system,
Wim Taymans and Edward Hervey worked on turning the mutex-emulated rwlocks
into OS native ones with quite positive results, for more information see this
bug.

As soon as my review request is fully processed mutrace will be available
in rawhide.

A snapshot tarball of mutrace you may find here
(despite the name of the tarball that’s just a snapshot, not the real release
0.1), for all those folks who are afraid of git, or don’t have a current
autoconf/automake/libtool installed.

Oh, and they named a unit after me.

Post Syndicated from Lennart Poettering original https://0pointer.net/blog/projects/mutrace.html

When naively profiling multi-threaded applications the time spent waiting
for mutexes is not necessarily visible in the generated output. However lock
contention can have a big impact on the runtime behaviour of applications. On
Linux valgrind’s
drd can be used to track down mutex contention. Unfortunately running
applications under valgrind/drd slows them down massively, often having the
effect of itself generating many of the contentions one is trying to track
down. Also due to its slowness it is very time consuming work.

To improve the situation if have now written a mutex profiler called
mutrace. In contrast to valgrind/drd it does not virtualize the
CPU instruction set, making it a lot faster. In fact, the hooks mutrace
relies on to profile mutex operations should only minimally influence
application runtime. mutrace is not useful for finding
synchronizations bugs, it is solely useful for profiling locks.

Now, enough of this introductionary blabla. Let’s have a look on the data
mutrace can generate for you. As an example we’ll look at
gedit as a bit of a prototypical Gnome application. Gtk+ and the other
Gnome libraries are not really known for their heavy use of multi-threading,
and the APIs are generally not thread-safe (for a good reason). However,
internally subsytems such as gio do use threading quite extensibly.
And as it turns out there are a few hotspots that can be discovered with
mutrace:

$ LD_PRELOAD=/home/lennart/projects/mutrace/libmutrace.so gedit
mutrace: 0.1 sucessfully initialized.

gedit is now running and its mutex use is being profiled. For this example
I have now opened a file with it, typed a few letters and then quit the program
again without saving. As soon as gedit exits mutrace will print the
profiling data it gathered to stderr. The full output you can see
here. The most interesting part is at the end of the generated output, a
breakdown of the most contended mutexes:

mutrace: 10 most contended mutexes:

 Mutex #   Locked  Changed    Cont. tot.Time[ms] avg.Time[ms] max.Time[ms]       Type
      35   368268      407      275      120,822        0,000        0,894     normal
       5   234645      100       21       86,855        0,000        0,494     normal
      26   177324       47        4       98,610        0,001        0,150     normal
      19    55758       53        2       23,931        0,000        0,092     normal
      53      106       73        1        0,769        0,007        0,160     normal
      25    15156       70        1        6,633        0,000        0,019     normal
       4      973       10        1        4,376        0,004        0,174     normal
      75       68       62        0        0,038        0,001        0,004     normal
       9     1663       52        0        1,068        0,001        0,412     normal
       3   136553       41        0       61,408        0,000        0,281     normal
     ...      ...      ...      ...          ...          ...          ...        ...

mutrace: Total runtime 9678,142 ms.

(Sorry, LC_NUMERIC was set to de_DE.UTF-8, so if you can’t make sense of
all the commas, think s/,/./g!)

For each mutex a line is printed. The ‘Locked’ column tells how often the
mutex was locked during the entire runtime of about 10s. The ‘Changed’ column
tells us how often the owning thread of the mutex changed. The ‘Cont.’ column
tells us how often the lock was already taken when we tried to take it and we
had to wait. The fifth column tell us for how long during the entire runtime
the lock was locked, the sixth tells us the average lock time, and the seventh
column tells us the longest time the lock was held. Finally, the last column
tells us what kind of mutex this is (recursive, normal or otherwise).

The most contended lock in the example above is #35. 275 times during the
runtime a thread had to wait until another thread released this mutex. All in
all more then 120ms of the entire runtime (about 10s) were spent with this
lock taken!

In the full output we can now look up which mutex #35 actually is:

Mutex #35 (0x0x7f48c7057d28) first referenced by:
	/home/lennart/projects/mutrace/libmutrace.so(pthread_mutex_lock+0x70) [0x7f48c97dc900]
	/lib64/libglib-2.0.so.0(g_static_rw_lock_writer_lock+0x6a) [0x7f48c674a03a]
	/lib64/libgobject-2.0.so.0(g_type_init_with_debug_flags+0x4b) [0x7f48c6e38ddb]
	/usr/lib64/libgdk-x11-2.0.so.0(gdk_pre_parse_libgtk_only+0x8c) [0x7f48c853171c]
	/usr/lib64/libgtk-x11-2.0.so.0(+0x14b31f) [0x7f48c891831f]
	/lib64/libglib-2.0.so.0(g_option_context_parse+0x90) [0x7f48c67308e0]
	/usr/lib64/libgtk-x11-2.0.so.0(gtk_parse_args+0xa1) [0x7f48c8918021]
	/usr/lib64/libgtk-x11-2.0.so.0(gtk_init_check+0x9) [0x7f48c8918079]
	/usr/lib64/libgtk-x11-2.0.so.0(gtk_init+0x9) [0x7f48c89180a9]
	/usr/bin/gedit(main+0x166) [0x427fc6]
	/lib64/libc.so.6(__libc_start_main+0xfd) [0x7f48c5b42b4d]
	/usr/bin/gedit() [0x4276c9]

As it appears in this Gtk+ program the rwlock type_rw_lock
(defined in glib’s gobject/gtype.c) is a hotspot. GLib’s rwlocks are
implemented on top of mutexes, so an obvious attempt in improving this could
be to actually make them use the operating system’s rwlock primitives.

If a mutex is used often but only ever by the same thread it cannot starve
other threads. The ‘Changed.’ column lists how often a specific mutex changed
the owning thread. If the number is high this means the risk of contention is
also high. The ‘Cont.’ column tells you about contention that actually took
place.

Due to the way mutrace works we cannot profile mutexes that are
used internally in glibc, such as those used for synchronizing stdio
and suchlike.

mutrace is implemented entirely in userspace. It
uses all kinds of exotic GCC, glibc and kernel features, so you might have a
hard time compiling and running it on anything but a very recent Linux
distribution. I have tested it on Rawhide but it should work on slightly older
distributions, too.

Make sure to build your application with -rdynamic to make the
backtraces mutrace generates useful.

As of now, mutrace only profiles mutexes. Adding support for
rwlocks should be easy to add though. Patches welcome.

The output mutrace generates can be influenced by various
MUTRACE_xxx environment variables. See the sources for more
information.

And now, please take mutrace and profile and speed up your application!

You may find the sources in my
git repository.

Post Syndicated from Lennart Poettering original https://0pointer.net/blog/projects/pthread-key-create.html

If you use pthread_key_create() with a non-NULL destructor
parameter (or an equivalent TLS construct) in a library/shared object then you
MUST link your library wth -z nodelete (or an equivalent
construct).

If you don’t, then you’ll have a lot of fun (like I just had) debugging
segfaults in the TLS destruction logic where functions are called that might
not even exist anymore in memory.

Now don’t tell me I hadn’t told you.

(Oh, and I hope I don’t need to mention that all GObject-based libraries should
link with -z nodelete anyway, for making sure the type system doesn’t
break.)

Post Syndicated from Lennart Poettering original https://0pointer.net/blog/projects/oh-nine-sixteen.html

#nocomments y

As a followup to Oh Nine
Fifteen here’s a little overview of the changes coming with PulseAudio 0.9.16 which will be part of
Fedora 12 (already in Rawhide; I think Ubuntu Karmic (?) will have it
too).

A New Mixer Logic

We now try to control more than just a single ALSA mixer element for volume
control. This increases the hardware volume range and granularity exposed and
should also help minimizing problems by incomplete or incorrect default mixer
initialization on the lower levels.

This also adds support for allowing selection of input/output ports for
sound cards. This is used to expose changing between Mic vs. Line-In for input
source selection and Headphones vs. Speaker for output selection (of course the
list of available port is strictly dependant on what you hardware supports).
The list of available ports is deliberately kept minimal.

Thanks to Bastien the newest GNOME Volume Control now exposes profile/port
switching quite nicely, which he
blogged about. This
screenshot shows how the port (here called ‘Connector’) can be selected
in the new dialog.

The mixer rework also allows us to handle semi-pro/pro sound cards a bit
more flexibly. For example, which profiles/ports are exposed in PulseAudio or
how specific mixer elements are handled can now be controlled by editing .ini
file like configuration files in /usr/share/pulseaudio/alsa-mixer/.
Read
this mail for more information about this.

UPnP MediaServer Support

PulseAudio now integrates with Zeeshan’s fabulous Rygel UPnP/DLNA MediaServer. If enabled
Rygel will automatically expose all local audio devices which are managed by
PulseAudio as UPnP/DLNA MediaServer items which your UPnP/DLNA MediaRenderers
can now tune into. (Meaning: you can now stream audio from your PC directly to
your UPnP DMP (Digital Media Player) device, such as the PS3.) Communication
between Rygel and PulseAudio follows our little Media Server Spec on the
GNOME Wiki. This nicely complements the RAOP (Apple Airport) support we
introduced in PulseAudio 0.9.15. In one of the next versions of
PulseAudio/Rygel we hope to add support for PulseAudio becoming a MediaRenderer
as well. This will then not only allow you to stream from your PC to your
DMP device, but also allows PulseAudio to act as
“networked speaker”, which can be used by any UPnP/AV/DLNA control point, such
as Windows’ Media Player.

Hotplug Support Improved

If you select a particular device as the default for a specific application
or class of streams, then when unplugging the device PulseAudio moves the stream
automatically to another audio device if one exists. New in PulseAudio 0.9.16
is that if you replug the audio device the stream will instantly be moved back,
requiring no further user intervention.

Also, PulseAudio now includes some implicit rules for doing the ‘right
thing’ when finding an audio device for an application. For example, unless
configured otherwise it will now route telephony applications automatically to
Bluetooth headsets if one is connected, in favour of the internal sound card of
the computer.

Surround Sound Support for Event Sounds

This is more a new feature of libcanberra than
of PulseAudio, but nonetheless: we now support surround for events sounds.
This allows us to play full 5.1 login sounds for example, in best THX cinema
fashion. We’d love to ship a 5.1 sound for login by default in sound-theme-freedesktop.
We’d be very thankful if you would be willing to contribute a sound
here, or two! A sound a bit less bombastic than the famous cinema THX effect
would probably be a good idea though.

And then there’s of course the usual batch of fixes and small improvements.
A substantial number of non-user visible changes have been made as well. For
example, as HAL is now obsolete PulseAudio now moved to udev for its device
discovery needs. We replaced our gdbm support by support for tdb. Also,
we stripped all security senstive code from PulseAudio, and ported it to use
RealtimeKit instead.
For the upcoming distributions that means that PulseAudio will run as real-time
process by default, improving drop-out safety.

And for some extra PA eye-candy, have a look on Impulse!

Post Syndicated from Lennart Poettering original https://0pointer.net/blog/projects/avahi-world-domination.html

#nocomments y

I hereby officially declare that I have reached my goal of world domination. Emacs 23
(apparently due today) ships with Avahi support out of the box. Obviously,
one of the most natural combinations of software thinkable.

After Emacs, there’s not much else I could win, or is there?