My thoughts on the future of Gnome-VFS

Post Syndicated from Lennart Poettering original https://0pointer.net/blog/projects/gnomevfs-future.html

One of the major construction sites in GNOME and all the other free
desktop environments is the VFS abstraction. Recently, there has been
some discussion about developing a replacement DVFS
as replacement for the venerable Gnome-VFS system. Here are my 5 euro
cent on this issue (Yepp, I am not fully up-to-date on the whole DVFS
discussion, but during my flight from HEL to HAM I wrote this up,
without being necesarily too well informed, lacking an Internet
connection. Hence, if you find that I am an uniformed idiot, you’re of
course welcome to flame me!):

First of all, we have to acknowledge that Gnome-VFS never achieved any major
adoption besides some core (not even all) GNOME applications. The reasons are
many, among them: the API wasn’t all fun, using Gnome-VFS added another
dependency to applications, KDE uses a different abstraction (KIO), and many
others. Adoption was suboptimal, and due to that user experience was
suboptimal, too (to say the least).

One of the basic problems of Gnome-VFS is that it is a (somewhat) redundant
abstraction layer over yet another abstraction layer. Gnome-VFS makes available
an API that offers more or less the same functionality as the (most of the
time) underlying POSIX API. The POSIX API is well accepted, relatively
easy-to-use, portable and very well accepted. The same is not true for
Gnome-VFS. Semantics of the translation between Gnome-VFS and POSIX are not
always that clear. Paths understood by Gnome-VFS (URLs) follow a different
model than those of the Linux kernel. Applications which understand Gnome-VFS
can deal with FTP and HTTP resources, while the majority of the applications
which do not link against Gnome-VFS does not understand it. Integration of
Gnome-VFS-speaking and POSIX-speaking applications is difficult and most of the
time only partially implementable.

So, in short: One one side we have that POSIX API which is a file system
abstraction API. And a (kernel-based) virtual file system behind it. And on the other
side we have the Gnome-VFS API which is also a file system abstraction API and
a virtual file system behind it. Hence, why did we decide to standardize on
Gnome-VFS, and not just on POSIX?

The major reason of course is that until recently accessing FTP,
HTTP and other protocol shares through the POSIX API was not doable
without special kernel patches. However, a while ago the FUSE system
has been merged into the Linux kernel and has been made available for
other operating systems as well, among them FreeBSD and MacOS X. This
allows implementing file system drivers in userspace. Currently there
are all kinds of these FUSE based file systems around, FTP and SSHFS
are only two of them. My very own fusedav tool
implements WebDAV for FUSE.

Another (*the* other?) major problem of the POSIX file system API is
its synchronous design. While that is usually not a problem for local
file systems and for high-speed network file systems such as NFS it
becomes a problem for slow network FSs such as HTTP or FTP. Having the
GUI block for various seconds while an application saves its documents
is certainly not user friendly. But, can this be fixed? Yes, definitely, it can!
Firstly, there already is the POSIX AIO interface — which however is
quite unfriendly to use (one reason is its use of Unix signals for
notification of completed IO operations). Secondly, the (Linux) kernel
people are working on a better asynchronous IO API (see the
syslets/fibrils discussion). Unfortunately it will take a while
before that new API will finally be available in upstream
kernels. However, there’s always the third solution: add an
asynchronous API entirely in userspace. This is doable in a clean (and
glib-ified) fashion: have a couple of worker threads which
(synchronously) execute the various POSIX file system functions and
add a nice, asynchronous API that can start and stop these threads,
feed them operations to execute, and so on.

So, what’s the grand solution I propose for the desktop VFS mess? First, kick
Gnome-VFS entirely and don’t replace it. Instead write a small D-Bus-accessible
daemon that organizes a special directory ~/net/. Populate that
directory with subdirectories for all WebDAV, FTP, NFS and SMB shares that can
be found on the local network using both Avahi-based browsing and native SMB
browsing. Now use the Linux automounting interface on top of that directory and
automount the respective share every time someone wants to access it. For
shares that are not announced via Avahi/Samba, add some D-Bus API (and a nice
UI) for adding arbitrary shares. NFS and CIFS/SMB shares are mounted with the
fast, optimized kernel filesystem implementation; WebDAV and FTP on the other
hand are accessed via userspace FUSE-based file systems. The latter should also
integrate with D-BUS in some way, to query the user nicely for access
credentials and suchlike, with gnome-keyring support and everything.

~/net/ itself can — but probably doesn’t need to — be a FUSE
filesystem itself.

A shared library should be made available that will implement a few
remaining things, that are not available in the POSIX file
system API directly:

  • As mentioned, some nice Glib-ish asynchronous POSIX file system
    API wrapper
  • High-level file system operations such as copying, moving,
    deleting (trash!) which show a nice GUI when they are long-running
    operations.
  • An API to translate and setup URL <-> filesystem
    mappings, i.e. something that translates
    ftp://test.local/a/certain/path/ to
    ~/net/ftp:test.local/a/certain/path and vice versa. (and
    probably also to a more user-friendly notation, maybe like “FTP Share
    on test.local
    ” or similar). (Needs to communicate with the ~/net/
    handling daemon to setup mappings if required)
  • Meta data extraction. It makes sense to integrate that with
    extended attribute support (EA) in the kernel file system layer, which should be used more often anyway.
  • Explicit mount operations (in contrast to implicit mounts, that
    are done through automounting) (this also needs to communicate with
    the ~/net/ daemon in some way)

Et voilá! Without a lot of new code you get a nice, asynchronous,
modern, well integrated file system, that doesn’t suck. (or at least,
it doesn’t suck as much as other solutions).

Also, this way we can escape the “abstraction trap”. Let’s KDE play
the abstraction game, maybe they’ll grow up eventually and learn that
abstracting abstracted abstraction layers is child’s play.

Yeah, sure, this proposed solution also has a few drawbacks, but be it that way. Here’s a short incomprehensive list:

  • The POSIX file system API sucks for file systems that don’t have “inodes” or that are attached to a specific user sessions. — Yes, sure, but both problems have been overcome by the FUSE project, at least partially.
  • Not that portable — Yes, but FUSE is now available for many systems besides Linux. The automount project is the bigger problem. But all you loose if you would run this proposed system on these (let’s say “legacy”) systems that don’t have FUSE or automounting is access to FTP and WebDAV shares. So what? Local files can still be accessed.
  • Translating between URLs and $HOME/net/ based paths sucks — yepp, it does. But much less than not being able to access FTP/WebDAV shares from some apps but not from others, as we have it right now.
  • Bah, you suck — Yes, I do. On a straw, taking a nip from my caipirinha, right at the moment.

I guess I don’t have to list all the advantages of this solution, do I?

BTW, pumping massive amounts of data through D-Bus sucks anyway.

And no, I am not going to hack on this. Too busy with other stuff.

The plane is now landing in HAM, that shall conclude our small rant.

Update: No, I didn’t get a Caipirinha during my flight. That line I
added in before publishing the blog story, which was when I was drinking my
Caipirinha. In contrast to other people from the Free Software community I don’t
own my own private jet yet, with two stewardesses that might fix me a
Caipirinha.