Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2015/09/15/email.html
[ This post
was cross-posted
on Conservancy’s blog. ]
In this post, I discuss one example of how a choice for software freedom
can cause many strange problems that others will dismiss. My goal here is
to explain in gory detail how proprietary software biases in the computing
world continue to grow, notwithstanding Open Source ballyhoo.
Two decades ago, nearly every company, organization, entity, and
tech-minded individual ran their own email server. Generally speaking,
even back then, nearly all the software for both
MTAs and
MUAs were Free
Software0. MTA’s are the mail
transport agents — the complex software that moves email around from
one Internet domain to another. MUAs are the mail user agents, sometimes
called mail clients — the local programs with which users manipulate
their own email.
I’ve run my own MTA since around 1993: initially with sendmail, then with
exim for a while, and with Postfix since 1999 or so. Also, everywhere I’ve
worked throughout my entire career since 1995, I’ve either been in charge
of — or been the manager of the person in charge of — the MTA
installation for the organization where I worked. In all cases, that MTA
has always been Free Software, of course.
However, the world of email has changed drastically during that period.
The most notable change in the email world is the influx of massive amounts
of spam, which has been used as an excuse to implement another disturbing
change. Slowly but surely, email service — both the MTA and the MUA
— have been outsourced for most organizations. Specifically, either
(a) organizations run proprietary software on their own computers to deal
with email and/or (b) people pay a third-party to run proprietary and/or
trade-secret software on their behalf to handle the email services. Email,
generally speaking, isn’t handled by Free Software all that much
anymore.
This situation became acutely apparent to me this earlier this month when
Conservancy moved its email server. I had plenty of warning that the move
was needed1, and I’d set up
a test site on the new server. We sent and received some of our email for
months (mostly mailing list traffic) using that server configured with a
different domain (sf-conservancy.org). When the shut-off day came, I moved
sfconservancy.org’s email officially. All looked good: I had a current
Debian, with a new version of Postfix and Dovecot on a speedier host, and
with better spam protection settings in Postfix and better spam filtering
with a newer version of SpamAssassin. All was going great, thanks to all
those great Free Software projects — until the proprietary software
vendors threw a spanner in our works.
For reasons that we’ll never determine for
sure2, the IPv4
number that our new hosting provide gave us was already listed on many
spam blacklists.
I won’t debate the validity of various blacklists here, but the fact is, for
nearly every public-facing, pure-blacklist-only service, delisting is
straightforward, takes about 24 hours, and requires at most answering some
basic questions about your domain name and answering a captcha-like
challenge. These services, even though some are quite dubious, are not the
center of my complaint.
The real peril comes from third-party email hosting companies. These
companies have arbitrary, non-public blacklisting rules. More importantly,
they are not merely blacklist maintainers, they are MTA (and in some cases,
even MUA) providers who sell their proprietary and/or trade-secret hosted
solutions as a package to customers. Years ago, the idea of giving up that
much control of what happens to your own email would be considered
unbelievable. Today, it’s commonplace.
And herein lies the fact that is obvious to most software freedom
advocates but indiscernible by most email users. As a Free Software user,
with your own MTA on your own machine, your software only functions if
everyone else respects your right to run that software yourself.
Furthermore, if the people you want to email are fully removed from their
hosting service, they won’t realize nor understand that their hosting site
might block your emails. These companies have their customers fully
manipulated to oppose your software freedom. In other words, you can’t
appeal to those customers (the people you want to email), because you’re
likely the only person to ever raise this issue with them (i.e., unless
they know you very well, they’ll assume you’re crazy). You’re left begging
to the provider, whom you have no business relationship with, to
convince them that their customers want to hear from you. Your voice rings
out indecipherable from the spammers who want the same permission to attack
their customers.
The upshot for Conservancy? For days, Microsoft told all its customers
that Conservancy is a spammer; Microsoft did it so subtly that the
customers wouldn’t even believe it if we told them. Specifically, every
time I or one of my Conservancy colleagues emailed organizations using
Microsoft’s “Exchange Online”, “Office 365” or
similar products to host email for their
domain4, we got the following
response:
Sep 2 23:26:26 pine postfix/smtp[31888]: 27CD6E12B: to=, relay=example-org.mail.protection.outlook.com[207.46.163.215]:25, delay=5.6, delays=0.43/0/0.16/5, dsn=5.7.1, status=bounced (host example-org.mail.protection.outlook.com[207.46.163.215] said: 550 5.7.1 Service unavailable; Client host [162.242.171.33] blocked using FBLW15; To request removal from this list please forward this message to [email protected] (in reply to RCPT TO command))
Oh, you ask, did you forward your message to the specified address
?
Of course I did; right away! I got back an email that said:
Hello ,
Thank you for your delisting request SRXNUMBERSID. Your ticket was
received on (Sep 01 2015 06:13 PM UTC) and will be responded to within 24
hours.
Once we passed the 24 hour mark with no response, I started looking around
for more information. I
also saw
a suggestion online that calling is the only way to escalate one of
those tickets, so I phoned 800-865-9408 and gave V-2JECOD my ticket number
and she told that I could only raise these issues with the “Mail Flow
Team”. She put me on hold for them, and told me that I was number 2
in the queue for them so it should be a few minutes. I waited on hold for
just under six hours. I finally reached a helpful representative, who said
the ticket was the lowest level of escalation available (he hinted that it
would take weeks to resolve at that level, which is consistent with other
comments about this problem I’ve seen online). The fellow on the phone
agreed to escalate it to the highest priority available, and said within
four hours, Conservancy should be delisted. Thus, ultimately, I did
resolve these issues after about 72 hours. But, I’d spent about 15 hours
all-told researching various blacklists, email hosting companies, and their
procedures3, and that was after I’d
already carefully configured our MTA and DNS to be very RFC-compliant
(which is complicated and confusing, but absolutely essential to stay off
these blacklists once you’re off).
Admittedly, this sounds like a standard Kafkaesque experience with a large
company that almost everyone in post-modern society has experienced.
However, it’s different in one key way: I had to convince Microsoft to
allow me to communicate with their customers who are paying Microsoft for
proprietary and/or trade-secret software and services, ostensibly to
improve efficiency of their communications. Plus, since Microsoft, by the
nature of their so-called spam blocking, doesn’t inform their customers whom
they’ve blocked, I and my colleagues would have just sounded crazy if we’d
asked our contacts to call their provider instead. (I actually considered
this, and realized that we might negatively impact relationships with
professional contacts.)
These problems do reduce email software freedom by network effects.
Most people rely on third-party proprietary email software
from Google, Microsoft, Barracuda, or others. Therefore, most people,
don’t exercise any software freedom regarding email
services. Since exercising software freedom for email slowly becomes a
rarer and rarer (rather than norm it once was), society slowly but surely
pegs those who do exercise software freedom as
“random crazy people”.
There are a few companies who are seeking to do email hosting in a way
that respects your software freedom. The real test of such companies is if
someone technically minded can get the same software configured on their
own systems, and have it work the same way. Yet, in most cases, you go to
one of these companies’ Github pages and find a bunch of stuff pushed
public, but limited information on how to configure it so that it functions
the same way the hosted service does. RMS wrote years ago
that Free
Software cannot properly succeed without Free Documentation, and in
many of these hosting cases: the hosting company is using fully
upstreamed Free Software, but has configured the software in a way that is
difficult to stumble upon by oneself. (For that reason, I’m committing to
writing up tutorials on how Conservancy configured our mail server, so at
least I’ll be part of the solution instead of part of the problem.)
BTW, as I dealt with all this, I couldn’t help but think
of John
Gilmore’s activism efforts regarding open mail relays. While I don’t
agree with all of John’s positions on this, his fundamental position is
right: we must oppose companies who think they know better how we should
configure our email servers (or on which IP numbers we should run those
servers). I’d add a corollary that there’s a serious threat to software
freedom, at least with regard to email software, if we continue to allow such
top-down control of the once beautifully decentralized email system.
The future of software freedom depends on issues like this. Imagine
someone who has just learned that they can run their own email server, or
bought some Free Software-based plug computing system that purports to be a
“home cloud” service with email. There’s virtually no chance
that such users would bother to figure all this out. They’d see their
email blocked, declare the “home cloud” solution useless, and
would just get a gmail.com, outlook.com, or some other third-party email
account. Thus, I predict that software freedom that we once had, for our
MTAs and MUAs, will eventually evaporate for everyone except those tiny few
who invest the time to understand these complexities and fight the
for-profit corporate power that curtails software freedom. Furthermore,
that struggle becomes Sisyphean as our numbers dwindle.
Email is the oldest software-centric communication system on the planet.
The global email system serves as a canary in the coalmine regarding
software freedom and network service freedom issues. Frighteningly,
software now controls most of the global communications systems. How long
will it be before mobile network providers refuse to terminate PSTN calls
or SMS’s sent from devices running modified Android firmwares like
Replicant? Perhaps those providers, like large email providers, will argue
that preventing robocalls (the telephone equivalent of SPAM) necessitates
such blocking. Such network effects place so many dystopias on
software freedom’s horizon.
I don’t deny that every day, there is more Free Software existing in the
world than has ever existed before — the P.T. Barnum’s of Open Source
have that part right. The part they leave out is that, each day, their
corporate backers make it a little more difficult to complete mundane tasks
using only Free Software. Open Source wins the battle while software
freedom loses the war.
0Yes, I’m intimately
aware that Elm’s license was non-free, and that the software
freedom of PINE’s license was in question. That’s slightly
relevant here but mostly orthogonal to this point, because Free
Software MUAs were still very common then, and there were
(ultimately successful) projects
to actively rewrite the ones whose software freedom was in
question
1For the last five
years, one of Conservancy’s Director Emeriti, Loïc Dachary,
has donated an extensive amount of personal time and
in-kind donations by providing Cloud server for Conservancy to
host its three key servers, including the email server. The
burden of maintaining this for us became too time consuming (very
reasonably), and Loïc’s asked us to find another provider. I
want, BTW, to thank Loïc his for years of volunteer work
maintaining infrastructure for us; he provided this service for
much longer than we could have hoped! Loïc also gave us
plenty of warning that we’d need to move. None of these problems
are his fault in the least!
2The
obvious supposition is that, because IPv4 numbers are so scarce,
this particular IP number was likely used previously by a spammer
who was shut down.
3I of
course didn’t count the time time on phone hold, as I was able to
do other work while waiting, but less efficiently because the hold
music was very distracting.
4If you want to
see if someone’s domain is a Microsoft customer, see if the MX
record for their domain (say, example.org) points to
example-org.mail.protection.outlook.com.
