Post Syndicated from Lennart Poettering original https://0pointer.net/blog/projects/pa-plugin-for-xine.html
Quick update for all K-lovers: apply this patch to xine-lib and enjoy PulseAudio in Amarok and other KDE apps without stability issues. It’s a race-free rework of Diego’s original patch.
Post Syndicated from Lennart Poettering original https://0pointer.net/blog/projects/bossa-2008.html
Just three words: awesome awesome awesome.
And for those asking for it, here are my
slides, in which I try to explain the new “glitch-free” audio scheduling
core of PulseAudio that I recently commited to the glitch-free branch
in PA SVN. I also try to make clear why this functionality is practically a
*MUST* for all people who want to have low-latency audio, minimal power
consumption and maximum drop-out safety for their audio playback. And thus, why
all those fancy embedded Linux devices should adopt it better sooner than
later. The slides might appear a bit terse if you don’t have that awesome guy
they usually come with presenting them to you.
Post Syndicated from Lennart Poettering original https://0pointer.net/blog/photos/petrified-dunes.html
Post Syndicated from Lennart Poettering original https://0pointer.net/blog/photos/a-whole-lot-of-nothing.html
Post Syndicated from Lennart Poettering original https://0pointer.net/blog/photos/kata-tjuta.html
Post Syndicated from Lennart Poettering original https://0pointer.net/blog/projects/lca2008.html
After coming back from my somewhat extended linux.conf.au trip I spent the
whole day grepping through email. Only 263 unprocessed emails left in my inbox.
Yay.
PRTPILU
Thanks to the LCA guys, video footage is now available of all talks,
including my talk
Practical Real-Time Programming in Linux Userspace (Theora,
Slides).
In my endless modesty I have to recommend: go, watch it, it contains some
really good stuff (including me not being able to divide 1 by 1000). Right now,
the real-time features of the Linux kernel are seldomly used on the desktop due
to a couple of reasons, among them general difficulty and unsafety to use them
but predominantly it’s probably just unawareness. There are a couple of
situations however, where scheduling desktop processes as RT makes a lot of
sense (think of video playback, mouse curse feedback, etc.), to decouple the
execution (scheduling) latency from the system load. This talk focussed mostly
on non-trivial technical stuff and all the limitations RT on Linux still has.
To fully grok what’s going on you thus need some insight into concurrent
programming and stuff.
My plan is to submit a related talk to GUADEC wich will focus more on
actually building RT apps for the desktop, in the hope we will eventually be
able to ship a desktop with audio and video that never skips, and where user
feedback is still snappy and quick even if we do the most complicated IO
intensive processing in lots of different processes in the background on slow
hardware.
I didn’t have time to go through all my slides (which I intended that way
and is perfectly OK), so you might want to browse through my slides even if you
saw the whole clip. The slides, however, are not particularly verbose.
Rumors
Regarding all those
rumors
that have been spread while I — the maintainer of PulseAudio — was in the
middle of the australian outback, fist-fighting with kangaroos near Uluru: I
am not really asking anyone to port their apps to the native PulseAudio API right now. While I do think
the API is quite powerful and not redundant, I also acknowledge that it is
very difficult to use properly (and very easy to misuse), (mostly) due to its
fully asynchronous nature. The mysterious libsydney project is
supposed to fix this and a lot more. libsydney is mostly the Dukem Nukem
Forever of audio APIs right now, but in contrast to DNF I didn’t really
announce it publicly yet, so it doesn’t really count. 😉 Suffice to
say, the current situation of audio APIs is a big big mess. We are working on
cleaning it up. For now: stick to the well established and least-broken APIs,
which boils down to ALSA. Stop using the OSS API now! Don’t program
against the ESD API (except for event sounds). But, most importantly: please
stop misusing the existing APIs. I am doing my best to allow all current APIs
to run without hassles on top of PA, but due to the sometimes blatant misues,
or even brutal violations of those APIs it is very hard to get that working
for all applications (yes, that means you, Adobe, and you, Skype). Don’t
expect that mmap is available on all audio devices — it’s not, and especially
not on PA. Don’t use /proc/asound/pcm as an API for enumerating audio
devices. It’s totally unsuitable for that. Don’t hard code device strings. Use
default as device string. Don’t make assumptions that are not and
cannot be true for non-hardware devices. Don’t fiddle around with period
settings unless you fully grok them and know what you are doing. In short: be
a better citizen, write code you don’t need to be ashamed of. ALSA has its
limitations and my compatibility code certainly as well, but this is not an
excuse for working around them by writing code that makes little children cry.
If you have a good ALSA backend for your program than this will not only fix
your issues with PA, but also with Bluetooth, you will have less code to
maintain and also code that is much easier to maintain.
Or even shorter: Fix. Your. Broken. ALSA. Client. Code. Thank you.
Oh, if you have questions regarding PA, just ping me on IRC (if I am
around) or write me an email, like everyone else. Mysterious, blogged pseudo
invitations to rumored meetings is not the best way to contact me.
Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2008/01/24/apt-mirror-2.html
When I started building our apt-mirror, I ran into a problem: the
machine was throttled against ubuntu.com’s servers, but I had completed
much of the download (which took weeks to get multiple distributions).
I really wanted to roll out the solution quickly, particularly because
the service from the remote servers was worse than ever due to the
throttling that the mirroring created. But, with the mirror incomplete,
I couldn’t so easily make available incomplete repositories.
The solution was to simply let apache redirect users on to the real
servers if the mirror doesn’t have the file. The first order of
business for that is to rewrite and redirect URLs when files aren’t
found. This is a straightforward Apache configuration:
RewriteEngine on
RewriteLogLevel 0
RewriteCond %{REQUEST_FILENAME} !^/cgi/
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-F
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !(Packages|Sources)\.bz2$
RewriteCond %{REQUEST_URI} !/index\.[^/]*$ [NC]
RewriteRule ^(http://%{HTTP_HOST})?/(.*) http://91.189.88.45/$2 [P]
Note a few things there:
I have to hard-code an IP number, because as I mentioned in
the last
post on this subject, I’ve faked out DNS
for archive.ubuntu.com and other sites I’m mirroring. (Note:
this has the unfortunate side-effect that I can’t easily take advantage
of round-robin DNS on the other side.)
I avoid taking Packages.bz2 from the other site, because
apt-mirror actually doesn’t mirror the bz2 files (although I’ve
submitted a patch to it so it will eventually).
I make sure that index files get built by my Apache and not
redirected.
I am using Apache proxying, which gives me Yet Another type of
cache temporarily while I’m still downloading the other packages. (I
should actually work out a way to have these caches used by apt-mirror
itself in case a user has already requested a new package while waiting
for apt-mirror to get it.)
Once I do a rewrite like this for each of the hosts I’m replacing with
a mirror, I’m almost done. The problem is that if for any reason my
site needs to give a 403 to the clients, I would actually like to
double-check to be sure that the URL doesn’t happen to work at the place
I’m mirroring from.
My hope was that I could write a RewriteRule based on what the
HTTP return code would be when the request completed. This was
really hard to do, it seemed, and perhaps undoable. The quickest
solution I found was to write a CGI script to do the redirect. So, in
the Apache config I have:
ErrorDocument 403 /cgi/redirect-forbidden.cgi
And, the CGI script looks like this:
#!/usr/bin/perl
use strict;
use CGI qw(:standard);
my $val = $ENV{REDIRECT_SCRIPT_URI};
$val =~ s%^http://(\S+).sflc.info(/.*)$%$2%;
if ($1 eq "ubuntu-security") {
$val = "http://91.189.88.37$val";
} else {
$val = "http://91.189.88.45$val";
}
print redirect($val);
With these changes, the user will be redirected to the original when
the files aren’t available on the mirror, and as the mirror gets more
accurate, they’ll get more files from the mirror.
I still have problems if for any reason the user gets a Packages or
Sources file from the original site before the mirror is synchronized,
but this rarely happens since apt-mirror is pretty careful. The only
time it might happen is if the user did an apt-get update when
not connected to our VPN and only a short time later did one while
connected.
Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2008/01/24/apt-mirror-2.html
When I started building our apt-mirror, I ran into a problem: the
machine was throttled against ubuntu.com’s servers, but I had completed
much of the download (which took weeks to get multiple distributions).
I really wanted to roll out the solution quickly, particularly because
the service from the remote servers was worse than ever due to the
throttling that the mirroring created. But, with the mirror incomplete,
I couldn’t so easily make available incomplete repositories.
The solution was to simply let apache redirect users on to the real
servers if the mirror doesn’t have the file. The first order of
business for that is to rewrite and redirect URLs when files aren’t
found. This is a straightforward Apache configuration:
RewriteEngine on
RewriteLogLevel 0
RewriteCond %{REQUEST_FILENAME} !^/cgi/
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-F
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !(Packages|Sources)\.bz2$
RewriteCond %{REQUEST_URI} !/index\.[^/]*$ [NC]
RewriteRule ^(http://%{HTTP_HOST})?/(.*) http://91.189.88.45/$2 [P]
Note a few things there:
I have to hard-code an IP number, because as I mentioned in
the last
post on this subject, I’ve faked out DNS
for archive.ubuntu.com and other sites I’m mirroring. (Note:
this has the unfortunate side-effect that I can’t easily take advantage
of round-robin DNS on the other side.)
I avoid taking Packages.bz2 from the other site, because
apt-mirror actually doesn’t mirror the bz2 files (although I’ve
submitted a patch to it so it will eventually).
I make sure that index files get built by my Apache and not
redirected.
I am using Apache proxying, which gives me Yet Another type of
cache temporarily while I’m still downloading the other packages. (I
should actually work out a way to have these caches used by apt-mirror
itself in case a user has already requested a new package while waiting
for apt-mirror to get it.)
Once I do a rewrite like this for each of the hosts I’m replacing with
a mirror, I’m almost done. The problem is that if for any reason my
site needs to give a 403 to the clients, I would actually like to
double-check to be sure that the URL doesn’t happen to work at the place
I’m mirroring from.
My hope was that I could write a RewriteRule based on what the
HTTP return code would be when the request completed. This was
really hard to do, it seemed, and perhaps undoable. The quickest
solution I found was to write a CGI script to do the redirect. So, in
the Apache config I have:
ErrorDocument 403 /cgi/redirect-forbidden.cgi
And, the CGI script looks like this:
#!/usr/bin/perl
use strict;
use CGI qw(:standard);
my $val = $ENV{REDIRECT_SCRIPT_URI};
$val =~ s%^http://(\S+).sflc.info(/.*)$%$2%;
if ($1 eq "ubuntu-security") {
$val = "http://91.189.88.37$val";
} else {
$val = "http://91.189.88.45$val";
}
print redirect($val);
With these changes, the user will be redirected to the original when
the files aren’t available on the mirror, and as the mirror gets more
accurate, they’ll get more files from the mirror.
I still have problems if for any reason the user gets a Packages or
Sources file from the original site before the mirror is synchronized,
but this rarely happens since apt-mirror is pretty careful. The only
time it might happen is if the user did an apt-get update when
not connected to our VPN and only a short time later did one while
connected.
Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2008/01/24/apt-mirror-2.html
When I started building our apt-mirror, I ran into a problem: the
machine was throttled against ubuntu.com’s servers, but I had completed
much of the download (which took weeks to get multiple distributions).
I really wanted to roll out the solution quickly, particularly because
the service from the remote servers was worse than ever due to the
throttling that the mirroring created. But, with the mirror incomplete,
I couldn’t so easily make available incomplete repositories.
The solution was to simply let apache redirect users on to the real
servers if the mirror doesn’t have the file. The first order of
business for that is to rewrite and redirect URLs when files aren’t
found. This is a straightforward Apache configuration:
RewriteEngine on
RewriteLogLevel 0
RewriteCond %{REQUEST_FILENAME} !^/cgi/
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-F
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !(Packages|Sources)\.bz2$
RewriteCond %{REQUEST_URI} !/index\.[^/]*$ [NC]
RewriteRule ^(http://%{HTTP_HOST})?/(.*) http://91.189.88.45/$2 [P]
Note a few things there:
I have to hard-code an IP number, because as I mentioned in
the last
post on this subject, I’ve faked out DNS
for archive.ubuntu.com and other sites I’m mirroring. (Note:
this has the unfortunate side-effect that I can’t easily take advantage
of round-robin DNS on the other side.)
I avoid taking Packages.bz2 from the other site, because
apt-mirror actually doesn’t mirror the bz2 files (although I’ve
submitted a patch to it so it will eventually).
I make sure that index files get built by my Apache and not
redirected.
I am using Apache proxying, which gives me Yet Another type of
cache temporarily while I’m still downloading the other packages. (I
should actually work out a way to have these caches used by apt-mirror
itself in case a user has already requested a new package while waiting
for apt-mirror to get it.)
Once I do a rewrite like this for each of the hosts I’m replacing with
a mirror, I’m almost done. The problem is that if for any reason my
site needs to give a 403 to the clients, I would actually like to
double-check to be sure that the URL doesn’t happen to work at the place
I’m mirroring from.
My hope was that I could write a RewriteRule based on what the
HTTP return code would be when the request completed. This was
really hard to do, it seemed, and perhaps undoable. The quickest
solution I found was to write a CGI script to do the redirect. So, in
the Apache config I have:
ErrorDocument 403 /cgi/redirect-forbidden.cgi
And, the CGI script looks like this:
#!/usr/bin/perl
use strict;
use CGI qw(:standard);
my $val = $ENV{REDIRECT_SCRIPT_URI};
$val =~ s%^http://(\S+).sflc.info(/.*)$%$2%;
if ($1 eq "ubuntu-security") {
$val = "http://91.189.88.37$val";
} else {
$val = "http://91.189.88.45$val";
}
print redirect($val);
With these changes, the user will be redirected to the original when
the files aren’t available on the mirror, and as the mirror gets more
accurate, they’ll get more files from the mirror.
I still have problems if for any reason the user gets a Packages or
Sources file from the original site before the mirror is synchronized,
but this rarely happens since apt-mirror is pretty careful. The only
time it might happen is if the user did an apt-get update when
not connected to our VPN and only a short time later did one while
connected.
Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2008/01/24/apt-mirror-2.html
When I started building our apt-mirror, I ran into a problem: the
machine was throttled against ubuntu.com’s servers, but I had completed
much of the download (which took weeks to get multiple distributions).
I really wanted to roll out the solution quickly, particularly because
the service from the remote servers was worse than ever due to the
throttling that the mirroring created. But, with the mirror incomplete,
I couldn’t so easily make available incomplete repositories.
The solution was to simply let apache redirect users on to the real
servers if the mirror doesn’t have the file. The first order of
business for that is to rewrite and redirect URLs when files aren’t
found. This is a straightforward Apache configuration:
RewriteEngine on
RewriteLogLevel 0
RewriteCond %{REQUEST_FILENAME} !^/cgi/
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-F
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !(Packages|Sources)\.bz2$
RewriteCond %{REQUEST_URI} !/index\.[^/]*$ [NC]
RewriteRule ^(http://%{HTTP_HOST})?/(.*) http://91.189.88.45/$2 [P]
Note a few things there:
I have to hard-code an IP number, because as I mentioned in
the last
post on this subject, I’ve faked out DNS
for archive.ubuntu.com and other sites I’m mirroring. (Note:
this has the unfortunate side-effect that I can’t easily take advantage
of round-robin DNS on the other side.)
I avoid taking Packages.bz2 from the other site, because
apt-mirror actually doesn’t mirror the bz2 files (although I’ve
submitted a patch to it so it will eventually).
I make sure that index files get built by my Apache and not
redirected.
I am using Apache proxying, which gives me Yet Another type of
cache temporarily while I’m still downloading the other packages. (I
should actually work out a way to have these caches used by apt-mirror
itself in case a user has already requested a new package while waiting
for apt-mirror to get it.)
Once I do a rewrite like this for each of the hosts I’m replacing with
a mirror, I’m almost done. The problem is that if for any reason my
site needs to give a 403 to the clients, I would actually like to
double-check to be sure that the URL doesn’t happen to work at the place
I’m mirroring from.
My hope was that I could write a RewriteRule based on what the
HTTP return code would be when the request completed. This was
really hard to do, it seemed, and perhaps undoable. The quickest
solution I found was to write a CGI script to do the redirect. So, in
the Apache config I have:
ErrorDocument 403 /cgi/redirect-forbidden.cgi
And, the CGI script looks like this:
#!/usr/bin/perl
use strict;
use CGI qw(:standard);
my $val = $ENV{REDIRECT_SCRIPT_URI};
$val =~ s%^http://(\S+).sflc.info(/.*)$%$2%;
if ($1 eq "ubuntu-security") {
$val = "http://91.189.88.37$val";
} else {
$val = "http://91.189.88.45$val";
}
print redirect($val);
With these changes, the user will be redirected to the original when
the files aren’t available on the mirror, and as the mirror gets more
accurate, they’ll get more files from the mirror.
I still have problems if for any reason the user gets a Packages or
Sources file from the original site before the mirror is synchronized,
but this rarely happens since apt-mirror is pretty careful. The only
time it might happen is if the user did an apt-get update when
not connected to our VPN and only a short time later did one while
connected.
Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2008/01/24/apt-mirror-2.html
When I started building our apt-mirror, I ran into a problem: the
machine was throttled against ubuntu.com’s servers, but I had completed
much of the download (which took weeks to get multiple distributions).
I really wanted to roll out the solution quickly, particularly because
the service from the remote servers was worse than ever due to the
throttling that the mirroring created. But, with the mirror incomplete,
I couldn’t so easily make available incomplete repositories.
The solution was to simply let apache redirect users on to the real
servers if the mirror doesn’t have the file. The first order of
business for that is to rewrite and redirect URLs when files aren’t
found. This is a straightforward Apache configuration:
RewriteEngine on
RewriteLogLevel 0
RewriteCond %{REQUEST_FILENAME} !^/cgi/
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-F
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !(Packages|Sources)\.bz2$
RewriteCond %{REQUEST_URI} !/index\.[^/]*$ [NC]
RewriteRule ^(http://%{HTTP_HOST})?/(.*) http://91.189.88.45/$2 [P]
Note a few things there:
I have to hard-code an IP number, because as I mentioned in
the last
post on this subject, I’ve faked out DNS
for archive.ubuntu.com and other sites I’m mirroring. (Note:
this has the unfortunate side-effect that I can’t easily take advantage
of round-robin DNS on the other side.)
I avoid taking Packages.bz2 from the other site, because
apt-mirror actually doesn’t mirror the bz2 files (although I’ve
submitted a patch to it so it will eventually).
I make sure that index files get built by my Apache and not
redirected.
I am using Apache proxying, which gives me Yet Another type of
cache temporarily while I’m still downloading the other packages. (I
should actually work out a way to have these caches used by apt-mirror
itself in case a user has already requested a new package while waiting
for apt-mirror to get it.)
Once I do a rewrite like this for each of the hosts I’m replacing with
a mirror, I’m almost done. The problem is that if for any reason my
site needs to give a 403 to the clients, I would actually like to
double-check to be sure that the URL doesn’t happen to work at the place
I’m mirroring from.
My hope was that I could write a RewriteRule based on what the
HTTP return code would be when the request completed. This was
really hard to do, it seemed, and perhaps undoable. The quickest
solution I found was to write a CGI script to do the redirect. So, in
the Apache config I have:
ErrorDocument 403 /cgi/redirect-forbidden.cgi
And, the CGI script looks like this:
#!/usr/bin/perl
use strict;
use CGI qw(:standard);
my $val = $ENV{REDIRECT_SCRIPT_URI};
$val =~ s%^http://(\S+).sflc.info(/.*)$%$2%;
if ($1 eq "ubuntu-security") {
$val = "http://91.189.88.37$val";
} else {
$val = "http://91.189.88.45$val";
}
print redirect($val);
With these changes, the user will be redirected to the original when
the files aren’t available on the mirror, and as the mirror gets more
accurate, they’ll get more files from the mirror.
I still have problems if for any reason the user gets a Packages or
Sources file from the original site before the mirror is synchronized,
but this rarely happens since apt-mirror is pretty careful. The only
time it might happen is if the user did an apt-get update when
not connected to our VPN and only a short time later did one while
connected.
Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2008/01/24/apt-mirror-2.html
When I started building our apt-mirror, I ran into a problem: the
machine was throttled against ubuntu.com’s servers, but I had completed
much of the download (which took weeks to get multiple distributions).
I really wanted to roll out the solution quickly, particularly because
the service from the remote servers was worse than ever due to the
throttling that the mirroring created. But, with the mirror incomplete,
I couldn’t so easily make available incomplete repositories.
The solution was to simply let apache redirect users on to the real
servers if the mirror doesn’t have the file. The first order of
business for that is to rewrite and redirect URLs when files aren’t
found. This is a straightforward Apache configuration:
RewriteEngine on
RewriteLogLevel 0
RewriteCond %{REQUEST_FILENAME} !^/cgi/
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-F
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !(Packages|Sources)\.bz2$
RewriteCond %{REQUEST_URI} !/index\.[^/]*$ [NC]
RewriteRule ^(http://%{HTTP_HOST})?/(.*) http://91.189.88.45/$2 [P]
Note a few things there:
I have to hard-code an IP number, because as I mentioned in
the last
post on this subject, I’ve faked out DNS
for archive.ubuntu.com and other sites I’m mirroring. (Note:
this has the unfortunate side-effect that I can’t easily take advantage
of round-robin DNS on the other side.)
I avoid taking Packages.bz2 from the other site, because
apt-mirror actually doesn’t mirror the bz2 files (although I’ve
submitted a patch to it so it will eventually).
I make sure that index files get built by my Apache and not
redirected.
I am using Apache proxying, which gives me Yet Another type of
cache temporarily while I’m still downloading the other packages. (I
should actually work out a way to have these caches used by apt-mirror
itself in case a user has already requested a new package while waiting
for apt-mirror to get it.)
Once I do a rewrite like this for each of the hosts I’m replacing with
a mirror, I’m almost done. The problem is that if for any reason my
site needs to give a 403 to the clients, I would actually like to
double-check to be sure that the URL doesn’t happen to work at the place
I’m mirroring from.
My hope was that I could write a RewriteRule based on what the
HTTP return code would be when the request completed. This was
really hard to do, it seemed, and perhaps undoable. The quickest
solution I found was to write a CGI script to do the redirect. So, in
the Apache config I have:
ErrorDocument 403 /cgi/redirect-forbidden.cgi
And, the CGI script looks like this:
#!/usr/bin/perl
use strict;
use CGI qw(:standard);
my $val = $ENV{REDIRECT_SCRIPT_URI};
$val =~ s%^http://(\S+).sflc.info(/.*)$%$2%;
if ($1 eq "ubuntu-security") {
$val = "http://91.189.88.37$val";
} else {
$val = "http://91.189.88.45$val";
}
print redirect($val);
With these changes, the user will be redirected to the original when
the files aren’t available on the mirror, and as the mirror gets more
accurate, they’ll get more files from the mirror.
I still have problems if for any reason the user gets a Packages or
Sources file from the original site before the mirror is synchronized,
but this rarely happens since apt-mirror is pretty careful. The only
time it might happen is if the user did an apt-get update when
not connected to our VPN and only a short time later did one while
connected.
Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2008/01/24/apt-mirror-2.html
When I started building our apt-mirror, I ran into a problem: the
machine was throttled against ubuntu.com’s servers, but I had completed
much of the download (which took weeks to get multiple distributions).
I really wanted to roll out the solution quickly, particularly because
the service from the remote servers was worse than ever due to the
throttling that the mirroring created. But, with the mirror incomplete,
I couldn’t so easily make available incomplete repositories.
The solution was to simply let apache redirect users on to the real
servers if the mirror doesn’t have the file. The first order of
business for that is to rewrite and redirect URLs when files aren’t
found. This is a straightforward Apache configuration:
RewriteEngine on
RewriteLogLevel 0
RewriteCond %{REQUEST_FILENAME} !^/cgi/
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-F
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !(Packages|Sources)\.bz2$
RewriteCond %{REQUEST_URI} !/index\.[^/]*$ [NC]
RewriteRule ^(http://%{HTTP_HOST})?/(.*) http://91.189.88.45/$2 [P]
Note a few things there:
I have to hard-code an IP number, because as I mentioned in
the last
post on this subject, I’ve faked out DNS
for archive.ubuntu.com and other sites I’m mirroring. (Note:
this has the unfortunate side-effect that I can’t easily take advantage
of round-robin DNS on the other side.)
I avoid taking Packages.bz2 from the other site, because
apt-mirror actually doesn’t mirror the bz2 files (although I’ve
submitted a patch to it so it will eventually).
I make sure that index files get built by my Apache and not
redirected.
I am using Apache proxying, which gives me Yet Another type of
cache temporarily while I’m still downloading the other packages. (I
should actually work out a way to have these caches used by apt-mirror
itself in case a user has already requested a new package while waiting
for apt-mirror to get it.)
Once I do a rewrite like this for each of the hosts I’m replacing with
a mirror, I’m almost done. The problem is that if for any reason my
site needs to give a 403 to the clients, I would actually like to
double-check to be sure that the URL doesn’t happen to work at the place
I’m mirroring from.
My hope was that I could write a RewriteRule based on what the
HTTP return code would be when the request completed. This was
really hard to do, it seemed, and perhaps undoable. The quickest
solution I found was to write a CGI script to do the redirect. So, in
the Apache config I have:
ErrorDocument 403 /cgi/redirect-forbidden.cgi
And, the CGI script looks like this:
#!/usr/bin/perl
use strict;
use CGI qw(:standard);
my $val = $ENV{REDIRECT_SCRIPT_URI};
$val =~ s%^http://(\S+).sflc.info(/.*)$%$2%;
if ($1 eq "ubuntu-security") {
$val = "http://91.189.88.37$val";
} else {
$val = "http://91.189.88.45$val";
}
print redirect($val);
With these changes, the user will be redirected to the original when
the files aren’t available on the mirror, and as the mirror gets more
accurate, they’ll get more files from the mirror.
I still have problems if for any reason the user gets a Packages or
Sources file from the original site before the mirror is synchronized,
but this rarely happens since apt-mirror is pretty careful. The only
time it might happen is if the user did an apt-get update when
not connected to our VPN and only a short time later did one while
connected.
Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2008/01/24/apt-mirror-2.html
When I started building our apt-mirror, I ran into a problem: the
machine was throttled against ubuntu.com’s servers, but I had completed
much of the download (which took weeks to get multiple distributions).
I really wanted to roll out the solution quickly, particularly because
the service from the remote servers was worse than ever due to the
throttling that the mirroring created. But, with the mirror incomplete,
I couldn’t so easily make available incomplete repositories.
The solution was to simply let apache redirect users on to the real
servers if the mirror doesn’t have the file. The first order of
business for that is to rewrite and redirect URLs when files aren’t
found. This is a straightforward Apache configuration:
RewriteEngine on
RewriteLogLevel 0
RewriteCond %{REQUEST_FILENAME} !^/cgi/
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-F
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !(Packages|Sources)\.bz2$
RewriteCond %{REQUEST_URI} !/index\.[^/]*$ [NC]
RewriteRule ^(http://%{HTTP_HOST})?/(.*) http://91.189.88.45/$2 [P]
Note a few things there:
I have to hard-code an IP number, because as I mentioned in
the last
post on this subject, I’ve faked out DNS
for archive.ubuntu.com and other sites I’m mirroring. (Note:
this has the unfortunate side-effect that I can’t easily take advantage
of round-robin DNS on the other side.)
I avoid taking Packages.bz2 from the other site, because
apt-mirror actually doesn’t mirror the bz2 files (although I’ve
submitted a patch to it so it will eventually).
I make sure that index files get built by my Apache and not
redirected.
I am using Apache proxying, which gives me Yet Another type of
cache temporarily while I’m still downloading the other packages. (I
should actually work out a way to have these caches used by apt-mirror
itself in case a user has already requested a new package while waiting
for apt-mirror to get it.)
Once I do a rewrite like this for each of the hosts I’m replacing with
a mirror, I’m almost done. The problem is that if for any reason my
site needs to give a 403 to the clients, I would actually like to
double-check to be sure that the URL doesn’t happen to work at the place
I’m mirroring from.
My hope was that I could write a RewriteRule based on what the
HTTP return code would be when the request completed. This was
really hard to do, it seemed, and perhaps undoable. The quickest
solution I found was to write a CGI script to do the redirect. So, in
the Apache config I have:
ErrorDocument 403 /cgi/redirect-forbidden.cgi
And, the CGI script looks like this:
#!/usr/bin/perl
use strict;
use CGI qw(:standard);
my $val = $ENV{REDIRECT_SCRIPT_URI};
$val =~ s%^http://(\S+).sflc.info(/.*)$%$2%;
if ($1 eq "ubuntu-security") {
$val = "http://91.189.88.37$val";
} else {
$val = "http://91.189.88.45$val";
}
print redirect($val);
With these changes, the user will be redirected to the original when
the files aren’t available on the mirror, and as the mirror gets more
accurate, they’ll get more files from the mirror.
I still have problems if for any reason the user gets a Packages or
Sources file from the original site before the mirror is synchronized,
but this rarely happens since apt-mirror is pretty careful. The only
time it might happen is if the user did an apt-get update when
not connected to our VPN and only a short time later did one while
connected.
Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2008/01/24/apt-mirror-2.html
When I started building our apt-mirror, I ran into a problem: the
machine was throttled against ubuntu.com’s servers, but I had completed
much of the download (which took weeks to get multiple distributions).
I really wanted to roll out the solution quickly, particularly because
the service from the remote servers was worse than ever due to the
throttling that the mirroring created. But, with the mirror incomplete,
I couldn’t so easily make available incomplete repositories.
The solution was to simply let apache redirect users on to the real
servers if the mirror doesn’t have the file. The first order of
business for that is to rewrite and redirect URLs when files aren’t
found. This is a straightforward Apache configuration:
RewriteEngine on
RewriteLogLevel 0
RewriteCond %{REQUEST_FILENAME} !^/cgi/
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-F
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !(Packages|Sources)\.bz2$
RewriteCond %{REQUEST_URI} !/index\.[^/]*$ [NC]
RewriteRule ^(http://%{HTTP_HOST})?/(.*) http://91.189.88.45/$2 [P]
Note a few things there:
I have to hard-code an IP number, because as I mentioned in
the last
post on this subject, I’ve faked out DNS
for archive.ubuntu.com and other sites I’m mirroring. (Note:
this has the unfortunate side-effect that I can’t easily take advantage
of round-robin DNS on the other side.)
I avoid taking Packages.bz2 from the other site, because
apt-mirror actually doesn’t mirror the bz2 files (although I’ve
submitted a patch to it so it will eventually).
I make sure that index files get built by my Apache and not
redirected.
I am using Apache proxying, which gives me Yet Another type of
cache temporarily while I’m still downloading the other packages. (I
should actually work out a way to have these caches used by apt-mirror
itself in case a user has already requested a new package while waiting
for apt-mirror to get it.)
Once I do a rewrite like this for each of the hosts I’m replacing with
a mirror, I’m almost done. The problem is that if for any reason my
site needs to give a 403 to the clients, I would actually like to
double-check to be sure that the URL doesn’t happen to work at the place
I’m mirroring from.
My hope was that I could write a RewriteRule based on what the
HTTP return code would be when the request completed. This was
really hard to do, it seemed, and perhaps undoable. The quickest
solution I found was to write a CGI script to do the redirect. So, in
the Apache config I have:
ErrorDocument 403 /cgi/redirect-forbidden.cgi
And, the CGI script looks like this:
#!/usr/bin/perl
use strict;
use CGI qw(:standard);
my $val = $ENV{REDIRECT_SCRIPT_URI};
$val =~ s%^http://(\S+).sflc.info(/.*)$%$2%;
if ($1 eq "ubuntu-security") {
$val = "http://91.189.88.37$val";
} else {
$val = "http://91.189.88.45$val";
}
print redirect($val);
With these changes, the user will be redirected to the original when
the files aren’t available on the mirror, and as the mirror gets more
accurate, they’ll get more files from the mirror.
I still have problems if for any reason the user gets a Packages or
Sources file from the original site before the mirror is synchronized,
but this rarely happens since apt-mirror is pretty careful. The only
time it might happen is if the user did an apt-get update when
not connected to our VPN and only a short time later did one while
connected.
Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2008/01/24/apt-mirror-2.html
When I started building our apt-mirror, I ran into a problem: the
machine was throttled against ubuntu.com’s servers, but I had completed
much of the download (which took weeks to get multiple distributions).
I really wanted to roll out the solution quickly, particularly because
the service from the remote servers was worse than ever due to the
throttling that the mirroring created. But, with the mirror incomplete,
I couldn’t so easily make available incomplete repositories.
The solution was to simply let apache redirect users on to the real
servers if the mirror doesn’t have the file. The first order of
business for that is to rewrite and redirect URLs when files aren’t
found. This is a straightforward Apache configuration:
RewriteEngine on
RewriteLogLevel 0
RewriteCond %{REQUEST_FILENAME} !^/cgi/
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-F
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !(Packages|Sources)\.bz2$
RewriteCond %{REQUEST_URI} !/index\.[^/]*$ [NC]
RewriteRule ^(http://%{HTTP_HOST})?/(.*) http://91.189.88.45/$2 [P]
Note a few things there:
I have to hard-code an IP number, because as I mentioned in
the last
post on this subject, I’ve faked out DNS
for archive.ubuntu.com and other sites I’m mirroring. (Note:
this has the unfortunate side-effect that I can’t easily take advantage
of round-robin DNS on the other side.)
I avoid taking Packages.bz2 from the other site, because
apt-mirror actually doesn’t mirror the bz2 files (although I’ve
submitted a patch to it so it will eventually).
I make sure that index files get built by my Apache and not
redirected.
I am using Apache proxying, which gives me Yet Another type of
cache temporarily while I’m still downloading the other packages. (I
should actually work out a way to have these caches used by apt-mirror
itself in case a user has already requested a new package while waiting
for apt-mirror to get it.)
Once I do a rewrite like this for each of the hosts I’m replacing with
a mirror, I’m almost done. The problem is that if for any reason my
site needs to give a 403 to the clients, I would actually like to
double-check to be sure that the URL doesn’t happen to work at the place
I’m mirroring from.
My hope was that I could write a RewriteRule based on what the
HTTP return code would be when the request completed. This was
really hard to do, it seemed, and perhaps undoable. The quickest
solution I found was to write a CGI script to do the redirect. So, in
the Apache config I have:
ErrorDocument 403 /cgi/redirect-forbidden.cgi
And, the CGI script looks like this:
#!/usr/bin/perl
use strict;
use CGI qw(:standard);
my $val = $ENV{REDIRECT_SCRIPT_URI};
$val =~ s%^http://(\S+).sflc.info(/.*)$%$2%;
if ($1 eq "ubuntu-security") {
$val = "http://91.189.88.37$val";
} else {
$val = "http://91.189.88.45$val";
}
print redirect($val);
With these changes, the user will be redirected to the original when
the files aren’t available on the mirror, and as the mirror gets more
accurate, they’ll get more files from the mirror.
I still have problems if for any reason the user gets a Packages or
Sources file from the original site before the mirror is synchronized,
but this rarely happens since apt-mirror is pretty careful. The only
time it might happen is if the user did an apt-get update when
not connected to our VPN and only a short time later did one while
connected.
Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2008/01/24/apt-mirror-2.html
When I started building our apt-mirror, I ran into a problem: the
machine was throttled against ubuntu.com’s servers, but I had completed
much of the download (which took weeks to get multiple distributions).
I really wanted to roll out the solution quickly, particularly because
the service from the remote servers was worse than ever due to the
throttling that the mirroring created. But, with the mirror incomplete,
I couldn’t so easily make available incomplete repositories.
The solution was to simply let apache redirect users on to the real
servers if the mirror doesn’t have the file. The first order of
business for that is to rewrite and redirect URLs when files aren’t
found. This is a straightforward Apache configuration:
RewriteEngine on
RewriteLogLevel 0
RewriteCond %{REQUEST_FILENAME} !^/cgi/
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-F
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !(Packages|Sources)\.bz2$
RewriteCond %{REQUEST_URI} !/index\.[^/]*$ [NC]
RewriteRule ^(http://%{HTTP_HOST})?/(.*) http://91.189.88.45/$2 [P]
Note a few things there:
I have to hard-code an IP number, because as I mentioned in
the last
post on this subject, I’ve faked out DNS
for archive.ubuntu.com and other sites I’m mirroring. (Note:
this has the unfortunate side-effect that I can’t easily take advantage
of round-robin DNS on the other side.)
I avoid taking Packages.bz2 from the other site, because
apt-mirror actually doesn’t mirror the bz2 files (although I’ve
submitted a patch to it so it will eventually).
I make sure that index files get built by my Apache and not
redirected.
I am using Apache proxying, which gives me Yet Another type of
cache temporarily while I’m still downloading the other packages. (I
should actually work out a way to have these caches used by apt-mirror
itself in case a user has already requested a new package while waiting
for apt-mirror to get it.)
Once I do a rewrite like this for each of the hosts I’m replacing with
a mirror, I’m almost done. The problem is that if for any reason my
site needs to give a 403 to the clients, I would actually like to
double-check to be sure that the URL doesn’t happen to work at the place
I’m mirroring from.
My hope was that I could write a RewriteRule based on what the
HTTP return code would be when the request completed. This was
really hard to do, it seemed, and perhaps undoable. The quickest
solution I found was to write a CGI script to do the redirect. So, in
the Apache config I have:
ErrorDocument 403 /cgi/redirect-forbidden.cgi
And, the CGI script looks like this:
#!/usr/bin/perl
use strict;
use CGI qw(:standard);
my $val = $ENV{REDIRECT_SCRIPT_URI};
$val =~ s%^http://(\S+).sflc.info(/.*)$%$2%;
if ($1 eq "ubuntu-security") {
$val = "http://91.189.88.37$val";
} else {
$val = "http://91.189.88.45$val";
}
print redirect($val);
With these changes, the user will be redirected to the original when
the files aren’t available on the mirror, and as the mirror gets more
accurate, they’ll get more files from the mirror.
I still have problems if for any reason the user gets a Packages or
Sources file from the original site before the mirror is synchronized,
but this rarely happens since apt-mirror is pretty careful. The only
time it might happen is if the user did an apt-get update when
not connected to our VPN and only a short time later did one while
connected.
Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2008/01/24/apt-mirror-2.html
When I started building our apt-mirror, I ran into a problem: the
machine was throttled against ubuntu.com’s servers, but I had completed
much of the download (which took weeks to get multiple distributions).
I really wanted to roll out the solution quickly, particularly because
the service from the remote servers was worse than ever due to the
throttling that the mirroring created. But, with the mirror incomplete,
I couldn’t so easily make available incomplete repositories.
The solution was to simply let apache redirect users on to the real
servers if the mirror doesn’t have the file. The first order of
business for that is to rewrite and redirect URLs when files aren’t
found. This is a straightforward Apache configuration:
RewriteEngine on
RewriteLogLevel 0
RewriteCond %{REQUEST_FILENAME} !^/cgi/
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-F
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !(Packages|Sources)\.bz2$
RewriteCond %{REQUEST_URI} !/index\.[^/]*$ [NC]
RewriteRule ^(http://%{HTTP_HOST})?/(.*) http://91.189.88.45/$2 [P]
Note a few things there:
I have to hard-code an IP number, because as I mentioned in
the last
post on this subject, I’ve faked out DNS
for archive.ubuntu.com and other sites I’m mirroring. (Note:
this has the unfortunate side-effect that I can’t easily take advantage
of round-robin DNS on the other side.)
I avoid taking Packages.bz2 from the other site, because
apt-mirror actually doesn’t mirror the bz2 files (although I’ve
submitted a patch to it so it will eventually).
I make sure that index files get built by my Apache and not
redirected.
I am using Apache proxying, which gives me Yet Another type of
cache temporarily while I’m still downloading the other packages. (I
should actually work out a way to have these caches used by apt-mirror
itself in case a user has already requested a new package while waiting
for apt-mirror to get it.)
Once I do a rewrite like this for each of the hosts I’m replacing with
a mirror, I’m almost done. The problem is that if for any reason my
site needs to give a 403 to the clients, I would actually like to
double-check to be sure that the URL doesn’t happen to work at the place
I’m mirroring from.
My hope was that I could write a RewriteRule based on what the
HTTP return code would be when the request completed. This was
really hard to do, it seemed, and perhaps undoable. The quickest
solution I found was to write a CGI script to do the redirect. So, in
the Apache config I have:
ErrorDocument 403 /cgi/redirect-forbidden.cgi
And, the CGI script looks like this:
#!/usr/bin/perl
use strict;
use CGI qw(:standard);
my $val = $ENV{REDIRECT_SCRIPT_URI};
$val =~ s%^http://(\S+).sflc.info(/.*)$%$2%;
if ($1 eq "ubuntu-security") {
$val = "http://91.189.88.37$val";
} else {
$val = "http://91.189.88.45$val";
}
print redirect($val);
With these changes, the user will be redirected to the original when
the files aren’t available on the mirror, and as the mirror gets more
accurate, they’ll get more files from the mirror.
I still have problems if for any reason the user gets a Packages or
Sources file from the original site before the mirror is synchronized,
but this rarely happens since apt-mirror is pretty careful. The only
time it might happen is if the user did an apt-get update when
not connected to our VPN and only a short time later did one while
connected.
Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2008/01/24/apt-mirror-2.html
When I started building our apt-mirror, I ran into a problem: the
machine was throttled against ubuntu.com’s servers, but I had completed
much of the download (which took weeks to get multiple distributions).
I really wanted to roll out the solution quickly, particularly because
the service from the remote servers was worse than ever due to the
throttling that the mirroring created. But, with the mirror incomplete,
I couldn’t so easily make available incomplete repositories.
The solution was to simply let apache redirect users on to the real
servers if the mirror doesn’t have the file. The first order of
business for that is to rewrite and redirect URLs when files aren’t
found. This is a straightforward Apache configuration:
RewriteEngine on
RewriteLogLevel 0
RewriteCond %{REQUEST_FILENAME} !^/cgi/
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-F
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !(Packages|Sources)\.bz2$
RewriteCond %{REQUEST_URI} !/index\.[^/]*$ [NC]
RewriteRule ^(http://%{HTTP_HOST})?/(.*) http://91.189.88.45/$2 [P]
Note a few things there:
I have to hard-code an IP number, because as I mentioned in
the last
post on this subject, I’ve faked out DNS
for archive.ubuntu.com and other sites I’m mirroring. (Note:
this has the unfortunate side-effect that I can’t easily take advantage
of round-robin DNS on the other side.)
I avoid taking Packages.bz2 from the other site, because
apt-mirror actually doesn’t mirror the bz2 files (although I’ve
submitted a patch to it so it will eventually).
I make sure that index files get built by my Apache and not
redirected.
I am using Apache proxying, which gives me Yet Another type of
cache temporarily while I’m still downloading the other packages. (I
should actually work out a way to have these caches used by apt-mirror
itself in case a user has already requested a new package while waiting
for apt-mirror to get it.)
Once I do a rewrite like this for each of the hosts I’m replacing with
a mirror, I’m almost done. The problem is that if for any reason my
site needs to give a 403 to the clients, I would actually like to
double-check to be sure that the URL doesn’t happen to work at the place
I’m mirroring from.
My hope was that I could write a RewriteRule based on what the
HTTP return code would be when the request completed. This was
really hard to do, it seemed, and perhaps undoable. The quickest
solution I found was to write a CGI script to do the redirect. So, in
the Apache config I have:
ErrorDocument 403 /cgi/redirect-forbidden.cgi
And, the CGI script looks like this:
#!/usr/bin/perl
use strict;
use CGI qw(:standard);
my $val = $ENV{REDIRECT_SCRIPT_URI};
$val =~ s%^http://(\S+).sflc.info(/.*)$%$2%;
if ($1 eq "ubuntu-security") {
$val = "http://91.189.88.37$val";
} else {
$val = "http://91.189.88.45$val";
}
print redirect($val);
With these changes, the user will be redirected to the original when
the files aren’t available on the mirror, and as the mirror gets more
accurate, they’ll get more files from the mirror.
I still have problems if for any reason the user gets a Packages or
Sources file from the original site before the mirror is synchronized,
but this rarely happens since apt-mirror is pretty careful. The only
time it might happen is if the user did an apt-get update when
not connected to our VPN and only a short time later did one while
connected.
Post Syndicated from Bradley M. Kuhn original http://ebb.org/bkuhn/blog/2008/01/24/apt-mirror-2.html
When I started building our apt-mirror, I ran into a problem: the
machine was throttled against ubuntu.com’s servers, but I had completed
much of the download (which took weeks to get multiple distributions).
I really wanted to roll out the solution quickly, particularly because
the service from the remote servers was worse than ever due to the
throttling that the mirroring created. But, with the mirror incomplete,
I couldn’t so easily make available incomplete repositories.
The solution was to simply let apache redirect users on to the real
servers if the mirror doesn’t have the file. The first order of
business for that is to rewrite and redirect URLs when files aren’t
found. This is a straightforward Apache configuration:
RewriteEngine on
RewriteLogLevel 0
RewriteCond %{REQUEST_FILENAME} !^/cgi/
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-F
RewriteCond /var/spool/apt-mirror/mirror/archive.ubuntu.com%{REQUEST_FILENAME} !-d
RewriteCond %{REQUEST_URI} !(Packages|Sources)\.bz2$
RewriteCond %{REQUEST_URI} !/index\.[^/]*$ [NC]
RewriteRule ^(http://%{HTTP_HOST})?/(.*) http://91.189.88.45/$2 [P]
Note a few things there:
I have to hard-code an IP number, because as I mentioned in
the last
post on this subject, I’ve faked out DNS
for archive.ubuntu.com and other sites I’m mirroring. (Note:
this has the unfortunate side-effect that I can’t easily take advantage
of round-robin DNS on the other side.)
I avoid taking Packages.bz2 from the other site, because
apt-mirror actually doesn’t mirror the bz2 files (although I’ve
submitted a patch to it so it will eventually).
I make sure that index files get built by my Apache and not
redirected.
I am using Apache proxying, which gives me Yet Another type of
cache temporarily while I’m still downloading the other packages. (I
should actually work out a way to have these caches used by apt-mirror
itself in case a user has already requested a new package while waiting
for apt-mirror to get it.)
Once I do a rewrite like this for each of the hosts I’m replacing with
a mirror, I’m almost done. The problem is that if for any reason my
site needs to give a 403 to the clients, I would actually like to
double-check to be sure that the URL doesn’t happen to work at the place
I’m mirroring from.
My hope was that I could write a RewriteRule based on what the
HTTP return code would be when the request completed. This was
really hard to do, it seemed, and perhaps undoable. The quickest
solution I found was to write a CGI script to do the redirect. So, in
the Apache config I have:
ErrorDocument 403 /cgi/redirect-forbidden.cgi
And, the CGI script looks like this:
#!/usr/bin/perl
use strict;
use CGI qw(:standard);
my $val = $ENV{REDIRECT_SCRIPT_URI};
$val =~ s%^http://(\S+).sflc.info(/.*)$%$2%;
if ($1 eq "ubuntu-security") {
$val = "http://91.189.88.37$val";
} else {
$val = "http://91.189.88.45$val";
}
print redirect($val);
With these changes, the user will be redirected to the original when
the files aren’t available on the mirror, and as the mirror gets more
accurate, they’ll get more files from the mirror.
I still have problems if for any reason the user gets a Packages or
Sources file from the original site before the mirror is synchronized,
but this rarely happens since apt-mirror is pretty careful. The only
time it might happen is if the user did an apt-get update when
not connected to our VPN and only a short time later did one while
connected.