Leon Weber, leon@leonweber.de
August 22, 2010
A few weeks ago, the server hosting the Chaosradio Podcast portal and the associated blog received some maintenance, which did not go too well. On bootup, the system reported a filesystem inconsistency, which resulted in random files and directories on the system being simply gone. This led to a couple of problems with the operating system installation, most of which could be resolved quite easily, however, the directory containing the main website's static HTML files was one of the victims – so I had to wait for Tim Pritlove to be reachable, who had a backup of the website.
In the meantime, I figured it might be a good idea to redirect all those HTTP requests for the main website to the blog, and about an hour later it came to my mind that there is actually no point in redirecting to the blog until there is an explanation of the outage there, so I changed the redirect to point an error message that I called fnord.txt.
Well, after a while, all problems with the systems were resolved and everything was working again – it seemed. After some time, users started complaining about subscribing to podcasts from iTunes not working anymore. Turns out: all our feed URLs in the iTunes Podcast Directory were wrong. This happened because when I installed the redirects to first the blog and then fnord.txt, I was using lighttpd's url.redirect feature, which in lighttpd 1.4 returns the HTTP 301 Moved Permanently status code. The more appropriate answer would have been 302 Found, which, according to RFC 2616, expresses that
The requested resource resides temporarily under a different URI.
which is exactly what was the case here.
The iTunes Podcast Directory regularly checks the feed URLs in its database, and apparently, it is an iTunes feature to permanently change the podcasts' URL when it receives a 301 Moved Permanently answer (which is correct behaviour, following RFC 2616, so unfortunately, we cannot blame iTunes here) – resulting in all of the 16 Chaosradio podcast feeds having an incorrect URL in the iTunes database, and, to make matters worse, they were not unique anymore. Those 16 podcasts all pointed either to http://chaosradio.ccc.de/fnord.txt or http://blog.chaosradio.ccc.de, neither of which is a valid RSS feed, of course.
So, this is the problem. But how do you solve it? Removing and re-adding the feeds to iTunes was not an option, as apparently there are statistics and comments that would have been lost. We could not simply redirect them backwards either, as they were not unique anymore. Thankfully, Finn Wilke came up with an idea to solve the situation.
And this is exactly what we did. We could recognize iTunes' request by its User-agent iTMS. PHP was already installed on the server anyways, so fnord.txt became the following PHP script:
<?php
if(trim($_SERVER['HTTP_USER_AGENT']) != 'iTMS') {
header("HTTP/1.0 404 Not Found");
}
else {
$file = uniqid("", true);
$hdl = fopen("/var/www/chaosradio.ccc.de/temp/$file", 'w');
fclose($hdl);
header("Location: http://chaosradio.ccc.de/temp/$file", true, 301);
}
?>
This would create an empty file with a unique name in http://chaosradio.ccc.de/temp/ and redirect there, if the request originates from iTunes. For those podcasts that pointed to http://blog.chaosradio.ccc.de/, I added a similar code snippet at the top of Wordpress's index.php file (without the 404 Not Found part, of course), and then we just had to wait about 24 hours.
After that, 16 files had been created in http://chaosradio.ccc.de/temp/, so next, we needed to figure out the random URL that each podcast pointed to. This was possible using the iTunes Ping method, which is actually designed to request iTunes to re-read the feed, but as a side-effect it also returns the current feed URL from the database.
Having all the information we needed, we could proceed with step 3: We replaced the created files with PHP scripts, redirecting to the respective feed URL, like this:
$ cat 4c6ef831a4acc1.71174537
<?php
header("Location: http://chaosradio.ccc.de/24c3_mp3-all.rss", true, 301);
?>
...and, after another 24 hours, all 16 podcasts in the iTunes Podcast Directory now point to their correct URLs. Yay!