The Transportation Security Administration has setup a new page on their website to react to the recent story about a woman being detained at DCA Reagan Airport for spilling her toddler's "sippy" cup. I'm not going to discuss the specifics of that case, as it's not important.
What is interesting, is that the new section of TSA's website is called "MythBusters". As many of you may know, MythBusters is the name of a hugely popular TV show on the Discovery Channel.
It turns out that Discovery Communications has registered the term MythBusters with the US Trademark office as: Entertainment services in the nature of non-fiction television programming featuring examination of popularly held beliefs and misinformation; information regarding same provided via a global computer network.
Trademark isn't really an area of IP law that I know too much about. My (limited) expertise thus far is mainly in copyright - as it is the area that I'm most likely to get myself in trouble with.
I know that the US government has carved itself a number of exceptions in other areas of the law. For example, the US government cannot commit the tort of "patent infringement" - and instead, merely has to pay "reasonable and entire compensation" to a patent owner for the unauthorized use of a patent.
So - I pose the question to those out there who know more about the law than me:
Can TSA name a section of their website MythBusters (note, even the same capitalization as the TV show) without breaking any laws? TSA's new website provides video footage that examines popularly held beliefs and misinformation, which is then delivered via a global computer network (the Internet).
Is this legit, or did TSA fail to run this by their in-house counsel?
Friday, June 15, 2007
Thursday, June 14, 2007
A Takedown Resistant, Unofficial Podcast Feed for "This American Life"
Update Sept 28, 2008:
I get a fair number of hits to this page from people looking for mp3 copies of This American Life episodes.
The short version, is that you can get each episode here: http://audio.thisamericanlife.org/jomamashouse/ismymamashouse/EPISODENUMBER.mp3
The longer explanation can be found here: New location for This American Life Mp3s
The podcast feed that I created, as described in this blog post, was taken down due to a letter sent by This American Life. To read it, go here: An Emotional Blackmail Takedown: Remove The Podcast, Or We Shoot This Puppy
The vanilla feed, which lists the 15 most recent episodes as broadcast by TAL.
A feed of only new episodes, which does not include the reruns as broadcast by TAL.
Please note that this was not done with the permission or consent of NPR, PRI, or the fine folks at This American Life. Show your support, go to the TAL website, and donate 20 bucks.
Introduction
I'm a big fan of the Chicago Public Radio show This American Life (TAL), which is broadcast on many US National Public Radio affiliates nationwide. For some reason, the public radio and TV business (in the US and elsewhere) has not yet figured out this whole Internet thing, and thus many of their online/podcast offerings are less than stellar.
Many young people do not listen to radio anymore. Even when I'm in the US - simply put - if This American Life, Democracy Now, Open Source, Radio Lab and other fantastic radio shows were not available as podcasts - there is no way I'd listen to them. I cannot concentrate on work while listening to an informative radio show, and thus for the most part, streaming them on my computer is just not practical. Instead, they are perfect for the tram ride/walk to work, airplane/bus trips, or walks in the park.
TAL's current business/podcasting model is as follows:
This is sub-optimal for many reasons - the most important of which is that I have no desire to enrich Steve Jobs. When I give money to TAL, it is through a direct donation on their website. I do not wish for Apple, or any other middleman to take 30-50%.
TAL has clearly been struggling to find a business plan that works - and are probably contractually bound by their recently renegotiated contract with Audible.com to not allow their back-catalog to be podcasted for free. As Current.org noted, "[t]he barrier to podcasting for years was the longstanding Audible deal. The vendor sold episodes of TAL for $3.95 a pop, barring the show from offering free downloads."
TAL has also recently had a podcasting fund-drive to pay for their yearly bandwidth bill of $108k. I've given them $20, but I can't figure out why they don't just put all of their content on archive.org (as Democracy Now has done), and thus do away with their bandwidth bill completely. Likewise, TAL is extremely popular amongst many tech geek circles - I am sure that a company or two would step up to the plate and give them free bandwidth if they asked.
I suspect that their unwillingness to let others host their media comes from their desire to somehow find a way to preserve their audible.com/iTunes deal. If they are making more than $108k through paid downloads, then perhaps this is a wise choice. The nicer, and smarter approach (in my humble opinion) would be to ditch the paid podcasting model, allow other organizations to host the TAL podcasts, and thus do away with that nasty bandwidth bill. In three weeks of fundraising, they were able to raise over $110k - more than enough to cover the costs of their 300,000 podcast downloads per week. If bandwidth were provided by others for free, this money could instead go towards TAL's other operating costs - and thus make up for the loss of the iTunes/audible.com revenue stream.
When This American Life first moved from Realaudio to MP3 storage of their show archive, the decision seems to be primarily due to issues with Realplayer - and not because they wished to enable listeners to save copies of shows to their machines. While many listeners petitioned TAL for a podcast feed, the show resisted.
Due to the fact that the episodes were being stored as MP3s, it was quite possible for users to download them to their own computers, and for others to create a podcast RSS feed.
In 2006, Jared Benedict and Jon Udell did just this, and made unofficial This American Life podcast feeds. Soon, links appeared in popular websites such as BoingBoing, and shortly after, the TAL webmaster sent a polite takedown request to both gentlemen. Jared sums it up by stating, "we received friendly emails from Ms. Meister, This American Life’s webmaster, making a request to take down the hyperlinks and RSS feeds, or she’d regrettably have to get lawyers involved. While Ms. Meister did miss the mark by accusing us of copyright infringement without a clear understanding of what we were actually doing, or what copyright law allows, she was trying to be polite and friendly which I appreciate."
Subsequently, both men took down their podcast feeds.
Last year, I discovered that TAL has all of their mp3s in one directory, amusingly located at http://audio.thisamericanlife.org/jomamashouse/ismymamashouse/episodenum.mp3. Clearly, their webmaster has a sense of humor.
Unfortunately, it is not possible to give itunes/other podcast clients a directory to download mp3 files from, so a podcast RSS feed is necessary. TAL's official podcast feed only has the most current episode available for download - which doesn't help me when I miss a week, or if I want to load up my iPod for a long journey.
I've learned enough about Internet law at this point to know that website owners are within their rights to get angry if you scrape their website on a regular basis. This comes down to a tort claim of trespass to chattels.
While TAL does provide a podcast RSS feed that I could scrape, I thought it best to avoid downloading data from there, if just to provide a bit of legal distance, and avoid any claim that I was hammering their servers.
Yesterday, I announced and made available a script that pulls data from Google's cache of popular RSS feeds. By using that script, I can be sure that I am not directly accessing TAL's webservers, nor am I increasing the load on their servers (assuming of course, that at least a couple other people using Google Reader to keep up to date with TAL episodes).
Regarding copyright - The content of the episodes, show logos, etc are protected by copyright law. However, episode names, broadcast dates and other info are not. It is for this same reason that one cannot copyright tv schedule listings.
The issue of deep linking (i.e. providing a direct link to a mp3 file on TAL's website) is a legal grey area. The case law here is by no means solid. A few years back, A California judged ruled that, "hyperlinking does not itself involve a violation of the Copyright Act" because "no copying is involved." I am fairly confident that I am on firm ground - since, afterall, the content I am linking to is perfectly legal (thus the DeCSS decision shouldn't apply) and accessible by any user who visits the TAL website and uses their embedded flash mp3 player.
With those legal issues dealt with for the most part, I decided to roll my own.
I've setup two unofficial podcast feeds for This American Life.
The vanilla feed, which lists the 15 most recent episodes as broadcast by TAL.
A feed of only new episodes, which does not include the reruns as broadcast by TAL.
Please note that this was not done with the permission or consent of NPR, PRI, or the fine folks at This American Life. Show your support, go to the TAL website, and donate 20 bucks.
I get a fair number of hits to this page from people looking for mp3 copies of This American Life episodes.
The short version, is that you can get each episode here: http://audio.thisamericanlife.
The longer explanation can be found here: New location for This American Life Mp3s
The podcast feed that I created, as described in this blog post, was taken down due to a letter sent by This American Life. To read it, go here: An Emotional Blackmail Takedown: Remove The Podcast, Or We Shoot This Puppy
The vanilla feed, which lists the 15 most recent episodes as broadcast by TAL.
A feed of only new episodes, which does not include the reruns as broadcast by TAL.
Please note that this was not done with the permission or consent of NPR, PRI, or the fine folks at This American Life. Show your support, go to the TAL website, and donate 20 bucks.
Introduction
I'm a big fan of the Chicago Public Radio show This American Life (TAL), which is broadcast on many US National Public Radio affiliates nationwide. For some reason, the public radio and TV business (in the US and elsewhere) has not yet figured out this whole Internet thing, and thus many of their online/podcast offerings are less than stellar.
Many young people do not listen to radio anymore. Even when I'm in the US - simply put - if This American Life, Democracy Now, Open Source, Radio Lab and other fantastic radio shows were not available as podcasts - there is no way I'd listen to them. I cannot concentrate on work while listening to an informative radio show, and thus for the most part, streaming them on my computer is just not practical. Instead, they are perfect for the tram ride/walk to work, airplane/bus trips, or walks in the park.
TAL's current business/podcasting model is as follows:
- The latest episode can be downloaded through a podcast feed for free from their website.
- Archived episodes can be streamed online (via a flash player) on their website but not downloaded.
- Listeners wishing to podcast archived episodes must buy them online - from either Apple's iTunes store, or audible.com for 99 cents
This is sub-optimal for many reasons - the most important of which is that I have no desire to enrich Steve Jobs. When I give money to TAL, it is through a direct donation on their website. I do not wish for Apple, or any other middleman to take 30-50%.
TAL has clearly been struggling to find a business plan that works - and are probably contractually bound by their recently renegotiated contract with Audible.com to not allow their back-catalog to be podcasted for free. As Current.org noted, "[t]he barrier to podcasting for years was the longstanding Audible deal. The vendor sold episodes of TAL for $3.95 a pop, barring the show from offering free downloads."
TAL has also recently had a podcasting fund-drive to pay for their yearly bandwidth bill of $108k. I've given them $20, but I can't figure out why they don't just put all of their content on archive.org (as Democracy Now has done), and thus do away with their bandwidth bill completely. Likewise, TAL is extremely popular amongst many tech geek circles - I am sure that a company or two would step up to the plate and give them free bandwidth if they asked.
I suspect that their unwillingness to let others host their media comes from their desire to somehow find a way to preserve their audible.com/iTunes deal. If they are making more than $108k through paid downloads, then perhaps this is a wise choice. The nicer, and smarter approach (in my humble opinion) would be to ditch the paid podcasting model, allow other organizations to host the TAL podcasts, and thus do away with that nasty bandwidth bill. In three weeks of fundraising, they were able to raise over $110k - more than enough to cover the costs of their 300,000 podcast downloads per week. If bandwidth were provided by others for free, this money could instead go towards TAL's other operating costs - and thus make up for the loss of the iTunes/audible.com revenue stream.
When This American Life first moved from Realaudio to MP3 storage of their show archive, the decision seems to be primarily due to issues with Realplayer - and not because they wished to enable listeners to save copies of shows to their machines. While many listeners petitioned TAL for a podcast feed, the show resisted.
Due to the fact that the episodes were being stored as MP3s, it was quite possible for users to download them to their own computers, and for others to create a podcast RSS feed.
In 2006, Jared Benedict and Jon Udell did just this, and made unofficial This American Life podcast feeds. Soon, links appeared in popular websites such as BoingBoing, and shortly after, the TAL webmaster sent a polite takedown request to both gentlemen. Jared sums it up by stating, "we received friendly emails from Ms. Meister, This American Life’s webmaster, making a request to take down the hyperlinks and RSS feeds, or she’d regrettably have to get lawyers involved. While Ms. Meister did miss the mark by accusing us of copyright infringement without a clear understanding of what we were actually doing, or what copyright law allows, she was trying to be polite and friendly which I appreciate."
Subsequently, both men took down their podcast feeds.
Last year, I discovered that TAL has all of their mp3s in one directory, amusingly located at http://audio.thisamericanlife.org/jomamashouse/ismymamashouse/episodenum.mp3. Clearly, their webmaster has a sense of humor.
Unfortunately, it is not possible to give itunes/other podcast clients a directory to download mp3 files from, so a podcast RSS feed is necessary. TAL's official podcast feed only has the most current episode available for download - which doesn't help me when I miss a week, or if I want to load up my iPod for a long journey.
I've learned enough about Internet law at this point to know that website owners are within their rights to get angry if you scrape their website on a regular basis. This comes down to a tort claim of trespass to chattels.
While TAL does provide a podcast RSS feed that I could scrape, I thought it best to avoid downloading data from there, if just to provide a bit of legal distance, and avoid any claim that I was hammering their servers.
Yesterday, I announced and made available a script that pulls data from Google's cache of popular RSS feeds. By using that script, I can be sure that I am not directly accessing TAL's webservers, nor am I increasing the load on their servers (assuming of course, that at least a couple other people using Google Reader to keep up to date with TAL episodes).
Regarding copyright - The content of the episodes, show logos, etc are protected by copyright law. However, episode names, broadcast dates and other info are not. It is for this same reason that one cannot copyright tv schedule listings.
The issue of deep linking (i.e. providing a direct link to a mp3 file on TAL's website) is a legal grey area. The case law here is by no means solid. A few years back, A California judged ruled that, "hyperlinking does not itself involve a violation of the Copyright Act" because "no copying is involved." I am fairly confident that I am on firm ground - since, afterall, the content I am linking to is perfectly legal (thus the DeCSS decision shouldn't apply) and accessible by any user who visits the TAL website and uses their embedded flash mp3 player.
With those legal issues dealt with for the most part, I decided to roll my own.
I've setup two unofficial podcast feeds for This American Life.
The vanilla feed, which lists the 15 most recent episodes as broadcast by TAL.
A feed of only new episodes, which does not include the reruns as broadcast by TAL.
Please note that this was not done with the permission or consent of NPR, PRI, or the fine folks at This American Life. Show your support, go to the TAL website, and donate 20 bucks.
Wednesday, June 13, 2007
Coding Around Trespass To Chattel with Google Feeds API
Begin Update July 27 2012:
Many popular websites now provide RSS/XML feeds for all kinds of data relating to their website. Digg headlines, CNN news, Craigslist for sale items, and millions of blogs.
What happens when you want to build on that data in a way that the website owner didn't plan for, and probably won't approve of?
Since they're providing an RSS feed to the general public, it's unlikely that they can stop you from downloading it. Depending on how you are using the data, they may or may not be able to use copyright law against you. However, they may be able to claim that you're hammering their servers , and thus taking up their resources. In such a situation, normally, if they tell you to stop and you continue, you could be in trouble. This comes under a claim of trespass to chattel - an arcane area of the law that has been used by companies such as Ebay and American Airlines to stop people from scraping their website.
Google has a very popular online RSS reader service, unsurprisingly named Google Reader. Instead of requesting any given website's RSS feed every time one of their millions of users wishes to view this feed in their browser, Google instead serves the user a cached copy. Google's feed crawler ("Feedfetcher") software retrieves feeds from most sites less than once every hour. Some frequently updated sites may be refreshed more often.
The benefits to website owners is clear: Instead of getting hit once every hour by hundreds of thousands of people, they get visited maybe once every hour by Google's software, who then deal with the bandwidth issues involved in getting that data to customers.
Google is even nice enough to modify the user agent string that the feed crawler sends to webservers, to tell website owners exactly how many people have subscribed to the feed.
If you are logged into a Google service (such as Gmail), you can view Google's cached copy of any RSS feed by going to: http://www.google.com/reader/atom/feed/the_feed_you_want
For example, the This American Life podcast feed can be seen by going to: http://www.google.com/reader/atom/feed/http://feeds.thisamericanlife.org/talpodcast
Unfortunately, this cannot be automated with a script - as it requires that you login to a Google Account first. While this can be automated somewhat using the Google provided ClientLogin/AuthSub mechanisms, it would still mean that you'd be scraping the query results. ClientLogin (I believe) also requires that you solve a CAPTCHA, which isn't going to work for a ruby script running on a remote server. Google's terms of service forbid users from automating and scraping data returned from Google queries. If you want to get data from Google with a script, you need to use one of their APIs.
Luckily - Google recently announced a AJAX Feed API that permits developers to embed data from a cached RSS feed in their websites. The new Feed API allows you to put a bit of javascript on your website - which will then automatically display the contents of an RSS feed. That's great, but it's not quite what I'm after. As I want to do this from a ruby/perl script, and not from within a javascript webpage.
Niall Kennedy reverse engineered the API a bit, and figured out how to get a JSON encoded version of a cached RSS feed from Google's servers. When Niall first started looking into the guts of the Reader and its APIs late last year, members of the Google Reader development team left comments on his blog commending him on his reverse engineering, and provided key bits of information. Thus, while the release of this information isn't 100% sanctioned by Google, its fair to assume that Google is aware it is out there. Most importantly, this method uses the API (and requires an API key that you can request from Google) - which means that in using it, I'm safer and more legit.
Thus, I whipped up a ruby script that will parse the JSON output from Google, and give you a real RSS feed. A live demo, and source code of the RSS/XML cache fetcher can be downloaded here.
Why is this useful? It means that you can scrape someone's RSS/XML feed without ever going to their website. While I'm no lawyer (and this is certainly not legal advice) - I'd imagine that it'd be fairly difficult for anyone to attempt a trespass to chattel claim, since they'll be unable to prove any harm, or consumption of resources.
Google has millions of customers. It's quite likely that their bot is already crawling most popular RSS feeds. Thus, it's very very unlikely that by pulling up a copy of the feed, that you'll cause Google to go and fetch a new copy.
Many websites already provide different (unpassword protected) data to Google than they do to anyone else visting their site. It may be possible, by using Google's RSS cache, to take advantage of this architecture flaw/design decision, and access data that one wouldn't normally be able to get.
Finally, since you won't be hitting anyone's webservers, there is no link (at least in their weblogs) between you and them. They have no way of knowing how often you're accessing their feed. You're hidden amongst the millions of Google users.
The day after I published this post in 2007, I published a podcast feed for the NPR radio show This American Life, which used the Google Cache method to build an RSS feed for the popular radio show (TAL only provides the most recent couple shows available to the public for free via a podcast, rather than older shows).
Soon after, I took that podcast feed down after I was directly contacted by someone working for the radio show.
Fast forward five years, and Craigslist is now suing a company called 3Taps for using a method which appears to be surprisingly similar to the technique I described in 2007.
As in 2007, the legal issues surrounding this technique are unclear at best. Eric DeMenthon, the founder of PadMapper, a company using 3Taps data which was also sued by Craigslist told Ars Technica that "Since I'm not actually re-posting the content of the listings, just the facts about the listings, I figured (with legal advice) that there was no real copyright issue there."
Perhaps the courts will finally resolve this interesting legal question.
End update
Check out the RSS/XML cache fetcher here.
Many popular websites now provide RSS/XML feeds for all kinds of data relating to their website. Digg headlines, CNN news, Craigslist for sale items, and millions of blogs.
What happens when you want to build on that data in a way that the website owner didn't plan for, and probably won't approve of?
Since they're providing an RSS feed to the general public, it's unlikely that they can stop you from downloading it. Depending on how you are using the data, they may or may not be able to use copyright law against you. However, they may be able to claim that you're hammering their servers , and thus taking up their resources. In such a situation, normally, if they tell you to stop and you continue, you could be in trouble. This comes under a claim of trespass to chattel - an arcane area of the law that has been used by companies such as Ebay and American Airlines to stop people from scraping their website.
Google has a very popular online RSS reader service, unsurprisingly named Google Reader. Instead of requesting any given website's RSS feed every time one of their millions of users wishes to view this feed in their browser, Google instead serves the user a cached copy. Google's feed crawler ("Feedfetcher") software retrieves feeds from most sites less than once every hour. Some frequently updated sites may be refreshed more often.
The benefits to website owners is clear: Instead of getting hit once every hour by hundreds of thousands of people, they get visited maybe once every hour by Google's software, who then deal with the bandwidth issues involved in getting that data to customers.
Google is even nice enough to modify the user agent string that the feed crawler sends to webservers, to tell website owners exactly how many people have subscribed to the feed.
If you are logged into a Google service (such as Gmail), you can view Google's cached copy of any RSS feed by going to: http://www.google.com/reader/atom/feed/the_feed_you_want
For example, the This American Life podcast feed can be seen by going to: http://www.google.com/reader/atom/feed/http://feeds.thisamericanlife.org/talpodcast
Unfortunately, this cannot be automated with a script - as it requires that you login to a Google Account first. While this can be automated somewhat using the Google provided ClientLogin/AuthSub mechanisms, it would still mean that you'd be scraping the query results. ClientLogin (I believe) also requires that you solve a CAPTCHA, which isn't going to work for a ruby script running on a remote server. Google's terms of service forbid users from automating and scraping data returned from Google queries. If you want to get data from Google with a script, you need to use one of their APIs.
Luckily - Google recently announced a AJAX Feed API that permits developers to embed data from a cached RSS feed in their websites. The new Feed API allows you to put a bit of javascript on your website - which will then automatically display the contents of an RSS feed. That's great, but it's not quite what I'm after. As I want to do this from a ruby/perl script, and not from within a javascript webpage.
Niall Kennedy reverse engineered the API a bit, and figured out how to get a JSON encoded version of a cached RSS feed from Google's servers. When Niall first started looking into the guts of the Reader and its APIs late last year, members of the Google Reader development team left comments on his blog commending him on his reverse engineering, and provided key bits of information. Thus, while the release of this information isn't 100% sanctioned by Google, its fair to assume that Google is aware it is out there. Most importantly, this method uses the API (and requires an API key that you can request from Google) - which means that in using it, I'm safer and more legit.
Thus, I whipped up a ruby script that will parse the JSON output from Google, and give you a real RSS feed. A live demo, and source code of the RSS/XML cache fetcher can be downloaded here.
Why is this useful? It means that you can scrape someone's RSS/XML feed without ever going to their website. While I'm no lawyer (and this is certainly not legal advice) - I'd imagine that it'd be fairly difficult for anyone to attempt a trespass to chattel claim, since they'll be unable to prove any harm, or consumption of resources.
Google has millions of customers. It's quite likely that their bot is already crawling most popular RSS feeds. Thus, it's very very unlikely that by pulling up a copy of the feed, that you'll cause Google to go and fetch a new copy.
Many websites already provide different (unpassword protected) data to Google than they do to anyone else visting their site. It may be possible, by using Google's RSS cache, to take advantage of this architecture flaw/design decision, and access data that one wouldn't normally be able to get.
Finally, since you won't be hitting anyone's webservers, there is no link (at least in their weblogs) between you and them. They have no way of knowing how often you're accessing their feed. You're hidden amongst the millions of Google users.
Subscribe to:
Posts (Atom)