comma space

MetaRSS Diagram

dml — Sun, 15 Oct 2006 01:58:52 +0000

Essentiall, MetaRSS is comprised of the following components:

A cron job
An indexer
Metafilter content
An HTML cache
The user interface

The process is initiated by a user request for an RSS feed. This is currently done with a Greasemonkey script or a bookmarklet. The request simply feeds the user interface a URL. The user interface caches the HTML and stores the url for user later by the indexer. The user interface parses the html and returns the RSS to the user. At fixed intervals, a cron jobs kicks off the indexer, which simply caches the HTML, which is parsed when a user requests the cache.

Pretty simple and open to a lot of improvement.

For example, the parser is very rigid. Essentially it’s hard coded, using some weird combo of HTML::Parser and regular expressions. If the parser could scrape more flexibly, MetaRSS could return more types of RSS feeds.

Additionally, the algorithm that indexes the requested Metafilter pages could be optimized by using last modified times to determine whether or not to grab the whole page.

SourceForge

dml — Sat, 14 Oct 2006 17:50:07 +0000

MetaRSS was approved, though DNS still has not updated.

More to come.

MetaRSS: The Open Source Project

dml — Tue, 10 Oct 2006 17:43:33 +0000

It’s no secret: MetaRSS was hacked together with the best of intentions in the least amount of time possible. As such, MetaRSS went live in alpha and never saw much further development. Since that time, the name has been taken by another company, several bugs have been discovered (more on that later), reported, and fixed. Given that there is, at the very least, a modicum of interest in extensible RSS feeds for Metafilter, and little promise that they’ll actually show up as a feature, I’ve decided to put the code out there.

Perhaps the project is too small of a niche to stir up much, if any, interest in the community, but I can’t know until I make the effort. Therefore, I have submitted a project proposal to sourceFORGE.net for hosting the repository and related collaborative tools. Though the project hasn’t been accepted yet, I’m going to begin to lay the groundwork so that when it finally does gain approval, we can hit the ground running.

First, let me take a few minutes to explain what MetaRSS does and should do. Currently, it provides RSS feeds for the main properties and indivudal threads in such a manner that reduces load on the MeFi servers to one hit, per feed, per hour. (With Matt’s permission, I’d be interested in increasing the frequency.) In the future, MetaRSS should provide per user and per generic search feeds, as well as other specialty feeds.

In addition to providing generic RSS feeds, MetaRSS caused me to look into FeedBurner’s API and an RSS splicer. It seems to me that providing integration of these services would add to the value of the product over all. The way I see it, MetaRSS should handle front-end requests for feed creation, splicing, and customization, as well as do all of the feed creation, but FeedBurner should bear the brunt of serving the actual feeds.

Clearly, this description isn’t even remotely close to a spec. It’s just a general description of the state of things as they are now and as I hope them to be. In the end, if a community of develipers emerge, the product will open to consensus development. The features that the developers and users have the most interest in will drive the development, not me. The code and the product will be free to all, though I should note that Matt has a significant say in all of this, despite remaining silent thus far.

MetaRSS moved!

dml — Sun, 04 Jun 2006 04:00:32 +0000

As much fun as playing with Joomla was, I just don’t have the time or patience to stick with it.

Not to mention, the end result of said futzing was, well, pretty unusable. Therefore, MetaRSS has moved to this mostly empty blog. None of the script locations have changed, but the user interface doesn’t need the complications of an obtuse CMS. All of the tools you may have had trouble finding previously are now handily available on the pages of comma space.

Here’s a quick reference for the existing MetaRSS resources

metarss

metarss bookmarklets

rss tools

What is MetaRSS?

dml — Sat, 03 Jun 2006 14:01:25 +0000

Metafilter offers RSS feeds for the front pages of MeFi and AskMe. While I am thankful for the feeds, they do not help me keep up with activity after I’ve seen a thread. Every hour MetaRSS updates threads that it knows about, for up to thirty days. When your RSS reader hits MetaRSS the next time, the newest comments will be visibile in your RSS reader. (Please don’t update your RSS more frequently than once an hour.)

How does it work?

When you see a thread you want to read in your RSS reader, simply use one of the bookmarklets to the left. If it is a new thread, it caches the thread immediately and displays the available RSS. If the thread exists, the existing RSS is displayed. If you use Bloglines, My Yahoo or Newsgator, you can use the specialized bookmarklets below to do the very same thing while adding it to your RSS reader at the same time.

A Word About Ownership

The content available via RSS feeds through this tool belongs to the respective posters. Additionally, I do not have any association, other than by membership, with Metafilter.