Find Top Blog Posts Using Yahoo Pipes with AideRSS

I’ve been really excited about the potential of Yahoo Pipes recently, and as a result I’ve spent quite a bit of time playing with Yahoo Pipes over the past couple of weeks. I recently put together a pipe that I am finding really useful, and I thought a few others might find it interesting, too.

The problem:

When reading my rss feeds, I tend to skip blog posts with titles that do not immediately catch my eye as something interesting. As a result, I sometimes miss important news or ideas that everyone else is talking about.

The solution:

I decided to put together a pipe that takes some of my favorite blogs as inputs and sends the posts through AideRSS to find the ones with the most comments, discussion, bookmarks, etc.

Details:

  • I put together a csv file with some of my favorite blogs formatted without the leading http:// to make them easier to process through AideRSS (alternatively, you could also bring in the complete URL and use pipes, to reformat the strings, but I was striving for simple). I then pulled this into the pipe as input using the Fetch CSV module.
  • I then used the Loop module with an embedded URL Builder module to append the appropriate string (blog url from the csv file) to an AideRSS URL (filtering on only the “great” posts). The output from this module produces a bunch of URLs each looking something like this:

    http://aiderss.com/rss/great/webworkerdaily.com

  • I ran this output through another Loop module with an embedded Fetch Feed module to fetch each individual blog post from each URL built in the previous step.
  • In order to filter out any duplicates, I then ran it through a Unique Filter module based on item link. You would only need this step if one or more of your original sources in the csv file aggregates feeds from other sources.
  • I also wanted to limit my results to blog posts from the past 5 days, so I used the Filter Module along with the Date Builder module to restrict the dates.
  • The result of the above steps gives you the basic information, but I decided that I also wanted to reformat the titles to add the AideRSS rating and post date directly into the title, so that I could easily see which ones were the most important. I used yet another Loop module with an embedded String Builder module to add additional data to the title. I then stored the output back into the item title, which results in titles like this:
    Rank 10.0 1-19 This is the blog post title
  • My final step was to sort the items using the Sort module to put the highest rated posts (using AideRSS rating) at the top with a secondary sort by date that puts the newest posts at the top when you have several posts with the same rating.

Viola! I have a pipe that finds the most important blog posts for me. Keep in mind that this will never help you find breaking news, since it usually takes a day or so for many posts to accumulate enough comments / links / etc. to have a high AideRSS rating, but it does keep you from missing really important news and ideas.

You can view the source of the Top Blog Posts pipe or get the RSS feed. You can also clone the pipe when viewing the source if you want to use it as a starting point for something else you want to do.

Related Fast Wonder Blog posts:

4 Responses to “Find Top Blog Posts Using Yahoo Pipes with AideRSS”


  • This is fantastic! Thanks for putting this together. This has been a great way for me to dive in and learn Pipes by taking what you have done apart a bit and figure out what is going on.

    Thanks again!

  • If you happen to use Google Reader + Firefox, they have an extension that lets you do all kinds of cool filtering. It rocks.

  • Hey, cool tool — nice job! I found this post while searching for a Yahoo! Pipe that would strip out all of the “near dupes” from a Summize search. In other words, I want to reduce all of the duplicate reposts of the same story to just one (even if the shortened URLs are different). Running the RSS of a Summize search for “phillies” through your tool and you’ll still see like 15 copies of the same article. Have any suggestions? Thanks!!

  • Hi Steve,

    Unfortunately, I don’t think you can use Yahoo Pipes to remove duplicates where people are using several shortened URLs linking back to the same source article. Yahoo Pipes is a great tool, but it is pretty limited in functionality, and what you want to do would require something a little more sophisticated.

Comments are currently closed.