Tag Archive for 'rss'

Hacking RSS at Webvisions

In March, I gave a talk at SXSW on Hacking RSS: Filtering & Processing Obscene Amounts of Information. Today, at Webvisions, I will be giving the short and sweet (12 minute) version of this presentation.

For more details, you can get the full presentation materials and listen to the audio from the SXSW talk.

Hacking RSS: Filtering & Processing Obscene Amounts of Information at SXSW

My SXSW session this year, Hacking RSS: Filtering & Processing Obscene Amounts of Information, is at the coveted 9:30 am session time on the final day of SXSW, so I thought that it might be a good idea to outline the presentation here in the hopes that I can entice a few people  to drag themselves out of bed early to attend.

Presentation

You can listen to the audio of my presentation.

Description

Information overload is less about having too much information and more about not having the right tools and techniques to filter and process information to find the pieces that are most relevant for you. This presentation will focus on showing you a variety of tips and techniques to get you started down the path of looking at RSS feeds in a completely different light. The default RSS feeds generated by your favorite blog or website are just a starting point waiting to be hacked and manipulated to serve your needs. Most people read RSS feeds, but few people take the time to go one step further to hack on those RSS feeds to find only the most interesting posts. I combine tools like Yahoo Pipes, BackTweets, PostRank and more with some simple API calls to be able to find what I need while automatically discarding the rest. You start with one or more RSS feeds and then feed those results into other services to gather more information that can be used to further filter or process the results. This process is easier than it sounds once you learn a few simple tools and techniques, and no “real” programming experience is required to get started. This session will show you some tips and tricks to get you started down the path of hacking your RSS feeds.

A few specific topics

Logistics

  • Tuesday March 15 at 9:30AM
  • Venue: Austin Convention Center Ballroom C
  • Tag: #hackingRSS

UPDATED to add embedded slides on 3/15/2011 and add audio link on 3/23/2011.

Want to see me talk about RSS Hacking at SXSW?

If you want to see me speak at SXSW, you can comment and vote on my session before 11:59 CDT on Friday, August 27. Here’s how SXSW panel selection is weighted: 30% popular vote (you!), 30% program staff and 40% advisory board, so I’ll need some votes before they will accept my session! Here are the details about my proposed session so you can decide on whether to vote for it.

Hacking RSS: Filtering & Processing Obscene Amounts of Information

Information overload is less about having too much information and more about not having the right tools and techniques to filter and process information to find the pieces that are most relevant for you. This presentation will focus on showing you a variety of tips and techniques to get you started down the path of looking at RSS feeds in a completely different light. The default RSS feeds generated by your favorite blog or website are just a starting point waiting to be hacked and manipulated to serve your needs. Most people read RSS feeds, but few people take the time to go one step further to hack on those RSS feeds to find only the most interesting posts. I combine tools like Yahoo Pipes, BackTweets, PostRank and more with some simple API calls to be able to find what I need while automatically discarding the rest. You start with one or more RSS feeds and then feed those results into other services to gather more information that can be used to further filter or process the results. This process is easier than it sounds once you learn a few simple tools and techniques, and no “real” programming experience is required to get started. This session will show you some tips and tricks to get you started down the path of hacking your RSS feeds.

Using Yahoo Pipes to Filter Results for Iterasi PositivePress

I’ve been beta testing Iterasi’s PositivePress for a while and was excited to finally see it released to the public! It’s a great way to track and archive blog posts and articles from across the web while creating reports with links to certain articles that can be delivered to clients or friends. CNET did a great job of covering the release, so I won’t rehash it here. I encourage you to read the CNET article for more information about PositivePress.

Since PositivePress uses RSS feeds as input, I decided to do a quick video showing you how to use Yahoo Pipes to filter results from your RSS feed to make sure that you are only archiving and reporting on the most relevant results. This video shows you how to take a list of feeds from a CSV file, fetch the results, and filter them for a set of keywords. The end result is fed into PositivePress and a new report is generated from the filtered results.

If you haven’t already watched the rest of the Yahoo Pipes Video series, you might benefit from watching these other videos first. They provide a little more background on the concepts used in this Yahoo Pipe.

This Yahoo Pipes video series is also available as a podcast on iTunes, and as a separate feed.

You can watch the embedded video above, but I recommend downloading the higher resolution Quicktime file (30 MB) to watch. The quality will be much better than the flash version above.

More Details on the Pipe:

  • The Demo Pipe. A copy of the Using Yahoo Pipes with Iterasi PositivePress pipe click “View Source” to see the modules.
  • Fetch CSV Module. Enter the URL of a CSV file. Here is the example we used in the video.
  • Loop module with Fetch Feed. Loops through each element in the CSV file and fetches the feed associated with the item.
  • Filter Module (Permit). Filter by a couple of keywords to permit only the results that match at least one of your keywords
  • Filter Module (Block). Filter out any results that match undesired keywords.
  • Sort Module. Sort by date in descending order to make sure things are sorted in a logical manner.
  • Pipe Output. The final module in every Yahoo Pipe.

I’ve created many Yahoo Pipes, and most of them have been published on my Yahoo Pipes and RSS Hacks page where you can also learn more about my Yahoo Pipes Training courses.

Demystifying Social Media Tools and Techniques

I was in Eugene today to talk to the Willamette Valley AMA about social media. The presentation was similar to the one that I gave earlier this year at WebVisions, but with a few more details on how to use some of the various tools. Here are the topics that I covered and a copy of my slides.

  • Guiding Principles & Strategy for Participation
  • Social Media Activities / Tools
    • Twitter
    • Facebook
    • LinkedIn
    • FriendFeed
    • Blogging
  • Monitoring
    • RSS
    • Monitoring Twitter
    • Yahoo Pipes
  • Managing your social media efforts

Contact me if you would like to have me train your company on online communities or social media.

Related Fast Wonder blog posts:

Introduction to Yahoo Pipes: Monitoring Conversations – June 25

My next Yahoo Pipes class is scheduled for June 25th. For those who aren’t familiar with Yahoo Pipes, it is a tool that I use to monitor conversations happening across various social media sites. While Yahoo Pipes can also do so much more, I have focused this introductory class on monitoring conversations.

Register for Introduction to Yahoo Pipes: Monitoring Conversations

When: Thursday, June 25, 2009 from 5:00pm – 7:30pm
City: Portland, OR
Location: CubeSpace 622 SE Grand Ave., Portland OR (no remote attendance)
Learn more: Prerequisites, Course Outline and Information

Costs:

  • Early Bird: $124.99
  • Late Registration (After June 18): $199.99
  • Students, Freelancers, or Unemployed can contact me for lower prices [UPDATED 6/5/09 with tweaks to pricing for the class]

Prerequisites

  • You will need to bring a laptop to the training class
  • You must have a Yahoo account (Flickr / Upcoming) and confirm that you can access Pipes with that account
  • You will also need to create a Google Spreadsheet with 5-6 keywords formatted like this prior to the class.
  • No prior knowledge of Yahoo Pipes is required

Why you need this course to learn about Yahoo Pipes

This course will teach you how using Yahoo Pipes can help you understand what people are saying about you, your industry, your competitors and more through smart filtering of blogs, news sources, Twitter, and other online sites. Your customers are talking about you and your competitors are revealing information that you want to know online. Can you find it quickly and efficiently now?

  • Become more responsive to your customers by knowing when and where people are talking about your company and products on blogs and Twitter. Find and respond more quickly and efficiently.
  • Use what people are saying about your company and your products to improve your products / services, marketing messages, web content, documentation and other communications.
  • Get insight into your competitors.
  • Keep up with important information about your industry by focusing on keyword filtering to find the most relevant content for your situation.
  • Use the information to get ideas for blog posts or other communication.
  • Tailor your online research to your specific needs and interest areas.

Visit the Introduction to Yahoo Pipes training page for the course outline and other information about the class.

Yahoo Pipes Training Class May 7th in Portland

I am holding my first Introduction to Yahoo Pipes training course on May 7th. This Yahoo Pipes training course is designed for people who are new to Yahoo Pipes. In 2 hours, we will cover the basics of building Yahoo Pipes from building your first Yahoo Pipe to some more advanced uses.

When: Thursday, May 7, 2009 from 3:00pm – 5:00pm
City: Portland, OR
Location: WebTrends 851 SW 6th Ave., Suite 1600 (no remote attendance)
Learn more: Prerequisites, Course Outline and Information

Register for the course

Pricing:

  • Students, freelancers, or unemployed: $100
  • Early Bird (prior to April 23rd): $150
  • Late registration (after April 23rd): $250

Prerequisites

  • You must have a Yahoo Pipes account.
  • No prior knowledge of Yahoo Pipes is required

Why you need this course to learn about Yahoo Pipes
This course will teach you how using Yahoo Pipes can help you understand what people are saying about you, your industry, your competitors and more through smart filtering of blogs, news sources, Twitter, and other online sites. Your customers are talking about you and your competitors are revealing information that you want to know online. Can you find it quickly and efficiently now?

  • Become more responsive to your customers by knowing when and where people are talking about your company and products on blogs and Twitter. Find and respond more quickly and efficiently.
  • Use what people are saying about your company and your products to improve your products / services, marketing messages, web content, documentation and other communications.
  • Get insight into your competitors.
  • Keep up with important information about your industry by focusing on keyword filtering to find the most relevant content for your situation.
  • Use the information to get ideas for blog posts or other communication.
  • Tailor your online research to your specific needs and interest areas.

Learn more: Prerequisites, Course Outline and Information

If you can’t attend this course or want to be notified about future Fast Wonder training, you can subscribe to my training notifications.


Register for the course

Accepting Google Docs Input: 2 Minute Yahoo Pipes Demo

This video shows how to use data from Google Docs as an input into Yahoo Pipes. In a previous video, I showed how to use a list of items in a CSV file. If you haven’t already watched the CSV input demo, you should start with it, since this pipe builds on the previous one. A CSV file is a great choice when you want to fetch a bunch of feeds and change them frequently without having to update your pipe, but a Google Doc is a great choice for when you want to easily update inputs to the pipe without needing to update a file on a webserver.

More Details

  • The Demo Pipe. A copy of the Using Google Docs Inputs in Yahoo Pipes Demo click “View Source” to see the modules.
  • Fetch CSV Module. Enter the URL of a Google Spreadsheet that has already been published as a web page and append &output=csv to the end of the URL. Here is the example we used in the video.
  • Loop module with Fetch Feed. Loops through each element in the CSV file and fetches the feed associated with the item.
  • Filter Module. Filter by a couple of keywords.
  • Sort Module. Sort by date in descending order to make sure things are sorted in a logical manner.
  • Pipe Output. The final module in every Yahoo Pipe.

I’ve created many Yahoo Pipes, and most of them have been published on my Yahoo Pipes and RSS Hacks page where you can also learn more about my Yahoo Pipes Training courses.

Related Fast Wonder Blog posts

YQL (Yahoo Query Language): 2 Minute Yahoo Pipes Video Demo

After learning that Yahoo Pipes introduced a new module called YQL (Yahoo Query Language) last week, I knew that I had to find a way to work it into one of my 2 minute videos. YQL uses an SQL-like syntax for more powerful and flexible inputs into Yahoo Pipes; however, working it into a 2 minute demo was quite challenging, since queries can be complicated. I cheated a little by glossing over the query language and focusing on how to use YQL effectively within Pipes. Honestly, I’m never going to teach someone to use an SQL syntax without extensive training, so this 2 minute demo is really geared toward people with database experience or programming backgrounds. I also did a more complicated version of this pipe with a more extensive explanation that you might be interested in reading to get more details about using YQL within Pipes.

More Details

  • The Demo Pipe. A copy of the YQL: 2 Minute Yahoo Pipes Demo pipe click “View Source” to see the modules.
  • YQL. The query:
    select * from flickr.photos.info(20) where photo_id in (select id from flickr.photos.search(20) where tags = “igniteportland”).
    Selects 20 photos from Flickr with the tag “igniteportland”. flickr.photos.search is used to find the images, but it doesn’t provide much information about the images, so I ran it through flickr.photos.info to pull in more data. If you omit the (20), YQL returns 10 results by default, and you can set this number to something appropriate for your needs.
  • Rename. Maps item.urls.url.content into the more standard item.link, which is recognized by RSS readers and the Yahoo Pipes output for clickable links back to the original content.
  • Pipe Output. The final module in every Yahoo Pipe.

I’ve created many Yahoo Pipes, and most of them have been published on my Yahoo Pipes and RSS Hacks page where you can also learn more about my Yahoo Pipes Training courses.

Related Fast Wonder Blog posts

Find Flickr Comments by Tag Using YQL in Yahoo Pipes

This week, Yahoo Pipes introduced a new module called YQL (Yahoo Query Language) allowing more powerful and flexible inputs into Yahoo Pipes using a SQL-like syntax.

The Flickr Comments by Tag pipe uses the new YQL module to look for any photos matching a certain tag that also have comments. In this pipe, I’m using the YQL module to pull some data out of Flickr that was not previously available in Flickr rss feeds or using the Flickr module in Yahoo Pipes. However, the data is available in the API and can be easily accessed via Yahoo Pipes using the new YQL module. I’ve also made the pipe configurable by prompting for user input, which allows other people to easily use the pipe whether or not they understand YQL.

Usage:

  1. Go to the Flickr Comments by Tag pipe.
  2. Enter a tag and click “run pipe”
  3. Grab the RSS feed output

The Technical Details on Using YQL in Yahoo Pipes

Caveat. The use of this module is better suited for developers, instead of casual users of Yahoo Pipes. If you’ve never done any command line database manipulation or programming, I suspect that there will be a steep learning curve associated with using the YQL module.

YQL Query. The query I’m using is a variation of the one below, but with the query built using the String Builder module, which includes a user input as part of the string. If you aren’t familiar with user inputs in Yahoo Pipes, you might want to watch the User Input: 2 Minute Yahoo Pipes Video Demo.

select * from flickr.photos.info(200) where photo_id in (select id from flickr.photos.search(200) where tags = “userinput”)

Basically, this query says that it is finding all photos where the tag matches the userinput string. By default, YQL returns only 10 items from a table, which is not sufficient for most uses within Yahoo Pipes, so I added a parameter to get 200 items. You need to change this parameter for each table you are using in the query. I also noticed that I was experiencing intermittent issues with pipe when I used a value over 200, so you will need to be careful when setting this parameter.

Filter. After the YQL module, I ran the output through a standard Filter module permitting only the items that matched: item.comments > 0.

Loop (feed modification). For those of you familiar with my Yahoo Pipes style, you know that I frequently use the loop module to modify the title of each item in the feed to include more information. In this case, I wanted to know the number of comments at a glance without having to click each item to get the numbers. This step is optional.

Rename. For some reason, the URL coming out of the flickr data is not automatically stored in item.link. The Yahoo Pipes output and RSS readers expect a feed to have the source url of the image stored in item.link, so you will need to manually rename item.urls.url.content to link using the Rename module. Without this step, you cannot click on any of the images to see the text of the comments.

Flickr Comments by Tag Using YQL in Yahoo Pipes

You’ll probably want to look at the source of the Flickr Comments by Tag pipe for more details.

The full YQL documentation is available on the Yahoo Developer Network. There also include several data sets available by default in YQL including Flickr, Upcoming, MyBlogLog, Yahoo Social, weather, geo / location, and more along with other standard data formats (JSON, RSS, XML, etc.)

Related Fast Wonder Blog posts