Tag Archives: Yahoo

rss

Find Flickr Comments by Tag Using YQL in Yahoo Pipes

2 Comments

This week, Yahoo Pipes introduced a new module called YQL (Yahoo Query Language) allowing more powerful and flexible inputs into Yahoo Pipes using a SQL-like syntax.

The Flickr Comments by Tag pipe uses the new YQL module to look for any photos matching a certain tag that also have comments. In this pipe, I’m using the YQL module to pull some data out of Flickr that was not previously available in Flickr rss feeds or using the Flickr module in Yahoo Pipes. However, the data is available in the API and can be easily accessed via Yahoo Pipes using the new YQL module. I’ve also made the pipe configurable by prompting for user input, which allows other people to easily use the pipe whether or not they understand YQL.

Usage:

  1. Go to the Flickr Comments by Tag pipe.
  2. Enter a tag and click “run pipe”
  3. Grab the RSS feed output

The Technical Details on Using YQL in Yahoo Pipes

Caveat. The use of this module is better suited for developers, instead of casual users of Yahoo Pipes. If you’ve never done any command line database manipulation or programming, I suspect that there will be a steep learning curve associated with using the YQL module.

YQL Query. The query I’m using is a variation of the one below, but with the query built using the String Builder module, which includes a user input as part of the string. If you aren’t familiar with user inputs in Yahoo Pipes, you might want to watch the User Input: 2 Minute Yahoo Pipes Video Demo.

select * from flickr.photos.info(200) where photo_id in (select id from flickr.photos.search(200) where tags = “userinput”)

Basically, this query says that it is finding all photos where the tag matches the userinput string. By default, YQL returns only 10 items from a table, which is not sufficient for most uses within Yahoo Pipes, so I added a parameter to get 200 items. You need to change this parameter for each table you are using in the query. I also noticed that I was experiencing intermittent issues with pipe when I used a value over 200, so you will need to be careful when setting this parameter.

Filter. After the YQL module, I ran the output through a standard Filter module permitting only the items that matched: item.comments > 0.

Loop (feed modification). For those of you familiar with my Yahoo Pipes style, you know that I frequently use the loop module to modify the title of each item in the feed to include more information. In this case, I wanted to know the number of comments at a glance without having to click each item to get the numbers. This step is optional.

Rename. For some reason, the URL coming out of the flickr data is not automatically stored in item.link. The Yahoo Pipes output and RSS readers expect a feed to have the source url of the image stored in item.link, so you will need to manually rename item.urls.url.content to link using the Rename module. Without this step, you cannot click on any of the images to see the text of the comments.

Flickr Comments by Tag Using YQL in Yahoo Pipes

You’ll probably want to look at the source of the Flickr Comments by Tag pipe for more details.

The full YQL documentation is available on the Yahoo Developer Network. There also include several data sets available by default in YQL including Flickr, Upcoming, MyBlogLog, Yahoo Social, weather, geo / location, and more along with other standard data formats (JSON, RSS, XML, etc.)

Related Fast Wonder Blog posts

Yahoo Pipes Videos

Modifying RSS Feeds: 2 Minute Yahoo Pipes Video Demo

6 Comments

While many people use Yahoo Pipes to filter RSS feeds, Pipes can also be used to modify RSS feeds to work better for your purposes. In this example, we will add the author name to the beginning of the title to make it easier to see the author without opening the item in your rss reader. I recommend watching the Introduction to Yahoo Pipes: 2 minute Yahoo Pipes Video Demo if you haven’t already, since we’re going use the basic pipe developed in that demo using fetch feed and sort modules, but without the filter module.

More Details

  • The Demo Pipe. A copy of the Modifying RSS Feeds demo pipe click “View Source” to see the modules.
  • Fetch Feed. Fetches the elements from 2 feeds: ReadWriteWeb and GigaOM.
  • Sort Module. Sort by date in descending order to make sure things are sorted in a logical manner.
  • Loop module with String Builder Module. Loops through each element in the feed and builds a string with item.author: item.title. This string is assigned back into item.title.
  • Pipe Output. The final module in every Yahoo Pipe.

I’ve created many Yahoo Pipes, and most of them have been published on my Yahoo Pipes and RSS Hacks page where you can also learn more about my Yahoo Pipes Training courses.

Related Fast Wonder Blog posts

portland

Portland Data Plumbing User Group: Discuss RSS, Yahoo Pipes, and More!

5 Comments

I’ve decided to resurrect the Portland Data Plumbing Group to give us a time and place to talk about RSS feed hacking, Yahoo Pipes, Dapper, and other related technologies.

We’ll be having these meetings on the 2nd Tuesday of every month at 6pm (location TBD). The first meeting will be on:
Tuesday, January 13th
6:00pm – 7:30pm
Location TBD
Please RSVP on Upcoming if you plan to attend.

The agenda for the first meeting:

  • Intros
  • Round table discussion: each person gets 3-5 min to talk about the coolest thing they’ve done to manipulate an RSS feed.
  • Talk about ideas for future agendas.

If you want to be notified of future meetings, you should subscribe to the Portland Data Plumbing Google Group. A huge thanks to Justin for starting this group and for encouraging me to schedule a new meeting!

Related Fast Wonder Blog posts

Yahoo Pipes Videos

CSV File Inputs: 2 Minute Yahoo Pipes Video Demo

1 Comment

This video shows how to get input from a list of items in a CSV file, and it introduces the loop module. A CSV file is a great choice when you want to fetch a bunch of feeds and change them frequently without having to update your pipe. If you haven’t already watched the 2 Minute Yahoo Pipes Introductory Demo, I strongly suggest that you watch it. This demo builds on the basic structure developed in the introductory demo. The CSV file I’m using also contains the same feeds from ReadWriteWeb and GigaOM that we used in the original demo plus a few others.

More Details

  • The Demo Pipe. A copy of the CSV File Input in Yahoo Pipes Demo click “View Source” to see the modules.
  • Fetch CSV Module. Enter the URL of the CSV file (make sure that it already exists in a publicly accessible location) along with information about column names and separation characters as needed.
  • Loop module with Fetch Feed. Loops through each element in the CSV file and fetches the feed associated with the item.
  • Filter Module. Filter by a couple of keywords.
  • Sort Module. Sort by date in descending order to make sure things are sorted in a logical manner.
  • Pipe Output. The final module in every Yahoo Pipe.

I’ve created many Yahoo Pipes, and most of them have been published on my Yahoo Pipes and RSS Hacks page where you can also learn more about my Yahoo Pipes Training courses.

Related Fast Wonder Blog posts

Yahoo Pipes Videos

Using The Debugger: 2 Minute Yahoo Pipes Video Demo

1 Comment

The debugger serves a couple of functions in Yahoo Pipes. The first one is pretty obvious – debugging. It allows you to see exactly what is happening in each component of your Yahoo Pipe to help with the debugging process when something isn’t working quite right.

The debugger also serves a less obvious purpose. You can use it to learn more about the elements of the rss feeds that you are using as input. For most blogs, the rss feeds are fairly standard, but other services store all kinds of useful information in their rss feeds in addition to standard elements like link, title, and date. The debugger can help you find this additional information in the feed.

More Details

  • The Demo Pipe. A copy of the Yahoo Pipes Debugger Demo Pipe. Click “View Source” to see the modules.
  • Fetch Feed Module. Contains 1 feed from Shizzow, which contains a wealth of location information within the feed.
  • Filter Module. Filters posts with a certain keyword out of the feed to demonstrate the debugger.
  • Pipe Output. The final module in every Yahoo Pipe.
  • The demo also spends time using the debugger to explore various elements of the feed.

I’ve created many Yahoo Pipes, and most of them have been published on my Yahoo Pipes and RSS Hacks page where you can also learn more about my Yahoo Pipes Training courses.

Previous videos in the series:

Yahoo Pipes Videos

User Input: 2 Minute Yahoo Pipes Video Demo

3 Comments

This video shows how to request a user input and use the keyword from the user as the filter in a Yahoo Pipe. If you haven’t already watched the 2 Minute Yahoo Pipes Introductory Demo, I strongly suggest that you watch it. The user input demo starts with the simple pipe created in the introductory demo and changes the filters from keywords hardcoded in the introductory pipe to the keyword from the user input.

Podcast Feed and iTunes

More Details

  • The Demo Pipe. A copy of the User Input Demo Pipe click “View Source” to see the modules.
  • Fetch Feed Module. Contains 2 feeds: ReadWriteWeb and GigaOM.
  • Text Input Module. Leave Name and Position as the defaults. Prompt: “enter a keyword” or other instructions for the user. Default and Debug: “apple” in this example. You can generally use the same term for default and debug, but make sure that you use a word commonly found in your feeds that will generate enough volume for testing and will generate default results for your users to view.
  • Filter Module. Filter by the keyword entered in the user input module by connecting the text input to the filter module as seen in the demo.
  • Sort Module. Sort by date in descending order to make sure things are sorted in a logical manner.
  • Pipe Output. The final module in every Yahoo Pipe.

I’ve created many Yahoo Pipes, and most of them have been published on my Yahoo Pipes and RSS Hacks page where you can also learn more about my Yahoo Pipes Training courses.

Related Fast Wonder Blog posts

Yahoo

Yahoo Pipes for Social Media Monitoring and Business Benefit

4 Comments

Many large companies can afford to spring for expensive software packages that monitor what people are saying about your company, industry, and competition online; however, for those of us looking for an inexpensive solution, Yahoo Pipes can be an important tool.

After I published the 2 minute Yahoo Pipes demo on Monday, I realized that I have done many posts about Yahoo Pipes without ever writing about why people should care about the technology and how it can used. I decided it was time to devote a post to explaining how people can use Yahoo Pipes and get actual business benefits.

Using Yahoo Pipes can help you understand what people are saying about you, your industry, your competitors and more through smart filtering of blogs, news sources, Twitter, and other online sites. Your customers are talking about you and your competitors are revealing information that you want to know online. Can you find it quickly and efficiently now?

A few business benefits from using Yahoo Pipes:

  • Become more responsive to your customers by knowing when and where people are talking about your company and products on blogs and Twitter. Find and respond more quickly and efficiently.
  • Use what people are saying about your company and your products to improve your products / services, marketing messages, web content, documentation and other communications.
  • Get insight into your competitors.
  • Keep up with important information about your industry by focusing on keyword filtering to find the most relevant content for your situation.
  • Monitoring dashboards will help you disseminate the information from Yahoo Pipes throughout the rest of your organization.

While I think that monitoring is always important, I usually emphasize it when I am working on blogging strategies with clients. Knowing what people say about your organization, your industry, and your competition is an important element of a blogging strategy. All of the information found via Yahoo Pipes for the purposes above can then be used to generate ideas for blog posts or other communications.

What business benefits do you get from Yahoo Pipes?

Related Fast Wonder Blog posts

Yahoo Pipes Videos

Yahoo Pipes Training: A 2 Minute Yahoo Pipes Demo

22 Comments

If you have been wanting to create Yahoo Pipes, but weren’t quite sure where to start when faced with the blank page on a new Yahoo Pipe, I created a very simple demo to get you started. This Yahoo Pipes demo will help you get started with Yahoo Pipes and show you what you need to know to create a basic Pipe. We use filtering for this example, since Yahoo Pipes is great at taking feeds and filtering to display only the posts that match (or exclude) certain keywords. The demo covers taking multiple feeds as inputs, filtering on keywords, sorting, and using the debugger to analyze the elements of your source feeds and your output. After watching this demo, you should be able to create a basic Yahoo Pipe and use it to filter to include or exclude items that match certain keywords.

Yahoo Pipes Training

I also wanted to take this opportunity to announce that I will begin conducting Yahoo Pipes training for organizations. These 2 hour classes will cover the basics of building Yahoo Pipes and provide some custom content if needed to help meet the specific needs of your organization.

More Details

You can view a copy of the 2 Minute Demo Pipe Source.

Modules Used:

  • Fetch Feed. Enter your source feeds here. These are the feeds that have the information you want to filter.
  • Filter. Filter the source feeds to include only posts that match certain keywords.
  • Sort. Sort by date (descending) to make sure that the output from your various source feeds appears in sorted order.
  • Pipe Output. The final module in every Yahoo Pipe.

Related Fast Wonder Blog posts

rss

Yahoo Pipes and RSS Hacks

I’ve been having a lot of fun with Yahoo Pipes lately (some would say slightly obsessed with using them) to hack up RSS feeds and tweak them to be more useful for me and others. Justin told me it would be a good idea to pull them all together in one page to make it easier for people to find them, so I created the Yahoo Pipes and RSS Hacks page. You can find it conveniently located in the Fast Wonder Blog menu bar, and I’ll try to keep it up to date with new Yahoo Pipes.

rss

Solution to Missed FriendFeed Comments: FriendFeed Comment Finder

A bunch of people have been talking about how FriendFeed allows people to comment on content within FriendFeed. This means that we have to log into friend feed every day and scour for comments, which remain fragmented from the source of the content. I can’t fix the fragmentation, but I think I have part of a solution (implemented as a Yahoo Pipe, of course).

The FriendFeed Comment Finder attempts to find content with comments or that people have marked as “liked”.

Important Caveats:

  • Consider this highly experimental right now. Suggestions are welcome!
  • FriendFeed has really convoluted feed structures, and this pipe is implemented in a crappy way right now.
  • The feeds from FriendFeed seem really truncated with only the few most recent entries appearing. If you are using something like NetNewsWire, you should set persistence for x days, and not until they disappear from feed.
  • I also suspect it might be missing a few comments, but haven’t been able to isolate this from the above problem. If you can find a pattern, please let me know.

To use the FriendFeed Comment Finder, enter your FriendFeed username, click “run pipe”, and then grab the RSS feed from “More Options”. Note that I think it is only picking up recent comments.

Related Fast Wonder Blog Posts: