Displaying RSS feeds using YQL, a Google Feed API alternative

Home

Tutorial posted: Dec 3rd, 15'

With the official deprecation of Google Feed API in April 15' and yesterday's complete outage (it seems to be back up today Dec 3rd 15'), it left many webmasters scrambling to find an alternative free solution for displaying RSS feeds from external sources. While doing my own research on the subject, I came across Yahoo's YQL, a service that lets you combine and filter data sources from around the web, including RSS feeds. It's by no means perfect with its restrictive query limits, and with the recent shut down of Yahoo Pipes, probably no more reliable than Google Feed API as a long term solution. For the time being, however, it seems to be the best free alternative to Google Feed API that's backed by a large company with ample resources to ensure a stable experience while the service is still available.

In this tutorial, we'll look at how to quickly get up and running with YQL in fetching and displaying RSS feeds from public external sources using simply JavaScript, for example.

Latest Digg news:

Latest BBC Headlines:

The 10 minute setup

Step 1: YQL is baked into Yahoo's YUI Library as a utility, which offers a convenient wrapper for making basic calls to YQL. To get started, first, include the latest version of YUI in the HEAD section of your page:

<script src="http://yui.yahooapis.com/3.18.1/build/yui/yui-min.js"></script>

Step 2: Once that's done, invoke YQI using YUI with the following barebones code:

<head>
<script src="http://yui.yahooapis.com/3.18.1/build/yui/yui-min.js"></script>

<script>
YUI().use('yql', function(Y){
	var query = ' yql_query_here '
	var q = Y.YQL(query, function(r) {
		//r now contains the result of the YQL Query
		//do something with r
	})
})
</script>
</head>

Notice the highlighted line- all the magic is conjured up inside the query string itself ('yql query here'), which should be replaced with a valid YQL query similar in syntax to SQL that instructs YQL to fetch the desired content from the web in a variety of ways. For example, to fetch the contents of a single RSS feed and return it as a JSON object, the appropriate query string would be:

select * from rss where url = "http://url_to_rss_feed"

So with that in mind, the following code accesses the RSS feed from QZ.com and returns it as a JSON object for further manipulation:

<head>
<script src="http://yui.yahooapis.com/3.18.1/build/yui/yui-min.js"></script>

<script>
YUI().use('yql', function(Y){
	var query = 'select * from rss where url = "http://qz.com/feed/"'
	var q = Y.YQL(query, function(r){
		//r now contains the result of the YQL Query as a JSON
		console.log(r)
	})
})
</script>
</head>

As you can see, the JSON result of the YQL query is made available inside the callback function of Y.YQL() as a parameter (ie: r) once it's loaded. We'll discuss the structure of the result plus more advanced YQL queries next.

Step 3: To display RSS feeds using YQL then, we simply have to construct the proper YQL query string that goes out to fetch the target RSS feed(s), optionally sort and filter them, before returning the result as a JSON object for molding into something human readable. The following shows the most common YQL queries strings for fetching RSS feed(s) in a variety of ways:

select * from rss where url = "rssurl" // get contents of 1 RSS feed
select * from rss where url in ("rssurl", "rssurl2", etc) // get contents of multiple RSS feeds
select * from rss(0, 5) where url = "rssurl" // get first 5 entries from RSS feed
select * from rss where url = "rssurl" | sort(field="pubDate", descending="true") // sort feed by a particular field (ie: pubDate)

There is the YQL Console you can use to input live YQL queries and see what gets returned, and also the YQL Guide for more info on constructing queries.

Once we've received the desired RSS feeds as a JSON object, the final step is to parse and display it. The following simple example shows the first 5 RSS entries from QZ.com:

The code:

<script src="http://yui.yahooapis.com/3.18.1/build/yui/yui-min.js"></script>

<div id="qznews"></div>

<script>
YUI().use('yql', function(Y){
	var query = 'select * from rss(0,5) where url = "http://qz.com/feed/"'
	var q = Y.YQL(query, function(r){
		//r now contains the result of the YQL Query as a JSON
		var feedmarkup = '<p>'
		var feed = r.query.results.item // get feed as array of entries
		for (var i=0; i<feed.length; i++){
			feedmarkup += '<a href="' + feed[i].link + '">'
			feedmarkup += feed[i].title + '</a><br />'
			feedmarkup += feed[i].description + '</p>'
		}
		document.getElementById('qznews').innerHTML = feedmarkup
	})
})
</script>

Demo:

As you can see, once the YQL query has completed, the parameter r contains the result as a JSON object. To access the actual feed, we reference r.query.results.item, which returns an array of RSS entries. The individual components of each entry such as "title", "description" etc are stored in the corresponding property within each array entry. Your browser's console is your best friend in examining the contents of the r parameter, including when there is an error in your YQL query or the feed fails to load.

Returning the results as a XML text instead of JSON

If you wish to handle the returned result from YQL as XML text instead of the default JSON, simply modify your call to YQL with the highlighted addition:

<head>
<script src="http://yui.yahooapis.com/3.18.1/build/yui/yui-min.js"></script>

<script>
YUI().use('yql', function(Y){
	var query = ' yql_query_here '
	var q = Y.YQL(query, function(r){
		//r now contains the result of the YQL Query
		//do something with r
	}, {
		format: 'xml'
	})
})
</script>
</head>

In this version of the YQL call, r.query.results.item, contains an array of XML strings like the following:

"<item><title>Spanish-speakers ...</title><link>...</link>..."

one of each RSS entry.