The solution was to take my “shared items” feed from Google Reader, massage it a bit using Yahoo Pipes and create an other feed which includes the distinct blogs that had postings in the feed and include the resulting RSS feed in the sidebar. Unfortunately Blogger renders the feed gadget asynchronously via javascript, which means that I won’t give link goodies to the blogs, even though I would like to :-(.

If you wish to use the pipe yourself, it can be found here. All you need is to plug your RSS feed into it and you get back an RSS feed containing the (distinc) blogs and links to them. Given that I continuously update my shared list with things I read, the list too will get updated frequently.

Picture taken from oddsock’s photostream with permission.

Luckily, on the Network Security Blog I saw the link to LongURL, which provides the same action for multiple services (in fact I think that they fetch the URL with something like cURL and observe the final destination, so in theory they should be able to support any service) and they provide a REST API. W00t!

So, I created an updated, simplified version of the DeShortify pipe and modified my Twitter Content pipe to use this instead (if you currently us the Twitter Content pipe, there is no need to change anything at your end).

PS. This service doesn’t support Shrinkster.com, presumably because they show a click-trough page, rather than sending a 3xx header. I can’t support them with a custom pipe either, because their tracking page needs a POST rather than a GET, and it is also based on the ASP viewstate (so you would need to do a rather complicated dance of fetch the page, get the viewstate and repost it). Offtopic rant: this is what you get for trying to create the “VB 6.0” illusion on the web Microsoft! No leaky abstractions please!

Picture taken from [niv]’s photostream with permission.

Below you can see the pipe, which is relatively simple:

The code consumes the result of the pipe as JSON and generates the flash object. It is adapted from the original include code. If you wish to use it, you should edit the Yahoo Pipes URL (replace it with your blog URL and the number of tags you wish to show), the text color (on the line with “tcolor”) and possibly the size of the flash. The weird method for constructing the links is necessary because Blogger seems to “muck” with the code otherwise.

<div id="tagCloudContainer"></div>
<script type="text/javascript">
var tagCloud = {
  'storeCloud' : function (obj) {
    var baseUrl = location.protocol + "//" + location.host;
    var cloud = '';
    for (var i in obj.value.items) {
      cloud += "<" + "a " + "href='" + baseUrl + "/search/label/" + obj.value.items[i].title + "' style='8'>" + obj.value.items[i].title + '<' + '/a>';
    }
    cloud += '<' + '/tags>';    
    tagCloud.cloudTags = cloud;
    setTimeout(tagCloud.renderTagcloudCallback, 100);
  },
  
  'renderTagcloudCallback' : function () {  
    try {
      swfobject.embedSWF('http://hype-free.googlecode.com/svn/trunk/blog/tagcloud.swf', 'tagCloudContainer', '230', '240', '7', '',
        { 'tcolor' : '0x666633', 'mode' : 'tags', 'distr' : 'true', 'tspeed' : '100', 'tagcloud' : tagCloud.cloudTags },
        { 'wmode' : 'transparent', 'allowScriptAccess' : 'always' });      
    } catch (err) {
      alert(err);
      setTimeout(tagCloud.renderTagcloudCallback, 100);
    }
  },
  
  'init' : function () {
    var script = document.createElement('script');
    script.src = 'http://ajax.googleapis.com/ajax/libs/swfobject/2.2/swfobject.js';
    document.getElementsByTagName('head')[0].appendChild(script);
    
    script = document.createElement('script');
    script.src = 'http://pipes.yahoo.com/pipes/pipe.run?BlogUrl=http%3A%2F%2Fhype-free.blogspot.com%2F&Topcount=10&_id=xKTkgQsT3hGl4j3nJpREag&_render=json&_callback=tagCloud.storeCloud';
    document.getElementsByTagName('head')[0].appendChild(script);    
    return this;
  }
}.init();
</script>

If you are reading this in your RSS reader, visit the blog to see it in action.

Update: it seems that the Flash file can’t interpret tag names with special symbols in them (like ‘). Because of this I modified the pipe so that such tags are filtered out. This will result in some top tags not being displayed (if they contain special characters), but I considered this the right solution, because even if they were displayed, clicking on them wouldn’t result in anything. Credit goes to Evie for finding this issue.

Update: the S3 instance hosting the flash file and javascript (halotemplates.s3.amazonaws.com) went away (thanks to Soufiane for pointing this out). So I’ve downloaded the latest version of the WP-Cumulus and uploaded the SWF file from to Google Code. The SWFObject library is also served from there. So to everyone using the old version: please update to the latest code posted above to make it work again. Sorry for the disruption!

Update: The update broke the “clickability” of the links, since it seems that the new SWF file requires absolute URLs for that. This is now fixed in the script above.

Update: Yahoo pipes changed their backend and thus some adjustment was needed in the pipe which was done.

• links are made clickable
• short links are transformed to their longer variants (the DeShortify pipe is used – this means currently there is support for TinyURL and bit.ly – also, it currently processes only the first link, however – given the 140 character limit of Twitter – this doesn’t seem to be a big limitation)

To use it, go to the Twitter Content Pipe Page, enter the Twitter username you want to subscribe to, press “Run Pipe” and get the resulting RSS feed. As a structure, this pipe is rather simple – is passes the fields trough a sub-pipe and assigns them to an other field:

Some people feel that URL shortening services like TinyURL are a security risk because they take away the user’s ability to hover over a link and find out where it is taking them. An example for such an argument can be read in the TinyURL and security post from the SANS diary. You might or might not agree with this statement, but here is a little Yahoo! Pipe to de-shortify such URL’s:

DeShortify

Currently it supports TinyURL and bit.ly, but it can be extended rather easily to other services, provided that they offer a preview feature. An other feature is that it doesn’t touch URL’s which it doesn’t know about. The structure of the pipe is somewhat complicate (as you can see it below), because – as I discovered – pipes don’t run if one component of theirs returns an error. This means that I’d had to set up different “cases” for each service and filter them:

Hope that somebody finds it useful.

PS. If you wish to consume Pipes from Javascript, they support a JSON callback style of fetching the data, even though it is not very well documented.

The flow is as follows:

• from the first element you get the date/time at the time of the feed “evaluation” (this might differ from the time the feed is fetched)
• the second step is to format it as a timestamp (number of seconds since 00:00:00 UTC on January 1, 1970), which also has the side-effect of creating a single number
• the third step is to normalize it. Taking modulo 100 means that the number will always be between 0 and 99. If you need an interval not starting at 0, just add an other steps which “shifts” the lower end of it (for example: if you need numbers between –150 and –100, you would implement the following expression: –150 + (timestamp % 50))

Now for the second version: this is based on the random.org REST API. Simply fetch the URL which outputs the random numbers as a CSV file:

The advantage of this method is that it is fairly easy to obtain high-quality  random numbers in large quantities. The disadvantage is that they might ban Yahoo pipes at one moment if too many people start using the service (since they state on their site that: “If you access RANDOM.ORG via an automated client, please make sure you observe the Guidelines for Automated Clients or your computer may be banned.” – and you have little control over the method the Pipes backend uses to fetch the data)

The steps performed by the pipe are the following:

1. It takes the blog URL from the user
2. It removes the beginning (“http://”) and end (“/”) of the URL. This is much easier than requiring the user to enter the URL in a particular format
3. It builds the URL for the comment feed of the given blog. We only fetch the summary of the last 500 comments to conserve bandwidth.
4. It fetches the given feed.
5. You can specify a given user who should not be shown (probably your own nick)
6. It filters out comments made by anonymous and by the specified user
7. It aggregates the comments based on the author names
8. It sorts the result in descending order based on the number of occurances
9. Puts the author name in the title and the author link in the link field for the resulting feed.
10. Finally it appends the count to each title

You can access the pipe here and create your own, customized version (different blog URL, different user to filter out).

To add it to your own blog, add a “Feed” widget with the resulting RSS feed (you can obtain the address of the feed by going to “More options” -> “Get as RSS”)

Update: I can’t figure out how to make the Blogger RSS fetcher to update the complete feed. Until I figure out, you can use the the alternative solution described here.