Skip to Content

SDN checks it’s “pipes”

It’s new and it just launched a couple of days ago, I just finally got some time to try it out and see how “good” it really is of course I am talking about Yahoo! Pipes. So what is Pipes and what does it have to do with you? Well that depends on what you do really, and how much you use data from various sources on the net.

There is a rapidly-growing body of well-structured data available online in the form of XML feeds. These feeds range from simple lists of blog entries and news stories to more structured, machine-generated data sources like the Yahoo! Maps Traffic RSS feed. Because of the dearth of tools for manipulating these data sources in meaningful ways, their use has so far largely been limited to feed readers.

source SDN and BPX are full of RSS feeds which of course are XML based feeds. Now SAP is full of them as well and you can get them with little effort, or with my favorite which is one of the first things I implemented from SDN. Now of course Pipes sits outside of your firewall so you are a bit limited but let’s take a look at bringing data into your system. For that I will take a recent issue that came up over a coffee (OK the two of us were a few thousands miles apart but still). We implemented or more so we enabled external bloggers to blog and bring the data to SDN and BPX not that a blogger has to redo their blogs or start a whole new one if they are already established often this is quite good when their focus is not really 100% what our community is looking for but occasionally they have something cool to share with us. You can find more details here. Quite a list of bloggers already there, 14 and counting! So one of the problems we have though is what happens when anyone starts using the tags? I mena we don’t want SPLOGS and SPAMMERS to just start using the tag and flooding us with garbarge right – so we make a quick check of who is who and ensure that they are at least “real” with legit information. So of course this meant custom development or did it? [Enter Pipes] Now here is a product out there than can do some filtering for me and give me the results as a new RSS feed which I could then use inside of our community. Now this is quite interesting but does it work? I decided to give it a try… image I did the whole drag and drop of the different components:

  • Fetch
  • Text Input
  • Filter
  • Sort
  • Output

For the final one I would leave out the Text input and just hard code the values we “approve” or I should say from the legitimate bloggers we know about to prevent the SPAMMERS. But this is just a test which I published so you can give a look yourself if you like, should even be able to clone it and modify your own copy all you like. image With it all together all that was left was to run it. image Without text input it returned the entire list. But with input and I choose that of Blag since he’s got a great little blog over there in Perú. image As you can see I got my filtered list and I have an RSS feed to subscribe to and have just saved myself time for developing a solution, not bad for 15 minutes of work.

You must be Logged on to comment or reply to a post.
  • Great tool Craig…I wasn’t aware of it -:) I’m not a big yahoo fan, but this tool is definitely something to look at -;)

    Also…Thanx for including my little blog on your example -:D That really makes my day -;)



  • I think this tools is a neat idea, the same way I think Visual Composer is a neat idea.  It’s a good attempt to remove some of the “coding” from the process.  As more fast changing data comes in RSS formats people who aren’t ace programmers might want to stick bits together to see the bigger picture and tools like this let me.  I would love to know more about their target audience for the application and who they were hoping to reach.

    I’m sure there are limits to this things usefulness but for me, I’d love to try and use it to replace tools like feedblendr.  One problem I have is i never know if i should post a blog here or on my site and i never want to do both because I don’t want to have it in my RSS feed twice, with this i bet i could filter for that.


      • Nice and that while chatting online with me!

        The audience I wonder about as well, but it does open up quite a few options for the age of the information worker now doesn’t it 🙂

        • The challenge for the next few years is how do companies like Yahoo and Google with great tools like this work with companies who need VPNs and need to keep their data secret?  Do we bring the tooling behind our firewall, or do we someone securely link to the outside world?
          If we leave the tooling outside but somehow open the enterprise to connect to these tools I think we empower the information worker.  If we try and bring the tooling side we are just locking them up again.
          • I wonder about this, is there money to be made by “middleware” companies here?  Coming up with a way to but a box inside the firewall that handles the movement of data and deals with authentication with any old system from that outside that can pass a query and credentials?
          • Interesting idea I wonder if Teqlo is going that route or if they are staying online?

            A box with a dynamic tunnel to get to data in or outside of the network – very interesting.