Got your attention? Good! Recently I used the "Yahoo Pipes" tool for the first time, and all I can say is "W-o-w!". Yahoo Pipes is the answer to many questions, and I highly recommend you to use it.
I subscribe to three major fashion newsfeeds for SL: the massive fashion.WorldOfSL feed, Kesserett Steeplechase' Metavirtual.us feed and the relatively new fashionfeedofsl.com. Scanning those three, I think I cover about 90% of the fashion blogging community. The problem is that more and more authors crosspost their blogposts, so in worst case I get the same post up to 4 times. A minor problem is that certain topics like hair or skin do simply not interest me. I looked for a dupecheck solution, but did not find any, until finally a forum post somewhere pointed me into the direction of Yahoo Pipes.
Yahoo Pipes is a toolset that takes feeds as input, manipulates them according to rules, and creates a new feed as output. The pipe gets constructed by connecting graphical boxes containing rules and conditions wiht some sort of ribbon, thus creating a flow of data from input and output. The pipe constructions themselves are by default open - everybody can not only view how a pipe is built, but can actually clone a pipe and use it as basis for their own pipes.
The first pipe I built adressed the topic of the dupes in the fashion feed. For this it takes the three feeds as input and applies a few filters to get rid of unwanted content (for example posts without images). In the next step, three filters check for potential duplicte postings. And in a final step the headlines of the individual posts get changed so that the originating blog is easily visible. The result is stunning. From 240 posts fed into the pipe, only 42 remained after all those manipulations! You can subscribe to this pipe with any feed reader here. In case you actually do like to see skin and hair related posts, feel free to clone the pipe and remove the filters.
The second pipe I built scans the new item feeds from OnRez and Xstreet SL for freshly created freebies. While OnRez had a special feed for freebies (which just needed some rinsing), the problem with Xstreet SL was to decide whether it is a freebie or not depending on the item title. For both shopping portals obvious demo items got deleted from the feed as well as items without picture or without description. Finally the source - OnRez or Xstreet - gets added to the title. You can subscribe to the freebie feed here.
Apart from those public pipes I created some personal pipes - not visible to anybody - for special purposes. You do not need a software development background for them although it does help. The visual interface makes it easy though to combine things.
Have fun while laying pipe! ;-)