Visualizing Wikipedia Update Stream With Redis Pub/Sub and Node.js
Fascinating story about the volume of edits happening on Wikipedia and how using Redis Pub/Sub and node.js helped building some visualization tools—wikistream and wikipulse—for the activity stream:
For both of these apps I needed to log into the wikimedia IRC server, listen on ~30 different channels, push all the updates through some code that helped visualize the data in some way, and then get this data out to the browser. I had heard good things about node for high concurrency network programming from several people. I ran across a node library called socket.io that reported to make it easy to stream updates from the server to the client, in a browser independent way, using a variety of transport protocols. Instinctively it felt like the pub/sub model would also be handy for connecting up the IRC updates with the webapp. I had been wanting to play around with the pub/sub features in redis for some time, and since there is a nice redis library for node I decided to give it a try.
Original title and link: Visualizing Wikipedia Update Stream With Redis Pub/Sub and Node.js (©myNoSQL)
