Three presentations covering the various NoSQL usages at Twitter:

1. Kevin Weil talking about data analysis using Scribe for logging, base analysis with Pig/Hadoop, and specialized data analysis with HBase, Cassandra, and FlockDB on InfoQ

2. Ryan King’s presentation from last year’s QCon SF NoSQL track on Gizzard, Cassandra, Hadoop, and Redis on InfoQ

3. Dmitriy Ryaboy on Hadoop from Devoxx 2010:

By looking at the powered by NoSQL page and my records, Twitter seems to be the largest adopter of NoSQL solutions. Here is an updated version of who is using Cassandra and HBase

• Twitter: Cassandra, HBase, Hadoop, Scribe, FlockDB, Redis
• Facebook: Cassandra, HBase, Hadoop, Scribe, Hive
• Netflix: Amazon SimpleDB, Cassandra
• Digg: Cassandra
• SimpleGeo: Cassandra
• StumbleUpon: HBase, OpenTSDB
• Yahoo!: Hadoop, HBase, PNUTS
• Rackspace: Cassandra

And probably many more missing from the list. But that could change if you leave a comment.

Original title and link: Hadoop and NoSQL Databases at Twitter (NoSQL databases © myNoSQL)

A not yet released ☞ book:

Wondering what would such a book cover. The tons of different APIs for each NoSQL database?

Original title and link: NoSQL The Ruby Way (NoSQL databases © myNoSQL)

A panel discussion on NoSQL, NoSQL databases, and relational databases, featuring Salvatore Sanfilippo^[1], Lenz Grimmer^[2], Filipe David Borba Manana^[3], and a forth person from SAPO whose name I couldn’t spell:

Original title and link: To SQL or not to SQL Panel at CODEBITS IV (NoSQL databases © myNoSQL)

Featuring Jacob Burch, Alex Gaynor, Eric Florenzano, Jacob Kaplan-Moss, Michael Richardson, Noah Silas^[1]:

You probably know already that Django and NoSQL is hot!

Original title and link: NoSQL and Django Panel Video (NoSQL databases © myNoSQL)

Story and script by Latha Annur Subramaniam:

In case you have a hard time reading it:

RDBMS (SQL) was worried when the news about a new technology called NoSQL

SQL: Oh, he is this new NoSQL guy. People say he is here to beat me out. Hmm. I just hate him!

NoSQL: Howdy, senior SQL. How are you?

SQL: um… uh… oh Hi young man. Looks like you are new to this place?

NoSQL: Oh yeah! Just out of the ‘Latest computing trends’ school.

SQL: He is just a fresher. But I am his great grand senior. He can never take me down.

SQL: Hey, your name NoSQL sounds strange. Sounds like you are an anti-SQL guy.

NoSQL: Hm… true. I fell so unfortunate of my name. But I am never al alternate to you. In short, I am a new solution for the fresh new problems of this computing era… the “WEBSCALE” era.

SQL: (Hey he sounds modest. Am kinda like this guy) Oh. Am hearing this term for the first time. What is this W-E-B SCALE thing all about?

NoSQL: Interestingly, these days humans lead a much active social life on the WEB only.

NoSQLL Just like in their real life, people always need more and more of everything. Tweet, Search, Maps, Blog… their needs never end ;-)

SQL: Hmmm. Now I get it. I’ve been the darling for the enterprises for their data storage needs. But maybe they will abandom me and choose you, when they need more scale?!?!

NoSQL: Partially true. I can help them in scaling massively. But you are still the best in a lot of things.

NoSQL: For example, you are the Superstar when it comes to ‘transaction based apps’. I can never beat you in your ACID qualities

NoSQL: Also, I am still not the best for ‘Reporting’ requirements. While my ‘schemaless’ quality helps dynamically add different types of data, it causes the drawback of not being helpful for reporting.

SQL: I fell you are the right fit for the modern social apps.

NoSQL: You are the right bet for the critical business apps… soon until I catch up with you

SQL: HaHaHa

SQL: Yup. I wish you good luck, young man.

NoSQL: Thank you, senior. Btw, my name doesn’t mean a NO to SQL!!! It is only that I am NOT only SQL :-)

SQL: and so I dedicate this song to you buddy:

Twinkle, twinkle NoSQL
Was wondering who you are
Out into this computing world,
I wish you success all around!!!

Definitely not as good as MongoDB is web scale.

Original title and link: SQL vs NoSQL: Twinkle, Twinkle, NoSQL! (NoSQL databases © myNoSQL)

Something to think about it:

• if you are using some caching in your application, would you call that the persistence layer?
• if you are using a distributed cache, would you call that your persistence layer?
• if you are using a replicated and distributed cache, would you call that your persistence?
• if your replicated and distributed cache does some sort of snapshotting to disk, would you call that your persistence?

Some are saying ☞ RAM is the new disk, so I’m wondering what their answers to the above questions are.

Original title and link: Questions about Caching and Persistence (NoSQL databases © myNoSQL)

Steve Klabnik (@steveklabnik):

Software Engineering is the art of choosing the correct set of tradeoffs

True, even if not specific to NoSQL databases alone.

Original title and link: NoSQL, Tradeoffs, and Software Engineering (NoSQL databases © myNoSQL)

Kenn Ejima ☞ posted his thoughts about NoSQL, scalability and O(N) as a preamble to some details about Redis persistence, concluding:

For people who don’t really grok what’s been said in this post (maybe because it was just too long to read), my recommended setup is: “Use Redis for small datasets that don’t grow fast (stay far less than 1GB). Have at least 2x memory than the dataset. Use default snapshotting and disable AOF.”

Considering this time I was one of those that didn’t really follow the first part of the article, filing it under “not sure what all these have to do with Redis persistence implementation”, I’ve found Jeff Darcy’s ☞ follow up adding a bit more context(!) to the discussion:

I’d rephrase above as “Use Redis for small datasets (less than 50GB this year) that don’t need to be highly available, have memory at least 2x your actual dataset (until the snapshot implementation improves), use frequent snapshotting or AOF (depending on your need for performance vs. durability – not both) and always avoid overcommit.” I also have nothing against Redis, it’s a fine tool for what it does, but I think its durability story is a bit confused and its reinvented VM can only serve a need that it’s not good for anyway. As always, the real answer is to use multiple data stores to serve multiple needs, with careful consideration of the tradeoffs each represents.

Original title and link: Redis: Thoughts and Counter-Thoughts (NoSQL databases © myNoSQL)