ALL COVERED TOPICS

NoSQL Benchmarks NoSQL use cases NoSQL Videos NoSQL Hybrid Solutions NoSQL Presentations Big Data Hadoop MapReduce Pig Hive Flume Oozie Sqoop HDFS ZooKeeper Cascading Cascalog BigTable Cassandra HBase Hypertable Couchbase CouchDB MongoDB OrientDB RavenDB Jackrabbit Terrastore Amazon DynamoDB Redis Riak Project Voldemort Tokyo Cabinet Kyoto Cabinet memcached Amazon SimpleDB Datomic MemcacheDB M/DB GT.M Amazon Dynamo Dynomite Mnesia Yahoo! PNUTS/Sherpa Neo4j InfoGrid Sones GraphDB InfiniteGraph AllegroGraph MarkLogic Clustrix CouchDB Case Studies MongoDB Case Studies NoSQL at Adobe NoSQL at Facebook NoSQL at Twitter

NAVIGATE MAIN CATEGORIES

Close

snowflake: All content tagged as snowflake in NoSQL databases and polyglot persistence

Flake: An Erlang Decentralized, K-Ordered Unique ID Generator by Boundary

After posting about the proposal of adding a 128-bit K-Order unique id generator to Redis, I’ve realized I haven’t linked to Boundary’s post about Flake, their decentralized k-ordered unique id generator project:

All that is required to construct such an id is a monotonically increasing clock and a location 3. K-ordering dictates that the most-significant bits of the id be the timestamp. UUID-1 contains this information, but arranges the pieces in such a way that k-ordering is lost. Still other schemes offer k-ordering with either a questionable representation of ‘location’ or one that requires coordination among nodes.

Original title and link: Flake: An Erlang Decentralized, K-Ordered Unique ID Generator by Boundary (NoSQL database©myNoSQL)

via: http://blog.boundary.com/2012/01/12/Flake-A-decentralized-k-ordered-id-generation-service-in-Erlang.html


Redis: Adding a 128-Bit K-Ordered Unique Id Generator

Interesting pull request submitted to Redis by Pierre-Yves Ritschard:

This is similar to what snowflake and the recent boundary solution do, but it makes sense to use redis for that type of use cases for people wanting a simple way to get incremental ids in distributed systems without an additional daemon requirement.

Snoflake is the network service for generating unique ID numbers at high scale used (and open sourced) by Twitter. Flake is an Erlang decentralized, k-ordered id generation service open sourced by Boundary.

Original title and link: Redis: Adding a 128-Bit K-Ordered Unique Id Generator (NoSQL database©myNoSQL)


MongoDB and Auto Increment

Chris Shiflett shares a solution to emulate the MySQL auto_increment with MongoDB. While you should read his post, the proposed solution is:

db.seq.findAndModify({    
    query: {"_id":"users"},   
     update: {$inc: {"seq":1}},   
     new: true
});

☞ shiflett.org

Kenny Gorman warns that the above solution is not exactly optimal:

When using this technique each insert would require both the insert as well as the findAndModify command which is a query plus an update. So now you have to perform 3 operations where it used to be one. Not only that, but there are 3 more logical I/O’s due to the query, and those might be physical I/O’s. This pattern is easily seen with the mongostat utility.

☞ www.kennygorman.com

I remember Twitter has tried to solve a similar problem — probably at a different, larger scale — and they came up with Snowflake:

We needed something that could generate tens of thousands of ids per second in a highly available manner. This naturally led us to choose an uncoordinated approach. These ids need to be roughly sortable, meaning that if tweets A and B are posted around the same time, they should have ids in close proximity to one another since this is how we and most Twitter clients sort tweets.

☞ engineering.twitter.com

If you have other ideas (even if not directly related to MongoDB) I’d love to hear them!

Original title and link for this post: MongoDB and Auto Increment (published on the NoSQL blog: myNoSQL)