snowflake: All content tagged as snowflake in NoSQL databases and polyglot persistence
Tuesday, 17 January 2012
Flake: An Erlang Decentralized, K-Ordered Unique ID Generator by Boundary
After posting about the proposal of adding a 128-bit K-Order unique id generator to Redis, I’ve realized I haven’t linked to Boundary’s post about Flake, their decentralized k-ordered unique id generator project:
All that is required to construct such an id is a monotonically increasing clock and a location 3. K-ordering dictates that the most-significant bits of the id be the timestamp. UUID-1 contains this information, but arranges the pieces in such a way that k-ordering is lost. Still other schemes offer k-ordering with either a questionable representation of ‘location’ or one that requires coordination among nodes.
Original title and link: Flake: An Erlang Decentralized, K-Ordered Unique ID Generator by Boundary (©myNoSQL)
Monday, 16 January 2012
Redis: Adding a 128-Bit K-Ordered Unique Id Generator
Interesting pull request submitted to Redis by Pierre-Yves Ritschard:
This is similar to what snowflake and the recent boundary solution do, but it makes sense to use redis for that type of use cases for people wanting a simple way to get incremental ids in distributed systems without an additional daemon requirement.
Snoflake is the network service for generating unique ID numbers at high scale used (and open sourced) by Twitter. Flake is an Erlang decentralized, k-ordered id generation service open sourced by Boundary.
Original title and link: Redis: Adding a 128-Bit K-Ordered Unique Id Generator (©myNoSQL)
Tuesday, 7 September 2010
MongoDB and Auto Increment
Chris Shiflett shares a solution to emulate the MySQL auto_increment with MongoDB. While you should read his post, the proposed solution is:
db.seq.findAndModify({ query: {"_id":"users"}, update: {$inc: {"seq":1}}, new: true });
Kenny Gorman warns that the above solution is not exactly optimal:
When using this technique each insert would require both the insert as well as the findAndModify command which is a query plus an update. So now you have to perform 3 operations where it used to be one. Not only that, but there are 3 more logical I/O’s due to the query, and those might be physical I/O’s. This pattern is easily seen with the mongostat utility.
I remember Twitter has tried to solve a similar problem — probably at a different, larger scale — and they came up with Snowflake:
We needed something that could generate tens of thousands of ids per second in a highly available manner. This naturally led us to choose an uncoordinated approach. These ids need to be roughly sortable, meaning that if tweets A and B are posted around the same time, they should have ids in close proximity to one another since this is how we and most Twitter clients sort tweets.
If you have other ideas (even if not directly related to MongoDB) I’d love to hear them!
Original title and link for this post: MongoDB and Auto Increment (published on the NoSQL blog: myNoSQL)
