LevelDB: All content tagged as LevelDB in NoSQL databases and polyglot persistence
Thursday, 25 April 2013
Quick Intro to LevelDB
If you haven’t heard of LevelDB before or you forgot some of the details, read Rod Vagg’s short post that will give you an overview of the basics and internals. Particularly interesting is the file organization in LevelDB (which also gives it the name).
Original title and link: Quick Intro to LevelDB (©myNoSQL)
Tuesday, 30 October 2012
Improvements and Benchmarks for LevelDB in Riak 1.2
Basho team started to investigate and optimize LevelDB, one of the supported storage engine for Riak and the engine for Riak 2i, and the results are already impressive:
- reduced stalls (from 10-90s every 3-5min to 10-30s every 2h)
- increased throughput (from 400 ops/s to 2000 ops/s)
- a better solution for dealing with an infinite loop during compaction against a corrupted data block
- LevelDB bloom filter for quickly identifying keys that don’t exist in the data store
The original posts also shows some charts of the throughput and maximum latency measured in Level 1.1 vs Level 1.2.
Original title and link: Improvements and Benchmarks for LevelDB in Riak 1.2 (©myNoSQL)
via: http://basho.com/blog/technical/2012/10/30/leveldb-in-riak-1p2/
Monday, 6 February 2012
LevelDB: SSTable and Log Structured Storage
Ilya Grigorik digs into LevelDB’s SSTable and log structured storage1:
If Protocol Buffers is the lingua franca of individual data record at Google, then the Sorted String Table (SSTable) is one of the most popular outputs for storing, processing, and exchanging datasets. As the name itself implies, an SSTable is a simple abstraction to efficiently store large numbers of key-value pairs while optimizing for high throughput, sequential read/write workloads.
Even if not very talked about, LevelDB is making notable contributions to the NoSQL space: the leveled compaction strategy in Cassandra 1.0 is based on LevelDB and Riak ships with LevelDB since 1.0.
-
Make sure you are not missing Writes Performance: B+Tree, LSM Tree, Fractal Tree ↩
Original title and link: LevelDB: SSTable and Log Structured Storage (©myNoSQL)
via: http://www.igvita.com/2012/02/06/sstable-and-log-structured-storage-leveldb/
Tuesday, 17 January 2012
LevelDB Might Be a Great Fit for MongoDB
Mark Callaghan:
MongoDB doesn’t need multi-statement transactions. Both are limited by 1-writer or N-reader concurrency, but writes to database files are much faster with LevelDB because it doesn’t do update in place. So LevelDB doesn’t lose as much performance for IO-bound workloads by doing 1-writer or N-readers and my guess is that this could make MongoDB much better at supporting IO-bound workloads.
If getting LevelDB as a storage engine for MySQL is doubtful and I wouldn’t hold my breath until MongoDB would be using it, keep in mind that Riak might actually use LevelDB at some point.
Update: Justin Sheehy (CTO, Basho) has kindly (again) corrected me: LevelDB is supported and ships with Riak since the release of Riak 1.0. Plus it is the backend for the new secondary indexes, as it was correctly mentioned in the comments by Philip Cristiano.
Original title and link: LevelDB Might Be a Great Fit for MongoDB (©myNoSQL)
via: http://mysqlha.blogspot.com/2012/01/who-wants-to-write-storage-engine.html
Wednesday, 3 August 2011
LevelDB and Kyoto Cabinet Benchmark
I’ve been pretty excited about Google’s LevelDB, not to mention there are some really old tanks already in the battle field like BDB, Tokyo Cabinet (Kyoto Cabinet as new one), HamsterDB etc. Fortunately I’ve already worked with Kyoto Cabinet and when I looked at the benchmarks I was totally blown away.
His benchmark results are radically different than the ones published in the LevelDB benchmark.
Original title and link: LevelDB and Kyoto Cabinet Benchmark (©myNoSQL)
via: http://maxpert.tumblr.com/post/8330476086/leveldb-vs-kyoto-cabinet-my-findings
Tuesday, 2 August 2011
LevelDB: Google’s Fast Persistent Key-Value Store Library
Google open sourced a while ago LevelDB , a C++ library that provides an ordered mapping key-value storage. LevelDB performance convinced Basho guys to experiment with adding LevelDB as a storage engine for Riak. And there’s also a benchmark comparing LevelDB with SQLite and Kyoto Cabinet.
The LevelDB project lists the following key features:
- Keys and values are arbitrary byte arrays.
- Data is stored sorted by key.
- Callers can provide a custom comparison function to override the sort order.
- The basic operations are
Put(key,value),Get(key),Delete(key). - Multiple changes can be made in one atomic batch.
- Users can create a transient snapshot to get a consistent view of data.
- Forward and backward iteration is supported over the data.
- Data is automatically compressed using the Snappy compression library.
- External activity (file system operations etc.) is relayed through a virtual interface so users can customize the operating system interactions.
- Detailed documentation about how to use the library is included with the source code.
You can check out also the old thread on Hacker News about LevelDB..
Original title and link: LevelDB: Google’s Fast Persistent Key-Value Store Library (©myNoSQL)
Tuesday, 5 July 2011
Riak Getting LevelDB as Storage Engine
After Innostore and Bitcask, Basho guys are currently experimenting with integrating Google’s LevelDB as a storage engine for Riak. Preliminary results are looking promising:
For most Riak users, Bitcask is the obvious right storage engine to use. It provides low latency, solid predictability, is robust in the face of crashes, and is friendly from a filesystem backup point of view. However, it has one notable limitation: total RAM use depends linearly (though via a small constant) on the total number of objects stored. For this reason, Riak users that need to store billions of entries per machine sometimes use Innostore (our wrapper around embedded InnoDB) as their storage engine instead. InnoDB is a robust and well-known storage engine, and uses a more traditional design than Bitcask which allows it to tolerate a higher maximum number of items stored on a given host.
It appears that LevelDB may become a preferred choice for Riak users whose data set has massive numbers of keys and therefore is a poor match with Bitcask’s model. Performance aside, it compares favorably to InnoDB on other issues such as permissive license and operational usability.
Original title and link: Riak Getting LevelDB as Storage Engine (©myNoSQL)
