NoSQL Benchmarks NoSQL use cases NoSQL Videos NoSQL Hybrid Solutions NoSQL Presentations Big Data Hadoop MapReduce Pig Hive Flume Oozie Sqoop HDFS ZooKeeper Cascading Cascalog BigTable Cassandra HBase Hypertable Couchbase CouchDB MongoDB OrientDB RavenDB Jackrabbit Terrastore Amazon DynamoDB Redis Riak Project Voldemort Tokyo Cabinet Kyoto Cabinet memcached Amazon SimpleDB Datomic MemcacheDB M/DB GT.M Amazon Dynamo Dynomite Mnesia Yahoo! PNUTS/Sherpa Neo4j InfoGrid Sones GraphDB InfiniteGraph AllegroGraph MarkLogic Clustrix CouchDB Case Studies MongoDB Case Studies NoSQL at Adobe NoSQL at Facebook NoSQL at Twitter



Neo4j Internals: Caching

Chris Gioran explains how Neo4j is using internal caching layers for optimizing access to nodes and relationships:

First note that there are two caches, one for nodes and one for relationships, which apart from storing different classes, they can have different maximum sizes (via max_node_cache_size and max_relationship_cache_size configuration parameters), albeit not different implementations. There are 4 different kinds of cache implementations, all in package org.neo4j.kernel.impl.cache. There is the degenerate NoCache, which stores nothing. LruCache uses a good old LinkedHashMap with an overridden removeEldestEntry() which makes it an LRU cache and provides adaptability (more on that later). Finally, {Soft,Weak}LruCache provide the exact same implementation, with Weak and Soft interchanged, utilizing {Soft,Weak}Values to store references and ReferenceQueues to store garbage collected references. They both extend org.neo4j.kernel.impl.cache.ReferenceCache, which defines a pollClearedValues() method for removing “dead” entries from the map. By the way, this is the end of the code for adaptability for ReferenceCaches, meaning that their adaptation to the current memory utilization is whatever the Weak/SoftReference + the gc can offer (which is a lot). All management of the content of the caches is performed exclusively by NodeManager, while the actual memory consumption is determined by the AdaptiveCacheManager and is influenced by the user-supplied configuration.

Original title and link: Neo4j Internals: Caching (NoSQL databases © myNoSQL)