ALL COVERED TOPICS

NoSQL Benchmarks NoSQL use cases NoSQL Videos NoSQL Hybrid Solutions NoSQL Presentations Big Data Hadoop MapReduce Pig Hive Flume Oozie Sqoop HDFS ZooKeeper Cascading Cascalog BigTable Cassandra HBase Hypertable Couchbase CouchDB MongoDB OrientDB RavenDB Jackrabbit Terrastore Amazon DynamoDB Redis Riak Project Voldemort Tokyo Cabinet Kyoto Cabinet memcached Amazon SimpleDB Datomic MemcacheDB M/DB GT.M Amazon Dynamo Dynomite Mnesia Yahoo! PNUTS/Sherpa Neo4j InfoGrid Sones GraphDB InfiniteGraph AllegroGraph MarkLogic Clustrix CouchDB Case Studies MongoDB Case Studies NoSQL at Adobe NoSQL at Facebook NoSQL at Twitter

NAVIGATE MAIN CATEGORIES

Close

Redis Bulk Load Tips and Tricks

Awesome solutions right from the source:

The fastest way to do this is the following: generate Redis protocol out of this data. The documentation to generate the Redis protocol is on the Redis.io site, it is a trivial protocol. Once you have that, just call it appendonly.log and start redis in append only mode.

You can even do a FLUSHALL command and finally push the data into your server with netcat, redirecting the output to /dev/null.

This will be super fast, there is no RTT to wait, it’s just a bulk loading of data.

Less hackish way, just insert things 1000 per time using pipelining. It’s almost as fast as generating the protocol, but much more clean :)

Original title and link: Redis Bulk Load Tips and Tricks (NoSQL database©myNoSQL)

via: http://stackoverflow.com/questions/7505011/bulk-ingest-into-redis