Redis Bulk Load Tips and Tricks
Awesome solutions right from the source:
The fastest way to do this is the following: generate Redis protocol out of this data. The documentation to generate the Redis protocol is on the Redis.io site, it is a trivial protocol. Once you have that, just call it appendonly.log and start redis in append only mode.
You can even do a
FLUSHALLcommand and finally push the data into your server with netcat, redirecting the output to /dev/null.This will be super fast, there is no RTT to wait, it’s just a bulk loading of data.
Less hackish way, just insert things 1000 per time using pipelining. It’s almost as fast as generating the protocol, but much more clean :)
Original title and link: Redis Bulk Load Tips and Tricks (©myNoSQL)
via: http://stackoverflow.com/questions/7505011/bulk-ingest-into-redis
