Neo4j Tips & Tricks: Handling Long Transactions
An answer to the question: is write performance influenced by the size of transactions? (nb the “popular” question though is: why does my write performance drops off when performing many operations in a single transaction?):
The reason is because Neo4j keeps the transaction’s operations in memory until commit, so your JVM will eventually run out of memory and start paging to disk.
There are two solutions:
- split your transactions into groups of 30,000 or so (obviously you give up the ability to do a full rollback)
- skip the transaction part and use the BatchInserter, which writes directly to the persistence layer rather than keeping everything in memory.
via: http://tobym.posterous.com/neo4j-write-performance-fast-then-massive-slo
