Gil Hildebrand (Squidoo):
What I’ve learned is that Cassandra is remarkably resilient. You can do unholy things to the data files and get away with it.
It took several days to clean up the mess, primarily because working with extremely large files can be a very slow process. Yet the entire time our cluster stayed online, and new data was being recorded without interruption. When a write was attempted on a server whose volumes were full, the write was retried on a server who had more space. And when we managed to clear some space, new writes were recorded normally even on the servers with corrupted data.
There’s both a good part and a bad part about it.
Original title and link: Cassandra Resiliency: How a data corruption problem made me more confident about Cassandra (NoSQL databases © myNoSQL)