A month ago I was writing about one of those catchy articles NoSQL wants to be elastic caching when it grows up arguing that if it is something to happen in this space, it will be that elastic caching solutions will look more seriously into persistency.
Nati Shalom (Gigaspaces CTO, @natishalom), has recently published a new article about RAM being the new enterprise persistence. As far as I can tell most of the decisions are based on the research paper The case for RAMClouds (pdf):
By integrating GigaSpaces XAP with the Cisco UCS machine we are demonstrating our ability to easily load hundreds of gigabytes into a single box, and to scale linearly with growing capacity without any performance degradation. This is a great example of how middleware that was built for memory from the ground up, combined with hardware that was equipped to provide terabytes of memory in a single box, can be game changing.
This exciting combination makes it possible to manage 15-20x the amount of data in-memory, per partition. This, in turn, makes it possible to store the entire application data set in‑memory, and gain not only 10x the performance but also great simplicity, because the application no longer needs to deal with a miss ratio in the cache; and, at the same time, there are no consistency issues because all the data resides in-memory.
This is indeed an interesting argument and one that I’m not going to argue against. But it still feels like elastic caching or in-memory elastic databases will remain just a part of the software equation:
- even if the price of RAM has continued to decrease, the machines mentioned do not sound like commodity hardware so you’ll have to balance the costs with the value of data
- it still sounds like vertical scaling (nb not saying that vertical scaling is always bad)
- there will always be data that will fit better on disk (e.g. video)
- the more data will be accumulated the more you’d like to make sure that querying it (nb online or offline) is not expensive
-  According to the original article the following solutions were considered as being part of elastic cache: IBM eXtremeScale, Gigaspaces, Terracotta, Microsoft Velocity, Hazelcast, NCache, Infinispan (↩)