hadoop: All content about hadoop in NoSQL databases and polyglot persistence
In my post about in-memory databases vs Aster Data and Greenplum vs Hadoop market share, I’ve proposed a scenario in which Aster Data and Greenplum could expand into the space of in-memory databases by employing hybrid storage.
What I haven’t covered in that post is the possibility of Hadoop, actually HDFS, expanding into hybrid storage.
But that’s happening already and Hortonworks is already working on introducing support for heterogeneous storages in HDFS:
We plan to introduce the idea of Storage Preferences for files. A Storage Preference is a hint to HDFS specifying how the application would like block replicas for the given file to be placed. Initially the Storage Preference will include:
- The desired number of file replicas (also called the replication factor) and;
- The target storage type for the replicas.
Even if the costs of memory will continue to decrease at the same rate as before 2012, when they flat-lined, a cost effective architecture will almost always rely on hybrid storage.
Original title and link: Heterogeneous storages in HDFS ( ©myNoSQL)
Donald Miner, author of MapReduce Design Patterns and CTO at ClearEdge IT Solutions discusses how he chooses between Pig and MapReduce, considering developer and processing time, maintainability and deployment, and repurposing engineers that are new to Java and Pig.
Video and slides after the break.