Performance Evaluation of Multilevel Storage Optimization using Dimension Reduction and Compression Framework in Distributed Clusters
The increase in the volume of Big Data and IoT environments are due to different types and complexity of data generated from multiple sources especially from sensors and IoT devices. There are many proven dimension reduction techniques and data compression techniques evolved over last two decades. This paper provides brief overview of some of the critical steps to apply PCA and Erasure Codes in Hadoop Environment and also how Multilevel Storage Optimization is a good approach when compared with other storage compression techniques and dimension reduction techniques. The case study and evaluation proved that the performance of the framework is better in terms of number of blocks, utilization of CPU, Memory, IO and storage efficiency.
Keywords - Evaluation, Performance, MLSO-DRAC, Framework, Storage, Erasure, PCA