Small file processing is a performance bottleneck in Hadoop big data platform. It creates huge storage overhead at Hadoop Namenode and exhausts computational resources by spawning multiple map tasks. Current solutions for small file problem can be categorized to merging small files, reusing java virtual machine instance, caching are not adaptive to user and application context. In most solutions, merging is based only on size and do not consider user access or application contexts, content characteristics and their semantic relation. As the result, processing and query latency increases. This work proposes a user and application context adaptive small file management solution to this problem. As part of solution, adaptive block categorization is proposed to categorize blocks based on context and merged. By this way processing speed is increased. Caching is done based on multi criteria optimization specific to context access pattern to reduce the latency. The proposed solution is able to provide 10% higher speed up and 15% lower latency compared to existing works.

#