With the invent of internet technology and increasing the use of digital technology, the storage system has been into a difficult mode of expansion as the growing data is a big concern for the storage system, typically the images which occupy large scale size of storage. Hence worldwide data Deduplication mechanism had been used enormously to improve the backup storage system. Lots of efforts had done worldwide for identifying the image Deduplication system to improve the storage utilization system. In spite of these continuous efforts the so-called existing system can only manage to remove the images which are the same in texture and size, but it fails to identify the Deduplication of images that may have the same visual perceptions but may have different effects. To solve the above issue of image Deduplication our thesis work proposed the mechanism to identify the duplicated images which are technically the same from the perspective of visual identification but different with some effects. With further improving to remove such duplicate images, we are contributing to improving storage space. Our proposed algorithm based on feature selection and extraction and accuracy optimization will not improve storage space but helps to perform such task quite efficiently and effectively.

Image Deduplication, Feature Extraction, Centroid selection, Storage space, Optimization, DHash, Hamming, SIFT Algorithm

[1] NEMIROVSKIY V.B., STOYANOV A.K. Near duplicate image recognition based on the rank distribution of the brightness clusters cardinality, COMPUTER OPTICS. – 2014. – VOL. 38(4). – P. 811-817.
[2] Dynamic Data Deduplication in Cloud Storage by Waraporn Leesakul, Paul Townend, Jie Xu. IEEE 8th International Symposium on Service-Oriented System Engineering (SOSE 2014). IEEE 8th International Symposium on Service-Oriented System Engineering (SOSE 2014), 07-11 Apr 2014, Oxford, UK. IEEE, pp. 320-325. ISBN 9781479925049
[3] A study on data Deduplication techniques for optimized storage by E. Manogar, S. Abirami, 2014 Sixth International Conference on Advanced Computing(lCoAC),pp 161-165, 978-1-4 799-8159-5114
[4] Eunji Lee, Jee E. Jang, Taeseok Kim, Hyokyung Bahn, "On-Demand Snapshot: An Efficient Versioning File System for Phase-Change Memory," IEEE Transactions On Knowledge And Data Engineering, Vol. 25, No. 12, December 2013.
[5] Q. He, Z. Li, X. Zhang, "Data deduplication techniques, "Future Information Technology and Management Engineering (FITME)," vol. I, pp. 430-433, 2010
[6] Maddodi.S, Attigeri G.V, Karunakar. A.K, "Data Deduplication Techniques and Analysis," Emerging Trends in Engineering and Technology (lCETET), pp 664 - 668, IEEE, 2010
[7] N. Mandagere, P. Zhou, M.A. Smith, and S. Uttamchandani."Demystifying data deduplication," In Proceedings of the ACM/IFlP/USENIX Middleware`08 Conference Companion, pages 12-17. ACM, 2008