Open Access   Article Go Back

The Hadoop Dispersed File system: Balancing Movability And Performance

K.U. Malar1 , D. Ragupathi2 , G.M. Prabhu3

Section:Research Paper, Product Type: Journal Paper
Volume-2 , Issue-9 , Page no. 166-177, Sep-2014

Online published on Oct 04, 2014

Copyright © K.U. Malar, D. Ragupathi, G.M. Prabhu . This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

View this paper at   Google Scholar | DPI Digital Library

How to Cite this Paper

  • IEEE Citation
  • MLA Citation
  • APA Citation
  • BibTex Citation
  • RIS Citation

IEEE Style Citation: K.U. Malar, D. Ragupathi, G.M. Prabhu, “The Hadoop Dispersed File system: Balancing Movability And Performance,” International Journal of Computer Sciences and Engineering, Vol.2, Issue.9, pp.166-177, 2014.

MLA Style Citation: K.U. Malar, D. Ragupathi, G.M. Prabhu "The Hadoop Dispersed File system: Balancing Movability And Performance." International Journal of Computer Sciences and Engineering 2.9 (2014): 166-177.

APA Style Citation: K.U. Malar, D. Ragupathi, G.M. Prabhu, (2014). The Hadoop Dispersed File system: Balancing Movability And Performance. International Journal of Computer Sciences and Engineering, 2(9), 166-177.

BibTex Style Citation:
@article{Malar_2014,
author = {K.U. Malar, D. Ragupathi, G.M. Prabhu},
title = {The Hadoop Dispersed File system: Balancing Movability And Performance},
journal = {International Journal of Computer Sciences and Engineering},
issue_date = {9 2014},
volume = {2},
Issue = {9},
month = {9},
year = {2014},
issn = {2347-2693},
pages = {166-177},
url = {https://www.ijcseonline.org/full_paper_view.php?paper_id=271},
publisher = {IJCSE, Indore, INDIA},
}

RIS Style Citation:
TY - JOUR
UR - https://www.ijcseonline.org/full_paper_view.php?paper_id=271
TI - The Hadoop Dispersed File system: Balancing Movability And Performance
T2 - International Journal of Computer Sciences and Engineering
AU - K.U. Malar, D. Ragupathi, G.M. Prabhu
PY - 2014
DA - 2014/10/04
PB - IJCSE, Indore, INDIA
SP - 166-177
IS - 9
VL - 2
SN - 2347-2693
ER -

VIEWS PDF XML
3501 3330 downloads 3554 downloads
  
  
           

Abstract

Hadoop is a general open-source application of map reduce for the examination of big datasets. To achieve storing capitals crosswise the cluster, Hadoop uses a dispersed user-level file system. This file system � HDFS � is written in java and envisioned for movability crosswise varied hardware and software platforms. This newspaper inspects the presentation of HDFS and uncovers numerous presentation issues. First, architectural blockages be in the Hadoop application that consequence in incompetent HDFS usage owing to postponements in preparation new map reduce tasks. Second, movability limits stop the java application after abusing topographies of the innate platform. Third, HDFS implicitly brands movability molds about how the innate phase achieves storing resources, smooth however innate file systems and I/O schedulers vary widely in arrangement and behavior. This newspaper examines the root details of these presentation blockages in instruction to assess tradeoffs among movability and presentation in the Hadoop dispersed file system.

Key-Words / Index Term

Component; Formatting; Style; Styling; Insert

References

[1] Chandrasekar, S. ; Dakshinamurthy, R. ; Seshakumar, P.G. ;Prabavathy, B. ; Babu, C. �A novel indexing scheme for efficient handling of small filesin Hadoop Distributed File System�Computer Communication and Informatics (ICCCI), 2013 International Conference on Publication Year: 2013, Page(s): 1 - 8
[2] Jing Zhang ; Gongqing Wu ; Xuegang Hu ; Xindong Wu �A Distributed Cache for Hadoop Distributed File System in Real-Time Cloud Services� Grid Computing (GRID), 2012 ACM/IEEE 13th International Conference on Publication Year: 2012 , Page(s): 12 - 21
[3 Kaushik, R.T. ; Bhandarkar, M. ; Nahrstedt, K. �Evaluation and Analysis of GreenHDFS: A Self-Adaptive, Energy-Conserving Variant of the Hadoop Distributed File System� Cloud Computing Technology and Science (CloudCom), 2010 IEEE Second International Conference on Publication Year: 2010 , Page(s): 274 - 287
[4] Hieu Hanh Le ; Hikida, S. ; Yokota, H. Dependable, �An Evaluation of Power-Proportional Data Placement forHadoop Distributed File Systems Autonomic and Secure Computing (DASC), 2011 IEEE Ninth International Conference on Publication Year: 2011 , Page(s): 752 - 759
[5] Attebury, G. ; Baranovski, A. ; Bloom, K. ; Bockelman, B. ; Kcira, D. ;Letts, J. ; Levshina, T. ; Lundestedt, C. ; Martin, T. ; Maier, W. ;Haifeng Pi ; Rana, A. ; Sfiligoi, I. ; Sim, A. ; Thomas, M. ;Wuerthwein, �Hadoop distributed file system for the Grid� F. Nuclear Science Symposium Conference Record (NSS/MIC), 2009 IEEE Publication Year: 2009 , Page(s): 1056 - 1061
[6] Youwei Wang ; WeiPing Wang ; Can Ma ; Dan Meng �Zput: A speedy data uploading approach for the Hadoop Distributed File System� Cluster Computing (CLUSTER), 2013 IEEE International Conference on Publication Year: 2013 , Page(s): 1 � 5
[7] Shahabinejad, M. ; Khabbazian, M. ; Ardakani, M. �An Efficient Binary Locally Repairable Code for Hadoop Distributed File System� Communications Letters, IEEE Volume: 18 , Issue: 8 Publication Year: 2014 , Page(s): 1287 � 1290
[8] Tomasic, I. ; Ugovsek, J. ; Rashkovska, A. ; Trobec, R. �Multicluster Hadoop Distributed File System�MIPRO, 2012 Proceedings of the 35th International Convention Publication Year: 2012 , Page(s): 301 - 305
[9] Hsiao-Ying Lin ; Shiuan-Tzuo Shen ; Wen-Guey Tzeng ; Lin, B.-S.P. �Toward Data Confidentiality via Integrating Hybrid Encryption Schemes and Hadoop Distributed File System� Advanced Information Networking and Applications (AINA), 2012 IEEE 26th International Conference on Publication Year: 2012, Page(s): 740 - 747
[10] Moraveji, R. ; Taheri, J. ; Farahabady, M.R.H. ; Rizvandi, N.B. ;Zomaya, A.Y. �Data-Intensive Workload Consolidation for the Hadoop Distributed File System� Grid Computing (GRID), 2012 ACM/IEEE 13th International Conference on Publication Year: 2012 , Page(s): 95 � 103 �The Hadoop Distributed File System�
[11] Shvachko, K. ; Hairong Kuang ; Radia, S. ; Chansler, R. Mass Storage Systems and Technologies (MSST), 2010 IEEE 26th Symposium on Publication Year: 2010 , Page(s): 1 - 10
[12] Xiayu Hua ; Hao Wu ; Shangping Ren �Enhancing Throughput of Hadoop Distributed File System for Interaction-Intensive Task Parallel, Distributed and Network-Based Processing (PDP), 2014 22nd Euromicro International Conference on Publication Year: 2014 , Page(s): 508 � 511
[13] Wei Zhou ; Jizhong Han ; Zhang Zhang ; Jiao Dai ,�Dynamic Random Access for Hadoop Distributed File System� Distributed Computing Systems Workshops (ICDCSW), 2012 32nd International Conference on Publication Year: 2012 , Page(s): 17 � 22
[14] Madaan, S. ; Agrawal, R.K. �Implementation of identity based distributed cloud storage encryption scheme using PHP and C for Hadoop File System� Tier 2 Federation Grid, Cloud & High Performance Computing Science (RO-LCG), 2012 5th Romania Publication Year: 2012 , Page(s): 74 - 77
[15] Yang Jin ; Tang Deyu ; Zhou Yi �A Distributed Storage Model for EHR Based on HBase� Information Management, Innovation Management and Industrial Engineering (ICIII), 2011 International Conference on Volume: 2 Publication Year: 2011 , Page(s): 369 - 372
[16] Yang Jin ; Tang Deyu ; Zheng Xianrong �Research on the distributed electronic medical records storage model� IT in Medicine and Education (ITME), 2011 International Symposium on Volume: 2 Publication Year: 2011 , Page(s): 288 - 292
[17] Krish, K.R. ; Anwar, A. ; Butt, A.R. �hatS: A Heterogeneity-Aware Tiered Storage for Hadoop� Cluster, Cloud and Grid Computing (CCGrid), 2014 14th IEEE/ACM International Symposium on Publication Year: 2014 , Page(s): 502 � 511
[18] Changjian Fu ; Zhihua Leng �A Framework for Recommender Systems in E-Commerce Based on Distributed Storage and Data-Mining� E-Business and E-Government (ICEE), 2010 International Conference on Publication Year: 2010 , Page(s): 3502 - 3505
[19] Bo Dong ; Xiao Zhong ; Qinghua Zheng ; Lirong Jian ; Jian Liu ; Jie Qiu ; Ying Li �Correlation Based File Prefetching Approach for Hadoop� Cloud Computing Technology and Science (CloudCom), 2010 IEEE Second International Conference on Publication Year: 2010 , Page(s): 41 - 48
[20] Bo Dong ; Jie Qiu ; Qinghua Zheng ; Xiao Zhong ; Jingwei Li ; Ying Li �A Novel Approach to Improving the Efficiency of Storing and Accessing Small Files on Hadoop: A Case Study by PowerPoint Files� Services Computing (SCC), 2010 IEEE International Conference on Publication Year: 2010 , Page(s): 65 - 72
[21] Martha, V.S. ; Weizhong Zhao ; Xiaowei Xu �h-MapReduce: A Framework for Workload Balancing in MapReduce� Advanced Information Networking and Applications (AINA), 2013 IEEE 27th International Conference on Publication Year: 2013 , Page(s): 637 - 644
[22] Li Wang ; Zhiwei Ni ; Yiwen Zhang ; Zhang Jun Wu ; �Liyang Tang Notice of Violation of IEEE Publication Principles �Pipelined-MapReduce: An Improved MapReduce Parallel Programing Model � Intelligent Computation Technology and Automation (ICICTA), 2011 International Conference on Volume: 1 Publication Year: 2011 , Page(s): 871 - 874
[23] Kaiqi Xiong ; Yuxiong He �Power-effiicent resource allocation in MapReduce clusters�Integrated Network Management (IM 2013), 2013 IFIP/IEEE International Symposium on Publication Year: 2013 , Page(s): 603 � 608
[24] Yanpei Chen ; Ganapathi, A. ; Griffith, R. ; Katz, R. �The Case for Evaluating MapReduce Performance Using Workload Suites� Modeling, Analysis & Simulation of Computer and Telecommunication Systems (MASCOTS), 2011 IEEE 19th International Symposium on Publication Year: 2011 , Page(s): 390 - 399 �Getting more for less in optimized MapReduce workflows�
[25] Zhuoyao Zhang ; Cherkasova, L. ; Boon Thau Loo Integrated Network Management (IM 2013), 2013 IFIP/IEEE International Symposium on Publication Year: 2013 , Page(s): 93 � 100
[26] Guigang Zhang ; Jian Wang ; Weixing Huang ; Chao Li ; Yong Zhang; Chunxiao Xing �A Semantic++ MapReduce: A Preliminary Report�Semantic Computing (ICSC), 2014 IEEE International Conference on Publication Year: 2014 , Page(s): 330 � 336
[27] Kewen Wang ; Wenzhong Predator An experience guided configuration optimizer for Hadoop MapReduce Tang Cloud Computing Technology and Science (CloudCom), 2012 IEEE 4th International Conference on Publication Year: 2012 , Page(s): 419 - 426
[28] Chu Huang ; Sencun Zhu ; Dinghao Wu Towards Trusted Services: Result Verification Schemes for MapReduce Cluster, Cloud and Grid Computing (CCGrid), 2012 12th IEEE/ACM International Symposium on Publication Year: 2012 , Page(s): 41 - 48
[29] Cura: A Cost-Optimized Model for MapReduce in a Cloud Palanisamy, B. ; Singh, A. ; Langston, B. Parallel & Distributed Processing (IPDPS), 2013 IEEE 27th International Symposium on Publication Year: 2013 , Page(s): 1275 � 1286
[30] M2M: A simple Matlab-to-MapReduce translator for cloud computing Zhang, Junbo ; Xiang, Dong ; Li, Tianrui ; Pan, Yi Tsinghua Science and Technology Volume: 18 , Issue: 1 Publication Year: 2013 , Page(s): 1-9