Zhao H.,Key Laboratory of Trustworthy Computing of Shanghai |
Zhao H.,East China Normal University |
Ai S.,Nanyang Technological University |
Lv Z.,East China Normal University |
Li B.,Key Laboratory of Trustworthy Computing of Shanghai
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) | Year: 2010
As a Network Common Data Format, NetCDF has been widely used in terrestrial, marine and atmospheric sciences. A new paralleling storage and access method for large scale NetCDF scientific data is implemented based on Hadoop. The retrieval method is implemented based on MapReduce. The Argo data is used to demonstrate our method. The performance is compared under a distributed environment based on PCs by using different data scale and different task numbers. The experiments result show that the parallel method can be used to store and access the large scale NetCDF efficiently. © 2010 Springer-Verlag Berlin Heidelberg.