Xue S.,Nanjing University of Information Science and Technology |
Huang J.,Nanjing University of Information Science and Technology |
Xu X.,Nanjing University of Technology |
Xu X.,Nanjing University of Science and Technology |
And 2 more authors.
Energy Education Science and Technology Part A: Energy Science and Research | Year: 2014
With the increase of the number of meteorological stations, the meteorological data is growing rapidly. Association analysis algorithm is widely applied to the analysis of meteorological data. The traditional association analysis algorithm is difficult to meet the needs of meteorological big data processing. In order to solve this problem, in this paper, we propose a candidate item reduce method named CIRM. It aims at reducing the times of searching data sets and the number of generating candidate sets. To improve its scalability and efficiency in big data environment, CIRM is implemented on Hadoop, a widely-adopted distributed computing platform using the MapReduce parallel processing paradigm. Finally, extensive experiments are conducted on meteorological data sets, and results demonstrate that CIRM significantly improves the efficiency and scalability of the Meteorological analysis data over existing approaches. © Sila Science. All rights reserved.