Evaluación de algoritmos de aprendizaje de máquina no supervisados sobre datos climáticos

dc.contributorDuque-Méndez, Néstor Darío
dc.contributorGAIA - Grupo Ambientes inteligentes Adaptativos
dc.creatorRamírez-Gómez, Juan Sebastián
dc.date.accessioned2020-03-04T20:56:06Z
dc.date.available2020-03-04T20:56:06Z
dc.date.created2020-03-04T20:56:06Z
dc.date.issued2019
dc.identifierhttps://repositorio.unal.edu.co/handle/unal/75848
dc.description.abstractFor the use of climate data, researchers have difficulties in determining which clustering algorithm may be the best performer for the processing and analysis of a specific dataset. There is no clear information that allows to give an approach about which clustering algorithm to use based on the quantity of records, quantity and types of variables and quality of the data set (datasets with missing data and outliers data) to get a good performance. In this research we present the evaluation of unsupervised machine learning algorithms with K-means, K-medoids and Linkage-complete, applied to three datasets with climatological variables (temperature, rainfall, relative humidity and solar radiation), for three meteorological stations located in the department of Caldas, Colombia, at different heights above sea level (municipality of Villamaría, the city of Manizales and Los Nevados National Park), which include 430,635 records, 530,802 records and 248,297 records respectively. 5 scenarios are defined for 2, 3 and 5 clusters for each of the two partitioned algorithms and 5 scenarios for the hierarchical algorithm, for each of the meteorological stations, and adding 3 more scenarios (one for each algorithm) using parallel computing, and applying a different quantity and grouping of variables for the different scenarios and using Euclidean distance as a measure of distance, Davis-Bouldin as a method of quality evaluation of clusters, application of normalization with techniques such as range-transformation and z -transformation, definition of number of iterations of the algorithm and reduction of dimensionality with Principal Components Analysis. In addition, the computational cost of the algorithms is evaluated according to the characteristics established for the scenarios. This research provides conclusions that can guide the researcher about some decisions in clusters analysis in meteorological data, as well as identify the most important characteristics to take into consideration when constructing the best performance scenario according to the particular needs and requirements.
dc.description.abstractPara el uso de datos climáticos los investigadores tienen dificultades al determinar qué algoritmo de agrupamiento puede ser el de mejor desempeño para el procesamiento y análisis de un dataset específico. No se cuenta con información clara que permita dar un acercamiento de qué algoritmo de clustering emplear basados en la cantidad de instancias, cantidad y tipos de variables y calidad del conjunto de datos (datasets con datos faltantes y datos atípicos) para obtener un buen desempeño. En esta investigación se presenta la evaluación de algoritmos de aprendizaje de máquina no supervisados con K-means, K-medoids y Agrupamiento jerárquico, aplicados a tres datasets con variables climatológicas (temperatura, precipitación, humedad relativa y radiación solar), para tres estaciones climáticas ubicadas en el departamento de Caldas, Colombia, a diferentes alturas sobre el nivel del mar (municipio de Villamaría, la ciudad de Manizales y el Parque Nacional Natural Los Nevados), los cuales comprenden 430.635 instancias, 530.802 instancias y 248.297 instancias respectivamente. Se definen 5 escenarios de ejecución con 2, 3 y 5 clústeres para cada uno de los dos algoritmos particionados y 5 escenarios de ejecución para el algoritmo jerárquico, para cada una de las estaciones climáticas, y adicionando 3 escenarios más (uno para cada algoritmo) empleando computación paralela, y aplicando para los diversos escenarios una variedad de tratamientos con diferente cantidad y agrupación de variables y utilizando distancia euclidiana como medida de distancia; Davis-Bouldin como método de evaluación de calidad de los clústeres; aplicación de normalización con técnicas como range-transformation y z-transformation; Definición de número de iteraciones del algoritmo y reducción de dimensionalidad con Análisis de Componentes Principales. Además, se evalúa el costo computacional de los algoritmos según las características establecidas para los escenarios. Esta investigación entrega conclusiones que pueden orientar al investigador sobre algunas decisiones en análisis de clústeres en datos meteorológicos, como también para identificar las características más importantes a tener en cuenta al momento de construir el escenario de mejor desempeño según las necesidades y requerimientos particulares. (Texto tomado de la fuente)
dc.languagespa
dc.publisherUniversidad Nacional de Colombia - Sede Manizales
dc.relationAbdul Halim, N. D., Latif, M. T., Ahamad, F., Dominick, D., Chung, J. X., Juneng, L., & Khan, M. F. (2018). The long-term assessment of air quality on an island in Malaysia. Heliyon, 4(12). https://doi.org/10.1016/j.heliyon.2018.e01054 Acciani, G., Chiarantoni, E., Fornarelli, G., & Vergura, S. (2003). A feature extraction unsupervised neural network for an environmental data set. Neural Networks, 16(3–4), 427–436. https://doi.org/10.1016/S0893-6080(03)00014-5 Aggarwal, C. C., & Reddy, C. K. (2013). DATA Custering Algorithms and Applications. (C. K. R. Charu C. Aggarwal, Ed.). CRC Press. Alpaydın, E. (2014). Introduction to machine learning. Methods in Molecular Biology, 1107, 105–128. https://doi.org/10.1007/978-1-62703-748-8-7 André, M., Perez, R., Soubdhan, T., Schlemmer, J., Calif, R., & Monjoly, S. (2019). Preliminary assessment of two spatio-temporal forecasting technics for hourly satellite-derived irradiance in a complex meteorological context. Solar Energy, 177(December 2018), 703–712. https://doi.org/10.1016/j.solener.2018.11.010 Arbolino, R., Carlucci, F., Cirà, A., Ioppolo, G., & Yigitcanlar, T. (2017). Efficiency of the EU regulation on greenhouse gas emissions in Italy: The hierarchical cluster analysis approach. Ecological Indicators, 81(May), 115–123. https://doi.org/10.1016/j.ecolind.2017.05.053 Arroyo, Á., Herrero, Á., Tricio, V., & Corchado, E. (2017). Analysis of meteorological conditions in Spain by means of clustering techniques. Journal of Applied Logic, 24, 76–89. https://doi.org/10.1016/j.jal.2016.11.026 Asadi Zarch, M. A., Sivakumar, B., & Sharma, A. (2015). Assessment of global aridity change. Journal of Hydrology, 520, 300–313. https://doi.org/10.1016/j.jhydrol.2014.11.033 Bador, M., Naveau, P., Gilleland, E., Castellà, M., & Arivelo, T. (2015). Spatial clustering of summer temperature maxima from the CNRM-CM5 climate model ensembles & E-OBS over Europe. Weather and Climate Extremes, 9, 17–24. https://doi.org/10.1016/j.wace.2015.05.003 Ben-David, S., & Shalev-Shwartz, S. (2014). Understanding Machine Learning: From Theory to Algorithms. Understanding Machine Learning: From Theory to Algorithms. https://doi.org/10.1017/CBO9781107298019 Bramer, M. (2007). Principles of Data Mining. Springer. https://doi.org/10.1007/978-1-4471-4884-5_1 Calliari, E., Farnia, L., Ramieri, E., & Michetti, M. (2019). A network approach for moving from planning to implementation in climate change adaptation: evidence from southern Mexico. Environmental Science and Policy, 93(November 2017), 146–157. https://doi.org/10.1016/j.envsci.2018.11.025 Carro-Calvo, L., Ordóñez, C., García-Herrera, R., & Schnell, J. L. (2017). Spatial clustering and meteorological drivers of summer ozone in Europe. Atmospheric Environment, 167, 496–510. https://doi.org/10.1016/j.atmosenv.2017.08.050 Carvalho, M. J., Melo-Gonçalves, P., Teixeira, J. C., & Rocha, A. (2016). Regionalization of Europe based on a K-Means Cluster Analysis of the climate change of temperatures and precipitation. Physics and Chemistry of the Earth, 94, 22–28. https://doi.org/10.1016/j.pce.2016.05.001 Cashman, S. A., Meyer, D. E., Edelen, A. N., Ingwersen, W. W., Abraham, J. P., Barrett, W. M., … Smith, R. L. (2016). Mining Available Data from the United States Environmental Protection Agency to Support Rapid Life Cycle Inventory Modeling of Chemical Manufacturing. Environmental Science and Technology, 50(17), 9013–9025. https://doi.org/10.1021/acs.est.6b02160 Chen, J., Song, M., & Xu, L. (2015). Evaluation of environmental efficiency in China using data envelopment analysis. Ecological Indicators, 52, 577–583. https://doi.org/10.1016/j.ecolind.2014.05.008 Chen, L., & Jia, G. (2017). Environmental efficiency analysis of China’s regional industry : a data envelopment analysis (DEA) based approach. Journal of Cleaner Production, 142, 846–853. https://doi.org/10.1016/j.jclepro.2016.01.045 Chen, Y., Wang, L., Li, F., Du, B., Choo, K. K. R., Hassan, H., & Qin, W. (2017). Air quality data clustering using EPLS method. Information Fusion, 36, 225–232. https://doi.org/10.1016/j.inffus.2016.11.015 Chidean, M. I., Caamaño, A. J., Ramiro-Bargueño, J., Casanova-Mateo, C., & Salcedo-Sanz, S. (2018). Spatio-temporal analysis of wind resource in the Iberian Peninsula with data-coupled clustering. Renewable and Sustainable Energy Reviews, 81(June), 2684–2694. https://doi.org/10.1016/j.rser.2017.06.075 Chidean, M. I., Muñoz-Bulnes, J., Ramiro-Bargueño, J., Caamaño, A. J., & Salcedo-Sanz, S. (2015). Spatio-temporal trend analysis of air temperature in Europe and Western Asia using data-coupled clustering. Global and Planetary Change, 129, 45–55. https://doi.org/10.1016/j.gloplacha.2015.03.006 Clay, N., & King, B. (2019). Smallholders’ uneven capacities to adapt to climate change amid Africa’s ‘green revolution’: Case study of Rwanda’s crop intensification program. World Development, 116, 1–14. https://doi.org/S0305750X18304285 Conradt, T., Gornott, C., & Wechsung, F. (2016). Extending and improving regionalized winter wheat and silage maize yield regression models for Germany: Enhancing the predictive skill by panel definition through cluster analysis. Agricultural and Forest Meteorology, 216, 68–81. https://doi.org/10.1016/j.agrformet.2015.10.003 Deogun, J. S., & Raghavan, V. V. (1999). Data Mining : Research Trends , Challenges , and Applications, 1–29. https://doi.org/10.1.1.52.337 Erman, N., & Suklan, J. (2015). Performance of selected agglomerative clustering methods. Innovative Issues and Approaches in Social Sciences, 8(January). https://doi.org/10.12959/issn.1855-0541.IIASS-2015-no1-art11 Falquina, R., & Gallardo, C. (2017). Development and application of a technique for projecting novel and disappearing climates using cluster analysis. Atmospheric Research, 197(July), 224–231. https://doi.org/10.1016/j.atmosres.2017.06.031 Farah, S., Whaley, D., Saman, W., & Boland, J. (2019). Integrating Climate Change into Meteorological Weather Data for Building Energy Simulation. Energy and Buildings, 183, 749–760. https://doi.org/S0378778818323296 Franceschi, F., Cobo, M., & Figueredo, M. (2018). Discovering relationships and forecasting PM10 and PM2.5 concentrations in Bogotá Colombia, using Artificial Neural Networks, Principal Component Analysis, and k-means clustering. Atmospheric Pollution Research, 9(5), 912–922. https://doi.org/10.1016/j.apr.2018.02.006 Franco, D. G. de B., & Steiner, M. T. A. (2018). Clustering of solar energy facilities using a hybrid fuzzy c-means algorithm initialized by metaheuristics. Journal of Cleaner Production, 191, 445–457. https://doi.org/10.1016/j.jclepro.2018.04.207 Gallo, C., Faccilongo, N., & La Sala, P. (2017). Clustering analysis of environmental emissions: A study on Kyoto Protocol’s impact on member countries. Journal of Cleaner Production. https://doi.org/10.1016/j.jclepro.2017.07.194 Gan, G., Ma, C., & Wu, J. (2007). Data Clustering: Theory, Algorithms, and Applications. Philadelphia, Pennsylvania: SIAM - Society for Industrial and Applied Mathematics. https://doi.org/10.1137/1.9780898718348 Ghayekhloo, M., Ghofrani, M., Menhaj, M. B., & Azimi, R. (2015). A novel clustering approach for short-term solar radiation forecasting. Solar Energy, 122, 1371–1383. https://doi.org/10.1016/j.solener.2015.10.053 Guo, Y., Gao, H., & Wu, Q. (2017). A meteorological information mining-based wind speed model for adequacy assessment of power systems with wind power. International Journal of Electrical Power and Energy Systems, 93, 406–413. https://doi.org/10.1016/j.ijepes.2017.05.031 Han, J., Kamber, M., & Pei, J. (2012). Data Mining: Concepts and Techniques. San Francisco, CA, itd: Morgan Kaufmann. https://doi.org/10.1016/B978-0-12-381479-1.00001-0 Han, S., Zhang, L. na, Liu, Y. qian, Zhang, H., Yan, J., Li, L., … Wang, X. (2019). Quantitative evaluation method for the complementarity of wind–solar–hydro power and optimization of wind–solar ratio. Applied Energy, 236(December 2018), 973–984. https://doi.org/10.1016/j.apenergy.2018.12.059 Hao, Y., Dong, L., Liao, X., Liang, J., Wang, L., & Wang, B. (2019). A novel clustering algorithm based on mathematical morphology for wind power generation prediction. Renewable Energy, 136, 572–585. https://doi.org/10.1016/j.renene.2019.01.018 Hidalgo, J., Dumas, G., Masson, V., Petit, G., Bechtel, B., Bocher, E., … Mills, G. (2019). Comparison between local climate zones maps derived from administrative datasets and satellite observations. Urban Climate, 27(November 2017), 64–89. https://doi.org/10.1016/j.uclim.2018.10.004 Jiang, J., Ye, B., Xie, D., & Tang, J. (2017). Provincial-level carbon emission drivers and emission reduction strategies in China: Combining multi-layer LMDI decomposition with hierarchical clustering. Journal of Cleaner Production, 169, 178–190. https://doi.org/10.1016/j.jclepro.2017.03.189 Kalteh, A. M., Hjorth, P., & Berndtsson, R. (2008). Review of the self-organizing map (SOM) approach in water resources: Analysis, modelling and application. Environmental Modelling & Software, 23(7), 835–845. https://doi.org/http://dx.doi.org/10.1016/j.envsoft.2007.10.001 Kanevski, M., Pozdnukhov, A., & Timonin, V. (2008). Machine Learning Algorithms for GeoSpatial Data. Applications and Software Tools. Iemss.Org, 320–327. Khedairia, S., & Khadir, M. T. (2012). Impact of clustered meteorological parameters on air pollutants concentrations in the region of Annaba, Algeria. Atmospheric Research, 113, 89–101. https://doi.org/10.1016/j.atmosres.2012.05.002 Lausch, A., Schmidt, A., & Tischendorf, L. (2015). Data mining and linked open data – New perspectives for data analysis in environmental research. Ecological Modelling, 295, 5–17. https://doi.org/10.1016/j.ecolmodel.2014.09.018 Lee, J., & Kim, K. Y. (2018). Analysis of source regions and meteorological factors for the variability of spring PM10 concentrations in Seoul, Korea. Atmospheric Environment, 175(April 2017), 199–209. https://doi.org/10.1016/j.atmosenv.2017.12.013 Li, C., Sun, L., Jia, J., Cai, Y., & Wang, X. (2016). Risk assessment of water pollution sources based on an integrated k-means clustering and set pair analysis method in the region of Shiyan, China. Science of the Total Environment, 557–558, 307–316. https://doi.org/10.1016/j.scitotenv.2016.03.069 Li, S., Ma, H., & Li, W. (2017). Typical solar radiation year construction using k-means clustering and discrete-time Markov chain. Applied Energy, 205(May), 720–731. https://doi.org/10.1016/j.apenergy.2017.08.067 Lin, P., Peng, Z., Lai, Y., Cheng, S., Chen, Z., & Wu, L. (2018). Short-term power prediction for photovoltaic power plants using a hybrid improved Kmeans-GRA-Elman model based on multivariate meteorological factors and historical power datasets. Energy Conversion and Management, 177(July), 704–717. https://doi.org/10.1016/j.enconman.2018.10.015 Lokers, R., Knapen, R., Janssen, S., van Randen, Y., & Jansen, J. (2016). Analysis of Big Data technologies for use in agro-environmental science. Environmental Modelling and Software, 84, 494–504. https://doi.org/10.1016/j.envsoft.2016.07.017 Marzban, C., & Sandgathe, S. (2006). Cluster Analysis for Verification of Precipitation Fields. Weather and Forecasting, 21(5), 824–838. https://doi.org/10.1175/WAF948.1 Mayer, A., Winkler, R., & Fry, L. (2014). Classification of watersheds into integrated social and biophysical indicators with clustering analysis. Ecological Indicators, 45, 340–349. https://doi.org/10.1016/j.ecolind.2014.04.030 Meghea, I., Mihai, M., Lacatusu, I., & Iosub, I. (2012). Evaluation of Monitoring of Lead Emissions in Bucharest by Statistical Processing. Journal of Environmental Protection and Ecology, 13(2), 746–755. Mokdad, F., & Haddad, B. (2017). Improved infrared precipitation estimation approaches based on k-means clustering: Application to north Algeria using MSG-SEVIRI satellite data. Advances in Space Research, 59(12), 2880–2900. https://doi.org/10.1016/j.asr.2017.03.027 Naik, A., & Samant, L. (2016). Correlation Review of Classification Algorithm Using Data Mining Tool: WEKA, Rapidminer, Tanagra, Orange and Knime. Procedia Computer Science, 85(Cms), 662–668. https://doi.org/10.1016/j.procs.2016.05.251 Nguyen, T. T., Kawamura, A., Tong, T. N., Nakagawa, N., Amaguchi, H., & Gilbuena, R. (2015). Clustering spatio-seasonal hydrogeochemical data using self-organizing maps for groundwater quality assessment in the Red River Delta, Vietnam. Journal of Hydrology, 522, 661–673. https://doi.org/10.1016/j.jhydrol.2015.01.023 Obradovic, V., Bjelica, D., Petrovic, D., Mihic, M., & Todorovic, M. (2016). Whether We are Still Immature to Assess the Environmental KPIs! Procedia - Social and Behavioral Sciences, 226(October 2015), 132–139. https://doi.org/10.1016/j.sbspro.2016.06.171 Oshana, R. (2015). Principles of Parallel Computing. Multicore Software Development Techniques, 1–30. https://doi.org/10.1016/b978-0-12-800958-1.00001-2 Parente, J., Pereira, M. G., & Tonini, M. (2016). Space-time clustering analysis of wildfires: The influence of dataset characteristics, fire prevention policy decisions, weather and climate. Science of the Total Environment, 559, 151–165. https://doi.org/10.1016/j.scitotenv.2016.03.129 Pechenizkiy, M., Puuronen, S., & Tsymbal, A. (2008). Does relevance matter to data mining research? In Studies in Computational Intelligence (Vol. 118, pp. 251–275). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-540-78488-3_15 Pitchayadejanant, K., & Nakpathom, P. (2017). Data mining approach for arranging and clustering the agro-tourism activities in orchard. Kasetsart Journal of Social Sciences. https://doi.org/10.1016/j.kjss.2017.07.004 Pokorná, L., Kučerová, M., & Huth, R. (2018). Annual cycle of temperature trends in Europe, 1961–2000. Global and Planetary Change, 170(August), 146–162. https://doi.org/10.1016/j.gloplacha.2018.08.015 Reduction, D., Comparative, M., Arroyo-hernández, J., & Rica, C. (2016). Métodos de reducción de dimensionalidad : Análisis comparativo de los métodos APC , ACPP y ACPK. Uniciencia, 30(1), 115–122. https://doi.org/http://dx.doi.org/10.15359/ru.30-1.7 Ruzmaikin, A., & Guillaume, A. (2014). Clustering of atmospheric data by the deterministic annealing. Journal of Atmospheric and Solar-Terrestrial Physics, 120, 121–131. https://doi.org/10.1016/j.jastp.2014.09.009 Schäfer, E., Heiskanen, J., Heikinheimo, V., & Pellikka, P. (2016). Mapping tree species diversity of a tropical montane forest by unsupervised clustering of airborne imaging spectroscopy data. Ecological Indicators, 64, 49–58. https://doi.org/10.1016/j.ecolind.2015.12.026 Schneider, T., Hampel, H., Mosquera, P. V., Tylmann, W., & Grosjean, M. (2018). Paleo-ENSO revisited: Ecuadorian Lake Pallcacocha does not reveal a conclusive El Niño signal. Global and Planetary Change, 168(February), 54–66. https://doi.org/10.1016/j.gloplacha.2018.06.004 Shaukat, S. S., Rao, T. A., & Khan, M. A. (2016). Impact of sample size on principal component analysis ordination of an environmental data set: Effects on Eigenstructure. Ekologia Bratislava, 35(2), 173–190. https://doi.org/10.1515/eko-2016-0014 Sheridan, S. C., & Lee, C. C. (2011). The self-organizing map in synoptic climatological research. Progress in Physical Geography, 35(1), 109–119. https://doi.org/10.1177/0309133310397582 Sivaramakrishnan, T. R., & Meganathan, S. (2012). Point rainfall prediction using data mining technique. Research Journal of Applied Sciences, Engineering and Technology, 4(13), 1899–1902. https://doi.org/10.5120/14467-2750 Soubdhan, T., Abadi, M., & Emilion, R. (2014). Time dependent classification of solar radiation sequences using best information criterion. Energy Procedia, 57, 1309–1316. https://doi.org/10.1016/j.egypro.2014.10.121 Wang, X., Huang, G., Lin, Q., Nie, X., Cheng, G., Fan, Y., … Suo, M. (2013). A stepwise cluster analysis approach for downscaled climate projection - A Canadian case study. Environmental Modelling and Software, 49, 141–151. https://doi.org/10.1016/j.envsoft.2013.08.006 Wrzesień, M., Treder, W., Klamkowski, K., & Rudnicki, W. R. (2018). Prediction of the apple scab using machine learning and simple weather stations. Computers and Electronics in Agriculture, (June 2017). https://doi.org/10.1016/j.compag.2018.09.026 Yadav, A. K., Malik, H., & Chandel, S. S. (2015). Application of rapid miner in ANN based prediction of solar radiation for assessment of solar energy resource potential of 76 sites in Northwestern India. Renewable and Sustainable Energy Reviews, 52, 1093–1106. https://doi.org/10.1016/j.rser.2015.07.156 Yahyaoui, H., & Own, H. S. (2018). Unsupervised clustering of service performance behaviors. Information Sciences, 422, 558–571. https://doi.org/10.1016/j.ins.2017.08.065 Zheng, Y., Han, J., Huang, Y., Fassnacht, S. R., Xie, S., Lv, E., & Chen, M. (2017). Vegetation response to climate conditions based on NDVI simulations using stepwise cluster analysis for the Three-River Headwaters region of China. Ecological Indicators, (September 2016), 0–1. https://doi.org/10.1016/j.ecolind.2017.06.040 Zheng, Y., Han, J., Huang, Y., Fassnacht, S. R., Xie, S., Lv, E., … Sharma, A. (2016). Assessment of global aridity change. Ecological Indicators, 75(September 2016), 151–165. https://doi.org/10.1016/j.scitotenv.2015.11.063 Zuo, X., Hua, H., Dong, Z., & Hao, C. (2017). Environmental Performance Index at the Provincial Level for China 2006–2011. Ecological Indicators, 75, 48–56. https://doi.org/10.1016/j.ecolind.2016.12.016
dc.rightsAtribución-NoComercial 4.0 Internacional
dc.rightsAcceso abierto
dc.rightshttp://creativecommons.org/licenses/by-nc/4.0/
dc.rightsinfo:eu-repo/semantics/openAccess
dc.rightsDerechos reservados - Universidad Nacional de Colombia
dc.titleEvaluación de algoritmos de aprendizaje de máquina no supervisados sobre datos climáticos
dc.typeOtro


Este ítem pertenece a la siguiente institución