2024
![The AIDE toolbox: Artificial intelligence for disentangling extreme events [software and data sets]](/assets/images/covers/grsm_2024.jpg)
The AIDE toolbox: Artificial intelligence for disentangling extreme events [software and data sets]
Gonzalez-Calabuig, Maria, Jordi Cortés-Andrés, Tristan Keith Ellis Williams, Mengxue Zhang, Oscar Jose Pellicer-Valero, Miguel-Ángel Fernández-Torres, Gustau Camps-Valls
IEEE Geoscience and Remote Sensing Magazine 2024
We introduce the Artificial Intelligence for Disentangling Extremes (AIDE) toolbox that allows for anomaly detection, extreme event analysis, and impact assessment in remote sensing and geoscience applications. AIDE integrates advanced machine learning (ML) models, ranging in complexity, assumptions, and sophistication, and can yield spatiotemporal explicit monitoring maps with probabilistic estimates. Supervised and unsupervised algorithms, deterministic and probabilistic, convolutional and recurrent neural networks (CNNs and RNNs), as well as methods based on density estimation, are covered by this framework. The open source toolbox is intended for scientists, engineers, and students with basic knowledge of remote sensing and geosciences working in anomaly detection, deep learning (DL), and explainable AI (XAI), and is available at https://github.com/IPL-UV/AIDE.
The AIDE toolbox: Artificial intelligence for disentangling extreme events [software and data sets]
Gonzalez-Calabuig, Maria, Jordi Cortés-Andrés, Tristan Keith Ellis Williams, Mengxue Zhang, Oscar Jose Pellicer-Valero, Miguel-Ángel Fernández-Torres, Gustau Camps-Valls
IEEE Geoscience and Remote Sensing Magazine 2024
We introduce the Artificial Intelligence for Disentangling Extremes (AIDE) toolbox that allows for anomaly detection, extreme event analysis, and impact assessment in remote sensing and geoscience applications. AIDE integrates advanced machine learning (ML) models, ranging in complexity, assumptions, and sophistication, and can yield spatiotemporal explicit monitoring maps with probabilistic estimates. Supervised and unsupervised algorithms, deterministic and probabilistic, convolutional and recurrent neural networks (CNNs and RNNs), as well as methods based on density estimation, are covered by this framework. The open source toolbox is intended for scientists, engineers, and students with basic knowledge of remote sensing and geosciences working in anomaly detection, deep learning (DL), and explainable AI (XAI), and is available at https://github.com/IPL-UV/AIDE.
2023
DSR-GCN: Differentiated-scale restricted graph convolutional network for few-shot hyperspectral image classification
Xue, Zhaohui, Zhiwei Liu, Mengxue Zhang
IEEE Transactions on Geoscience and Remote Sensing 2023
Graph convolutional networks (GCNs) have shown great potential for few-shot hyperspectral image (HSI) classification. Mainstream GCNs construct graphs according to single-scale segmentation, which usually ignores subtle adjacency relations between small regions, leading to an unreliable initial local graph. To overcome the above issue, we propose a differentiated-scale restricted GCN (DSR-GCN) for HSI classification. First, we propose a differentiated-scale graph construction method considering both the subtle and relative wider range spectral–spatial relation. Second, restricted fusion loss is designed to restrict the fusion of features extracted with differentiated-scale GCN branches. Finally, we design a lightweight spectral-spatial siamese network (S3Net) to remedy local pixel-level features. The proposed DSR-GCN can better model spatial structure with a reliable and refined graph, and it can capture more discriminate features in few-shot learning (FSL) scenarios. Extensive experiments conducted on four benchmark datasets demonstrate that DSR-GCN outperforms the other deep learning methods in terms of classification accuracy and generalization performance, with improvements in terms of overall accuracy (OA) around 6.20% ∼ 23.41% (Indian Pines), 4.45% ∼ 16.48% (University of Pavia), 4.25% ∼ 11.85% (Salinas), and 2.0% ∼ 17.23% (University of Houston) under five labeled samples per class.
DSR-GCN: Differentiated-scale restricted graph convolutional network for few-shot hyperspectral image classification
Xue, Zhaohui, Zhiwei Liu, Mengxue Zhang
IEEE Transactions on Geoscience and Remote Sensing 2023
Graph convolutional networks (GCNs) have shown great potential for few-shot hyperspectral image (HSI) classification. Mainstream GCNs construct graphs according to single-scale segmentation, which usually ignores subtle adjacency relations between small regions, leading to an unreliable initial local graph. To overcome the above issue, we propose a differentiated-scale restricted GCN (DSR-GCN) for HSI classification. First, we propose a differentiated-scale graph construction method considering both the subtle and relative wider range spectral–spatial relation. Second, restricted fusion loss is designed to restrict the fusion of features extracted with differentiated-scale GCN branches. Finally, we design a lightweight spectral-spatial siamese network (S3Net) to remedy local pixel-level features. The proposed DSR-GCN can better model spatial structure with a reliable and refined graph, and it can capture more discriminate features in few-shot learning (FSL) scenarios. Extensive experiments conducted on four benchmark datasets demonstrate that DSR-GCN outperforms the other deep learning methods in terms of classification accuracy and generalization performance, with improvements in terms of overall accuracy (OA) around 6.20% ∼ 23.41% (Indian Pines), 4.45% ∼ 16.48% (University of Pavia), 4.25% ∼ 11.85% (Salinas), and 2.0% ∼ 17.23% (University of Houston) under five labeled samples per class.
XAIDA4Detection: A Toolbox for the Detection and Characterization of Spatio-Temporal Extreme Events
Cortés-Andrés, Jordi, Gonzalez-Calabuig, Maria, Zhang, Mengxue, Williams, Tristan, Fernández-Torres, Miguel-Ángel, Pellicer-Valero, Oscar J., Camps-Valls, Gustau
In EGU General Assembly Conference Abstracts. EGU-4816 2023
The automatic anticipation and detection of extreme events constitute a major challenge in the current context of climate change, which has changed their likelihood and intensity. One of the main objectives within the EXtreme Events: Artificial Intelligence for Detection and Attribution (XAIDA) project (https://xaida.eu/) is related to developing novel approaches for the detection and localization of extreme events, such as tropical cyclones and severe convective storms, heat waves and droughts, as well as persistent winter extremes, among others. Here we introduce the XAIDA4Detection toolbox that allows for tackling generic problems of detection and characterization. The open-source toolbox integrates a set of advanced ML models, ranging in complexity, assumptions, and sophistication, and yields spatio-temporal explicit detection maps with probabilistic heatmap estimates. We included supervised and unsupervised methods, deterministic and probabilistic, neural networks based on convolutional and recurrent nets, and density-based methods. The toolbox is intended for scientists, engineers, and students with basic knowledge of extreme events, outlier detection techniques, and Deep Learning (DL), as well as Python programming with basic packages (Numpy, Scikit-learn, Matplotlib) and DL packages (PyTorch, PyTorch Lightning). This presentation will summarize the available features and their potential to be adapted to multiple extreme event problems and use cases.
XAIDA4Detection: A Toolbox for the Detection and Characterization of Spatio-Temporal Extreme Events
Cortés-Andrés, Jordi, Gonzalez-Calabuig, Maria, Zhang, Mengxue, Williams, Tristan, Fernández-Torres, Miguel-Ángel, Pellicer-Valero, Oscar J., Camps-Valls, Gustau
In EGU General Assembly Conference Abstracts. EGU-4816 2023
The automatic anticipation and detection of extreme events constitute a major challenge in the current context of climate change, which has changed their likelihood and intensity. One of the main objectives within the EXtreme Events: Artificial Intelligence for Detection and Attribution (XAIDA) project (https://xaida.eu/) is related to developing novel approaches for the detection and localization of extreme events, such as tropical cyclones and severe convective storms, heat waves and droughts, as well as persistent winter extremes, among others. Here we introduce the XAIDA4Detection toolbox that allows for tackling generic problems of detection and characterization. The open-source toolbox integrates a set of advanced ML models, ranging in complexity, assumptions, and sophistication, and yields spatio-temporal explicit detection maps with probabilistic heatmap estimates. We included supervised and unsupervised methods, deterministic and probabilistic, neural networks based on convolutional and recurrent nets, and density-based methods. The toolbox is intended for scientists, engineers, and students with basic knowledge of extreme events, outlier detection techniques, and Deep Learning (DL), as well as Python programming with basic packages (Numpy, Scikit-learn, Matplotlib) and DL packages (PyTorch, PyTorch Lightning). This presentation will summarize the available features and their potential to be adapted to multiple extreme event problems and use cases.
2022
Bag-of-features-driven spectral-spatial siamese neural network for hyperspectral image classification
Zhaohui Xue, Tianzhi Zhu, Yiyang Zhou, Mengxue Zhang
IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 2022
Deep learning (DL) exhibits commendable performance in hyperspectral image (HSI) classification because of its powerful feature expression ability. Siamese neural network further improves the performance of DL models by learning similarities within-class and differences between-class from sample pairs. However, there are still some limitations in siamese neural network. On the one hand, siamese neural network usually needs a large number of negative pair samples in the training process, leading to computing overhead. On the other hand, current models may lack interpretability because of complex network structure. To overcome the above limitations, we propose a spectral-spatial siamese neural network with bag-of-features (S3BoF) for HSI classification. First, we use a siamese neural network with 3-D and 2-D convolutions to extract the spectral-spatial features. Second, we introduce stop-gradient operation and prediction head structure to make the siamese neural network work without negative pair samples, thus reducing the computational burden. Third, a bag-of-features (BoF) learning module is introduced to enhance the model interpretability and feature representation. Finally, a symmetric loss and a cross entropy loss are respectively used for contrastive learning and classification. Experiments results on four common hyperspectral datasets indicated that S3BoF performs better than the other traditional and state-of-the-art deep learning HSI classification methods in terms of classification accuracy and generalization performance, with improvements in terms of OA around 1.40%–30.01%, 0.27%–8.65%, 0.37%–6.27%, 0.22%–6.64% for Indian Pines, University of Pavia, Salinas, and Yellow River Delta datasets, respectively, under 5% labeled samples per class.
Bag-of-features-driven spectral-spatial siamese neural network for hyperspectral image classification
Zhaohui Xue, Tianzhi Zhu, Yiyang Zhou, Mengxue Zhang
IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 2022
Deep learning (DL) exhibits commendable performance in hyperspectral image (HSI) classification because of its powerful feature expression ability. Siamese neural network further improves the performance of DL models by learning similarities within-class and differences between-class from sample pairs. However, there are still some limitations in siamese neural network. On the one hand, siamese neural network usually needs a large number of negative pair samples in the training process, leading to computing overhead. On the other hand, current models may lack interpretability because of complex network structure. To overcome the above limitations, we propose a spectral-spatial siamese neural network with bag-of-features (S3BoF) for HSI classification. First, we use a siamese neural network with 3-D and 2-D convolutions to extract the spectral-spatial features. Second, we introduce stop-gradient operation and prediction head structure to make the siamese neural network work without negative pair samples, thus reducing the computational burden. Third, a bag-of-features (BoF) learning module is introduced to enhance the model interpretability and feature representation. Finally, a symmetric loss and a cross entropy loss are respectively used for contrastive learning and classification. Experiments results on four common hyperspectral datasets indicated that S3BoF performs better than the other traditional and state-of-the-art deep learning HSI classification methods in terms of classification accuracy and generalization performance, with improvements in terms of OA around 1.40%–30.01%, 0.27%–8.65%, 0.37%–6.27%, 0.22%–6.64% for Indian Pines, University of Pavia, Salinas, and Yellow River Delta datasets, respectively, under 5% labeled samples per class.
2021
Attention-based second-order pooling network for hyperspectral image classificationAttention-based second-order pooling network for hyperspectral image classification
Xue, Zhaohui, Mengxue Zhang, Yifeng Liu, Peijun Du
IEEE Transactions on Geoscience and Remote Sensing 2021
Deep learning (DL) has exhibited huge potentials for hyperspectral image (HSI) classification due to its powerful nonlinear modeling and end-to-end optimization characteristics. Although the superior performance of DL-based methods has been witnessed, some limitations can still be found. On the one hand, existing DL frameworks usually resorted to first-order statistical features, whereas they rarely considered second-order or higher order statistical features. On the other hand, the optimization of complex hyperparameters (e.g., the layer number and convolutional kernel size) is time-consuming and a very tough task, making the designed DL framework unexplainable. To overcome these challenges, we propose a novel attention-based second-order pooling network (A-SPN). First, a first-order feature operator is designed to model the spectral–spatial information of HSI. Second, an attention-based second-order pooling (A-SOP) operator is designed to model discriminative and representative features. Finally, a fully connected layer with softmax loss is used for classification. The proposed framework can obtain second-order statistical features in an end-to-end manner. In addition, A-SPN is free of complex hyperparameters tuning, making it more explainable and easily equipped for classification tasks. Experimental results based on three common hyperspectral data sets demonstrate that A-SPN outperforms other traditional and state-of-the-art DL-based HSI classification methods in terms of generalization performance with limited training samples, classification accuracy, convergence rate, and computational complexity.
Attention-based second-order pooling network for hyperspectral image classificationAttention-based second-order pooling network for hyperspectral image classification
Xue, Zhaohui, Mengxue Zhang, Yifeng Liu, Peijun Du
IEEE Transactions on Geoscience and Remote Sensing 2021
Deep learning (DL) has exhibited huge potentials for hyperspectral image (HSI) classification due to its powerful nonlinear modeling and end-to-end optimization characteristics. Although the superior performance of DL-based methods has been witnessed, some limitations can still be found. On the one hand, existing DL frameworks usually resorted to first-order statistical features, whereas they rarely considered second-order or higher order statistical features. On the other hand, the optimization of complex hyperparameters (e.g., the layer number and convolutional kernel size) is time-consuming and a very tough task, making the designed DL framework unexplainable. To overcome these challenges, we propose a novel attention-based second-order pooling network (A-SPN). First, a first-order feature operator is designed to model the spectral–spatial information of HSI. Second, an attention-based second-order pooling (A-SOP) operator is designed to model discriminative and representative features. Finally, a fully connected layer with softmax loss is used for classification. The proposed framework can obtain second-order statistical features in an end-to-end manner. In addition, A-SPN is free of complex hyperparameters tuning, making it more explainable and easily equipped for classification tasks. Experimental results based on three common hyperspectral data sets demonstrate that A-SPN outperforms other traditional and state-of-the-art DL-based HSI classification methods in terms of generalization performance with limited training samples, classification accuracy, convergence rate, and computational complexity.