Interpretable Solar Flare Prediction with Sliding Window Multivariate Time Series Forests
Over the past few decades, the synergy of physics-based feature engineering and data-intensive methods, including machine learning and deep learning, has ushered in a new era in the analysis and prediction of space weather forecasting, specifically for solar flare prediction. These sophisticated approaches play a pivotal role in understanding the complex mechanisms leading to solar flares, with a primary focus on forecasting these events and mitigating potential risks they pose to our planet. While current methodologies have made substantial advancements, they are not without limitations, and one particularly glaring limitation is the neglect of temporal evolution characteristics within the active regions from which solar flares originate. This oversight impairs the capacity of these methods to capture the intricate relationships among high-dimensional features of these active regions, thereby constraining their practical utility. Our study focuses on two key objectives: the development of interpretable classifiers for multivariate time series data and the introduction of an innovative feature ranking method using sliding window-based sub-interval ranking.
The central contribution of our work lies in bridging the gap between complex, less interpretable "black-box" models typically employed for high-dimensional data and the exploration of pertinent sub-intervals within multivariate time series data, with a specific emphasis on solar flare forecasting. Our findings underscore the efficacy of our sliding-window time series forest classifier in solar flare prediction, achieving a True Skill Statistic of over 85%. Our approach is capable of pinpointing the most critical features and sub-intervals relevant to any given learning task. These results indicate a significant progress towards improving the interpretability and accuracy of flare prediction models, further advancing our understanding of these impactful events.
A Solar Flare Forecasting with Deep Learning-based Time Series Classifiers
Over the past two decades, machine learning and deep learning techniques for forecasting solar flares have generated great impact due to their ability to learn from a high dimensional data space. However, lack of high quality data from flaring phenomena becomes a constraining factor for such tasks. One of the methods to tackle this complex problem is utilizing trained classifiers with multivariate time series of magnetic field parameters. In this work, we compare the exceedingly popular multivariate time series classifiers applying deep learning techniques with commonly used machine learning classifiers (i.e., SVM). We intend to explore the role of data augmentation on time series oriented flare prediction techniques, specifically the deep learning-based ones.
We utilize four time series data augmentation techniques and couple them with selected multivariate time series classifiers to understand how each of them affects the outcome. In the end, we show that the deep learning algorithms as well as augmentation techniques improve our classifiers performance. The resulting classifiers’ performance after augmentation outplayed the traditional flare forecasting techniques.
See the publication here
A Systematic Magnetic Polarity Inversion Line Data Set from SDO/HMI Magnetograms
Magnetic polarity inversion lines (PILs) detected in solar active regions have long been recognized as arguably the most essential feature for triggering instabilities such as flares and eruptive events (i.e., eruptive flares and coronal mass ejections). In recent years, efforts have been focused on using features engineered from PILs for solar eruption prediction. However, PIL rasters and metadata are often generated as by-products and are not accessible for public use, which limits their utilization in data-intensive space weather analytics applications. We introduce a large-scale publicly available PIL data set covering practically the entire solar cycle 24 for applying to various space weather forecasting and analytics tasks. The data set is created using both radial magnetic field (B_r) and line-of-sight (B_LoS) magnetograms from the Solar Dynamics Observatory's Helioseismic and Magnetic Imager Active Region Patches (HARP) that involve 4090 HARP series ranging from 2010 May to 2019 March. This data set includes three PIL-related binary masks of rasters: the actual PILs as per the spatial analysis of the magnetograms, the region of polarity inversion, and the convex hull of PILs, along with time-series-structured metadata extracted from these masks. We also provide a preliminary exploratory analysis of selected features aiming to correlate time series of feature metadata and eruptive activity originating from active regions. We envision that this comprehensive PIL data set will complement existing data sets used for space weather forecasting and benefit research in related areas, specifically in better understanding the PIL structure, evolution, and role in eruptions.
See the publication here
A Modular Approach to Building Solar Energetic Particle Event Forecasting Systems
Solar energetic particle (SEP) events, as one of the most dangerous manifestations of solar activity, can generate severe hazardous radiation when accelerated by solar flares or shock waves formed aside coronal mass ejections (CMEs). Unlike common predictions that focus on the occurrence of an event, an All-Clear forecast puts more emphasis on predicting the absence of an event. Such forecasts, while usually not addressed directly, can be crucial in operational environments. We have developed an All-Clear SEP event prediction system utilizing active region-based prediction methods together with active region scenarios (i.e., location and complexity). Within our All-Clear forecast system, signals are generated only when requested as binary predictions of YES or NO indicating “All Clear” or “Not All Clear”, respectively.
Such signals referred to the potential possibility of the occurrence of any events in the next prediction window, in our cases, the next 24 hours. Four major space weather event forecasting modules are established corresponding to the flare prediction (FP), eruptive flare prediction (ERP), CME speed prediction, and full-disk aggregation methodology, where all of them are loosely coupled without direct communications between each other. Our system design follows a modular approach for flexibility, maintainability, and extensibility that can be configured to utilize file storage or any data access mechanisms, such as file storage or database systems, outside the confines of our system.
See the publication here
All-clear Flare Prediction Using Interval-based Time Series Classifiers
An all-clear flare prediction is a type of solar flare forecasting that puts more emphasis on predicting non-flaring instances (often relatively small flares and flare quiet regions) with high precision while still maintaining valuable predictive results. While many flare prediction studies do not address this problem directly, all-clear predictions can be useful in operational context. However, in all-clear predictions, finding the right balance between avoiding false negatives (misses) and reducing the false positives (false alarms) is often challenging. Our study focuses on training and testing a set of interval-based time series named Time Series Forest (TSF). These classifiers will be used towards building an all-clear flare prediction system by utilizing multivariate time series data. Throughout this paper, we demonstrate our data collection, predictive model building and evaluation processes, and compare our time series classification models with baselines using our benchmark datasets.
Our results show that time series classifiers provide better forecasting results in terms of skill scores, precision and recall metrics, and they can be further improved for more precise all-clear forecasts by tuning model hyperparameters.
See the publication here
Explainable Deep Learning Models for Solar Flare Prediction with Attribution and Post Hoc Attention
We present a deep learning-based full-disk model for solar flare prediction, emphasizing near-limb flares with model explanations for the predictions. The model utilizes hourly full-disk line-of-sight magnetogram images and employs a binary prediction mode for forecasting ≥M-class flares within 24 hours. The predictions are interpreted using attribution methods such as Guided Gradient-weighted Class Activation Mapping, Integrated Gradients, and Deep Shapley Additive Explanations. The analysis reveals that the model predictions of solar flares are linked to active regions (ARs). In particular it is observed that the model identifies and utilizes AR features in central and in near-limb locations from full-disk magnetograms, demonstrating novel operational forecasting capabilities. We also conduct a post hoc analysis of a deep learning-based full-disk solar flare prediction model as a case study. We used hourly full-disk line-of-sight magnetogram images and selected binary prediction mode to predict the occurrence of ≥M1.0-class flares within 24 hours. We interpret our model using three post hoc attention methods: (i) Guided Gradient-weighted Class Activation Mapping, (ii) Deep Shapley Additive Explanations, and (iii) Integrated Gradients. Our analysis shows that full-disk predictions of solar flares align with characteristics related to the active regions.
The key findings of this study are: (1) We demonstrate that our full disk model can tangibly locate and predict near-limb solar flares, which is a critical feature for operational flare forecasting, (2) Our candidate model achieves an average TSS=0.51±0.05 and HSS=0.38±0.08, and (3) Our evaluation suggests that these models can learn conspicuous features corresponding to active regions from full-disk magnetograms.
See the publication here
Towards Coupling Full-disk and Active Region-based Flare Prediction for Operational Space Weather Forecasting
we present a heterogeneous ensemble model that combines a full-disk and an active region-based model for solar flare prediction utilizing a set of new heuristic approaches to train and deploy an operational system for ≥M1.0-class flares. In full-disk mode, predictions are performed on full-disk line-of-sight magnetograms using deep learning models whereas in active region-based models, predictions are issued for each active region individually using multivariate time series data instances. The outputs from individual active region forecasts and full-disk predictors are combined to a final full-disk prediction result with a meta-model. We utilized an equal weighted average ensemble of two base learners’ flare probabilities as our baseline meta learner and improved the capabilities of our two base learners by training a logistic regression model.
An illustration of our ensemble flare prediction pipeline showing two base learners (AR-based FP) and (Full-disk FP) and the ensemble (Meta-FP) followed by full-disk aggregation of AR-based FP’s flare probabilities.
Our result analysis suggests that the logistic regression-based ensemble (Meta-FP) improves on the full-disk model (base learner) by 9% in terms TSS and 10% in terms of HSS. Similarly, it improves on the AR-based model ∼ (base learner) by 17% and 20% in terms of TSS and HSS respectively. Finally, when ∼compared to the baseline meta model, it improves on TSS by 10% and HSS by 15%.
See the publication here
Solar Flare Forecasting with Deep Neural Networks using Full-disk Magnetograms
we present a deep-learning based model for full-disk solar flare prediction using compressed magnetogram images to perform operations-ready flare forecasts. We selected two prediction modes, which are both binary for predicting the occurrence of ≥M1.0 and ≥C1.0 class flares within the next 24 hours. We followed two time- segmented cross-validation strategies: chronological and non-chronological, to effectively understand the predictive skill of our models. Our candidate model achieves an average TSS of 0.47±0.06 for ≥M1.0 mode and 0.63±0.05 for ≥C1.0 mode, and HSS of 0.35±0.05 for ≥M1.0 and 0.62±0.05 for ≥C1.0 mode. Our experimental evaluation suggests that training a flare prediction model is heavily influenced by the sampling strategies involved due to the imbalanced nature of the datasets and predicting ≥M1.0 class flares is a more challenging task compared to ≥C1.0 ones.
This work progresses the growing body of research on full-disk solar flare prediction using deep learning approaches. We utilized three well-known pretrained CNN models—AlexNet, VGG16 and ResNet34 and train our models in three prediction modes, among which two are binary for predicting the occurrence of ≥M1.0 and ≥C4.0 class flares and one is a multi-class mode for predicting the occurrence of <C4.0, [≥C4.0, <M1.0] and ≥M1.0 within the next 24 hours. Our candidate model for multi-class flare prediction achieves an average TSS of 0.36 and average HSS of 0.31. Similarly, for binary prediction in (i)≥C4.0 mode: we achieve an average TSS score of 0.47 and HSS score of 0.46 (ii)≥M1.0 mode: we achieve an average TSS score of 0.55 and HSS score of 0.43.
4-fold aggregated confusion matrices for multi-class predictions where NF, MF, SF indicates no-flare, mild-flare, and strong-flare respectively for three models (a) ResNet34 (b) VGG16 (c) AlexNet
A pictorial representation of the line-of-sight magnetogram as observed by SDO/HMI. The colors show the magnetic field strength in Gauss.
See the publication here
Data-driven Labels for Solar Flare Prediction
We introduce new data-driven labels for solar flare prediction. Current prediction tools heavily rely on the GOES classification system, using the highest X-ray flux measurement in the prediction window to label instances. However, this approach can lead to false alarms, particularly during periods of solar minimum. To overcome the limitations of the GOES class, we first define the relative X-ray flux increase, which refers to the ratio between the peak X-ray flux and calibrated background X-ray flux.
In the process of creating instances from time series data using a sliding window, we derive a set of new labels:
Maximum relative X-ray flux increase (rxfi Max)
Cumulative weighted GOES subclass values (GC )
Cumulative rxfi values (rxfi )
For rxfi Max, the maximum relative X-ray flux increase in the prediction window is used as a label. For cumulative indices, the sum of the weighted GOES subclass values or rxfi values are used as the index. We also implement a case study on an active region-based model. The Time Series Forest classifier (TSF) is trained with these three new labels. Our results indicate that new labels provide comparable skill scores to well-known techniques and thus can be considered as a new approach in solar flare forecasting.
Magnetic Polarity Inversion Lines-API
We have developed a RESTful API to more easily search and access information about various Magnetic Polarity Inversion Line datasets. Researchers can retrieve rasters and relevant metadata from 2010 to 2019 by searching for a specific ID, a time range, or a spatiotemporal range. The API is designed to be extensible and to incorporate other MPIL datasets, identification formats, and raster data.
While useful as a standalone service, this project was created as a support for future analysis and prediction operation systems for central space weather events. It will also provide a resource for future machine learning training and applications.
An Extended Noise-Aware Method for Detecting Magnetic Polarity Inversion Lines
We introduce a powerful software framework designed to detect solar Magnetic Polarity Inversion Lines (PILs), critical boundaries that separate regions with opposing magnetic polarity. PILs are pivotal in triggering solar instabilities like flares and eruptive events. Recognizing the computational complexity of this task, we've developed an innovative toolkit optimized for GPU use, enabling researchers to create multi-resolution PILs efficiently. Alongside this toolkit, we provide an open-source GPU-accelerated detection toolkit and an extensive publicly accessible PIL dataset, covering the entirety of Solar Cycle 24. This dataset serves as a valuable resource for various space weather forecasting and analytics endeavors, constructed using line-of-sight magnetograms from the Solar Dynamics Observatory's (SDO) Helioseismic and Magnetic Imager (HMI) Active Region Patches (HARPs). Our PIL detection approach hinges on a well-defined methodology involving a size and magnetic field strength threshold filter. Initially, we identify positive and negative polarity regions using a magnetic field strength threshold. Subsequently, we apply a size filter to both positive and negative regions, effectively eliminating small areas considered as noise. Further refining the process, we employ morphological operations on both positive and negative regions to identify coarse PILs and seamlessly connect sufficiently close ones by filling the gaps between them. Ultimately, we generate PILs by applying magnetic field strength and PIL size filters to the coarse PILs.
The example raster data products generated from PIL detection tool from an active region patch (shown in (a) HARP 377 (NOAA AR 11158) at 2011-02-13 16:00:00 UT): (b) Positive polarity region, (c) Negative polarity region, (d) Union of positive and negative polarity regions (colored for illustration -- white areas for positive and black for negative regions), (e) regions of polarity inversion, (f) refined polarity inversion lines (PILs), (g) convex hull of PILs, (h) PILs overlaid on original magnetogram raster
The dataset encompasses six PIL-related binary masks, including actual PILs, Regions of Polarity Inversion (RoPI), Positive Polarity Regions, Negative Polarity Regions, Unsigned Polarity Region (both positive and negative), and the convex hull of PILs. Additionally, the dataset includes structured metadata extracted from these masks in the form of multivariate time series. Our overarching goal is to enhance space weather forecasting by providing this comprehensive PIL dataset and advancing research in the field, particularly in understanding PIL structure, evolution, and their role in solar eruptions.
Space Weather Analytics Dataset
We have built a large-scale dataset of the solar active region was specifically for space weather analytics for Solar Flares (SWAN-SF). The dataset contains metadata from over 4,000 trajectories of the solar active region patches, which are integrated with carefully curated solar flare data over the best part of one complete solar cycle, almost as long as SDO/HMI commissioning time.
This dataset is machine learning ready. It allows different machine learning tools and techniques to uniformly use specific sections of the dataset for training, testing, and validation, while accurately comparing their performance to determine the most promising ones
See the publication here
Overall workflow of the local trajectory outlier detection method.
Outlier detection has become one of the core tasks in spatio-temporal data mining. It plays an essential role in data quality improvement for the machine learning models and recognizing the anomalous patterns, which may remarkably deviate from expected patterns among the trajectory datasets. In this work, we propose a clustering-based technique to detect local outliers in trajectory datasets by utilizing spatial and temporal attributes of moving objects.
See the publication here
Local Outlier Detection
for Spatiotemporal Trajectories
This local outlier detection involves three phases. In the first phase, we apply a temporal partition procedure to divide the raw trajectory into multiple trajectory segments and extract trajectory features from spatial and temporal attributes for each trajectory segment. Then, we generate template features of trajectory segments by applying a clustering schema in the second phase. Finally, we use the abnormal score - a novel dissimilarity measure, which quantifies the disparity among the query and template trajectory segments in terms of trajectory features and hence determines the local outliers based on the distribution of abnormal score. To demonstrate the effectiveness of our method, we conduct three case studies on the real-life spatio-temporal trajectory datasets from the solar astroinformatics domain (i.e., solar active regions, coronal mass ejections, polarity inversion lines (PIL)). Our experimental results show that our local outlier detection approach can effectively discover the erroneous reports from the reporting module and abnormal phenomenon in various spatio-temporal trajectory datasets.
Polarity Inversion Lines Detection
Magnetic polarity inversion line (PIL) in solar active regions have been recognized as essential features for the occurrence of solar flares and the prediction of the flaring phenomenon. In this work, we provide a software framework that detects PILs from the line-of-sight (LoS) or the radial component of the magnetic field vector in active region magnetogram patches. The PIL detection procedure is based on an edge detection technique along with magnetic field strength and PIL size filter. First, we identify positive and negative polarity regions with a magnetic field strength threshold. Then, we utilize the Canny edge detector and morphological operations to both positive and negative regions to identify coarse PILs. Finally, we generate PILs by applying magnetic field strength and PIL size filter to the coarse PILs as mentioned above.
Three binary masks overlay the magnetogram with a normalized magnetogram field strength map. The color bar indicates the intensity of magnetic field strength range from -1500 Gauss to +1500 Gauss.
Moreover, we provide feature extraction functions to obtain the properties of PILs (i.e., PIL size, the area of polarity inversion, the masked unsigned flux of enclosing PIL, convexity, eigenvalues, fractal dimension, and Hu moments of PIL shape), and produce three PIL-related binary masks (i.e., PIL, the region of polarity inversion, and the convex hull of PIL) for each Longitudinal magnetogram patch.
See the publication here
Qualitative evaluation of SubPixel CNN and ResNet models trained using cropped (2k by 2k) magnetograms
We show that through our experimental evaluation our models perform better than baselines and Sub-Pixel CNN super resolution model provides viable results for magnetogram super resolution.
Magnetograms Super-Resolution
Image super-resolution is a branch of image processing that is concerned with enhancing the spatial resolution and quality of images by learning the intrinsic details and relations between the lower resolution input and the higher resolution output images. It is widely accepted as an ill-posed problem, which has seen tremendous advancements with deep learning based models. In this work, we present two magnetogram super resolution models, Sub-Pixel Convolutional Neural Network (CNN) and Enhanced Deep Residual Networks (ResNet), which can be used for improving the spatial resolution of solar magnetograms. While the ill-posed nature of problem is still a challenge, there are several application areas, including space weather prediction, which can greatly benefit from the improved spatial resolution of solar magnetograms.
See the publication here
Rare-Event Time Series Prediction
We present a case study for time series prediction models in extreme class-imbalance problems. We have extracted multiple properties from the Space Weather Analytics for Solar Flares (SWAN-SF) benchmark dataset which comprises of magnetic features from over 4075 active regions over a period of 9 years to create the forecasting dataset used in this study. In the extracted dataset, the class-imbalance ratio is 1:60, where the minority class is formed by instances of strong solar flares (GOES M-and X-class). This ratio reaches 1:800 if we only consider the strongest class of flares (GOES X-class). We have explored remedies to tackle the class-imbalance issue such as undersampling, oversampling, and misclassification weights.
Frequency and imbalance ratio of all five flare classes across different partitions of SWAN-SF benchmark dataset.
In the process, we elaborate on common mistakes and pitfalls caused by ignoring the side effects of these remedies, including how and why they weaken the robustness of the trained models while seemingly improving the performance.
See the publication here