10:30
S7: Exploring Explainable AI, Predictive Models, and Federated Learning for CBM in Aerospace at Amphi Fournel (AF)
Chair: Bernardete Ribeiro and Derk Daverschot
10:30
20 mins
|
Deep Learning-Based Explanations for Remaining Useful Life Prediction of Aircraft Turbofan Engines
Fatemeh Hosseinpour, Asteris Apostolidis, Konstantinos P. Stamoulis
Abstract: In commercial aviation, maintenance costs comprise about 11% of an airline’s expenditures [1], while up to 30% of these costs are related to engine overhaul [2]. Predictive maintenance is a strategy which aims to anticipate future maintenance needs for aircraft components and predict their Remaining Useful Life (RUL). However, the complex, non-linear nature of the time series data that represent the performance parameters of gas turbine engines impose numerous challenges for many traditional machine learning models, as predictions may be compromised.
This work aims to employ Deep Neural Networks (DNN) for the prediction of RUL, with a focus in enhancing model interpretability through explainable artificial intelligence (XAI) techniques [3]. Two DNN methods, Convolutional Neural Networks (CNN) [4] and Long Short-Term Memory (LSTM) [5] have been considered. In addition, different post-processing methods such as Shapley additive explanation (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME) [3] are adopted to determine variables that significantly affect the prediction results.
As a case study, three datasets are utilized, which are part of the Commercial Modular Aero-Propulsion System Simulation (C-MAPSS) library [6], to represent increasing levels of complexity in turbofan engine degradation. Moreover, the results have been compared with transparent machine learning models, such as Generalized Additive Model (GAM) [7] and symbolic regression (SR) [8]. The validation process demonstrates that the proposed approach outperforms existing methods considering various performance metrics.
Furthermore, the integration of XAI techniques provides detailed insights into the decision-making process of the deep learning models, enhancing their interpretability. Future work will be devoted to using the obtained deep learning results in conjunction with transparent models, aiming to further enhance predictive accuracy and interpretability.
KEYWORDS: Black-box model, Explainable AI, Explanations post-processing, Remaining Useful life.
REFERENCES
[1] W. J. C. Verhagen et al., “Condition-Based Maintenance in Aviation: Challenges and Opportunities,” Aerospace, vol. 10, no. 9, 2023, doi: 10.3390/aerospace10090762.
[2] M. Dixon, “The Maintenance Costs of Aging Aircraft. Insights from Commercial Aviation,” RAND Corporation, MG-486-AF, 2006
[3] A. Barredo Arrieta et al., “Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI,” Information Fusion, vol. 58, 2020, doi: 10.1016/j.inffus.2019.12.012.
[4] G. S. Babu, P. Zhao, and X. L. Li, “Deep convolutional neural network-based regression approach for estimation of remaining useful life,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2016. doi: 10.1007/978-3-319-32025-0_14.
[5] S. Zheng, K. Ristovski, A. Farahat, and C. Gupta, “Long Short-Term Memory Network for Remaining Useful Life estimation,” in 2017 IEEE International Conference on Prognostics and Health Management, ICPHM 2017, 2017. doi: 10.1109/ICPHM.2017.7998311.
[6] M. A. Chao, C. Kulkarni, K. Goebel, and O. Fink, “Aircraft engine run-to-failure dataset under real flight conditions for prognostics and diagnostics,” Data (Basel), vol. 6, no. 1, 2021, doi: 10.3390/data6010005.
[7] A. Apostolidis, N. Bouriquet, and K. P. Stamoulis, “AI-Based Exhaust Gas Temperature Prediction for Trustworthy Safety-Critical Applications,” Aerospace, vol. 9, no. 11, 2022, doi: 10.3390/aerospace9110722.
[8] M. Kefalas, J. de S. Rojo, A. Apostolidis, D. van den Herik, B. van Stein, and T. Bäck, “Explainable Artificial Intelligence for Exhaust Gas Temperature of Turbofan Engines,” Journal of Aerospace Information Systems, vol. 19, no. 6, 2022, doi: 10.2514/1.I011058.
|
10:50
20 mins
|
Functional Time Series Regression in Hybrid Federated Learning
Raul Llasag Rosero, Catarina Silva, Bernardete Ribeiro
Abstract: #Introduction
Collaborative prognosis, a decentralized learning approach from the Prognostics and Health Management (PHM) discipline, has envisaged Federated Learning. This is a promising decentralized machine learning technology capable of guaranteeing the data privacy of collaborative parties by sensing equivalent machines. Thus collaboratively improving the prediction performance of data-driven predictive models, e.g., anomaly detectors and remaining lifetime estimators. Models such as training data, due to sensor heterogeneity, not only differ in sensor space but may also differ in input space.
#Problem
Due to safety policies, Federated Learning has not been extensively studied in the cargo/passenger transport industry, such as aviation or maritime. Estimating vehicle systems' Remaining Useful Life (RUL) requires accurate predictive models to prevent catastrophic accidents while minimizing maintenance costs of safety-critical systems, e.g. jet turbofan units. A recently Federated Learning, FedLabSync, evaluated the RUL estimation of turbofan engines using condition monitoring data from the Turbofan Engine Degradation Simulation Data-set. However, this study used a basic Multilayer Perceptron (MLP), the output of which must be filtered to correct inaccurate estimations.
#Approach
This study aims to use Functional Time Series Regression techniques to predict the RUL of turbofans more accurately in a federated scenario. Particularly, this study uses a Functional Multilayer Perceptron (FMLP) architecture which adds an input functional layer before the classical MLP [1]. Neuron mapping of these neurons considers functions as inputs as follows: sigma(W(X)+b), where "sigma" corresponds to the activation function, "W" to the functional weight and "b" to the bias term. This consideration requires using a set of finite curves as FMLP inputs. Therefore, T-dimensional curves were constructed to have inputs X of shape (T,M), where M refers to the number of sensor signals and T is the length of the input curves.
The success of this approach arises from enabling the weight function of the functional input layer to select the "P" most relevant components by applying Principal Component Analysis (PCA). Those components are automatically selected during the forward propagation step by defining the maximum Fraction Variance Explained (FVE) "Pfve" as an FMLP hyperparameter.
#Scenario
These experiments used condition monitoring data from turbofan units from the Turbofan Engine Degradation Simulation Dataset, simulated under the run-to-failure philosophy. To simulate a federated scenario, run-to-failure data from turbofan units of the FD004 dataset, which comprises 2 failure modes and 6 operating conditions, is divided equally into four parties $j$. Those parties initially trained their models using private data, using 18 of the 21 available sensor signals to simulate a hybrid data partitioning. Then, parties use the same data to participate in the federation, which is managed by the FedLabSync algorithm, whose success arises from propagating a collaborative error that is calculated through the synchronization of similar samples sharing the same label [2].
#Results and discussion
FMLP improves the RUL estimation of MLP in a federated scenario. It was possible after setting the FVE hyperparameter at 0.9 and restricting the parties to missing data from a maximum of 3 sensors. By using 85% and 15% of data for training and evaluation purposes, the RUL estimation error of the 100% of units from the testing dataset is presented in the attached Table. Those results suggest that more parties are learning from others when configuring an FMLP architecture.
|
11:10
20 mins
|
Exploring Uncertainty Visualization in Condition-Based Maintenance Planning: A Playful Simulation Approach
Jorge Ribeiro, Iordanis Tseremoglou, Licínio Roque, Bruno dos Santos
Abstract: This article aims to explore a playful simulation integrating a Machine Learning (ML) scheduling agent in the context of designing a planning tool for condition-based maintenance (CBM) in aircraft maintenance planning (AMP). This simulation environment facilitates the study of visualizing uncertainty indicators of prognostic maintenance in estimating the Remaining Useful Life (RUL) of aircraft components within a dynamic planning environment.
Designing for an emerging AI-assistant culture raises socio-technical challenges related to the evolving practices in the field, alongside designing for human appropriation and control (Bødker & Kyng, 2018) over state-of-the-art ML scheduling algorithms. In such a context, traditional ethnographic research methods, which involve studying practices that still need to be created, and conventional requirements elicitation approaches, are not applicable. Playful probing refers to a design research approach that uses creative and interactive methods, such as simulations and games, to explore topics in a playful and experimental manner (Bernhaupt et al., 2007). It can serve as a research approach to address the planning dialectics between developing new practices for the CBM planning approach and designing new tools (Ribeiro & Roque, 2022). This computer simulator serves as an intermediate step in building the maintenance simulation game, enabling us to further apply playful probing and delve deeper into the research of this context (Klabbers, 2006).
A key aspect of the CBM paradigm involves changes in the uncertainty of component life indicators over time, and dynamically visualizing this evolution can be crucial for using and trusting ML algorithm solutions (Ribeiro et al., 2022). Our use case involves simulating the maintenance schedule for a commercial aircraft fleet. Each aircraft has multiple monitored systems, for which prognostics algorithms generate RUL predictions with uncertainty. We consider these RUL predictions as the prognostics-driven tasks. Furthermore, we consider the multiple routine/non-routine tasks included in the MPD (Maintenance Planning Document). Utilizing a deep reinforcement learning approach within a two-stage framework (Tseremoglou & Santos, 2024), our model aims to optimize maintenance scheduling processes by aligning available resources and maintenance opportunities with the different maintenance task requirements of the whole aircraft fleet. This framework has been adapted to a human-in-the-loop approach, allowing human schedulers to decide on recalculations within the algorithm in a simulated environment.
Using a Design Science Research (DSR) approach (Vaishnavi et al., 2009), we applied a playful probing protocol, evaluating results through Participatory Design Workshops (PDW). Participants engaged in the simulation of scheduling maintenance prognostic tasks, generating insights into how to interpret and manage uncertainty in daily maintenance simulations where uncertainty dynamically changes over time. This work provides practical insights into how planners can manage uncertainty using a CBM tool. The simulation offers users the ability to interact with the RL agent, customizing the tasks they wish to recalculate or fix the date. On the other hand, it allows users to contextualize, track, and understand the meaning and reliability of RUL prognostics.
Through an iterative process aimed at developing a CBM simulation game, we created this specialised simulator and utilised it as a research instrument in participatory workshops. We gained insights into visualizing uncertainty in a dynamic planning environment, obtaining a comprehensive view of fleet prognostics, and understanding how to effectively represent and interact with uncertainty within the context of CBM maintenance.
|
11:30
20 mins
|
Bayesian modelling of stochastic effects to the useful life of structures subject to single and multi-imperfect post-repairs
Panagiotis Komninos, Georgios Galanopoulos, Dimitrios Zarouchas
Abstract: In recent years, there has been rapid development in health monitoring systems aimed at enhancing maintenance decisions in industrial operations, driven by the increasing number and variety of sensors on structures, leading to more precise diagnosis and prognosis. This evolution aligns with the fundamentals of Condition-based maintenance (CbM), emphasizing proactive maintenance decisions based on data to prevent structural failures. Prognostics and Health Management (PHM) is emerging as a crucial engineering discipline within CbM, involving the analysis of a structure's health condition and making predictions about impending failures. There's a growing shift towards CbM from corrective and preventive maintenance, with CbM contributing to strategic maintenance planning to prevent potential failures in structures. Machine Learning techniques are integral in analyzing sensory data to estimate a structure's current state via Remaining Useful Life (RUL), although RUL predictions inherently carry some level of uncertainty due to the stochastic nature of future forecasting.
Within industries implementing CbM, two primary forms of maintenance are prevalent: replacement and repair. The conventional approach typically involves replacing a structure or component with a new one aimed at preempting unforeseen damages or failures. Nonetheless, the exorbitant costs associated with replacements have prompted extensive exploration into the viability of repair methodologies. A common misunderstanding about repairs is expecting them to be perfect, which might result in opting for a replacement. In reality, however, practical repair scenarios rarely achieve the perfect state condition. In other words, the actual RUL often does not reach the expected value that comes under an ideal condition, which suggests that repair processes usually come with imperfections, defined as imperfect repairs.
Existing approaches to handling imperfect maintenance under the CbM concept are to incorporate its stochastic behavior into a predefined deterioration or degradation model. This is achieved by assuming either a Gamma stochastic process of the deterioration after an imperfect maintenance action or the degradation is improved by following a Normal or Gamma distribution. Another family of approaching imperfect maintenance scenarios considers Bayesian inference on a parameterized model where the parameters are defined as random variables can be updated mainly via Markov Chain Monte Carlo (MCMC) algorithms to avoid limitations associated with assuming conjugate prior distributions.
Undoubtedly, modeling imperfect maintenance significantly influences the formulation of optimal decisions. In CbM, choosing decisions directly based on RUL measurements underscores a significant aspect. All the abovementioned works assumed that a deteriorating system follows a predefined stochastic process with a stationary transition function from one state to another and the potential recovery of the system after an imperfect repair is based on a single distribution, such as a Normal distribution. However, it has been noted that since a time-dependent or non-stationary deterioration process is usually observed, the above methodologies might not accurately capture the recovery of the structure after an imperfect repair. The temporal variability emerges when alterations in the deterioration rate are observed, leading to potentially multiple transitions from a single state. To overcome this limitation, it becomes crucial to incorporate characteristics associated with RUL. This process effectively transforms the intricate non-linear dynamics of deterioration into singular (stochastic) data points able to construct simplified trajectories.
In this regard, this work aims to model imperfect repairs by employing the stochastic nature of RUL without affecting the prognostics phase. Thereby, the modeling of imperfect repairs naturally follows the establishment of an independent prognostic model. In such a setup, the modeling of imperfect repairs serves as the connection between the RUL estimation and decision-making, both integral components of PHM. This integration aligns with the common industry practice, where RUL models may already exist from prior models trained using existing datasets, as well as aiming to be compatible with the existing decision-making algorithms. Our focus lies on structures characterized by limited available data, which further compounds the complexity of the problem at hand.
Our proposed model utilizes the acquired data before and after the imperfect maintenance action to estimate the distribution of the structure’s recovery after single and multiple imperfect repairs. Since both RUL and the effects of imperfect maintenance actions are stochastic and a few samples of data are available, Bayesian inference is chosen as the ultimate solution. In Bayesian inference, the choice of specific prior and likelihood distributions is unavoidable. In practical approaches, to receive analytical solutions, conjugate priors are chosen. Possessing a generalizable model implies the availability of various prior-likelihood distributions that align with domain-specific knowledge. This necessitates the selection of a more sophisticated Bayesian inference technique, the MCMC approach, to accommodate these diverse choices effectively.
The effectiveness of the single and multi-repair models is assessed through evaluation in a real-case scenario involving an experimental campaign with tension-tension fatigue experiments on open-hole 7075-T6 aerospace-grade aluminum coupons. The purpose of the tests is to induce and monitor the fatigue crack growth as well as repair the damage, restore some of the parent material's properties, and evaluate the extent of the fatigue life. Each repair is performed via a rectangular CFRP patch, secondarily bonded using an Araldite 2015-1 two-part epoxy adhesive, at a predetermined percentage of the fatigue life. Results show that we were able to accurately estimate the distribution of recovery with both single and multi-imperfect repairs, even without having to enforce the structure to reach failure, thus ensuring reliability and safety.
|
11:50
20 mins
|
Vector Quantized Anomaly Detection applied to Condition Based Maintenance
Miguel Fernandes, Catarina Silva, Alberto Cardoso, Bernardete Ribeiro
Abstract: Condition-Based Maintenance (CBM) is paramount in sectors like aerospace and manufacturing, where the timely identification of potential system failures is of the most importance to operational continuity and safety. The prediction of the Remaining Useful Life (RUL) of system components underpins not only effective operational schedules but also proactive maintenance strategies. Traditionally, maintenance regimes have favoured a preventive approach, scheduling service activities at predetermined, often conservative, intervals. This approach may lead to unnecessary maintenance, high financial costs, and increased operational costs. By contrast, CBM leverages from historical sensor data to craft predictive models that accurately predict when maintenance should be performed, thus aligning interventions with the actual condition of the system. This alignment of maintenance activities with real-time component status avoids the inefficient conventional methods and paves the way for more cost-effective operations.
In CBM, there are two main types of strategies: physics-based models and data-driven approaches. Physics-based models, known for their precision, harness intricate insights derived from a system's physical properties and are most effective when meticulously calibrated. However, their practical application is often encumbered by the necessity for extensive, system-specific knowledge, making them intrinsically customized and limited in flexibility. On the other hand, data-driven approaches offer versatility by utilizing extensive sensor data to track a system's degradation over time. The effectiveness of these models depends on having comprehensive datasets that accurately describe a system’s wear and tear, which limits their usefulness when such data are not available.
When it comes to techniques for predicting the RUL of system components, current research primarily utilizes advanced neural networks. These methods effectively process raw and minimally processed sensor data to anticipate system degradation. Despite their potential, they sometimes struggle to model the nonlinear interactions within the data that are characteristic of complex degradation processes, leading to challenges in model generalizability and potential performance discrepancies across varied operational scenarios. In pursuit of more robust solutions, recent research has gravitated towards the application of Autoencoders, which adeptly transform input data into a more abstract, latent representation. This transformation is pivotal, as it unravels the intricate patterns embedded in the data that are vital for developing accurate RUL estimates. By transforming this complexity into Health Indicators, Autoencoders become a key component in predictive maintenance. Our research contributes to this evolving field by employing Autoencoders to refine input data into a latent form, extracting essential features and patterns that are crucial for reliable RUL prediction.
In this work, we employ a Transformer-based Vector Quantized Variational Autoencoder (T-VQ-VAE) [5] to integrate latent data into a Markov Chain (MC) model, enhancing our capacity for anomaly detection. The T-VQ-VAE generates a sequence of discrete latent vectors that serve as the building blocks for our MCs. Prior research has explored the utilization of MCs for posterior estimation methods, analysing data similarity to infer a system's RUL. We aim to extend this by examining the application of healthy state matrices, P_h, alongside matrices that indicate system degradation, P_hf. P_h relates to sequences of latent vectors that reflect the system's normal operation, derived from healthy system data via the T-VQ-VAE. In contrast, P_hf is indicative of data beginning to deviate from this established norm, marking a transition towards potential failure. An anomaly is flagged when the discrepancy between the transition matrices, P_h and P_hf, exceeds a specific threshold. This divergence indicates the system's departure from its established normal operational conditions, signalling a possible onset of failure.
The validation of our methodology on the C-MAPSS dataset indicates its potential as an alternative to current state-of-the-art methods. By employing both root mean squared error (RMSE) and a Score function defined by the International Conference on Prognostics and Health Management, our approach has demonstrated a unique ability to capture an engineering system's health and degradation. While our model showcases a different angle of predictive accuracy, the key advancement lies in its novel utilization of latent data structures and their probabilistic modelling through MCs. Our work suggests that considering these alternative approaches can expand the capabilities of predictive maintenance, offering insights that could lead to enhanced reliability and efficiency. The findings from the C-MAPSS dataset serve as a preliminary yet promising indication that this novel method holds interest for further investigation and could inform future advances in CBM strategies.
|
12:10
20 mins
|
NN and KF hybrid methods for estimating health parameters of aircraft engine in underdetermined and non-linear systems
Solène Thepaut, Dong Quan Vu, Sébastien Razakarivony
Abstract: To perform Condition Based Maintenance on physical systems may require solving an inverse problem. Indeed, physical models give measurements (such as temperature or pressure) given the state of health of a system (for example efficiencies of modules) while we need to obtain the state of health knowing such measurements.
Kalman Filter is a state of the art method to solve such inverse problems in a dynamic way, and proved to be very efficient in industry challenges such as aircraft engine (Simon 2009, Feng 2018, Lu 2018) or lithium battery health parameters estimation from noisy measurements (Seongjun Lee et al 2008). It performs well in linear cases but faces challenges when the relation between health parameters and measurement is not linear, or when considering an underdetermined case. To overcome the difficulties inherent to non-linear cases, underdetermined or peculiar systems, alternative algorithms and improvements such as, but not limited to, EKF, UKF (Julier & Ulhmann 1997) and AKF (Guo et al. 2014, Huang et al. 2017) variations rose in the literature with outstanding results.
In the meantime, neural network has gained in popularity in the last years. By considering non-linear activation function, NN models produced impressive results when fitting non-linear models (Sharma et al. 2017). In the aircraft industry, neural network models are very efficient for the inversion of performance models and can compete with Kalman Filters (Volponi 2003). In any case, these two approaches have their pros & cons.
On these premises, we first implemented our own Kalman Filters methods such as EKF and SR-UKF. Then we took interest in building a neural network algorithm aiming to estimate health parameters from measurements with promising results. We used a SAFRAN owned simulator to produce the dataset for training the model. The paper will be presented at IEEE CAI 2024.
Leveraging these two approaches, we intend to mix both worlds. Based on the survey of (Feng, et al, 2023), many hybrid approaches with neural network and Kalman filters exist and have proven to be effective on a wide range of applications from robotics to wind prediction. Different types of hybrids models can be found in the literature such as succession models (KF then NN or NN then KF) or, less often, using neural networks to manage uncertainty in the system of a Kalman filter model.
In our work, we aimed to improve our estimation obtained from our neural network model and Kalman filters methods. We will present two hybrid methods that combine neural networks and kalman filters adapted to inversion problem for estimating health parameters in aircraft engines. In a first method, we predict health parameters based on observed measurements using our neural network model. The outputs of the neural network can be considered as noisy estimation of the true health parameters and become inputs for the Kalman filter along with the measurements. In our underdetermined set up, the addition of inputs in our Kalman Filter is an opportunity to reduce or eliminate non-observability in our system. A second approach is to use neural network neural to compute the optimal value for Q and R noise matrices along time in an adaptative kalman filter fashion. Indeed, calibrating Q and R can be challenging for Kalman Filters, and are of crucial importance in the performances of the method.
Estimating health parameters from noisy measurements is still a challenge of major importance in the aircraft industry and in need of improvement. Hybrid methods are a promising research focus for improving the performance of our already implemented solutions. Such algorithms help decreasing the non-observability of the system through the addition of information in the inputs and/or adapting the parameters of our models to overcome non-linearity or convergence issues.
|
|