Publicação
Ensemble Learning Methodologies for Soft Sensor Development in Industrial Processes
| Resumo: | Increasing demands for on-line monitoring and control of industrial processes and their associated variables, and the limitations of the available measuring systems have led to the development of predictive models called Soft Sensors (SSs). SSs use computational intelligence methods to estimate difficult-to-measure variables based on some easy-to-measure variables in industrial applications. However, SS development has some difficulties. The performance of the SSs relies on the quality of the data used to extract knowledge during the identification procedure. Other problem is that industrial systems have many complex characteristics (e.g. nonlinearity and time-variance). Thus, bringing SSs to real-world industrial applications is a challenge. This thesis focuses on the development of computational learning methods applied to SSs, with particular emphasis on methodologies for improving the prediction accuracy and the system adaptation, in order to achieve adaptivity and stability in time-varying processes and reduce the maintenance costs. To deal with these issues, this thesis investigates the use of combinations of multiple learning models, a type of structure called ensemble system. These methodologies have demonstrated ability to improve the performance and stability of the systems. However, efficient mechanisms for balancing the diversity, adaptivity, and performance of the models should be investigated and proposed. For this purpose, four main research objectives and research directions are considered. The first objective is to develop methodologies for the automatic design of Neural Network (NN) ensembles in regression problems. Genetic Algorithm (GA) and Simulated Annealing (SA) methodologies are proposed and compared to select the best subset of models (from a set of models) to be aggregated to the ensemble, taking into account the key factors of ensemble systems (i.e. diversity, number of models, and combination strategy). First, a set of models with high degree of diversity is generated. That is, each model is trained with a different training data set by applying bootstrap, and the best NN architecture is selected by varying the number of hidden neurons, the activation function, and the weight initialization. Second, GA and SA are employed to select the best subset of models and the optimal combination. The second objective is to design an adaptive ensemble regression which is able to learn samples in the presence of several types of changes and simultaneously retain old information in scenarios where changes may recur. The key idea is to keep a moving data window that slides when a new sample is available. To handle recurring and non-recurring changes, the proposed ensemble uses a new assignment of models' weights that takes into account the models' errors on the past and current windows using a discounting factor that decreases or increases the contribution of old windows. New models are launched if the accuracy of the system is decreasing, and inaccurate models can be excluded over time. The third objective is to design an adaptive ensemble regression with fast adaptation capability for on-line prediction of variables in time-varying applications. The properties of the proposed ensemble are: on-line inclusion and removal of models to keep only the most accurate models with respect to the current state of the system; dynamic adaptation of the model's weights based on their on-line predictions on the most recent samples; and on-line adaptation of the models' parameters. The fourth objective is to design an on-line ensemble regression that selects dynamically the best subset models (from a set of models) to form the ensemble. The proposed method employs ordered aggregation to choose the ensemble size and the subset of models based on the minimization of the ensemble error on the newest sample. It is also proposed an adaptive NN using a variable forgetting factor. The performance and effectiveness of the proposed methodologies are validated and demonstrated using real-world industrial applications, including the estimation of the free lime in a cement kiln process, and other benchmarks for evaluating real-world SS applications. Additionally, experimental results using artificial data sets with several types of changes are presented to demonstrate the effectiveness and accuracy of the proposed methodologies that deal with time-varying environments. |
|---|---|
| Autores principais: | Soares, Symone Gomes |
| Assunto: | Soft Sensors Ensemble Learning |
| Ano: | 2015 |
| País: | Portugal |
| Tipo de documento: | tese de doutoramento |
| Tipo de acesso: | acesso embargado |
| Instituição associada: | Universidade de Coimbra |
| Idioma: | inglês |
| Origem: | Estudo Geral - Universidade de Coimbra |
| Resumo: | Increasing demands for on-line monitoring and control of industrial processes and their associated variables, and the limitations of the available measuring systems have led to the development of predictive models called Soft Sensors (SSs). SSs use computational intelligence methods to estimate difficult-to-measure variables based on some easy-to-measure variables in industrial applications. However, SS development has some difficulties. The performance of the SSs relies on the quality of the data used to extract knowledge during the identification procedure. Other problem is that industrial systems have many complex characteristics (e.g. nonlinearity and time-variance). Thus, bringing SSs to real-world industrial applications is a challenge. This thesis focuses on the development of computational learning methods applied to SSs, with particular emphasis on methodologies for improving the prediction accuracy and the system adaptation, in order to achieve adaptivity and stability in time-varying processes and reduce the maintenance costs. To deal with these issues, this thesis investigates the use of combinations of multiple learning models, a type of structure called ensemble system. These methodologies have demonstrated ability to improve the performance and stability of the systems. However, efficient mechanisms for balancing the diversity, adaptivity, and performance of the models should be investigated and proposed. For this purpose, four main research objectives and research directions are considered. The first objective is to develop methodologies for the automatic design of Neural Network (NN) ensembles in regression problems. Genetic Algorithm (GA) and Simulated Annealing (SA) methodologies are proposed and compared to select the best subset of models (from a set of models) to be aggregated to the ensemble, taking into account the key factors of ensemble systems (i.e. diversity, number of models, and combination strategy). First, a set of models with high degree of diversity is generated. That is, each model is trained with a different training data set by applying bootstrap, and the best NN architecture is selected by varying the number of hidden neurons, the activation function, and the weight initialization. Second, GA and SA are employed to select the best subset of models and the optimal combination. The second objective is to design an adaptive ensemble regression which is able to learn samples in the presence of several types of changes and simultaneously retain old information in scenarios where changes may recur. The key idea is to keep a moving data window that slides when a new sample is available. To handle recurring and non-recurring changes, the proposed ensemble uses a new assignment of models' weights that takes into account the models' errors on the past and current windows using a discounting factor that decreases or increases the contribution of old windows. New models are launched if the accuracy of the system is decreasing, and inaccurate models can be excluded over time. The third objective is to design an adaptive ensemble regression with fast adaptation capability for on-line prediction of variables in time-varying applications. The properties of the proposed ensemble are: on-line inclusion and removal of models to keep only the most accurate models with respect to the current state of the system; dynamic adaptation of the model's weights based on their on-line predictions on the most recent samples; and on-line adaptation of the models' parameters. The fourth objective is to design an on-line ensemble regression that selects dynamically the best subset models (from a set of models) to form the ensemble. The proposed method employs ordered aggregation to choose the ensemble size and the subset of models based on the minimization of the ensemble error on the newest sample. It is also proposed an adaptive NN using a variable forgetting factor. The performance and effectiveness of the proposed methodologies are validated and demonstrated using real-world industrial applications, including the estimation of the free lime in a cement kiln process, and other benchmarks for evaluating real-world SS applications. Additionally, experimental results using artificial data sets with several types of changes are presented to demonstrate the effectiveness and accuracy of the proposed methodologies that deal with time-varying environments. |
|---|