Amal Gunatilake

and 2 more

Amal Gunatilake

and 4 more

Nalika Ulapane

and 3 more

Identification of static nonlinear elements (i.e., nonlinear elements whose outputs depend only on the present value of inputs) is crucial for the success of system identification tasks. Identification of static nonlinear elements though can pose several challenges. Two of the main challenges are: (1) mathematical models describing the elements being unknown and thus requiring black-box identification; and (2) collection of sufficiently informative measurements. With the aim of addressing the two challenges, we propose in this paper a method of predetermining informative measurement points offline (i.e., prior to conducting experiments or seeing any measured data), and using those measurements for online model calibration. Since we deal with an unknown model structure scenario, a high order polynomial model is assumed. Over fit and under fit avoidance are achieved via checking model convergence via an iterative means. Model dependent information maximization is done via a D-optimal design of experiments strategy. Due to experiments being designed offline and being designed prior to conducting measurements, this method eases off the computation burden at the point of conducting measurements. The need for in-the-loop information maximization while conducting measurements is avoided. We conclude by comparing the proposed D-optimal design method with a method of in-the-loop information maximization and point out the pros and cons. The method is demonstrated for the single-input-single-output (SISO) static nonlinear element case. The method can be extended to MISO systems as well.

Nalika Ulapane

and 2 more

Hyper-parameter optimization is an essential task in the use of machine learning techniques. Such optimizations are typically done starting with an initial guess provided to hyperparameter values followed by optimization (or minimization) of some cost function via gradient-based methods. The initial values become crucial since there is every chance for reaching local minimums in the cost functions being minimized, especially since gradient-based optimizing is done. Therefore, initializing hyper-parameters several times and repeating optimization to achieve the best solutions is usually attempted. Repetition of optimization can be computationally expensive when using techniques like Gaussian Process (GP) which has an O(n3) complexity, and not having a formal strategy to initialize hyperparameter values is an additional challenge. In general, reinitialization of hyper-parameter values in the contexts of many machine learning techniques including GP has been done at random over the years; some recent developments have proposed some initialization strategies based on the optimization of some meta loss cost functions. To simplify this challenge of hyperparameter initialization, this paper introduces a data-dependent deterministic initialization technique. The specific case of the squared exponential kernel-based GP regression problem is focused on, and the proposed technique brings novelty by being deterministic as opposed to random initialization, and fast (due to the deterministic nature) as opposed to optimizing some form of meta cost function as done in some previous works. Although global suitability of this initialization technique is not proven in this paper, as a preliminary study the technique’s effectiveness is demonstrated via several synthetic as well as real data-based nonlinear regression examples, hinting that the technique may have the effectiveness for broader usage.

Nalika Ulapane

and 2 more

Systems containing linear first-order dynamics and static nonlinear elements (i.e., nonlinear elements whose outputs depend only on the present value of inputs) are often encountered; for example, certain automobile engine subsystems. Therefore, system identification of static nonlinear elements becomes a crucial component that underpins the success of the overall identification of such dynamical systems. In relation to identifying such systems, we are often required to identify models in differential equation form, and consequently, we are required to describe static nonlinear elements in the form of functions in time domain. Identification of such functions describing static elements is often a black-box identification exercise; although the inputs and outputs are known, correct mathematical models describing the static nonlinear elements may be unknown. Therefore, with the aim of obtaining computationally efficient models, calibrating polynomial models for such static elements is often attempted. With that approach comes several issues, such as long time requirements to collect adequate amounts of measurements to calibrate models, having to test different models to pick the best one, and having to avoid models over-fitting to noisy measurements. Given that premise, this paper proposes an approach to tackle some of those issues. The approach involves collecting measurements based on an uncertainty-driven Active Learning scheme to reduce time spent on measurements, and simultaneously fitting smooth models using Gaussian Process (GP) regression to avoid over-fitting, and subsequently picking best fitting polynomial models using GP-regressed smooth models. The principles for the single-input-single-output (SISO) static nonlinear element case are demonstrated in this paper through simulation. These principles can easily be extended to MISO systems.