Optimal control of nonlinear systems with input constraints using linear time varying approximations

Abstract. We propose a new method to solve input constrained optimal control problems for autonomous nonlinear systems affine in control. We then extend the method to compute the bangbang control solutions under the symmetric control constraints. The most attractive aspect of the proposed technique is that it enables the use of linear quadratic control theory on the input constrained linear and nonlinear systems. We illustrate the effectiveness of our technique both on linear and nonlinear examples and compare our results with those of the literature.


Introduction
Control of nonlinear systems is of great importance since many systems have inherent nonlinearities.Design and implementation of nonlinear controllers is a difficult task compared to linear controller design.Moreover this task becomes more demanding when systems have bounds on the control input, which is referred to as actuator saturation.In the presence of actuator saturation, a given nominal controller may loose its performance or even make the system unstable unless the constraints are considered a priori in the design process.Therefore, the effects of input constraints are very important in nonlinear control design and as a consequence, have attracted much interest in the area of nonlinear and optimal control [1,2,5,6,12,13].
In some control problems such as minimum-time optimal control problems, if the control input is bounded and the system is affine in the control, an optimal nonsingular solution to the control problem is of bang-bang type, i.e. control input switches between its lower and upper limits [9].In general, when the solution to the control problem is bangbang, the problem reduces to finding the switching instants or bang-bang arc durations.Optimal control problems whose solutions are of bang-bang, are solved by using indirect shooting methods to solve the multi-point boundary value problem obtained from the c Vilnius University, 2016 Pontryagin's minimum principle (PMP).Solving these problems is extremely difficult since very good initial guesses for switching arc durations and the values of the states at the switching instants are required.Direct optimization methods, such as nonlinear programming, can also be used to find bang-bang control solutions for nonlinear systems.This method works by converting the optimal control problem to a very large, finite dimensional optimization problem and it requires again good initial guesses for the initial value of the control and the number of switchings [8,10].
In this paper, input constrained control of nonlinear systems are studied by using a special kind of approximation technique.In this method, the nonlinear system is represented in the form of a series of linear time-varying (LTV) systems whose responses converge to the nonlinear system's response in the limit.This property enables one to use the well-known linear control methods on nonlinear systems.The approximation technique, which is also called as Approximating Sequence of Riccati Equations (ASRE), has been modified in order to be applied to input constrained systems since unbounded control input is considered in the general theory [3,7,11].More importantly, we show that using the input constrained theory, one can also obtain near minimum-time bangbang type control solutions for both linear and nonlinear systems without any requirement of the knowledge for the initial value of the control input and the number of bang-bang switchings since they are directly obtained from the solutions of the differential Riccati equations for the approximate linear time-varying systems.We also indicate the condition for the minimum convergence time of the LTV approximations.
The organization of the paper is as follows.In Section 2, we introduce the approximation method and then show how to incorporate hard constraints in the control problem.In Section 3, bang-bang type control for the nonlinear system type under consideration is studied.We apply our technique on both a linear and nonlinear system examples and show the simulation results in Section 4. Finally, we draw the conclusions in Section 5.

Control of input constrained nonlinear systems using LTV approximations
Consider the general control-affine autonomous nonlinear system where The functions f (x) and g i (x) are assumed to be sufficiently smooth.Without loss of generality, assuming that the origin is an equilibrium point, i.e. f (0) = 0 and g i (0) = 0, then system (2) can be expressed in the factored (extended-linearized) form Nonlinear Anal.Model.Control, 21(3):400-412 where A(x) ∈ R n×n and G(x) = [g 1 (x), g 2 (x), . . ., g m (x)] ∈ R n×m are matrix valued functions.Note that the factored representation f (x) = A(x)x is not unique unless the system is scalar.The cost functional for the finite-time quadratic optimal control problem to be minimized is where F, Q ∈ R n×n are positive semi-definite and R ∈ R m×m is positive definite symmetric matrices.We note that the weighting matrices Q and R can also be chosen state dependent such that Q(x) 0 and R(x) > 0 for all x(t), where t ∈ [t 0 , t f ].However, state-dependent selection of the weighting matrices yields a non-quadratic cost functional, which requires a special treatment.
The quadratic optimal control problem (3)-( 4) cannot be solved by using linear quadratic regulator (LQR) control theory directly, since the differential constraint (3) is nonlinear.Moreover, control solutions obtained by using the LQR theory is unbounded and in order to satisfy the control constraints the designer must tweak the weighting matrices Q and R.However, this is cumbersome and often it is difficult to find the correct weighting matrices, which satisfy the design specifications.Here we propose incorporating the input constraints into the optimal control problem priori by using a saturation function such that Here φ i (•) is a sufficiently smooth function and ū(t) = [ū 1 , ū2 , . . ., ūm ] T is the new control input vector.Also, φ i (•) must be chosen such that the constraint given by Eq. ( 2) is satisfied.If we multiply and divide Eq. ( 5) by ūi and substitute it in Eq. ( 3), then we get We then define a new control matrix Therefore, Eq. ( 6) becomes The new system is now control non-affine and moreover as ūi (t) → 0, ḡi (•) → ∞, which is not desired in any control problem.Let us define the function φ i (•) as follows: where Moreover, when ūi → 0, we get http://www.mii.lt/NAlim ūi→0 ρ i arctan(ū i )/ū i = ρ i .Therefore, the new control problem becomes along with the cost functional to be minimized where R ∈ R m×m is a positive definite symmetric matrix.In general, solving the optimal control problems of minimizing the cost (8) subject to nonlinear differential constraint (7) requires solving the two point boundary-value problem of the state and co-state equations that results from calculus of variations, by shooting techniques or nonlinear programming.
We propose a different approach, which represents the nonlinear system (7) as a sequence of Linear Time-Varying (LTV) approximations whose responses converge to the nonlinear system in the limit.Then one can implement LQR control theory on the LTV system of each approximation.For initiating the iterations, let us define a base system together with the quadratic cost functional where Ḡ(x, u) = [ḡ 1 , ḡ2 , . . ., ḡm ].We obtain a linear time invariant (LTI) base system for the initiation of the LTV sequences and hence, the optimal control for the base system is u where P [0] ∈ R n×n can be obtained by solving the well-known Algebraic Riccati Equation (ARE).
For the forthcoming approximations (k 1), where the superscript [k] denotes the iteration number and x [k] (0) = x 0 .Thus, the nonlinear optimal control problem becomes a LTV quadratic optimal control problem for each iteration of the approximation sequences.The optimal control input for each approximation is where P [k] (t) ∈ R n×n is the solution of the matrix-differential Riccati equation given in Eq. ( 14) backwards in time from t f to t 0 : where Then kth closed-loop dynamic system can be written as For the LTV sequences (12) to converge, i.e.
we only need local Lipschitz continuity of the nonlinear system (7), which is a very mild condition [4].

Computation procedure
The LTV sequences cannot be solved analytically, hence, numerical computations must be used.The computation procedure can be summarized as follows: 1. Use the initial state vector x [k−1] (t) = x 0 and the control input vector ū[k−1] (t) = 0 for k = 0 in ( 9) and obtain a LTI system.2. Solve the LQR problem ( 10)-( 11) for the LTI system obtained in the first step and get a stable solution for the state vector Eqs. ( 12)-( 14), then solve the matrix-differential Riccati equation ( 14) backwards in time to obtain P [k] (t). 4. Use P [k] (t) to solve (12) with the initial condition x [k] (0) = x 0 and obtain the solution x [k] (t) simultaneously with the control input ū[k] (t) from (13). 5. Repeat the steps 3-4 for k = 1, 2, . . ., , where is the number of last iteration, until the LTV approximations converge, i.e.
, where is a small positive number.6. Calculate u (t) = φ (ū (t), u max ) and use it in Eq. (1).http://www.mii.lt/NAFor the first step, one can also linearize the nonlinear system (7) around the origin and solve the LQR problem obtained from the linearized system.If the LTI system is uncontrollable, the differential Riccati equation must be solved instead of the algebraic Riccati equation to obtain x [0] (t).We note that the technique proposed here differs from the State-Dependent Riccati Equation (SDRE) approach in that it solves the differential Riccati equation for each LTV approximation systems.Therefore, the proposed method does not require pointwise controllability of the (A(x), B(x)) pair for all t ∈ [t 0 , t f ].
In the next section, we shall show how to obtain near-minimum time bang-bang type solutions for the nonlinear system (1).

Stability of the ASRE control
In the literature, although the proof of convergence for the ASRE control has been discussed [4], its stability has not been studied yet.Therefore, we shall prove the stability of the ASRE control in the following theorem.
Theorem 1.Consider the nonlinear control problem and let with x [k] (0) = x 0 be an approximating scheme as above.Then if the nonlinear system (15) is pointwise controllable almost everywhere, then the solution of problems (16)-( 17) converges to a solution of the nonlinear problem as k → ∞.Moreover, if we take F = αI and denote the solution of (16)-( 17) by x [k] (t; α), then as α → ∞, Proof.The idea comes from receding horizon control.First, note that the linear quadratic solution to (16)-( 17) is given by Nonlinear Anal.Model.Control, 21(3):400-412 where Here ).We would like to set F = αI, and let α → ∞.In order to do this, we set where W (t f ) = I/α.Hence, as α → ∞, where ) is controllable i.e.W (t) > 0, independently of α for almost all t < t f .Hence, the solution exists, and we have x [k] (t f ; α) → 0 as t → t f .

Obtaining bang-bang control solutions
We use a design parameter to increase the steepness of the sigmoid obtained by the arctan function (see Fig. 1) to obtain bang-bang type bounded control solutions for the given control problem.The modified control input becomes where r i > 1 is a design parameter.Let us define the function φ i (•) as follows: where ρ i = 2u max i /π.When ūi → 0, we get lim ūi→0 ρ i arctan(r i ūi )/ū i = ρ i r i .When we substitute the new control variable into the control problem, we get We note that the bounded control obtained from the saturation function (21) is continuous since we solve a continuous-time differential Riccati equation in each sequence of the LTV approximations.If r i is selected large enough, the solutions to the optimal control problem results in bang-bang type controls.Generally speaking, in order to obtain http://www.mii.lt/NAbang-bang type solutions to a constrained optimal control problem, two-point boundary value problem of the state and co-state equations are solved by using indirect shooting techniques, or the control problem is transformed into an optimization problem by discretization and recruiting a mathematical programming technique such as dynamic programming or nonlinear programming.If the minimum-time solutions are desired, the problem becomes more difficult as initial values for optimization problem parameters such as the number of switchings, switching arc lengths or durations, initial value of the control input must be guessed to start the optimization problem.Since multi-parameter guesses are required at the same time, in the case of a bad initial guess for even one parameter, the solutions to the optimization problem may converge very slowly or even may not converge at all.The proposed method finds the bang-bang solutions to a class of input constrained nonlinear optimal control problems for a given time t f .We note that a search algorithm may be recruited to find a sub-optimal final time t * f , which is close to t min (minimum-time), by using an initial guess for t * min and repeatedly solving the LTV approximations for the updated t f values.Moreover, the solution of the approximation sequences directly gives the number of the switchings, the switching arc lengths and the initial control value, i.e. whether the initial value of the control is u max or u min for the problem under consideration, which are the required input parameters to solve the optimization problem by shooting methods or nonlinear programming.
In order to find a solution to the nonlinear optimal control problem ( 7)-( 8) by using the proposed LTV sequences, the specified t f cannot be less then the minimum-time t min , which is necessary to drive the system states from an initial condition to a desired final condition.This can be explained by simple contradiction.If the sequences of controls and states did converge, then we would have a control, which drives the system to zero in less than the optimal time t min .Since the control is bounded by u max , we have a contradiction and hence, t f t min condition must be satisfied.

Numerical results
In this section, we shall implement the proposed method first on a nonlinear system, which is known as Rayleigh equations, also studied by [13,14], to obtain input constrained quadratic control results.In the second part of the simulations, we shall find the bangbang type near optimal control solutions of the Goddard problem and then the Rayleigh system, respectively, by using LTV approximations.

Unbounded and bounded quadratic control simulations
Rayleigh equations given by Eq. ( 22) represent the dynamics of an electric circuit (tunnel diode oscillator), where the state variable x 1 (t) denotes the electric current and the control input u(t) is the suitable transformation of the voltage at the generator [14].The control problem is to stabilize at the origin the system subject to the quadratic cost functional (3).The initial states are x 1 (0) = x 2 (0) = −5, and the control input is bounded as The factored form of system (22) for unbounded control design is selected as In the absence of control, the Rayleigh system shows oscillatory dynamics.We first implement the unbounded control obtained by solving the LTV approximations, to the system.Figures 2 and 3 illustrate the system states x 1 (t) and x 2 (t), respectively, in the absence of a bound on the control input.The design parameters for the quadratic control are selected as F = I 2×2 , Q = diag(100, 10) and R = 0.1, where I n×n is n dimensional identity matrix and diag denotes diagonal matrix.
We then incorporate the bound function φ(•) along with the new transformed control input ū(t) and obtain the system as follows:    where ḡ(x, ū) = ρ arctan(ū)/ū and ρ = 2u max /π, where the maximum value of the control input u max = 4.The initial state vector is x 0 = [−5, −5].The responses of the stabilized states in the presence of bounded control are given by Figs. 4 and 5.The comparison of the unconstrained and constrained control inputs is depicted in Fig. 6.It takes the system approximately 5 s to be stabilized at the origin with the constrained control, whereas this time is about 1.5 s (s denotes the time in seconds) if the control is unbounded.The design parameters for the quadratic constrained control are selected as F = I 2×2 , Q = diag(100, 10), R = 0.1.

Bang-bang control simulations
We first consider a linear system, which is also called as the Goddard problem where the initial states are x 1 (0) = 1, x 2 (0) = −1 and desired final states are The analytical solution to the minimum-time problem is found to be t min = 1.45 s with a switch at 0.225 s starting with a negative control then switches to positive.Incorporating the bound on the control results in a new system, which is nonlinear, and it is given as follows: where ḡ(x, ū) = ρ arctan(rū)/ū and ρ = 2u max /π, where the maximum value of the control input u max = 1.We aim to obtain the bang-bang type control solution for the original system (24) by using the modified system (25) and the cost functional (8).
As we have indicated before, the condition t f t min must be satisfied in order for lim k→∞ x [k] (t) − x [k−1] (t) = 0.In the case of t f > t min , the solutions obtained from the LTV approximations will converge to an arbitrary bang-bang solution.However, a simple update algorithm may be recruited, which gradually reduces the initially selected t f to obtain closer solutions to minimum-time solutions by checking whether the LTV approximations converge or not.Here, in order to get a better convergence, we choose t f = 1.46 s, which is very close to the analytical solution t min .We note that when t f = t min is reached, numerical issues arises such as low convergence rate or reduced accuracy of the final values of the states.In this example, the control parameters are selected as F = I 2×2 , Q = 0 2×2 , where 0 2×2 denotes the 2 by 2 null matrix, R = 1 and r = 2 × 10 3 .Figure 7 illustrates the system response in the presence of the bang-bang control.The control input u(t) is depicted in Fig. 8.We obtain x 1 (t f ) = 0.0000158 and http://www.mii.lt/NAx 2 (t f ) = −0.00203and the switch in the control occurs when the time is 0.227 s.The comparison of the analytical solution and the approximated solution is given in Fig. 9 on the phase plane.
We also obtain bang-bang type control solutions for the Rayleigh problem (22).Time optimal control is studied for the same system in [14].In order to get better convergence results, we select our final time t f = 3.7 s, which is bigger than the minimum-time t min = 3.66817 s computed in [14].The control parameters are selected as F = I 2×2 , Q = 0 2×2 , R = 1 and r = 2 × 10 4 .We obtained near-time optimal solutions to the Rayleigh problem (22) by using the modified system (23).The time evolution of the states under near bang-bang control is illustrated in Fig. 10, and the control input is in Fig. 11.The switching times are obtained as 1.353 s and 3.245 s.The comparison of the near minimum-time and minimum-time solutions for the states is illustrated on the phase plane in Fig. 12.

Conclusions
We have studied input constrained control of a class of nonlinear system by using LTV approximations.We show that the algorithm can be effectively used to find bang-bang type control solutions to the nonlinear control problem for a specified time without any Nonlinear Anal.Model.Control, 21(3):400-412

Figure 10 .
Figure 10.Convergence of the system states.

Figure 11 .
Figure 11.Control inputs of the 20th and 50th iterations.

Figure 12 .
Figure 12.Comparison of minimum-time and approximated trajectories for Rayleigh system.