Adaptive setpoint control system for microbial cultivation processes

Abstract. A control system for set-point control of microbial cultivation process parameters is developed, in which a tendency model is applied for controller adaptation to process nonlinearity and time-varying operating conditions. The tendency model is updated on-line and introduced into control algorithm for prediction of steady-state control action and returning of feedback controller. The control system was tested for controlling dissolved oxygen concentration in batch operating mode bioreactor under extreme operating conditions. In simulation experiments, the control system demonstrates fast adaptation, robust behaviour and significant improvement in control performance compared to that of fixed gain controller.


Introduction
Maintaining in bioreactors a specific state of microorganisms' culture is commonly implemented by set-point control of key technological parameters, in particular, concentrations of feeding substrates.Accurate control of technological parameters reduces the process deviations that often result in lost or poor product quality.However, the set-point control in batch operating mode bioreactors is not a trivial control problem due to nonlinearity and nonstationarity of bioprocesses.Because of significant variations in process dynamics over the course of microbial cultivation, the ordinary control systems with fixed gain linear controllers are not adequate to cope with the accurate control task.
Various approaches have been proposed for controlling microbial cultivation process parameters under time-varying operating conditions.The controller gain scheduling technique has been applied for design of batch bioreactor controllers with the oxygen uptake rate (OUR) as scheduling variable [4].The control systems of pH with the carbon dioxide evolution rate (CER) as scheduling variable [8] and dissolved oxygen concentration (DOC) with OUR as scheduling variable [12] have been applied in recombinant protein production processes.Adaptation of PI controller parameters in DOC control system based on using a first principle model is shown [13].Authors [11] developed DOC control system, in which the PID controller gain is adapted using a heuristic rule based on supervision in a moving window of three performance indices: the output error covariance, the average value of error, and the input covariance.Authors [6] applied the self-tuning generalized minimum variance controller and the autoregressive moving average with exogenous input (ARMAX) model for controlling DOC in batch bioreactor.Authors [1] presented a feed forward-feedback controller designed to keep glucose concentration at fixed set-point during fed-batch process of recombinant protein production.In the feed-forward part of the controller an extended Kalman filter is introduced to estimate the process variables that are used for calculation of the feed-forward control action.Authors [14] developed control systems of pH and DOC based on using artificial neural network (ANN) models.The forward ANN is trained off-line to predict dynamics of the controlled process and the inverse ANN is used as a direct feedback controller.Authors [9] reported application of model-based geometric control algorithm for controlling DOC in bioreactors.The algorithm incorporates two components -an estimator that predicts the system states and parameters one time step ahead, and a controller that uses the predicted states to compute the control action minimizing the predicted error.Application of a nonlinear model predictive controller for controlling glucose concentration in a fed-batch bioreactor is presented in [5].The control algorithm includes solving the model-based optimization problem at each time discretization step.Authors [3] proposed a cascade control system for controlling the DOC in fed-batch fungal cultivation process, which incorporates auxiliary measurements (OUR, CER and volume) to improve the control performance.In the control system, the substrate feed rate is used as manipulated variable and the inner control loop controls the OUR.In [17], an adaptive control system of fed-batch bioreactor is presented, which realizes a heuristic adaptation of PID controller parameters based on analysis of the control error.The adaptation, however, is available if the control error exceeds particular level.
Capability of the proposed control systems in many cases is investigated and illustrated by experiments carried out at typical cultivation process conditions with slow rate of process state change.However, testing of the control system performance under extreme operating conditions is required to better evaluate and compare advantages and shortcomings of various approaches.
The reasoned sight is that better performance can be expected by using control systems, which exploit more available information about controlled process.Thus, a relevant objective persists to develop and improve universal and reliable control systems for accurate control of cultivation process parameters that employ a priori knowledge of the controlled process and available on-line measurements.This objective is consistent with the Process Analytical Technology (PAT) initiative [10] and the efforts of steering the PAT initiative towards realistic and attainable industrial applications.
In this contribution, an adaptive control system is developed based on exploiting a tendency model of the cultivation process dynamics and on-line measurements of process variables applied for updating the model-based control algorithm.Capability of the control system is demonstrated by computer simulation of the system performance for http://www.mii.lt/NAAdaptive set-point control system for microbial cultivation processes 155 controlling the DOC in batch operating mode bioreactor under extreme conditions.The results are compared with those of a standard PI controller and the gain scheduling-based adaptive control system developed for the DOC control in [12].

Development of adaptive control system
Development of the adaptive control algorithm is based on an assumption that the basic dynamics of controlled process can be described by a simple mass balance model where c is concentration of desired substance (controlled variable); V is working volume of bioreactor; R and Q are consumption and compensation rates of the substance, respectively; and t is time.
The consumption rate R, along with the other factors, commonly depends on a level of the controlled substance concentration.At higher levels, the controlled substance does not influence the consumption rate.At lower levels, it becomes a limiting factor of microorganisms' culture growth and thus decreases the consumption rate.Dependence of the rate R on the concentration c is usually described by the Monod model [16] where R 0 is the substance consumption rate under unlimited (with respect to controlled variable) conditions, k c is a constant specific to particular substance and culture.Application of the model in control algorithm supposes on-line estimation of R.
The compensation rate Q is related with the control variable u by a particular functional relationship Q = Q(u, c, x), where x is a vector of measurable process variables.
With the above assumptions, the state model (1) takes the following structure: Model (3), although limited, by updating it on-line through continuous incorporation of new process data (u(t k ), c(t k ), x(t k ), t k is the updating time), is able to reveal tendencies of the process dynamics variations.
Control action that holds the controlled variable at desired set-point (c(t k ) = c set (t k ), (dc/dt) t=t k = 0) has to satisfy the steady-state condition so, the stabilizing control action (u 0 ) at time point t k can be calculated from equation (4).The control action u 0 based on condition (4) is essentially a feed-forward action that takes into account variations of process variables.However, the calculated value u 0 is unavoidably corrupted by the model inaccuracy and the measurement errors.An error of the calculated control action u 0 can be eliminated by feedback controller adjusted to operate in the vicinity of current state point.Linearization of (3) around the process state point at time t k with respect to variables c and u leads to the following equation: where ∆c, ∆u denote small deviations of c and u form the current state point.Parameters of the linear equation (3) at time t k are calculated using measured values of process variables c(t k ), u(t k ), x(t k ) and the indirectly estimated value R(t k ), which is usually calculated from the measurements of relevant variables.
In the vicinity of process state at time point t k the process dynamics can be described by linear differential equation or by a first-order transfer function model where s is the Laplace operator; ∆c(s), ∆u(s) are the Laplace transforms of ∆c and ∆u; K ∆c/∆u (t k ), T ∆c/∆u (t k ) are gain coefficient and time constant at time point t k , respectively, u 0 (t k ) is value of control variable satisfying condition (4).
Values of the dynamic parameters K ∆c/∆u (t k ) and T ∆c/∆u (t k ) depend on process state and can vary at each time point, therefore, adaptation of process controller is needful to cope with the process dynamics variations.
Adaptation of the feedback controller is performed with the reference to the resultant transfer function model of the controlled process, in which along with the process dynamics, the time-invariant dynamics of the control system elements (measurement device, etc.) is taken into account.The model is updated on-line with the measured or estimated values of process variables (c set (t k ), u 0 (t k ), x(t k ), R(t k )) and then reduced to a typical structure transfer function model in order to directly apply well-developed PI and PID controller tuning rules for simple dynamic models [2,16].In this way, the controller parameters are recalculated on-line at each time discretization point in accordance with the estimated values of the dynamic model parameters.The above controller adaptation procedure safeguards against instability of the closed-loop control system as the tuning rules guarantee a sufficient stability margin.
The adaptive set-point control can be implemented by the control system presented in Fig. 1.
Substantial advantage of the presented control system is instantaneous adaptation of process controller to time-varying operating conditions.

Application example: Control of dissolved oxygen concentration in batch operating mode bioreactor
Dissolved oxygen concentration (DOC) is one of the most important technological parameters of aerobic cultivation processes influencing physiological state of microorganisms' culture and production of desired product.Control of DOC in bioreactors is performed by manipulating stirring speed and (or) air flow rate.In the application example, the proposed control system (Fig. 1) is adopted for set-point control of DOC in batch operating mode bioreactor.The basic dynamics of the DOC control process in bioreactor can be described by a tendency model (1), in which where c is DOC, %; c * is saturation value of DOC, %; K L a is volumetric oxygen transfer coefficient from gas to liquid phase, s −1 ; OUR and OTR are oxygen uptake and transfer rates, respectively, mmol s −1 ; V is volume of fermentation broth, L; k 1 is proportionality coefficient, k 1 = 0.21 (100 • H), H is Henry's constant, L mmol −1 and t is time, s.
The oxygen transfer coefficient depends on bioreactor design and the properties of cultural liquid.Oxygen transfer correlations are generally power-law correlations of the form [16] K where u is stirring speed (control action) s −1 ; q is air supply rate, L s −1 ; α, β, γ are parameters estimated in advance from the batch culture experiments.
The oxygen uptake rate is modelled by the relationship where parameter k c is estimated from early experiments.Assuming quasi-steady state conditions of DOC at moving window ∆T (dc/dt ≈ 0, ∆T is length of moving window), the OTR and OUR are in equilibrium and can be indirectly estimated on-line using measurements of air supply rate and oxygen percentage in exhaust gas [7]: where q is mean value of the air supply rate in moving window, L/s; y O2 is mean value of oxygen percentage in exhaust gas in moving window, %; k 2 is proportionality coefficient, The saturation value of DOC is estimated using measurements of oxygen percentage in exhaust gas:

Development of control algorithm
With the estimated and measured values of process variables, condition (4) for calculation of the steady-state control action u 0 at time point t k is as follows: where c set is set-point value of DOC, %; α e , β e , γ e are pre-estimated values of the tendency model parameters α, β, and γ, respectively.The gain coefficient and the time constant of the process transfer function ( 8) are calculated from equations ( 9) and ( 10), using the functional relationships (2), ( 11)- (13).After some algebraic manipulations, the transfer function parameters can be written as http://www.mii.lt/NA where k ce is pre-estimated value of the tendency model parameter k c ; c * (t k ) is saturation value of DOC estimated from the oxygen percentage in exhaust gas (16); c(t k ) is measured value of DOC obtained from the dissolved oxygen (DO) electrode.The DOC control system performance is highly influenced by dynamical characteristic of DO electrode, which supplies with the feedback signal.Dynamics of the DO electrode is suitably described by a first-order plus time delay transfer function where c el is output signal of the DO electrode, %; T el and τ el are time constant and time delay of DO electrode, respectively, s.
In the presented investigation, dynamics of the motor-stirrer system is assumed to be fast compared to that of DO electrode, so it is not taken into account in the resultant transfer function model.Under necessity of considering the dynamics of motor-stirrer system (in large-scale bioreactors), the resultant transfer function should be extended with a required term.
By linking the transfer functions of the controlled process ( 8), ( 19)-( 21) and the DO electrode (22), the resultant transfer function is obtained that provides with the relevant information for process control: The transfer function (23) parameters K ∆c/∆u and T ∆c/∆u are estimated at each time discretization point using the functional relationships (19)-(21), respectively, the estimated value of OUR(t k ), the measured values of process variables c set (t k ), q(t k ), and the control action value u 0 (t k ) calculated from equation (18).With the updated parameters K ∆c/∆u and T ∆c/∆u , the transfer function (23) captures time-varying dynamics of the controlled process.
In order to apply controller tuning rules, developed for a first-order plus time delay (FOPTD) model, the process model ( 23) is approximated by the FOPTD model in which T pr (t k ) and τ pr (t k ) are resultant time constant and resultant time delay of the controlled process at time t k , respectively.The model parameters T pr (t k ) and τ pr (t k ) are updated by fitting the FOPTD model (24) to the simulated step response of the process model (23) at each sampling time.The Smith's approximation [15] is applied for fitting the FOPTD model.
In the control system (Fig. 1), the adaptive feedback controller is realized using the velocity form of a modified discrete PI control algorithm where ∆u is an increment/decrement of stirring speed, s −1 ; K c is controller gain coefficient, % −1 s −1 ; T i is controller integration constant, s; b is set-point weighting; ∆t is time discretization step of control action, s; c el is measured value of DOC, %.The controller parameters K c (t i ), T i (t i ), and b(t i ) are recalculated at each sampling instant using updated values of the FOPTD model parameters K ∆c/∆u (t k ), T pr (t k ), τ pr (t k ) and tuning rules developed for the FOPTD model.In the application example, the Kappa-Tau tuning rules for maximum sensitivity M s = 2.0 [2] are applied: It should be stressed that the step set-point change in the control system causes stepwise change of the control action u 0 and simultaneously the control action of the feedback controller.To obviate the double control action at the set-point change points, the calculated step change of u 0 is subtracted from the total control action.This refinement ensures good performance of the control system by tracking the set-point.

Simulation of the control system performance
Performance of the adaptive control system is investigated via computer simulation implemented in Matlab/Simulink environment.In the simulation experiments, the controlled process is modelled by a set of equations http://www.mii.lt/NA where q set is set value of air supply rate, L s −1 ; u set is set value of stirring speed (control variable), s −1 ; y O2 is portion of oxygen in exhaust gas, -; OUR v is volumetric oxygen uptake rate unlimited by DOC, mmol L −1 s −1 ; c a is DOC in absolute units, mmol L −1 ; a el is auxiliary variable, %; c el is signal from DO electrode, %; H is Henry's constant, L mmol −1 ; v mol is volume of mmol of gas, L mmol −1 ); T q , T u , T el1 , T el2 are time constants of air supply system, motor-stirrer system, and DO electrode, respectively, s; ε is gas holdup in the gas-liquid dispersion, -.
The model equations ( 31), (32) represent dynamics of air supply and stirring systems respectively, equations ( 33), (34) stand for mass balances on oxygen in liquid and gaseous phases, respectively, and equations ( 35), (36) represent second-order dynamics of DO electrode.Parameters of the model equations are taken from the ranges reported in literature [16].The parameter values are given in Table 1.
In the discrete PI control algorithm (25), (26), the sampling time ∆t = 0.2 s was used throughout the control experiments.
The tendency model parameters that appear in the controller adaptation formulas (18)-( 23) are identified from early experiments.The parameters estimation errors affect the control action calculation results, so, performance of the control system is investigated taking into account possible inaccuracy of the predetermined parameter values.In the simulation experiments, the tendency model parameter values deviate by 5-10% from the process model parameter values given in Table 1: k ce = 1.1%, α e = 0.0016, β e = 2.1, γ e = 0.18.A second-order dynamics of the DO electrode (equations ( 35), (36)) is approximated by the FOPTD model ( 22) with the parameter values T el = 11 s, τ el = 2 s.
In the control system simulation experiments, disturbances of the set-point and the air supply rate step changes under time-varying oxygen uptake rate (load disturbance) have been applied.Performance of the control system under simulated disturbances is presented in Fig. 2. Step changes of the set-point (from 10 to 30% occurring at t = 1000 s, from 30 to 50% occurring at t = 1750 s, from 50 to 20% occurring at t = 2500 s, and from  20 to 5% occurring at t = 3500 s) are presented by dotted lines in Fig. 2g, h.Step-changes of the air supply rate (from 6 to 3 L s −1 occurring at t = 500 s, from 3 to 7 L s −1 occurring at t = 3000 s, and from 7 to 1 L s −1 occurring at t = 4500 s) are shown in Fig. 2a.Time profile of the OUR v variation, presented in Fig. 2b, is chosen to simulate close to realistic operating conditions at batch cultivation processes.Adaptation of the controller parameters K c , T i , b to the time-varying operating conditions is demonstrated by the parameter time trajectories in Fig. 2c-e, respectively.Variation of the control variable u (stirring speed) is shown in Fig. 2f.Responses of the controlled DOC to the set-point changes and the disturbances are presented in Fig. 2g.Performance of the control system was also investigated under process noise added to the feedback signal and the estimated values of OUR.Responses of the controlled DOC at 2% multiplicative noise are presented in Fig. 2h.
From Fig. 2c-h, it can be seen that the adaptive control system demonstrates stable performance, instantaneous adaptation of controller parameters and quite accurate setpoint control under noisy measurements and variations of process disturbances in fairly wide ranges.
For comparison, performances of the conventional PI control system and the gainscheduling adaptive control system [12] have been investigated.
Performance of the control system with a fixed gain PI controller is presented by dashed lines in Fig. 2c-g.The values of controller tuning parameters are set to mean values of the adaptive controller parameter variation ranges presented in Fig. 2c-e (indicated by dashed lines, K c = 0.16 % −1 s −1 , T i = 160 s, b = 0.465).It can be seen from Fig. 2g that the conventional control system exhibits poor performance under time varying operating conditions.The simulation experiments (not presented) show that by setting the PI controller parameters that are optimal at the initial conditions of simulated process, the control system is sluggish and not able to track the set point changes and to compensate the disturbances at further operating conditions.
In the other case, by setting the controller parameters that are optimal at the end stage, high amplitude oscillations around the set-point occur at the initial operating conditions.
Performance of the DOC adaptive control system with the gain-scheduled PI controller and the OUR as scheduling variable (K c = 0.12+0.5•OUR,T i = 200−500•OUR, b = 0.465) is shown by dash-dotted line in Fig. 2g.The simulation results demonstrate that the gain scheduling control system does not ensure an accurate control of the DOC under fast changing operating conditions.
Comparison of the mean absolute deviations (MAD) of the DOC from the set-point time-trajectory calculated from the control system performance simulation results proves obvious advantage of the developed control system (MAD = 0.764%) compared with the fixed gain PI control system (MAD = 2.185%) and the gain-scheduled control system (2.035%).
The investigated DOC control system can be implemented in practice by using standard measurement devices of the air flow rate, the percentage of oxygen in exhaust gas, and the dissolved oxygen concentration.The adaptive control algorithm can be realized in industrial distributed process control systems such as Siemens PCS7, Emerson PM System DeltaV, etc.It should be stressed that the presented approach of the control systems design for microbial cultivation processes is versatile and can be directly applied in development of control systems of various biotechnological parameters: pH, dissolved carbon dioxide concentration, glucose and glutamine concentrations, etc.

Conclusions
In this contribution, an adaptive control system is developed for controlling cultivation process parameters at desired set-point under permanent and sudden changes in process dynamics.The proposed control system exploits a priori knowledge of the controlled process variable relationships and on-line measurements of relevant variables.
The control system adaptation algorithm uses information contained in a simple firstprinciples model that is updated on-line with the measured values of process variables and applied at each control discretization step for prediction the steady-state control action and retuning parameters of feedback controller.
The control system has been tested by controlling the dissolved oxygen concentration in batch operating mode bioreactor.In simulation experiments performance of the control system was investigated under extreme operating conditions, noisy measurements and inaccuracy of the tendency model parameter estimates.The investigation results show that even the dynamic model of a limited accuracy applied in the proposed control system provides with relevant information for controller adaptation.The control system demonstrates fast adaptation and robust behavior by tracking the set-point under sudden changes in operating conditions.

Figure 2 .
Figure 2. Performance of the DOC control system at set-point and air supply rate step changes and permanent change of oxygen uptake rate.Shown here are time profiles of simulated disturbances: air supply rate (a), OUR (b); adaptation of controller parameters: gain (c), integration time constant (d), and set-point weighting parameter (e); stirring speed (control action) (f); DOC controlled at set-point (g), (h).