On Advances in Noise Modeling in Stochastic Systems, Control and Adaptive Control

Abstract

In these chapter describes various advanced models for noise in systems. Some results associated with them and the related control and adaptive control problems are formulated and solved. Some problems are proposed for future investigation. The processes used for modeling a system noise include Brownian motion, fractional Brownian motion, and Rosenblatt processes. The Rosenblatt processes are non-Gaussian processes that have long-range dependence, an important property for current applications. They can be considered as a non-Gaussian generalization of fractional Brownian motions that also have a long-range dependence property. Some results are presented for adaptive control of partially known linear stochastic systems with Brownian motion and with fractional Brownian motion. Some optimal control results are given for scalar linear stochastic systems with Rosenblatt noise.

1. Introduction

Many physical systems experience perturbations or have unmodeled dynamics in the systems. These perturbations can often be modeled by white noise or other noise processes. Examples show that noise may have a stabilizing or a destabilizing effect. Some stochastic models are considered here with different types of noise. The noise processes include Brownian motion, fractional Brownian motion, and Rosenblatt processes. These latter two processes often seem more appropriate in a model given the data than using a Brownian motion. All three noise processes are used to drive linear and semilinear systems. However, each type of noise process for a control system needs somewhat distinct methods of analysis for solutions to linear-quadratic control and adaptive control problems. The general approach to adaptive control that is described here exhibits a splitting or separation of identification and adaptive control. Adaptive control results are given for partially known linear and semilinear systems with Brownian motion [1] and linear and semilinear systems with fractional Brownian motion [2, 3, 4, 5]. The solution of an ergodic control problem for a scalar linear system with a Rosenblatt noise is also given [6]. Stochastic calculus methods were developed for fractional Brownian motion [7, 8, 9, 10] and for Rosenblatt processes [11]. Stochastic calculus provides powerful tools: stochastic integrals, Itô’s differential formula, and martingales.

2. Noise modeled by Brownian motion

Historically, noise in continuous-time physical systems has been modeled by Brownian motion. This Gaussian process was a natural choice from the Central Limit Theorem and was used for the early applications by Kolmogorov and Wiener. Furthermore, a stochastic calculus had been developed particularly by K. Itô to allow for the analysis of the continuous-time stochastic systems. The system models were typically ordinary differential equations driven by a Brownian motion denoted as stochastic differential equations. Since the stochastic systems for these problems were typically only partially known, there was a problem to identify the appropriate values for the unknown parameters in the equations. These problems are typically described as identification problems for stochastic systems. The linearity of the systems provided explicit solutions and various testing methods were used, such as least-squares or weighted least-squares estimation, to obtain the estimates of unknown parameters in the systems. A general model is given as

$$\begin{array}{lll}\mathit{dX}\left(t\right)& =& \mathit{AX}\left(t\right)\mathit{dt}+\mathit{BU}\left(t\right)+\mathit{CdW}\left(t\right)\\ X\left(0\right)& =& x_0\end{array}$$E1

where $X\left(t\right)\in R^n$ and $W\left(t\right)$ is a Brownian motion. Assuming that the matrices $A,B,C$ are unknown or at least only partially known, there is a problem with identification of these unknowns. A typical method has been the least-squares method, which can provide explicit and computationally feasible algorithms for the estimates. In particular, the estimates are least-squares parameter estimation for the matrices $A,B$. An estimate for the matrix $C$ can be obtained from the quadratic variation of the solution $X\left(t\right)$, which means the quadratic variation of the stochastic term $\left(\mathit{CdW}\left(t\right)\right)$ that uses the quadratic variation of a Brownian motion.

The cost functional for the adaptive control problem is

$$J\left(U\right)=\underset{T\to \infty }{limsup}\frac{1}{T}{\int }_0^T\left(X^T\left(t\right)Q_{1}X\left(t\right)++U^T\left(t\right)Q_{2}U\left(t\right)\right)\mathit{dt}$$E2

where $Q_1,Q_2$ and symmetric and positive definite matrices. A weighted least-squares estimation procedure is done for ${\theta }^T=\left[AB\right]$ and $\varphi$ is the vector formed from $X\left(t\right)$ and $U\left(t\right)$.

The equation for $X$ can be rewritten as

$$\mathit{dX}\left(t\right)={\theta }^T\varphi \left(t\right)\mathit{dt}+\mathit{CdW}\left(t\right)$$E3

where the weighted least-squares estimation scheme is given by the following equations

$$\begin{array}{lll}\mathit{d\theta }\left(t\right)& =& a\left(t\right)P\left(t\right)\varphi \left(t\right)\left[{\mathit{dX}}^T\left(t\right)-{\varphi }^T\left(t\right)\theta \left(t\right)\mathit{dt}\right]\\ \mathit{dP}\left(t\right)& =& -a\left(t\right)P\left(t\right)\varphi \left(t\right){\varphi }^T\left(t\right)P^T\left(t\right)P\left(t\right)\mathit{dt}\end{array}$$E4

It can be shown that these processes form a family of strongly consistent estimators meaning that the estimators of the unknown terms $A,B$ converge almost surely [1]. It follows that a family of certainty equivalence controls is self-optimizing, that is, they provide convergence to the optimal long-run average cost for the true system. This approach eliminated some other assumptions that have previously been used that are unnecessary for the control problem for a known system and are often difficult to verify. For the adaptive control, a diminishing excited certainty equivalence control is used. To obtain a strong consistency for the family of estimators, a diminishing excitation is added to the adaptive control. The complete solution to the adaptive control problem with the most natural assumptions has been obtained for continuous-time linear and some nonlinear systems [1].

3. Noise modeled by a fractional Brownian motion for a linear system

Since many physical phenomena have a long-range dependence, it is natural to consider a Gaussian process with a long-range dependence property, that is, a fractional Brownian motion, and then to consider adaptive control for a linear stochastic equation driven by a fractional Brownian motion. If the equation has unknown parameters, then one has a problem of adaptive control. This problem is described here for a scalar linear system driven by a fractional Brownian motion. These processes are indexed by the Hurst parameter $H\in \left(0,1\right)$. Initially, a fractional Brownian motion is defined to fix the notation.

Fractional Brownian motion (FBM) is a collection of Gaussian processes with continuous sample paths and a stochastic calculus [7, 8, 10, 12] whose definition is given now. These processes and stochastic calculus tools had been developed for the purpose of being applied to linear and semilinear control systems with some unknowns [7, 8].

Definition 3.1. For each $H\in \left(0,1\right)$, a real-valued standard fractional Brownian motion $\left(B\left(t\right),t\ge 0\right)$ is a Gaussian process with continuous sample paths that satisfies the following conditions:

$$\mathbb{E}\left[B\left(t\right)\right]=0$$E5

$$\mathbb{E}\left[B\left(s\right)B\left(t\right)\right]=\left(1/2\right)\left(t^{2H}+s^{2H}-{\left|t-s\right|}^{2H}\right)$$E6

for all $s,t\in {\mathbb{R}}_{+}$.

This process has been used to model rainfall along the Nile River Valley. The long-range dependence of the rainfall was determined by Harold Hurst.

As is the case with Brownian motion it is sometimes useful to consider the formal time derivative of a fractional Brownian motion. Some particularly useful properties of a FBM are noted now. If $a>0$, then the processes $\left(B\left(\mathit{at}\right),t\ge 0\right)$ and $\left(a^{H}B\left(t\right),t\ge 0\right)$ have the same probability law. This property is called a sell-similarity property. A long-range dependence for a FBM occurs if the Hurst parameter $H$ is in the interval $\left(\frac{1}{2},1\right)$.

For the Hurst parameter $H\in \left(1/2,1\right)$, a FBM has a long-range dependence property, which means that

$$\underset{n=0}{\overset{\infty }{\mathrm{\Sigma }}}r\left(n\right)=\infty$$E7

where

$$r\left(n\right)=\mathbb{E}\left[B\left(1\right)\left(B\left(n+1\right)-B\left(n\right)\right)\right]$$E8

This property provides an indication that the distant past has a nontrivial influence on the present behavior of the process.

Now the linear-quadratic control problem is reviewed [2, 3, 4, 5]. Let $\left(X\left(t\right),t\ge 0\right)$ be the real-valued process that satisfies the stochastic differential equation

$$\begin{array}{lll}\mathit{dX}\left(t\right)& =& {\alpha }_{0}X\left(t\right)\mathit{dt}+\mathit{bU}\left(t\right)\mathit{dt}+\mathit{dB}\left(t\right)\\ X\left(t\right)& =& X_0\end{array}$$E9

where ${\alpha }_0,b,X_0$ are constants, $\left(B\left(t\right),t\ge 0\right)$ is a standard fractional Brownian motion with the Hurst parameter $H\in \left(1/2,1\right)$, ${\alpha }_0\in \left[a_1,a_2\right]$, where $a_2<0$, $b\in \mathbb{R}\backslash \left\{0\right\}$.

Consider the optimal control problem where the state $X\left(t\right)$ satisfies Eq. (9) and the ergodic (or average cost per unit time) cost function $J$ is

$$J\left(U\right)=\underset{T\to \infty }{limsup}\frac{1}{T}{\int }_0^T\left({\mathit{qX}}^2\left(t\right)+{\mathit{rU}}^2\left(t\right)\right)\mathit{dt}$$E10

where $q>0$ and $r>0$ are constants. The family $\mathcal{U}$ of admissible controls is all $\left({\mathcal{F}}_t\right)$-adapted processes such that Eq. (9) has one and only one solution.

Let $H\in \left(1/2,1\right)$ be fixed and $B\left(t\right)$ be a fractional Brownian motion with Hurst parameter $H$. For the applications in this paper, only a few results from a stochastic calculus are necessary. Let $f:\left[0,T\right]\to \mathbb{R}$ be a Borel measurable function. If $f$ satisfies the condition:

$${\left|f\right|}_{L_H^2}^2=\rho \left(H\right){\int }_0^T{\left(u_{1/2-H}\left(s\right)\left|I_{T-}^{H-1/2}\left(u_{H-1/2}f\right)\left(s\right)\right|\right)}^2\mathit{ds}<\infty$$E11

then $f\in L_H^2$ and ${\int }_0^{T}f\mathit{dB}$ is a zero mean Gaussian random variable with second moment

$$\mathbb{E}\left[{\left({\int }_0^{T}f\mathit{dB}\right)}^2\right]={\left|f\right|}_{L_H^2}^2$$E12

where $u_a\left(s\right)=s^a$ for $a>0$ and $s\ge 0$, $I_{T-}^{H-1/2}$ is a fractional integral defined almost everywhere and given by

$$\left(I_{T-}^{H-1/2}f\right)\left(x\right)=\frac{1}{\mathrm{\Gamma }\left(\alpha \right)}{\int }_x^T\frac{f\left(t\right)}{{\left(t-x\right)}^{3/2-H}}\mathit{dt}$$E13

for $x\in \left[0,T\right)$, $f\in L^1\left(\left[0,T\right]\right)$ and $\mathrm{\Gamma }\left(\cdot \right)$ is the gamma function and $\rho$ in Eq. (11) is

$$\rho \left(H\right)=\frac{H\mathrm{\Gamma }\left(H+1/2\right)\mathrm{\Gamma }\left(3/2-H\right)}{\mathrm{\Gamma }\left(2-2H\right)}$$E14

If ${\alpha }_0$ is unknown, then it is important to find a family of strongly consistent estimators of the unknown parameter ${\alpha }_0$ in Eq. (9). A method is used that is called pseudo-least squares because it uses the least-squares estimation for ${\alpha }_0$ assuming $H=\frac{1}{2}$, that is $B$, is then a standard Brownian motion. It can be shown that the family of estimators $\left(\widehat{\alpha }\left(t\right)\right),t\ge 0)$ is strongly consistent $H\in \left(1/2,1\right)$ for where

$$\widehat{\alpha }\left(t\right)={\alpha }_0+\frac{{\int }_0^t{X}^0\left(s\right)\mathit{dB}\left(s\right)}{{\int }_0^t{\left(X^0\left(s\right)\right)}^2\mathit{ds}}$$E15

and

$${\mathit{dX}}^0\left(t\right)={\alpha }_0{X}^0\left(t\right)\mathit{dt}+\mathit{dB}\left(t\right)$$E16

$$X^0\left(0\right)=X_0$$E17

An adaptive control $\left(U^{\wedge }\left(t\right),t\ge 0\right)$, is obtained from the certainty equivalence principle, that is, at time $t$, the estimate $\widehat{\alpha }\left(t\right)$ is assumed to be the correct value for the parameter. Thus, the stochastic equation for the system (9) with the control $U^{\wedge }$ is

$$\begin{array}{lll}{\mathit{dX}}^{\wedge }\left(t\right)& =& \left({\alpha }_0-\widehat{\alpha }\left(t\right)-\delta \left(t\right)\right)X^{\wedge }\left(t\right)\mathit{dt}\\ & & -\frac{\mathit{b\rho }\left(t\right)}{r}{V}^{\wedge }\left(t\right)\mathit{dt}+\mathit{dB}\left(t\right)\\ & =& \left(-{\alpha }_0-\widehat{\alpha }\left(t\right)-\delta \left(t\right)\right)X^{\wedge }\mathit{dt}\\ & & -\left(\alpha \left(t\right)+\delta \left(t\right)\right)V^{\wedge }\left(t\right)\mathit{dt}+\mathit{dB}\left(t\right)\\ X^{\wedge }\left(0\right)& =& X_0\end{array}$$E18

where

$$\begin{array}{lll}\delta \left(t\right)& =& {\left[{\alpha }^2\left(t\right)+\frac{b^2}{r}q\right]}^1/2\\ U^{\wedge }\left(t\right)& =& \frac{\mathit{b\rho }\left(t\right)}{r}\left[X^{\wedge }\left(t\right)+V^{\wedge }\left(t\right)\right]\\ \rho \left(t\right)& =& \frac{r}{b^2}\left[\alpha \left(t\right)+\delta \left(t\right)\right]\\ V^{\wedge }\left(t\right)& =& {\int }_0^t\delta \left(s\right)V^{\wedge }\left(s\right)\mathit{ds}\\ & & +{\int }_0^t\left[\tilde{k}\left(t,s\right)-1\right]\left[{\mathit{dX}}^{\wedge }\left(s\right)-\alpha \left(s\right)X^{\wedge }\left(s\right)\mathit{ds}-{\mathit{bU}}^{\wedge }\left(s\right)\mathit{ds}\right]\\ & =& {\int }_0^t\tilde{\delta }\left(s\right)V^{\wedge }\left(s\right)\mathit{ds}\\ & =& {\int }_0^t\left[\tilde{k}\left(t,s\right)-1\right]\left[\mathit{dB}\left(s\right)+\left({\alpha }_0-\alpha \left(t\right)\right)X^{\wedge }\left(s\right)\mathit{ds}\right]\\ \mathit{\text{and}}\\ \tilde{\delta }\left(t\right)& =& \delta \left(t\right)+\alpha \left(t\right)-{\alpha }_0\end{array}$$E19

The following result is obtained using the above [2].

Theorem 3.2. Let the scalar-valued control system satisfy the system equation given above. Let $\left({\alpha }^{\wedge }\left(t\right),t\ge 0\right)$ be the family of estimators of ${\alpha }_0$ given above, let $(U^{\wedge }\left(t\right)$ be the associated adaptive control given above and let $(X^{\wedge }\left(t\right)$ be the solution of the system with the control $U^{\wedge }$. Then the following limits exist:

$${\mathit{lim}}_{t\to \infty }\frac{1}{t}\mathbf{E}{\int }_{\mathit{0}}^{\mathbf{t}}{\left|{\mathbf{U}}^{\ast }\left(\mathbf{s}\right)-{\mathbf{U}}^{\wedge }\left(\mathbf{s}\right)\right|}^{\mathbf{2}}\mathbf{ds}=\mathbf{0}$$E20

$${\mathit{lim}}_{t\to \infty }\frac{1}{t}\mathbf{E}{\int }_{\mathbf{0}}^{\mathbf{t}}{\left|{\mathbf{X}}^{\ast }\left(\mathbf{s}\right)-{\mathbf{X}}^{\wedge }\left(\mathbf{s}\right)\right|}^{\mathbf{2}}\mathbf{ds}=\mathbf{0}$$E21

so

$${\mathit{lim}}_{t\to \infty }\frac{1}{t}\mathbf{E}{\int }_{\mathbf{0}}^{\mathbf{t}}\left(\mathbf{q}{\left({\mathbf{X}}^{\wedge }\left(\mathbf{s}\right)\right)}^{\mathbf{2}}+\mathbf{r}{\left({\mathbf{U}}^{\wedge }\left(\mathbf{s}\right)\right)}^{\mathbf{2}}\right)\mathbf{ds}=\lambda$$E22

where $\lambda$ is the optimal cost for the known system.

A family of estimators can be obtained from Eq. (9) by removing the control term. The family of estimators ${\alpha }^{\wedge }$ is modified here using the fact that ${\alpha }_0\in \left[a_1,a_2\right]$ as

$${\alpha }^{\wedge }\left(t\right)=\widehat{\alpha }\left(t\right){11}_{\left[a_1,a_2\right]}\left(\widehat{\alpha }\left(t\right)\right)$$E23

$$+a_1{11}_{\left(-\infty ,a_1\right)}\left(\widehat{\alpha }\left(t\right)\right)+a_2{11}_{\left(a_2,\infty \right)}\left(\widehat{\alpha }\left(t\right)\right)$$E24

For $t\ge 0$. $\widehat{\alpha }\left(0\right)$ is chosen arbitrarily in $\left[a_1,a_2\right]$. Then, it can be verified that a family of weighted least-squares estimates is strongly consistent for the unknown system parameter and that the family of running costs converges to the optimal cost. Thus, there is an optimal adaptive control for the system with a fractional Brownian motion. These results are summarized in the following theorem see T. E. Duncan et al. [2] for details and references.

Theorem 3.3. The family of estimators $\left({\alpha }^{\wedge }\left(t\right),t\ge 0\right)$ is strongly consistent for the unknown system parameter and that the family of running costs converge to the optimal cost. Thus there is an optimal adaptive control for the partially known linear system with a fractional Brownian motion.

4. Noise modeled by a Rosenblatt process

Since there is significant evidence from physical systems (Domanski [13]) that Gaussian processes are inadequate for modeling noise in interconnected physical control systems, the family of Rosenblatt processes is considered. Some of the evidence is industrial processes and actuators are non-linear, they are highly interconnected and they have real external disturbances.

Initially, some definitions of a Rosenblatt process [14], a related fractional Brownian motion and some differential operators are given that fix notation and that are used in a change of variables formula and the solution of questions of absolute continuity.

Some fractional Brownian motions are naturally associated with a Rosenblatt process. Similar to fractional Brownian motion, Rosenblatt processes are indexed by the Hurst parameter $H\in \left(0,1\right)$. Recall that both of these processes are members of the Hermite family of processes which are indexed by the number of multiple Wiener-Itô stochastic integrals. They are of order one and order two in the Hermite family reflecting the number of multiple Wiener-Itô integrals that are used to define them.

Let $h_1^H$ and $h_2^H$ be given as

$$h_1^H\left(u,y\right)={\left(u-y\right)}^{H-\frac{3}{2}}$$E25

$$h_2^H\left(u,y_1,y_2\right)={\left(u-y_1\right)}^{\frac{H}{2}-1}{\left(u-y_2\right)}^{\frac{H}{2}-1}$$E26

that are two singular kernels defined on the real line. These two kernels are used to define fractional Brownian motions and Rosenblatt processes. Rosenblatt processes like fractional Brownian motions are parametrized by the parameter $H$.

Definition 4.1. Let $H\in \left(1/2,1\right)$ be fixed. A real-valued fractional Brownian motion, $B_H=\left(B_H\left(t\right),t\in R\right)$, is defined as follows

$$B_H\left(t\right)=C_H^B{\int }_R\left({\int }_0^t{h}_1^H\left(u,y\right)\mathit{du}\right)\mathit{dW}\left(y\right)$$E27

For $t\ge 0$ (and similarly for $t<0$), where $C_H^B$ is a constant given below such that $\mathbb{E}\left(B_H^2\left(1\right)\right)=1$ and $W$ is a standard Wiener process (Brownian motion) on a fixed complete probability space denoted $\left(\mathrm{\Omega },\mathcal{F},\mathbb{P}\right)$ that is used throughout this paper.

Definition 4.2. Let $H\in \left(1/2,1\right)$ be fixed. A real-valued (standard) Rosenblatt process, $R_H=\left(R_H\left(t\right),t\in \mathbb{R}\right)$, is defined as follows

$$R_H\left(t\right)=C_H^R{\int }_{{\mathbb{R}}^2}\left({\int }_0^t{h}_2^H\left(u,y_1,y_2\right)\mathit{du}\right)\mathit{dW}\left(y_1\right)\mathit{dW}\left(y_2\right)$$E28

For $t\ge 0$ (and similarly for $t<0$), where $C_H^R$ is a constant such that $\mathbb{E}\left(R_H^2\left(1\right)\right)=1$, and the double stochastic integral is a Wiener-Itô multiple integral of order two with respect to the Wiener process (standard Brownian motion) $W$. This double Wiener integral is the definition given by Itô so the integral has expectation zero.

The normalizing constants $C_H^B$ and $C_H^R$ in the above two definitions are

$$C_H^B=\sqrt{\frac{H\left(2H-1\right)}{\beta \left(2-2H,H-\frac{1}{2}\right)}},C_H^R=\frac{\sqrt{2H\left(2H-1\right)}}{2\beta \left(1-H,\frac{H}{2}\right)}$$E29

where $\beta$ is the Beta function. For the subsequent change of variables (Itô-type) formula, it is also convenient to define the following constants

$$c_H^B=C_H^B\mathrm{\Gamma }\left(H-\frac{1}{2}\right),c_H^R=C_H^R{\mathrm{\Gamma }}^2\left(\frac{H}{2}\right),$$E30

and

$$c_H^{B,R}=\frac{c_R^H}{c_{\frac{H}{2}+\frac{1}{2}}^B}=\sqrt{\frac{\left(2H-1\right)}{\left(H+1\right)}\frac{\mathrm{\Gamma }\left(1-\frac{H}{2}\right)\mathrm{\Gamma }\left(\frac{H}{2}\right)}{\mathrm{\Gamma }\left(1-H\right)}}$$E31

where $\mathrm{\Gamma }$ is the Gamma function.

Some of the stochastic calculus for Rosenblatt processes is given in Refs. [11, 15]. A change of variables (Itô-Doeblin) formula is given in Čoupek et al. [11] that is used here to determine explicit Radon-Nikodym derivatives analogous to an approach for Wiener measure. The subsequent application of the change of variables formula for Rosenblatt processes uses the following two differential operators,

$${\nabla }^{\frac{H}{2}}=I_{+}^{\frac{H}{2}}D$$E32

$${\nabla }^{\frac{H}{2},\frac{H}{2}}=I_{+,+}^{\frac{H}{2},\frac{H}{2}}{D}^2.$$E33

where $D$ is the Malliavin derivative and the other terms are the following fractional integrals

$$I_{+}^{\alpha }\left(f\left(x\right)\right)={\int }_{-\infty }^{x}f\left(u\right){\left(x-u\right)}^{\alpha -1}\mathit{du}$$E34

$$\begin{array}{lll}\left(I_{+,+}^{{\alpha }_1,{\alpha }_2}f\right)\left(x_1,x_2\right)& =& \frac{1}{\mathrm{\Gamma }\left({\alpha }_1\right)\mathrm{\Gamma }\left({\alpha }_2\right)}{\int }_{-\infty }^{x_1}{\int }_{-\infty }^{x_2}f\left(u,v\right)\\ & & {\left(x_1-u\right)}^{{\alpha }_1-1}{\left(x_2-v\right)}^{{\alpha }_2-1}\mathrm{d}u\mathrm{d}v\end{array}$$E35

These operators reflect the singular integral definition of a Rosenblatt process.

It follows from the direct use of the differential operator (33) that

$$\left({\nabla }^{\frac{H}{2},\frac{H}{2}}{R}_H\left(t\right)\right)\left(u,u\right)={\tilde{C}}_H{\int }_0^t{\left|u-r\right|}^{2H-2}\mathit{dr}$$E36

where the constant ${\tilde{C}}_H$, is given by

$$\begin{array}{lll}{\tilde{C}}_H& =& 2c_H^R\frac{{\beta }^2\left(\frac{H}{2},1-H\right)}{{\mathrm{\Gamma }}^2\left(\frac{H}{2}\right)}\\ & =& \frac{1}{\mathrm{\Gamma }\left(\frac{H}{2}\right)}{\int }_0^u{\left(u-x\right)}^{\frac{H}{2}-1}{D}_{x}W\left(s\right)\mathit{dx}\end{array}$$E37

and $\beta$ is the Beta function.

The case where $t=u$ above is particularly useful here, so this case is explicitly given by

$$\begin{array}{lll}\left({\nabla }^{\frac{H}{2},\frac{H}{2}}{R}_H\left(t\right)\right)\left(t,t\right)& =& {\tilde{C}}_H{\int }_0^t{\left|t-r\right|}^{2H-2}\mathit{dr}={\tilde{C}}_H\frac{t^{2H-1}}{2H-1}\\ & =& \sqrt{\frac{2H}{2H-1}}\beta \left(\frac{H}{2},1-H\right)t^{2H-1}\end{array}$$E38

It is also necessary to note another derivative that appears in the change of variables formula associated with a Rosenblatt process.

$$\begin{array}{lll}\left({\nabla }^{\frac{H}{2}}W\left(s\right)\right)\left(u\right)& =& \frac{1}{\mathrm{\Gamma }\left(\frac{H}{2}\right)}{\int }_{-\infty }^u{\left(u-x\right)}^{\frac{H}{2}-1}{D}_{x}W\left(s\right)\mathit{dx}\end{array}$$E39

$$\begin{array}{lll}& =& \frac{1}{\mathrm{\Gamma }\left(\frac{H}{2}\right)}{\int }_{-\infty }^u{1}_{\left[0,s\right]}\left(x\right){\left(u-x\right)}^{\frac{H}{2}-1}\mathit{dx}\end{array}$$E40

$$\begin{array}{l}=\frac{2}{H\mathrm{\Gamma }\left(\frac{H}{2}\right)}\left[{\left(u-s\right)}_{+}^{\frac{H}{2}}-u_{+}^{\frac{H}{2}}\right]\end{array}$$E41

where $W\left(t\right)$ is a standard Wiener process.

The change of variables results for stochastic equations driven by Rosenblatt processes is described now following [11]. Consider a scalar process $\left(y\left(t\right),t\ge 0\right)$ for the description of the change of variables result. Let $H\in \left(\frac{1}{2},1\right)$ be fixed, $y_0\in \mathbb{R}$ be a constant, $\theta \in L_{\mathit{loc}}^1\left(0,\infty ;D^{2,2}\right)$, where $D^{2,2}$ is a Sobolev-Watanabe space, and define the process $\left(y\left(t\right),t\ge 0\right)$ by the equation

$$y\left(t\right)=y_0+{\int }_0^t\theta \left(s\right)\mathit{ds}+R_H\left(t\right).$$E42

If some natural conditions [6] are satisfied for each $t>0$, then the (scalar) process $\left(Y\left(t\right),t\ge 0\right)$ defined by $Y_t=f\left(t,y_t\right)$, where $f:{\mathbb{R}}_{+}\times \mathbb{R}\to \mathbb{R}$ is a smooth function and satisfies the following equation

$$Y_t=Y_0+{\int }_0^t{\tilde{\theta }}_s\mathit{ds}+2c_H^{B,R}{\int }_0^t{\tilde{\varphi }}_s{\mathit{dB}}_s^{\frac{H}{2}+\frac{1}{2}}+{\int }_0^t{\tilde{\psi }}_s{\mathit{dR}}_s^H$$E43

For each $t\ge 0$, (where)

$$\begin{array}{lll}{\tilde{\theta }}_s& =& \frac{\partial f}{\partial s}\left(s,y_s\right)+\frac{\partial f}{\partial x}\left(s,y_s\right){\theta }_s\\ & & +c_H^R\frac{{\partial }^{2}f}{\partial x^2}\left(s,y_s\right)\left({\nabla }^{\frac{H}{2},\frac{H}{2}}{y}_s\right)\left(s,s\right)\\ & & +c_H^R\frac{{\partial }^{3}f}{\partial x^3}\left(s,y_s\right){\left[\left({\nabla }^{\frac{H}{2}}{y}_s\right)\left(s\right)\right]}^2,\\ {\tilde{\varphi }}_s& =& \frac{{\partial }^{2}f}{\partial x^2}\left(s,y_s\right)\left({\nabla }^{\frac{H}{2}}{y}_s\right)\left(s\right),\\ {\tilde{\psi }}_s& =& \frac{\partial f}{\partial x}\left(s,y_s\right).\end{array}$$E44

It is useful to note that a third derivative occurs in the drift term and that a stochastic integral with respect to a fractional Brownian motion occurs.

As is the case for Brownian motion, this change of variables formula plays an important role in many stochastic calculations for a Rosenblatt process.

To indicate the ability to obtain explicit solutions for a Rosenblatt process, the explicit solution of an ergodic control problem for a scalar linear system is given, see P. Čoupek et al. [6] for details and references.

$$\begin{array}{lll}\mathit{dX}\left(t\right)& =& \mathit{aX}\left(t\right)\mathit{dt}+\mathit{bU}\left(t\right)\mathit{dt}+{\mathit{dR}}^H\left(t\right)\\ X\left(0\right)& =& x_0\end{array}$$E45

The solution of this scalar stochastic system is

$$X_t=x_0+{\int }_0^t\left({\mathit{aX}}_s+{\mathit{bU}}_s\right)\mathit{ds}+R_t^H$$E46

where $R^H$ is a Rosenblatt process with $H\in \left(\frac{1}{2},1\right)$ and $a,b,x_0\in \mathbb{R}$, $b\ne 0$, are known constants.

The family of admissible controls, $\mathcal{U}$, is the collection of constant scalar linear feedback operators $K$, that is,

$$\mathcal{U}=\left\{{\left(U_t\right)}_{t\ge 0}:U_t={\mathit{KX}}_t\text{with}K\in \mathbb{R}\right\},$$E47

The following result is given in P. Čoupek et al. [6]:

Theorem. The optimal gain $\widehat{K}$ in the family of admissible feedbacks is given by

$$\widehat{K}=-\frac{a+\sqrt{a^2+4H\left(1-H\right)\frac{b^{2}q}{r}}}{2b\left(1-H\right)},$$E48

and the optimal cost $J_{\infty }\left(\widehat{K}\right)$ is given by

$$J_{\infty }\left(\widehat{K}\right)=\frac{\mathrm{\Gamma }\left(2H\right)}{{\left[-\left(a+b\widehat{K}\right)\right]}^{2H-1}}\left(\frac{-r\widehat{K}}{b}\right).$$E49

The verification of this result for the optimal cost and an optimal control for a Rosenblatt noise is significantly more difficult than the corresponding results by replacing the Rosenblatt noise by a Brownian motion or a fractional Brownian motion, see Čoupek et al. [6] for details and many useful references.

5. Some future work

There are a number of natural directions to proceed for subsequent investigation. For multidimensional systems with either fractional Brownian motions or Rosenblatt processes, there is the basic problem of identification of unknown parameters. The basic parameters for estimation are those from the linear transformations $A$ in the controlled equations for systems with either fractional Brownian motion or Rosenblatt noise or a combination of these noise processes. Furthermore, there are questions of optimal control for linear or nonlinear systems with fractional Brownian motions or Rosenblatt processes. Even an ergodic control problem for a scalar linear system with a Rosenblatt process with unknown system parameters needs to be addressed. Another family of models is those for infinite dimensional systems, especially for modeling some types of partial differential equations (PDEs). Many physical models are described by stochastic partial differential especially parabolic differential equations of second order. These equations often describe the physical situation of diffusion. Classically, a Brownian motion was developed to describe the diffusion of a liquid or gas. It seems quite likely that stochastic processes, such as fractional Brownian motions and Rosenblatt processes, will provide some of the mathematics for analysis of other physical phenomena.

6. Conclusions

This paper provides a comparison of results for adaptive control for linear systems driven by a Brownian motion, a fractional Brownian motion, and a Rosenblatt process. The results for a Brownian motion represent the initial noise process in a linear system and the results are fairly well developed. While Brownian motion for linear systems has been well developed over many years, the results for a fractional Brownian motion are significantly less developed. While these latter processes are Gaussian because they are Brownian motion processes, the fractional Brownian motion processes are not martingales and have a long-range dependence. These properties make it more difficult for analysis as contrasted with Brownian motion and their martingale properties. The Rosenblatt processes introduce more difficulties for analysis because they are neither martingales nor Gaussian processes. The development of useful tools for a Rosenblatt process is still in its infancy. Nevertheless, they are important and provide insight to the family of Hermite processes of which fractional Brownian motions are Hermite of order one and Rosenblatt processes are of order two.

Remark. For more background on Rosenblat‘’s processes, see Refs. [14, 15].

Acknowledgments

The authors thank F. Leve and F. Fahroo program directors at AFOSR for their support. Furthermore, the authors acknowledge many colleagues who have collaborated with the authors on identification and adaptive control, such as H. F. Chen, P. Coupek, L. Guo, Y. Hu, J. Jakubowski, B. Maslowski, L. Stettner, and many other colleagues and students.