Some Remarks on Vector Markov Chains and Their Applications to the Description of Many-Particle Systems

Abstract

Although in the literature on the topic, this point is never expressed, but regularly implied, an important and widely studied scheme of models to describe and predict, at least in stochastic terms, the behavior of many-particle systems is based on Markov Chains. In fact, Markov Chains are special stochastic processes with important properties that have revealed to be shared with many natural processes. This fact, though almost obvious, requires to be carefully discussed, at least to point out that the very notion of a Markov Chain must be completed and somehow generalized to understand the way in which Markov Chains intervene in the equations describing the stochastic behavior of any system. This is not a purely critical and “foundational” task but aims to develop a new very plausible and likely way to introduce possible effects of external world on a non-isolated many-particle system. This paper aims to offer at least a basic discussion of this perspective.

1. Introduction

The aim of the present chapter is twofold.

On one hand, we want to consider explicitly some properties of a kind of Markov Chains [1, 2, 3], which is of course taken implicitly into account in the whole literature about these important stochastic processes but—as far as we are aware—has been never examined explicitly, that is, the Markov Chains whose states are n-tuples of integers or of real numbers, which can be called vector Markov Chains. In particular, we will also pay attention in particular to nonstationary Markov Chains, which we recall in some details in Section 1.2.

On the other hand, we want to examine the equations of kinetic-theoretic models [4] that since almost forty years seem to be diffusing ever more in the research about the evolution of many-particle systems and appear to be a particularly effective, expressive, and versatile tool to describe in stochastic terms such evolution [5, 6, 7, 8, 9, 10]. Rather strange to say, these equations are firmly based on the use of Markov Chains, since in them a basic rôle is played by transition matrices, that just characterize these important stochastic processes, but in the literature about the kinetic-theoretic models Markov Chains are never even mentioned (at least to our knowledge). The researchers in this field probably consider this reference superfluous. But the question spontaneously arises whether if we exploit the rôle of Markov Chains then our understanding of the terms of equations could be improved, suggesting to us some modifications that could help us to produce a good and effective description of the behavior of systems that have till now escaped the classical equations used up to the last few years (in this connection, see [11, 12], where an accurate stochastic description of the evolution of a system is obtained renouncing the Principle of Inertia, and [13, 14, 15] where new perspectives about the description of non-isolated systems have been obtained by introducing «forcing» terms in the equations).

Accordingly, after giving some definitions recalling some types of Markov Chains and introducing explicitly the vector Markov Chains, of which we shall discuss some basic properties that could be useful to understand the rôle of transition matrices in the equations of the kinetic-theoretic models, we shall first recall the structure of these latter, giving a critical description of the terms involved therein, and finally, we will suggest the ways in which, in virtue of our knowledge of the complete meaning of the transition probabilities, and in view of catching new natural processes for non-isolated systems, the transition probabilities (together with the so-called encounter rates, see Sections 1.6 and 1.7) can be modified.

2. Some background about Markov Chains

As is well-known, a random process [2] is a sequence ${\left\{X_{\eta }\right\}}_{\eta \in \mathrm{ℍ}}$ of random variables with a common range S of possible values (the states) called the state space of the process. Both $\mathrm{ℍ}$ and S can be either discrete or continuous sets. If $\mathrm{ℍ}$ is discrete, that is $\mathrm{ℍ}=\mathrm{\mathbb{N}}=\left\{\mathrm{0,1,2},\dots ,n,\dots \right\}$, then the process is said to be time-discrete, while, if $\mathrm{ℍ}$ is continuous, that is, $\mathrm{ℍ}=\left[0,+\infty \right)$, then the process is said to be time-continuous. Analogously if S is discrete, that is, $S={\left\{z_h\right\}}_{h\in \mathbb{I}}\subseteq \mathrm{ℝ}$ with $\mathbb{I}\subseteq \mathrm{\mathbb{Z}}$, then the process is said to be discrete, while, if S is continuous (it is any real interval), then the process is said to be continuous. From now to the end of this section, we shall first consider discrete and time-discrete processes, and at a second step time-continuous discrete processes. In both cases, for the sake of simplicity, the state space S will be assumed to be finite. In this choice, there is no loss of generality, and we only avoid to write matrices with infinitely many rows and columns.

Consider then a discrete and time-discrete random process ${\left\{X_h\right\}}_{h\in \mathrm{\mathbb{N}}}$ with $S=\left\{z_1,z_2,\dots ,z_n\right\}$. . As usual, we shall denote by a vector ${\mathbf{p}}_h$ (called the state vector of the process at time h) the (absolute) probability distribution on S according to the random variable $X_h$. In other words, ${\mathbf{p}}_h\equiv \left(p_{h,1},p_{h,2},\dots ,p_{h,n}\right)$ with $p_{h,k}=\mathbf{P}\left(X_h=z_k\right)$.¹

Now, a discrete time-discrete Markov Chain ${\left\{X_h\right\}}_{h\in \mathrm{\mathbb{N}}}$, with $R\left(X_h\right)=S\subset \mathrm{\mathbb{Z}}$, is a random process such that the Markov condition

$$\begin{array}{l}\mathbf{P}\left(X_h=z_h|X_1=z_1,X_2=z_2,\dots ,X_{h-1}=z_{h-1}\right)=\\ =\mathbf{P}\left(X_h=z_h|X_{h-1}=z_{h-1}\right),\forall h\in \mathrm{\mathbb{N}}\backslash \left\{0\right\}\end{array}$$E1

holds (where we have used the usual symbol $\mathbf{P}\left(A|B\right)$ to denote the conditional probability of an event A under the assumption that an event B has taken place). This means that, for any $h\in \mathrm{\mathbb{N}}$, the values of the h-th random variable of the process depend only on the values of the (h – 1)-th, not on the values of any previous variable. Now, setting $P_{\mathit{ij}}\left(h\right)=\mathbf{P}\left(X_h=z_j|X_{h-1}=z_i\right)$ (with $\left(i,j\right)\in {\left\{1,2,\dots ,n\right\}}^2$), $P_{\mathit{ij}}\left(h\right)$ is the transition probability from state $z_i$ to state $z_j$ at the h-th step, and the matrix

$${\mathbf{P}}_h\equiv {\left(P_{\mathit{ij}}\left(h\right)\right)}_{1\le i,j\le n}\equiv \left(\begin{array}{cccc}{P}_{11}\left(h\right)& P_{12}\left(h\right)& \dots & P_{1n}\left(h\right)\\ P_{21}\left(h\right)& P_{22}\left(h\right)& \dots & P_{2n}\left(h\right)\\ ⋮& ⋮& \ddots & ⋮\\ P_{n1}\left(h\right)& P_{n2}\left(h\right)& \dots & P_{\mathit{nn}}\left(h\right)\end{array}\right),\forall h\in \mathrm{\mathbb{N}}\backslash \left\{0\right\}.$$E2

is called the transition matrix at h-th step. It is well-known (and also obvious) that

$$\sum _{j=1}^n{P}_{\mathit{ij}}\left(h\right)=1,\forall h\in \mathrm{\mathbb{N}}\backslash \left\{0\right\}$$E3

and, in virtue of the law of alternatives,

$${\mathbf{p}}_h={\mathbf{p}}_{h-1}{\mathbf{P}}_h,\forall h\in \mathrm{\mathbb{N}}\backslash \left\{0\right\},$$E4

where ${\mathbf{p}}_h$ is a row vector for any h, so that the product at right-hand side is a row-by-column product. For the sake of completeness, we also recall that, for any couple (r, s) of nonnegative integers,

$${\mathbf{p}}_{r+s}={\mathbf{p}}_r{\mathbf{P}}_{r+1}{\mathbf{P}}_{r+2}\dots {\mathbf{P}}_{r+s},$$E5

and in particular, when the Markov Chain is stationary, that is, a transition matrix P exists such that ${\mathbf{P}}_h=\mathbf{P}$ for any $h\in \mathrm{\mathbb{N}}\backslash \left\{0\right\}$,

$${\mathbf{p}}_{r+s}={\mathbf{p}}_r{\mathbf{P}}^s,$$E6

where the power at right-hand side must be interpreted in the sense of the row-by-column product of matrices (for further details, see e.g. [2, 3]).

3. Joint and marginal transition probabilities

Now, bearing in mind these premises, we want to observe that in the literature about Markov Chains, the structure of the state space S is usually completely disregarded. With only one exception at our knowledge [16], it is simply assumed to be an ordered set, mainly to be allowed to write simple transition matrices. But we want to ask ourselves which new features should be taken into account if we considered some kinds of structure, and how should develop the study. In particular, we want to examine the case in which $S={\mathrm{\Omega }}^m$ (where $\mathrm{\Omega }=\left\{x_1,x_2,\dots ,x_l\right\}$ so that $\mid {\mathrm{\Omega }}^m\mid =l^m$ and ${\mathbf{z}}_i\equiv \left(x_{i_1},x_{i_2},\dots ,x_{i_m}\right)$ for any $i\in \left\{1,2,\dots ,n\right\}$ and $\left(i_1,i_2,\dots ,i_m\right)\in {\left\{1,2,\dots ,l\right\}}^m$) and the process on consideration should be described as a sequence ${\left\{\left(X_{h,1},X_{h,2},\dots ,X_{h,m}\right)\right\}}_{h\in \mathrm{\mathbb{N}}}$, which will be written in vector notation as ${\left\{{\mathbf{X}}_h\right\}}_{h\in \mathrm{\mathbb{N}}}$. In such a case, we should consider the transition probabilities

$$\begin{array}{l}\mathbf{P}\left({\mathbf{X}}_h={\mathbf{z}}_h|{\mathbf{X}}_{h-1}={\mathbf{z}}_{h-1}\right)=\\ =\mathbf{P}\left(X_{h,1}=x_{h,1},\dots ,X_{h,m}=x_{h,m}|X_{h-1,1}=x_{h-1,1},\dots ,X_{h-1,m}=x_{h-1,m}\right)\end{array}$$E7

that have 2m indexes, that is, we can set

$$\begin{array}{l}\mathbf{P}\left(X_{h,1}=x_{j_1},\dots ,X_{h,m}=x_{j_m}|X_{h-1,1}=x_{i_1},\dots ,X_{h-1,m}=x_{i_m}\right)\equiv \\ \equiv P_{i_1,i_2,\dots ,i_m,j_1,j_2,\dots ,j_m}\left(h\right).\end{array}$$E8

So, for any $h\in \mathrm{\mathbb{N}}$, the transition matrix ${\mathbf{P}}_h$ at h-th time is a 2m-dimensional matrix. It will be called the joint transition matrix, and each of its elements $P_{i_1,i_2,\dots ,i_m,j_1,j_2,\dots ,j_m}\left(h\right)$ expresses the probability that the chain passes from the state $\left(x_{i_1},\dots ,x_{i_m}\right)$ to the state $\left(x_{j_1},\dots ,x_{j_m}\right)$ at h-th time. These will be called the joint transition probabilities.

But, of course, together with these joint probabilities, we are now allowed to consider also the many different marginal probabilities. More precisely, for any $h\in N$, we can choose $r<m$ of the random variables $X_{h,1},X_{h,2},\dots ,X_{h,m}$, say $X_{h,k_1},X_{h,k_2},\dots ,X_{h,k_r}$ and look for the probabilities $\mathbf{P}\left(X_{h,k_1}=x_{j_{k_1}},\dots ,X_{h,k_r}=x_{j_{k_r}}|X_{h-1,1}=x_{i_1},\dots ,X_{h-1,m}=x_{i_m}\right)$. For any choice, we have a $\left(r+m\right)$-dimensional matrix with $l^{r+m}$ entries. Of course, the matrices are $\left(\begin{array}{l}n\\ r\end{array}\right)$. We shall denote by $P_{i_1,i_2,\dots ,i_m,j_{k_1},j_{k_2},\dots ,j_{k_r}}\left(h\right)$ the entries of each of such matrices. It should also be noted that, setting $s=m-r$, and agreeing to denote by $X_{h,{\mathrm{\ell }}_1},X_{h,{\mathrm{\ell }}_2},\dots ,X_{h,{\mathrm{\ell }}_s}$ the remaining s variables, we have

$${\mathit{P}}_{i_1,i_2,\dots ,i_m,j_{k_1},j_{k_2},\dots ,j_{k_r}}\left(h\right)=\sum _{j_{{\mathrm{\ell }}_1},\dots ,j_{{\mathrm{\ell }}_s}}^{1\dots n}{\mathit{P}}_{i_1,i_2,\dots ,i_m,j_{k_1},\dots ,j_{k_r},j_{{\mathrm{\ell }}_1},\dots ,j_{{\mathrm{\ell }}_s}}\left(h\right)$$E9

Now, for the sake of simplicity, assume that $k_1=1,k_2=2,\dots ,k_r=r$ and ${\mathrm{\ell }}_1=r+1,{\mathrm{\ell }}_2=r+2,\dots ,{\mathrm{\ell }}_s=m$. Then, we shall also agree to set $S={\left\{{\mathbf{z}}_{\mathit{ij}}\right\}}_{\begin{array}{l}1\le i\le p\\ 1\le j\le q\end{array}}\equiv {\mathrm{\Omega }}^r\times {\mathrm{\Omega }}^s\equiv {\left\{\left({\mathit{x}}_i,{\mathit{y}}_j\right)\right\}}_{\begin{array}{l}1\le i\le p\\ \begin{array}{c}1\le j\le q\end{array}\end{array}}\left(p=l^r,q=l^s\right)$, where $r+s=m$, $x_i\equiv \left(x_i^1,x_i^2,\dots ,x_i^r\right)$ and ${\mathit{y}}_j\equiv \left(x_j^{r+1},x_j^{r+2},\dots ,x_j^m\right)$². Then, we can write relation (9) in the form

$$P_{{\mathbf{z}}_{\mathit{hk}},{\mathit{x}}_i}\left(h\right)=\sum _{j=1}^q{P}_{{\mathbf{z}}_{\mathit{hk}},{\mathbf{z}}_{\mathit{ij}}}\left(h\right)=\sum _{j=1}^q{P}_{{\mathbf{z}}_{\mathit{hk}},\left({\mathit{x}}_i,{\mathit{y}}_j\right)}\left(h\right),$$E10

and introduce the joint state vector ${\mathbf{p}}_h\equiv \left(p_h^{\mathit{ij}}\right)$, with $p_h^{\mathit{ij}}\equiv p_h\left({\mathbf{z}}_{\mathit{ij}}\right)\equiv p_h\left({\mathit{x}}_i,{\mathit{y}}_j\right)$, and the marginal state vector ${\mathbf{p}}_{\mathit{x},h}\equiv {\left(p_{\mathit{x},h}\left({\mathit{x}}_i\right)\right)}_{1\le i\le p}$, where

$$p_{\mathit{x},h}\left({\mathit{x}}_i\right)=\sum _{j=1}^q{p}_h\left({\mathbf{z}}_{\mathit{ij}}\right)=\sum _{j=1}^q{p}_h\left({\mathit{x}}_i,{\mathit{y}}_j\right)$$E11

and, in virtue of the law of alternatives

$$p_h\left({\mathbf{z}}_{\mathit{ij}}\right)=\sum _{k=1}^p\sum _{l=1}^q{p}_{h-1}\left({\mathbf{z}}_{\mathit{kl}}\right)P_{{\mathbf{z}}_{\mathit{kl}},{\mathbf{z}}_{\mathit{ij}}}\left(h\right).$$

Hence, replacing this last relation in relation (11), and taking into account relation (10), we get

$$\begin{array}{l}{p}_{\mathit{x},h}\left({\mathit{x}}_i\right)=\sum _{j=1}^q\sum _{k=1}^p\sum _{l=1}^q{p}_{h-1}\left({\mathit{x}}_k,{\mathit{y}}_l\right)P_{{\mathbf{z}}_{\mathit{kl}},\left({\mathit{x}}_i,{\mathit{y}}_j\right)}\left(h\right)=\\ =\sum _{k=1}^p\sum _{l=1}^q{p}_{h-1}\left({\mathit{x}}_k,{\mathit{y}}_l\right)\left(\sum _{j=1}^q{P}_{{\mathbf{z}}_{\mathit{kl}},\left({\mathit{x}}_i,{\mathit{y}}_j\right)}\left(h\right)\right)=\\ =\sum _{k=1}^p\sum _{l=1}^q{p}_{h-1}\left({\mathit{x}}_k,{\mathit{y}}_l\right)P_{{\mathbf{z}}_{\mathit{kl}},{\mathit{x}}_i}\left(h\right).\end{array}$$

Now, for reasons that will be clear in the sequel, we write this last equation in the form

$$p_{\mathit{x},h}\left({\mathit{x}}_i\right)-p_{\mathit{x},h-1}\left({\mathit{x}}_i\right)=\sum _{k=1}^p\sum _{l=1}^q{p}_{h-1}\left({\mathbf{z}}_{\mathit{kl}}\right)P_{{\mathbf{z}}_{\mathit{kl}},{\mathit{x}}_i}\left(h\right)-p_{\mathit{x},h-1}\left({\mathit{x}}_i\right).$$E12

In addition, for any $l\in \left\{1,2,\dots ,q\right\},$

$$\sum _{i^{\prime }=1}^p\sum _{l^{\prime }=1}^q{P}_{\left({\mathit{x}}_i,{\mathit{y}}_l\right),\left({\mathit{x}}_{i^{\prime }},{\mathit{y}}_{l^{\prime }}\right)}\left(h\right)=1,$$

so that we can rewrite equation (12) in the final form

$$\begin{array}{c}\begin{array}{l}{p}_{\mathit{x},h}\left({\mathit{x}}_i\right)-p_{\mathit{x},h-1}\left({\mathit{x}}_i\right)=\sum _{l=1}^q\left\{\sum _{k\ne i}^{1\dots p}{p}_{h-1}\left({\mathbf{z}}_{\mathit{kl}}\right)P_{{\mathbf{z}}_{\mathit{kl}},{\mathit{x}}_i}\left(h\right)+\right.\end{array}-\left.p_{h-1}\left({\mathbf{z}}_{\mathit{il}}\right)\sum _{i^{\prime }\ne i}^{1\dots p}{P}_{\left({\mathit{x}}_i,{\mathit{y}}_l\right),{\mathit{x}}_{i^{\prime }}}\left(h\right)\right\}.\end{array}$$E13

Finally, if for any $h\in \mathrm{\mathbb{N}}$, the random variables $X_{h,i}$ and $X_{h,j}$ are independent for any couple (i, j), and the chain is stationary, then we find

$$\begin{array}{l}{p}_{\mathit{x},h}\left({\mathit{x}}_i\right)-p_{\mathit{x},h-1}\left({\mathit{x}}_i\right)=\sum _{l=1}^q\left\{\sum _{k\ne i}^{1\dots p}{p}_{\mathit{x},h-1}\left({\mathit{x}}_k\right)p_{\mathit{y},h-1}\left({\mathit{y}}_l\right)P_{{\mathbf{z}}_{\mathit{kl}},{\mathit{x}}_i}+\right.-\left.p_{\mathit{x},h-1}\left({\mathit{x}}_i\right)\sum _{k\ne i}^{1\dots p}{p}_{\mathit{y},h-1}\left({\mathit{y}}_l\right)P_{{\mathbf{z}}_{\mathit{il}},{\mathit{x}}_k}\right\}.\end{array}$$E14

The first term at right-hand side will be called the gain term of the subset of states $S_i={\left\{{\mathbf{z}}_{\mathit{il}}\right\}}_{1\le l\le q}$ since it accounts for all the possible transitions from other states to a state of $S_i$, while the last term is called the loss term of $S_i$ since it accounts for all the possible transitions from a state of $S_i$ to any state outside $S_i$.

4. Time-continuous vector Markov Chains

We now want to turn our attention to the case of time-continuous Markov Chains. But before treating that case, we want first to consider the random process ${\left\{\left({\mathbf{X}}_h,{\chi }_h\right)\right\}}_{h\in \mathrm{\mathbb{N}}}$, where, for any $h\in \mathrm{\mathbb{N}}$, ${\chi }_h$ is the classical Bernoulli variable, with range $R\left({\chi }_h\right)=\left\{0,1\right\}$. The variables ${\chi }_h$ are independent and identically distributed, with $\mathbf{P}\left({\chi }_h=0\right)=v$ and $\mathbf{P}\left({\chi }_h=1\right)=u$ for any $h\in \mathrm{\mathbb{N}}$. For any $h\in \mathrm{\mathbb{N}}$, ${\mathbf{X}}_h$, the state vector ${\mathbf{p}}_h$ and the transition matrix ${\mathbf{P}}_h$ are assumed to depend only on ${\chi }_h$. So, using again the notation introduced above, we see at once that $p_h\left({\mathbf{z}}_{\mathit{ij}}\right)=\mathbf{P}\left({\mathbf{X}}_h={\mathbf{z}}_{\mathit{ij}},{\chi }_h=1\right)+\mathbf{P}\left({\mathbf{X}}_h={\mathbf{z}}_{\mathit{ij}},{\chi }_h=0\right)$, that is,

$$p_h\left({\mathbf{z}}_{\mathit{ij}}\right)=\mathbf{P}\left({\mathbf{X}}_h={\mathbf{z}}_{\mathit{ij}}|{\chi }_h=1\right)u+\mathbf{P}\left({\mathbf{X}}_h={\mathbf{z}}_{\mathit{ij}}|{\chi }_h=0\right)v.$$E15

Analogously, for any $h\in \mathrm{\mathbb{N}}$, the transition matrix at h-th step (or time) should be written according to the condition

$$P_{{\mathbf{z}}_{\mathit{ij}},{\mathbf{z}}_{\mathit{kl}}}\left(h\right)=P_{{\mathbf{z}}_{\mathit{ij}},{\mathbf{z}}_{\mathit{kl}}\mid {\chi }_h=1}\left(h\right)u+P_{{\mathbf{z}}_{\mathit{ij}},{\mathbf{z}}_{\mathit{kl}}\mid {\chi }_h=0}\left(h\right)v,$$

where $P_{{\mathbf{z}}_{\mathit{ij}},{\mathbf{z}}_{\mathit{kl}}\mid {\chi }_h=1}\left(h\right)$ is the transition probability from the state ${\mathbf{z}}_{\mathit{ij}}$ to the state ${\mathbf{z}}_{\mathit{kl}}$ at h-th step under the assumption ${\chi }_h=1$, and $P_{{\mathbf{z}}_{\mathit{ij}},{\mathbf{z}}_{\mathit{kl}}\mid {\chi }_h=0}\left(h\right)$ is the transition probability from the state ${\mathbf{z}}_{\mathit{ij}}$ to the state ${\mathbf{z}}_{\mathit{kl}}$ at h-th step under the assumption ${\chi }_h=0$. In particular, we shall set

$$P_{{\mathbf{z}}_{\mathit{ij}},{\mathbf{z}}_{\mathit{kl}}\mid {\chi }_h=0}\left(h\right)={\delta }_k^i{\delta }_l^j,\forall h\in \mathrm{\mathbb{N}}:$$E16

this means that the chain remains in the state occupied at time h and also at time h + 1 when ${\chi }_h=0$. Also, notice that the Markov Chain is stationary if and only if the matrix $P_{{\mathbf{z}}_{\mathit{ij}},{\mathbf{z}}_{\mathit{kl}}\mid {\chi }_h=1}$ is constant with respect to h.

By replacing the well-known relations

$$\begin{array}{l}\mathbf{P}\left({\mathbf{X}}_{h+1}={\mathbf{z}}_{\mathit{ij}}|{\chi }_h=1\right)=\sum _{k=1}^p\sum _{l=1}^q{p}_h\left({\mathbf{z}}_{\mathit{kl}}\right)P_{{\mathbf{z}}_{\mathit{kl}},{\mathbf{z}}_{\mathit{ij}}\mid {\chi }_h=1}\left(h\right),\\ \mathbf{P}\left({\mathbf{X}}_{h+1}={\mathbf{z}}_{\mathit{ij}}|{\chi }_h=0\right)=\sum _{k=1}^p\sum _{l=1}^q{p}_h\left({\mathbf{z}}_{\mathit{kl}}\right)P_{{\mathbf{z}}_{\mathit{kl}},{\mathbf{z}}_{\mathit{ij}}\mid {\chi }_h=0}\left(h\right),\end{array}$$

in equation (15), and taking into account assumption (16) and the obvious relation v = 1 − u, we get

$$p_{h+1}\left({\mathbf{z}}_{\mathit{ij}}\right)-p_h\left({\mathbf{z}}_{\mathit{ij}}\right)=\sum _{k=1}^p\sum _{l=1}^q{p}_h\left({\mathbf{z}}_{\mathit{kl}}\right)P_{{\mathbf{z}}_{\mathit{kl}},{\mathbf{z}}_{\mathit{ij}}\mid {\chi }_h=1}\left(h\right)u-p_h\left({\mathbf{z}}_{\mathit{ij}}\right)u.$$E17

These are of course pq equations, one for any couple $\left({\mathit{x}}_i,{\mathit{y}}_j\right)$. By taking the sum over j of both sides, we obtain the following system of p equations:

$$\begin{array}{l}{p}_{\mathit{x},h+1}\left({\mathit{x}}_i\right)-p_{\mathit{x},h}\left({\mathit{x}}_i\right)=\\ =\sum _{k=1}^p\sum _{l=1}^q{p}_h\left({\mathbf{z}}_{\mathit{kl}}\right)P_{{\mathbf{z}}_{\mathit{kl}},{\mathit{x}}_i\mid {\chi }_h=1}\left(h\right)u-p_{\mathit{x},h}\left({\mathit{x}}_i\right)u,\end{array}\left(i=1,2,\dots ,p\right).$$E18

Finally, it is readily seen that, under the independence assumption on the random variables $X_{h,i}$ and $X_{h,j}$ for any h, we obtain the system

$$\begin{array}{l}{p}_{\mathit{x},h+1}\left({\mathit{x}}_i\right)-p_{\mathit{x},h}\left({\mathit{x}}_i\right)=\\ =u\sum _{l=1}^q\left\{\sum _{k\ne i}^{1\dots p}{p}_{\mathit{x},h}\left({\mathit{x}}_k\right)p_{\mathit{y},h}\left({\mathit{y}}_l\right)P_{{\mathbf{z}}_{\mathit{kl}},{\mathit{x}}_i\mid \chi =1}+\right.\\ -\left.p_{\mathit{x},h}\left({\mathit{x}}_i\right)\sum _{k\ne i}^{1\dots p}{p}_{\mathit{y},h}\left({\mathit{y}}_l\right)P_{{\mathbf{z}}_{\mathit{il}},{\mathit{x}}_k\mid \chi =1}\right\},\end{array}\left(i=1,2,\dots ,p\right).$$E19

if the transition matrix ${\mathbf{P}}_{\chi =1}\equiv \left(P_{{\mathbf{z}}_{\mathit{ij}},{\mathbf{z}}_{\mathit{kl}}\mid \chi =1}\right)$ is stationary.

We are now ready to consider the case of a time-continuous Markov Chain analogous to the one; we have just described, namely a random process of the type ${\left\{\left({\mathbf{X}}_t,{\chi }_t\right)\right\}}_{t\in \left[0,T\right)}$, where $T>0$ may be either finite or infinite. In this case, the state vectors obviously depend on time, but the transition probabilities may depend on time or not, as well as the probabilities associated to the Bernoulli variables ${\chi }_t$. But, similarly to what has been done above, these variables will be assumed to be identically distributed. In this connection, we have to recall to the reader that it is impossible to assign a positive probability u such that $\mathbf{P}\left({\chi }_t=1\right)=u$ for any t because in such case the probability that at least one of the variables ${\chi }_t$ could take the value 1 for t in any interval contained in [0, T) would be infinite. What we can do is to assign a constant probability density $\tau$ such that $\tau \mathrm{\Delta }t$ is the probability to find in any interval of length $\mathrm{\Delta }t$ some points such that ${\chi }_t=1$. Roughly speaking and referring to the interpretation of probability as relative frequency, we can state that, for any $s\in \left[0,T\right)$, $\tau \mathrm{\Delta }t$ is the length of the set $\left\{t\in \left[s,s+\mathrm{\Delta }t\right]|{\chi }_t=1\right\}$ so that $\tau$ is the ratio of that length to the length $\mathrm{\Delta }t$. Accordingly, $\tau$ is the measure of the set of points for which ${\chi }_t=1$ in any interval of unit length.

This stated, we shall write system (19) as follows:

$$\begin{array}{l}{p}_{\mathit{x},t+\mathrm{\Delta }t}\left({\mathit{x}}_i\right)-p_{\mathit{x},t}\left({\mathit{x}}_i\right)=\\ =\tau \mathrm{\Delta }t\sum _{l=1}^q\left\{\sum _{k\ne i}^{1\dots p}{p}_{\mathit{x},t}\left({\mathit{x}}_k\right)p_{\mathit{y},t}\left({\mathit{y}}_l\right)P_{{\mathbf{z}}_{\mathit{kl}},{\mathit{x}}_i\mid \chi =1}+\right.\\ -\left.p_{\mathit{x},t}\left({\mathit{x}}_i\right)\sum _{k\ne i}^{1\dots p}{p}_{\mathit{y},t}\left({\mathit{y}}_l\right)P_{{\mathbf{z}}_{\mathit{il}},{\mathit{x}}_k\mid \chi =1}\right\}.\end{array}\left(i=1,2,\dots ,p\right),$$E20

where $\tau \mathrm{\Delta }t$ replaces the probability u in this continuous framework.

Next, dividing both sides by $\mathrm{\Delta }t$ and letting $\mathrm{\Delta }t\to 0$, we get

$$\begin{array}{l}\frac{p_{\mathit{x},t}}{t}\left({\mathit{x}}_i\right)=\tau \sum _{l=1}^q\left\{\sum _{k\ne i}^{1\dots p}{p}_{\mathit{x},t}\left({\mathit{x}}_k\right)p_{\mathit{y},t}\left({\mathit{y}}_l\right)P_{{\mathbf{z}}_{\mathit{kl}},{\mathit{x}}_i\mid \chi =1}+\right.\\ -\left.p_{\mathit{x},t}\left({\mathit{x}}_i\right)\sum _{k\ne i}^{1\dots p}{p}_{\mathit{y},t}\left({\mathit{y}}_l\right)P_{{\mathbf{z}}_{\mathit{il}},{\mathit{x}}_k\mid \chi =1}\right\}.\end{array}\left(i=1,2,\dots ,p\right).$$E21

We will show that the above «mixed» process we have just discussed above, obtained by coupling a Markov Chain with a uniform process, can be further generalized, by taking into account vector uniform processes of an appropriate dimension. But we will do this in a different context, namely in the context of the models of collective phenomena.

5. Some background about the stochastic description of many-particle systems

For reasons that will be clear in a very short, while we shall now focus our attention on the double indexing of the elements on S, and assume r = s is so that m = 2s, $S={\mathrm{\Omega }}^{2s}$, $p=q=l^s$ and $n=l^{2s}$ and consider the random process ${\left\{\left({\mathbf{X}}_t,{\mathit{\chi }}_t\right)\right\}}_{t\in \left[0,T\right)}$, where $0<T\le +\infty$ and ${\mathit{\chi }}_t\equiv {\left({\chi }_t^{\mathit{ij}}\right)}_{1\le i,j\le l^s}$ for any $t\in \left[0,T\right)$. According to our notation conventions, ${\mathit{\chi }}_t$ is in fact for any t a vector with n components that are double-indexed just to reproduce the (conventional) double indexing of the elements of S. We can treat it as a square matrix with ${\mathit{\chi }}_t$ rows and $l^s$ columns. For each entry ${\chi }_t^{\mathit{ij}}$ of this matrix, we shall set

$${\mathit{\chi }}_t^{\mathit{ij}}\equiv \left\{{\chi }_t^{\mathit{kl}}\in {\mathit{\chi }}_t|\text{either}k\ne i\text{or}l\ne j\right\}.$$

For any t and for any couple (i, j), ${\chi }_t^{\mathit{ij}}$ is a Bernoulli random variable, and we assume that

1. For any t and $t^{\prime }$ in [0, T), ${\chi }_t^{\mathit{ij}}$ and ${\chi }_{t^{\prime }}^{\mathit{ij}}$ are independent and identically distributed. Accordingly, $\mathbf{P}\left({\chi }_t^{\mathit{ij}}=1\right)=u_{\mathit{ij}}$ and $\mathbf{P}\left({\chi }_t^{\mathit{ij}}=0\right)=1-u_{\mathit{ij}}$ for any $t\in \left[0,T\right)$.

2. For any $t\in \left[0,T\right)$ and for any $\left(i,j\right)\in {\left\{1,2,\dots ,l^s\right\}}^2$, the probability of state ${\mathbf{z}}_{\mathit{ij}}$ at time t depends only on ${\chi }_t^{\mathit{ij}}$.

3. For any $t\in \left[0,T\right)$ and for any $\left(i,j\right)\in {\left\{1,2,\dots ,l^s\right\}}^2$, the transition probabilities from the state ${\mathbf{z}}_{\mathit{ij}}$ to any other state depend only on ${\chi }_t^{\mathit{ij}}$; accordingly, we are allowed to consider only the conditional transition probabilities $P_{{\mathbf{z}}_{\mathit{ij}},{\mathbf{z}}_{\mathit{kl}}\mid {\chi }_t^{\mathit{ij}}=0}\left(t\right)$ and $P_{{\mathbf{z}}_{\mathit{ij}},{\mathbf{z}}_{\mathit{kl}}\mid {\chi }_t^{\mathit{ij}}=1}\left(t\right)$.

4. As for the process considered in the previous section,

$$P_{{\mathbf{z}}_{\mathit{ij}},{\mathbf{z}}_{\mathit{kl}}\mid {\chi }_t^{\mathit{ij}}=0}\left(t\right)={\delta }_k^i{\delta }_l^j,\forall t\in \left[0,T\right),$$

so that we are allowed again to state that the chain is stationary if and only if $P_{{\mathbf{z}}_{\mathit{ij}},{\mathbf{z}}_{\mathit{kl}}\mid {\chi }_t^{\mathit{ij}}=1}\left(t\right)$ is in fact independent of t.

This stated that we are allowed to reproduce the reasonings of the previous sections, with obvious changes leading us to replace the system (21) with the system

$$\begin{array}{l}\frac{p_{\mathit{x},t}}{t}\left({\mathit{x}}_i\right)=\sum _{l=1}^q\left\{\sum _{k\ne i}^{1\dots p}{\tau }_{\mathit{kl}}{p}_{\mathit{x},t}\left({\mathit{x}}_k\right)p_{\mathit{y},t}\left({\mathit{y}}_l\right)P_{{\mathbf{z}}_{\mathit{kl}},{\mathit{x}}_i\mid {\chi }_t^{\mathit{kl}}=1}+\right.\\ -\left.p_{\mathit{x},t}\left({\mathit{x}}_i\right)\sum _{k\ne i}^{1\dots p}{\tau }_{\mathit{il}}{p}_{\mathit{y},t}\left({\mathit{y}}_l\right)P_{{\mathbf{z}}_{\mathit{il}},{\mathit{x}}_k\mid {\chi }_t^{\mathit{il}}=1}\right\},\left(i=1,2,\dots ,l^s\right).\end{array}$$E22

Now, once we have reconstructed the way in which this last system of equations is originated by coupling a stationary vector Markov Chain with an additional vector random process actually modifying the final form of the transition matrix maintaining; however, its stationarity (that is its independence on time), we shall devote the rest of this section to recall a classical interpretation (see [8]) of system (22), which has become in the last forty years ever more important as it can be applied to a wide range of different phenomena, from mechanics of gases (which is its true historical origin, since Boltzmann proposed his celebrated kinetic theory [4]) to the behavior of biological systems, with particular concern to the interaction between healthy and tumor tissues (see [6], also for more complete bibliographic references), from social sciences, with particular concern to the diffusion of competing opinions [14], to economics [9], and to the behavior of swarms and crowds [5, 7]. This interpretation has in fact given rise to a general model for the description of the behavior of a large number of many-particle systems, we would dare say of almost all possible types and in almost all possible contexts.

The key idea upon which the kinetic-theoretic description of the behavior of many-particle systems is based and is the following. A set $\mathcal{S}$ of a very large number N of objects is given. The objects of $\mathcal{S}$, usually called «particles» or «individuals» according to the context, can interact pairwise³ with each other and are identified (further than by some common «physical» properties they can possess to different degrees, which do not appear explicitly in the mathematical description of the behavior of $\mathcal{S}$, but influence the values of parameters) by their «states». According to the context, the set of possible states of all particles can be finite, countably infinite, or continuous. We cannot (and do not) claim to know exactly the state of each particle of the system at any time so that even the prescription of N precise initial conditions on the states of particles would be quite unrealistic. What we assume to be allowed to state is that, if the number of all possible different states is q, then all the particles of $\mathcal{S}$ can be thought of as divided into n different classes ${\mathcal{S}}_1,{\mathcal{S}}_2,\dots ,{\mathcal{S}}_q$, each containing all and only the particles sharing the same state; if $n_i\left(t\right)$ (with $i\in \left\{1,2,\dots n\right\}$ is the number of particles sharing the i-th state at time t, then $p_i\left(t\right)=n_i\left(t\right)/N$ is the probability to pick at random in $\mathcal{S}$ a particle in the i-th state, and we can construct a state vector $\mathbf{p}\equiv \left(p_1,p_2,\dots ,p_q\right)$. In addition, each particle belonging to a class ${\mathcal{S}}_k$ can jump into another class ${\mathcal{S}}_i$ if and only if it «interacts» with some other particle: the state (that is, the class ${\mathcal{S}}_l^{\ast }$) of this latter does not matter, if not as influencing the assessment of the different probabilities of jump $P_{\mathit{ki}}=\mathbf{P}\left({\mathcal{S}}_k,{\mathcal{S}}_i\right)$, that accordingly are denoted $P_{\mathit{ki};l}=\mathbf{P}\left({\mathcal{S}}_k,{\mathcal{S}}_i;{\mathcal{S}}_l^{\ast }\right)$. Moreover, if a particle has no interactions with other particles, then it will not change its state.

The above almost purely qualitative descriptions lead us at once to conclude that, in the above scheme, any many-particle system is in fact a random process ${\left\{\left({\mathbf{X}}_t,{\mathit{\chi }}_t\right)\right\}}_{t\in \left[0,T\right)}$ of the above-described type, where $\left\{{\mathbf{X}}_t\right\}$ is a Markov Chain, characterized by a state space, which in our previous description is ${\mathrm{\Omega }}^s\equiv \left\{{\mathit{x}}_1,{\mathit{x}}_2,\dots ,{\mathit{x}}_q\right\}$ (where, we recall, $q=l^s$), a state (probability) vector ${\mathbf{p}}_x$ expressing in percentage the distribution of particles over all possible different states, an interaction rate ${\tau }_{\mathit{ij}}$ (the average number of interactions between a particle of ${\mathcal{S}}_i$ and a particle of ${\mathcal{S}}_j$ occurring in any unit time interval, that is, the measure of the set $I_1\equiv \left\{t\in I|{\chi }_t^{\mathit{ij}}=1\right\}$ in any inerval I of unit length), and each probability of jump $P_{\mathit{ki};l}=\mathbf{P}\left({\mathcal{S}}_k,{\mathcal{S}}_i;{\mathcal{S}}_l^{\ast }\right)$, conditional to the occurrence of an interaction between a particle in the state ${\mathit{x}}_k$ and a particle in the state ${\mathit{y}}_l$ is given by $P_{\left({\mathit{x}}_k,{\mathit{y}}_l\right),{\mathit{x}}_i\mid {\chi }_t^{\mathit{kl}}=1}=P_{{\mathbf{z}}_{\mathit{kl}},{\mathit{x}}_i\mid {\chi }_t^{\mathit{kl}}=1}$. So, relations (22) turn out to be exactly the equations governing, in stochastic terms, the evolution of isolated many-particle systems.

6. Equations for many-particle systems with time-dependent transition matrices

At a first glance, the above reconstruction of the equations of kinetic theory in terms of a random process ${\left\{\left({\mathbf{X}}_t,{\mathit{\chi }}_t\right)\right\}}_{t\in \left[0,T\right)}$ coupling a stationary Markov Chain and a Bernoullian process could seem a purely academic exercise, though interesting as showing the correspondence between two different languages, born in different contexts, with different aims and in different times⁴. But our exercise is not an end in itself but is very useful to extend our model for many-particle systems to cover the case of systems interacting with the external world. It is in fact not by chance that we have stressed that equations (22) hold for isolated many-particle systems. These equations are based on the assumption that the instantaneous variation of the distribution of particles over the state space is uniquely due to mutual interactions between particles and that these interactions modify the states of the involved particles always in the same way, independently of the time at which they occur⁵. But when the system perceives the influence of the external world, we are allowed to assume that both the interaction rate and the results of any interaction are more or less strongly modified from time to time by events that happen outside the system.

More precisely, our stochastic process ${\left\{\left({\mathbf{X}}_t,{\mathit{\chi }}_t\right)\right\}}_{t\in \left[0,T\right)}$ must be replaced by another process ${\left\{\left({\mathbf{X}}_t,{\mathit{\chi }}_t,{\mathbf{E}}_t\right)\right\}}_{t\in \left[0,T\right)}$, where, for any $t\in \left[0,T\right)$, ${\mathbf{E}}_t$ is another «strange» vector random variable⁶ with values in $\mathbb{W}\equiv {\mathrm{ℝ}}^d\times {\mathbb{K}}^h$ (where $\mathbb{K}$ is a finite subset of $\mathrm{\mathbb{Z}}$), such that, for any $t\in \left[0,T\right)$, there exists an $\mathbf{e}\left(t\right)=\left(\mathbf{u}\left(t\right),\mathbf{v}\left(t\right)\right)\equiv (u_1\left(t\right),u_2\left(t\right),\dots ,u_d\left(t\right),v_1\left(t\right)$, $v_2\left(t\right),\dots ,v_h\left(t\right))\in \mathbb{W}\left[\mathbf{u}\left(t\right)\equiv \left(u_1\left(t\right),u_2\left(t\right),\dots ,u_d\left(t\right)\right),\mathbf{v}\left(t\right)\equiv \left(v_1\left(t\right),v_2\left(t\right),\dots ,v_h\left(t\right)\right)\right]$ for which the probability density function of ${\mathbf{E}}_t$ is expressed, for any $\mathit{\epsilon }\equiv \left(\mathit{\xi },\mathit{\eta }\right)\equiv \left({\xi }_1,\dots ,{\xi }_d,{\eta }_1,\dots ,{\eta }_h\right)$, with $\mathit{\xi }\equiv \left({\xi }_1,\dots ,{\xi }_d,\right)$ and, $\mathit{\eta }\equiv \left({\eta }_1,\dots ,{\eta }_h\right)$, by the relation

$$\begin{array}{l}{\rho }_{{\mathbf{E}}_t}\left(\mathit{\epsilon }\right)=\delta \left({\xi }_1-u_1\left(t\right)\right)\delta \left({\xi }_2-u_2\left(t\right)\right)\dots \delta \left({\xi }_d-u_d\left(t\right)\right)\times \\ \times \delta \left({\eta }_1-v_1\left(t\right)\right)\delta \left({\eta }_2-u_2\left(t\right)\right)\dots \delta \left({\eta }_h-v_h\left(t\right)\right)=,\forall t\in \left[0,T\right)\\ =\delta \left(\mathit{\xi }-\mathbf{u}\left(t\right)\right)\delta \left(\mathit{\eta }-\mathbf{v}\left(t\right)\right).\end{array}$$E23

This means that the process ${\left\{{\mathbf{E}}_t\right\}}_{t\in \left[0,T\right)}$ is a deterministic external process. For instance, we can imagine that our system consists of all the inhabitants of a given region, whose states are vectors, one component of which is the amount of money of each individual, and the other components are quantities of food. We consider only interactions consisting in purchase and sale of apricots. In the late summer, in the autumn, and in the winter the interaction rate will be zero (no apricots in these seasons); in the early springtime, the interaction rate will be small, and the transition probabilities will describe changes of states with little increases or decreases of the quantity of apricots, and large increases or decreases of the amount of money (in that period, apricots are hardly available and very expensive); in the late springtime and in the early summer, the interaction rate will attain its maximum value and the transition probabilities will describe changes of states with large increases or decreases of the quantity of apricots, and small increases or decreases of the amount of money (in that period, many apricots available at low prices). So, both the interaction rates and the transition probabilities depend on seasons, that is, are functions of time, and system (22) must be simply written in the form

$$\begin{array}{l}\frac{p_{\mathit{x},t}}{t}\left({\mathit{x}}_i\right)=\sum _{l=1}^q\left\{\sum _{k\ne i}^{1\dots p}{\tau }_{\mathit{kl}}\left(t\right)p_{\mathit{x},t}\left({\mathit{x}}_k\right)p_{\mathit{y},t}\left({\mathit{y}}_l\right)P_{{\mathbf{z}}_{\mathit{kl}},{\mathit{x}}_i\mid {\chi }_t^{\mathit{kl}}=1}\left(t\right)+\right.\\ -\left.p_{\mathit{x},t}\left({\mathit{x}}_i\right)\sum _{k\ne i}^{1\dots p}{\tau }_{\mathit{il}}\left(t\right)p_{\mathit{y},t}\left({\mathit{y}}_l\right)P_{{\mathbf{z}}_{\mathit{il}},{\mathit{x}}_k\mid {\chi }_t^{\mathit{il}}=1}\left(t\right)\right\},\left(i=1,2,\dots ,l^s\right).\end{array}$$E24

Accordingly, the introduction of a dependence on time of both interaction rates and transition probabilities seems to be the most appropriate way to link the behavior of a many-particle system to deterministic external influences. Of course, the interesting aspect, and also the most difficult one, of this kind of model, is the choice of functions ${\tau }_{\mathit{kl}}\left(t\right)$ and $P_{{\mathbf{z}}_{\mathit{kl}},{\mathit{x}}_i\mid {\chi }_t^{\mathit{kl}}=1}\left(t\right)$ in such a way as to express faithfully the changes of the environment, as well as the way in which these changes are perceived by the individuals of the system. At an initial stage, we can adopt a purely hypothetical method and make any assumption we want, but when we have in mind some precise applications, then we need a preliminary statistical analysis to get the right hints toward the most plausible assignment of the above functions.

7. Equations for many-particle systems with stochastic transition matrices

The case treated in the previous section is rather trivial, and its main interest lies in the choice of the dependence of all the interaction rates ${\tau }_{\mathit{kl}}\left(t\right)$ and of all transition probabilities $P_{{\mathbf{z}}_{\mathit{kl}},{\mathit{x}}_i\mid {\chi }_t^{\mathit{kl}}=1}\left(t\right)$ on time, which is connected with the application we have in mind, that is, much more with experimental and statistical analysis than with theoretical reasonings. Much more interesting is the case in which the changes of external world influencing the behavior of the system are not assigned at each time, but stochastic events. This means that we have to consider again a stochastic process ${\left\{\left({\mathbf{X}}_t,{\mathit{\chi }}_t,{\mathbf{E}}_t\right)\right\}}_{t\in \left[0,T\right)}$, but now, for any $t\in \left[0,T\right)$, ${\mathbf{E}}_t$ is a random variable in the “strict” sense of the word. More precisely, for any $t\in \left[0,T\right)$,

1. We denote again by $\mathbb{W}\equiv {\mathrm{ℝ}}^d\times {\mathrm{\mathbb{Z}}}^h$ the range of all possible values of ${\mathbf{E}}_t$.

2. We denote by $\mathit{\epsilon }\equiv \left(\mathit{\xi },\mathit{\eta }\right)\equiv \left({\xi }_1,\dots ,{\xi }_d,{\eta }_1,\dots ,{\eta }_h\right)$ any unspecified element of $\mathbb{W}$.

3. We set $\mathit{a}\equiv \left(a_1,a_2,\dots ,a_d\right)$, $\mathit{b}\equiv \left(b_1,b_2,\dots ,b_d\right)$_, and $I\left(\mathit{a},\mathit{b}\right)\equiv \left[a_1,b_1\right]\times \left[a_2,b_2\right]\times \dots \times \left[a_d,b_d\right];$

4. We set ${\mathbf{E}}_t\equiv \left({\mathbf{E}}_{t,1},{\mathbf{E}}_{t,2}\right)$ where the range of values of ${\mathbf{E}}_1$ is ${\mathrm{ℝ}}^d$ and the range of values of ${\mathbf{E}}_2$ is ${\mathrm{\mathbb{Z}}}^h$, and assume that ${\mathbf{E}}_{t,1}$ and ${\mathbf{E}}_{t,2}$ are independent;

5. To the (continuous) random variable ${\mathbf{E}}_1$ is associated a probability density function ${\rho }_{t,1}\left(\mathit{\xi }\right)\equiv {\rho }_1\left(\mathit{\xi },t\right)$ such that

   $$\mathbf{P}\left({\mathbf{E}}_{t,1}\in I\left(\mathit{a},\mathit{b}\right)\right)={\int }_{I\left(\mathit{a},\mathit{b}\right)}{\rho }_1\left(\mathit{\xi },t\right)\mathrm{d}\mathit{\xi }<1$$

   for any $\left(\mathit{a},\mathit{b}\right)\in {\mathrm{ℝ}}^d\times {\mathrm{ℝ}}^d$ such that ${\mathrm{ℝ}}^d\backslash I\left(\mathit{a},\mathit{b}\right)\ne \varnothing ;$

6. To the (discrete) random variable ${\mathbf{E}}_2$ is associated a probability distribution function ${\mathbf{P}}_{t,2}$ such that

   $$\mathbf{P}\left({\mathbf{E}}_{t,2}=\mathit{\eta }\right)={\mathbf{P}}_{t,2}\left(\mathit{\eta }\right)\equiv {\mathbf{P}}_2\left(\mathit{\eta },t\right).$$

   For any $\mathit{\eta }\in {\mathrm{\mathbb{Z}}}^h$, ${\mathbf{P}}_2\left(\mathit{\eta },t\right)<1$.

7. we denote by the symbol ${\tau }_{\mathit{ij}}\left(\mathit{\epsilon },t\right)$ the density (on the interval [0, T)) of the probability that ${\chi }_t^{\mathit{ij}}=1$ conditional to the event ${\mathbf{E}}_t=\mathit{\epsilon }$.

Accordingly, for any $\mathit{\epsilon }\in \mathbb{W}$, the associated probability density of ${\mathbf{E}}_t$ at $\mathit{\epsilon }$ is

$${\rho }_{{\mathbf{E}}_t}\left(\mathit{\epsilon }\right)={\rho }_1\left(\mathit{\xi },t\right){\mathbf{P}}_2\left(\mathit{\eta },t\right).$$

This stated, according to the law of alternatives, and under the conditions imposed in Section 1.5, we find that each transition probability depends on time, and we have

$$P_{{\mathbf{z}}_{\mathit{il}},{\mathit{x}}_k\mid {\chi }_{\mathit{il},t}=1,t}=\sum _{\mathit{\eta }\in \mathrm{\mathbb{Z}}}{\mathbf{P}}_2\left(\mathit{\eta },t\right){\int }_{{\mathrm{ℝ}}^d}\mathbf{P}\left({\mathbf{z}}_{\mathit{il}},{\mathit{x}}_k|{\chi }_{\mathit{il},t}=1,{\mathbf{E}}_1=\mathit{\xi },{\mathbf{E}}_2=\mathit{\eta }\right){\rho }_1\left(\mathit{\xi },t\right){\tau }_{\mathit{ij}}\left(\mathit{\epsilon },t\right)\mathrm{d}\mathit{\xi }.$$E25

Accordingly, the system of equations governing the evolution of the many-particle system we have decided to describe by means of the present language and of the notions of random processes and, in particular, of Markov Chains, takes the form

$$\begin{array}{l}\frac{\mathrm{d}{p}_{\mathit{x},t}}{t}\left({\mathit{x}}_i\right)=\sum _{l=1}^q\sum _{k\ne i}^{1\dots p}\sum _{\mathit{\eta }\in \mathrm{\mathbb{Z}}}{\mathbf{P}}_2\left(\mathit{\eta },t\right)\times \left\{p_{\mathit{x},t}\left({\mathit{x}}_k\right)p_{\mathit{y},t}\left({\mathit{y}}_l\right){\int }_{{\mathrm{ℝ}}^d}{\mathbf{P}}_{{\mathbf{z}}_{\mathit{kl}},{\mathit{x}}_i\mid {\chi }_t^{\mathit{kl}}=1,{\mathbf{E}}_t\mathit{=}\mathit{\epsilon }}{\rho }_1(\mathit{\xi },t\left){\tau }_{\mathit{kl}}\right(\mathit{\epsilon },t)\mathrm{d}\mathit{\xi }+\right.\\ -\left.p_{\mathit{x},t}\left({\mathit{x}}_i\right)p_{\mathit{y},t}\left({\mathit{y}}_l\right){\int }_{{\mathrm{ℝ}}^d}{\mathbf{P}}_{{\mathbf{z}}_{\mathit{il}},{\mathit{x}}_k\mid {\chi }_t^{\mathit{il}}=1,{\mathbf{E}}_t\mathit{=}\mathit{\epsilon }}{\rho }_1(\mathit{\xi },t\left){\tau }_{\mathit{il}}\right(\mathit{\epsilon },t)\mathrm{d}\mathit{\xi }\right\},\left(i=1,2,\dots ,l^s\right)\end{array}$$E26

where ${\mathbf{P}}_{{\mathbf{z}}_{\mathit{kl}},{\mathit{x}}_i\mid {\chi }_t^{\mathit{kl}}=1,{\mathbf{E}}_t=\mathit{\epsilon }}\equiv \mathbf{P}\left({\mathbf{z}}_{\mathit{kl}},{\mathit{x}}_i|{\chi }_t^{\mathit{kl}}=1,{\mathbf{E}}_1=\mathit{\xi },{\mathbf{E}}_2=\mathit{\eta }\right)$ and ${\mathbf{P}}_{{\mathbf{z}}_{\mathit{il}},{\mathit{x}}_k\mid {\chi }_t^{\mathit{il}}=1,{\mathbf{E}}_t\mathit{=}\mathit{\epsilon }}\equiv \mathbf{P}\left({\mathbf{z}}_{\mathit{il}},{\mathit{x}}_k|{\chi }_t^{\mathit{il}}=1,{\mathbf{E}}_1=\mathit{\xi },{\mathbf{E}}_2=\mathit{\eta }\right)$, when $\mathrm{\Omega }$ (the state space common to all components of the vector Markov Chain X) is a discrete set (as we have assumed throughout the whole chapter till now). When $\mathrm{\Omega }$ is instead a continuous set (typically, a real interval), then the sums in relations (26) must be replaced by integrals and — with the obvious correspondences ${\mathit{x}}_i\to \mathit{x},$ ${\mathit{x}}_k\to {\mathit{x}}^{\prime },$ ${\mathit{y}}_l\to \mathit{y},$ ${\mathbf{z}}_{\mathit{kl}}\to \left({\mathit{x}}^{\prime },\mathit{y}\right),$ ${\mathbf{z}}_{\mathit{il}}\to \left(\mathit{x},\mathit{y}\right),$ ${\tau }_{\mathit{kl}}\left(\mathit{\epsilon },t\right)\to \tau \left({\mathit{x}}^{\prime },\mathit{y};\mathit{\epsilon },t\right),$ ${\tau }_{\mathit{il}}\left(\mathit{\epsilon },t\right)\to \tau \left(\mathit{x},\mathit{y};\mathit{\epsilon },t\right),$ ${\chi }_t^{\mathit{kl}}\to \chi \left({\mathit{x}}^{\prime },\mathit{y},t\right),$ and ${\chi }_t^{\mathit{il}}\to \chi \left(\mathit{x},\mathit{y},t\right)$ — we get the equation⁷

$$\begin{array}{l}\frac{\partial p_{\mathit{x}}}{\partial t}\left(\mathit{x},t\right)=\sum _{\mathit{\eta }\in \mathrm{\mathbb{Z}}}{\mathbf{P}}_2\left(\mathit{\eta },t\right)\times {\int }_{\mathrm{\Omega }}{\int }_{\mathrm{\Omega }}\\ \left\{p_{\mathit{x}}\left({\mathit{x}}^{\prime },t\right)p_{\mathit{y}}(\mathit{y},t\left){\int }_{{\mathrm{ℝ}}^d}{\mathbf{P}}_{\left({\mathbf{x}}^{\prime },\mathit{y}\right),\mathit{x}\mid \chi \left({\mathit{x}}^{\prime },\mathit{y},t\right)=1,{\mathbf{E}}_t=\mathit{\epsilon }}{\rho }_1\right(\mathit{\xi },t\left)\tau \right({\mathit{x}}^{\prime },\mathit{y};\mathit{\epsilon },t)\mathrm{d}\mathit{\xi }+\right.\\ -\left.p_{\mathit{x}}\left(\mathit{x},t\right)p_{\mathit{y}}(\mathit{y},t\left){\int }_{{\mathrm{ℝ}}^d}{\mathbf{P}}_{\left(\mathbf{x},\mathit{y}\right),{\mathit{x}}^{\prime }\mid \chi \left(\mathit{x},\mathit{y},t\right)=1,{\mathbf{E}}_t=\mathit{\epsilon }}{\rho }_1\right(\mathit{\xi },t\left)\tau \right(\mathit{x},\mathit{y});\mathit{\epsilon },t)\mathrm{d}\mathit{\xi }\right\}\mathrm{d}{\mathit{x}}^{\prime }\mathit{y}\end{array}$$E27

where we have also agreed to write

$$p_{\mathit{x},t}\left(\mathit{x}\right)=p_{\mathit{x}}\left(\mathit{x},t\right),p_{\mathit{x},t}\left({\mathit{x}}^{\prime }\right)=p_x\left({\mathit{x}}^{\prime },t\right),p_{\mathit{y},t}\left(\mathit{y}\right)=p_{\mathit{y}}\left(\mathit{y},t\right).$$

Equation (27) is a generalization of system (24). As a matter of fact, this latter can be easily obtained from the former by simply replacing the probability density function ${\rho }_1$ with the one defined by relation (23). And, of course, the equations governing the evolutions of isolated many-particle systems can be derived at once by choosing $\left\{{\mathbf{E}}_t\right\}$ as a deterministic and constant process (i.e. a process in which all the random variables ${\mathbf{E}}_t$ have the same probability density function

$$\rho \left(\mathit{\xi }\right)=\delta \left(\mathit{\xi }-\mathbf{e}\right)$$

on $\mathbb{W}$, however, e is fixed). So, system (27) seems to be the most general description of the behavior of any one of such systems. Recently, however, other types of external influence have been considered, acting directly on the state vectors and not on the interactions between particles [13, 14]. This point will be discussed in some details in the last section of the chapter.

8. Subsystems

This section is almost a pure aside, just to give a complete exposition of the development of the whole theory of many-particle systems as based on the theory of stochastic processes. We want to recall that a more general formulation of the theory takes into consideration the case in which a system $\mathcal{S}$ is the join of a number N of subsystems ${\mathcal{S}}_1,{\mathcal{S}}_2,\dots ,{\mathcal{S}}_N$, called the functional subsystems of $\mathcal{S}$, that can share the same state space or not, and are such that, for any $\lambda \in \left\{1,2,\dots ,N\right\}$, the particles of ${\mathcal{S}}_{\lambda }$ are not constrained to share the same state. In order to show the way in which the equations describing the behavior of $\mathcal{S}$ must be modified under this assumption, in order to avoid an undue multiplication of indexes, and for the sake of a better readability, we shall assume that all the subsystems share a continuous state space $\mathrm{\Omega }\equiv \left(a,b\right)\subseteq \mathrm{ℝ}$. The point of the present development is that now we must allow interactions between particles of different subsystems so that we must consider a mixed stochastic process ${\left\{\left({\mathbf{X}}_t,{\mathit{\chi }}_t,{\mathbf{E}}_t\right)\right\}}_{t\in \left[0,T\right)}$ where ${\mathit{\chi }}_t\equiv \left({\chi }_{\lambda ,\mu }\left(\mathit{x},\mathit{y},t\right)\right)$, and for any $t\in \left[0,T\right)$ and for any 4-tuple $\left(\lambda ,\mu ,\mathit{x},\mathit{y}\right)\in {\left\{1,2,\dots ,N\right\}}^2\times {\mathrm{\Omega }}^{2s}$ the random variable ${\chi }_{\lambda ,\mu }\left(\mathit{x},\mathit{y}\right)$ is again a Bernoulli variable, with probability density function ${\tau }_{\mathit{\lambda \mu }}\left(\mathit{u},\mathit{v}\right)\mathrm{\Delta }t$, in view of the continuity of variables x and y. By reproducing almost without changes the reasonings of the previous two sections, we obtain the system

$$\begin{array}{l}\frac{\partial p_{\mathit{x}}^{\lambda }}{\partial t}\left(\mathit{x},t\right)=\sum _{\mathit{\eta }\in \mathrm{\mathbb{Z}}}{\mathbf{P}}_2\left(\mathit{\eta },t\right)\times \sum _{\mu =1}^{\nu }{\int }_{\mathrm{\Omega }}{\int }_{\mathrm{\Omega }}\\ \{,p_{\mathit{x}}^{\lambda }\left({\mathit{x}}^{\prime },t\right)p_{\mathit{y}}^{\mu }(\mathit{y},t\left){\int }_{{\mathrm{ℝ}}^d}{\mathbf{P}}_{\left({\mathit{x}}^{\prime },\mathit{y}\right),\mathit{x}\mid {\chi }_{\mathit{\lambda \mu }}\left({\mathit{x}}^{\prime },\mathit{y},t\right)=1,{\mathbf{E}}_t=\mathit{\epsilon }}{\rho }_1\right(\mathit{\xi },t\left){\tau }_{\mathit{\lambda \mu }}\right({\mathit{x}}^{\prime },\mathit{y};\mathit{\epsilon },t)\mathrm{d}\mathit{\xi }+\\ -p_{\mathit{x}}^{\lambda }\left(\mathit{x},t\right)p_{\mathit{y}}^{\mu }(\mathit{y},t\left){\int }_{{\mathrm{ℝ}}^d}{\mathbf{P}}_{\left(\mathbf{x},\mathit{y}\right),{\mathit{x}}^{\prime }\mid {\chi }_{\lambda ,\mu }\left(\mathit{x},\mathit{y},t\right)=1,{\mathbf{E}}_t=\mathit{\epsilon }}{\rho }_1\right(\mathit{\xi },t){\tau }_{\mathit{\lambda \mu }}(\mathit{x},\mathit{y});\mathit{\epsilon },t)\mathrm{d}\mathit{\xi }\}\\ \mathrm{d}{\mathit{x}}^{\prime }\mathit{y}\left(\lambda =1,2,\dots ,N\right).\end{array}$$E28

We have now again a system of equations. Each of them describes the evolution of the probability density function of a corresponding subsystem in consequence of the interaction of both interactions between particles of the same subsystem and of particles of different subsystems. But, of course, we have written the equations in the final form they have when also external actions that modify both the rates and the effects of interactions.

9. Alternative views, conclusions and perspectives

In almost all the literature about many-particle systems, external actions are completely disregarded, and the system of equations governing their evolution is written in its simplest form, where both the transition probabilities and the interaction rates are constant. Only recently, a number of papers have taken into account external «forcing» events that are capable to modify the distribution of states on each subsystem of a system $\mathcal{S}$ independently of possible interactions between particles [11, 12, 13, 14, 15]. These external events, however, are assumed to act directly on the functions $p_{\mathit{x}}^{\lambda }\left(\mathit{x},t\right)$ (or $p_{\mathit{x}}^{\lambda }\left(\mathit{x},t\right)$, when the state space is discrete). The origin of this viewpoint is in [11] where, dealing with a description of possible evolutions of an epidemics, an additional term describing spontaneous healing or death was necessary to get a faithful picture. This also led to general considerations about the implicit use of an analog of the Principle of Inertia in the scheme depicted in Sections 1.5 and 1.6. Taking into account an additional term of this type, system (28) becomes

$$\begin{array}{l}\frac{\partial p_{\mathit{x}}^{\lambda }}{\partial t}\left(\mathit{x},t\right)=\sum _{\mathit{\eta }\in \mathrm{\mathbb{Z}}}{\mathbf{P}}_2\left(\mathit{\eta },t\right)\times \sum _{\mu =1}^{\nu }{\int }_{\mathrm{\Omega }}{\int }_{\mathrm{\Omega }}\\ \{,p_{\mathit{x}}^{\lambda }\left({\mathit{x}}^{\prime },t\right)p_{\mathit{y}}^{\mu }(\mathit{y},t\left){\int }_{{\mathrm{ℝ}}^d}{\mathbf{P}}_{\left({\mathit{x}}^{\prime },\mathit{y}\right),\mathit{x}\mid {\chi }_{\mathit{\lambda \mu }}\left({\mathit{x}}^{\prime },\mathit{y},t\right)=1,{\mathbf{E}}_t=\mathit{\epsilon }}{\rho }_1\right(\mathit{\xi },t\left){\tau }_{\mathit{\lambda \mu }}\right({\mathit{x}}^{\prime };\mathit{y},\mathit{\epsilon },t)\mathrm{d}\mathit{\xi }+\\ -\left.p_{\mathit{x}}^{\lambda }\left(\mathit{x},t\right)p_{\mathit{y}}^{\mu }(\mathit{y},t\left){\int }_{{\mathrm{ℝ}}^d}{\mathbf{P}}_{\left(\mathbf{x},\mathit{y}\right),{\mathit{x}}^{\prime }\mid {\chi }_{\lambda ,\mu }\left(\mathit{x},\mathit{y},t\right)=1,{\mathbf{E}}_t=\mathit{\epsilon }}{\rho }_1\right(\mathit{\xi },t){\tau }_{\mathit{\lambda \mu }}(\mathit{x},\mathit{y});\mathit{\epsilon },t)\mathrm{d}\mathit{\xi }\right\}\}\mathrm{d}{\mathit{x}}^{\prime }\mathit{y}+\\ +F^{\lambda }\left[p_{\mathit{x}}^1,p_{\mathit{x}}^2,\dots ,p_{\mathit{x}}^N\right]\left(t\right)\left(\lambda =1,2,\dots ,N\right).\end{array}$$E29

where the form of the term $F^{\lambda }\left[p_{\mathit{x}}^1,p_{\mathit{x}}^2,\dots ,p_{\mathit{x}}^N\right]\left(t\right)$ depends on the context and on the type of external solicitation producing a «spontaneous» variation of the distributions.

But now, to conclude this overview of the possible developments of the modern generalization of kinetic theory, that becomes evident when it is analyzed and formalized in terms in terms of n-tuples of joint random processes (one of which is a Markov Chain, but to this point we will return at the very end of this final section), going back to the discrete version of the equation, from which we started to reconstruct the logic the equations of the theory are based upon, we want to point out an interesting new application of the theory in the scope of experimental procedures. This application was presented in [17] and its theoretical formulation in [18].

In order to outline the above-mentioned development, we first assume to deal with a system $\mathcal{S}$ split into two subsystems ${\mathcal{S}}_1=\left\{\mathrm{\Sigma }\right\}$ and ${\mathcal{S}}_2={\left\{X_h\right\}}_{h\in \mathrm{\mathbb{N}}}$. So, ${\mathcal{S}}_1$ contains only one element, which will be called the «subject», while the elements of ${\mathcal{S}}_2$, that are random variables⁸will be called the experiments or questions about $\mathrm{\Sigma }$. The «state space of the subject $\mathrm{\Sigma }»$ is a set ${\mathrm{\Omega }}_1\equiv \left\{{\mathit{x}}_1,{\mathit{x}}_2,\dots ,{\mathit{x}}_p\right\}$ while, for any $h\in \mathrm{\mathbb{N}}$, the «state space of question $X_h»$, that is, the set of all of its possible outcomes, is the set ${\mathrm{\Omega }}_{2h}\equiv \left\{y_{h1},y_{h2},\dots ,y_{{\mathit{hq}}_h}\right\}\subset \mathrm{\mathbb{Z}}$. The set ${\mathrm{\Omega }}_2={\cup }_{h\in \mathrm{\mathbb{N}}}{\mathrm{\Omega }}_{2h}$ will be the state space common to all the random variables of the sequence ${\left\{X_h\right\}}_{h\in \mathrm{\mathbb{N}}}$. We consider the discrete (stepwise) stochastic process ${\left\{\left({\mathbf{X}}_h,{\chi }_h\right)\right\}}_{h\in \mathrm{\mathbb{N}}}$, where $\left\{{\mathbf{X}}_h\right\}$, such as in Section 1.3, is a Markov Chain with the state space ${\mathrm{\Omega }}_1\times {\mathrm{\Omega }}_2$ and

$${\mathit{\chi }}_h\equiv \left(\begin{array}{cc}{\chi }_h^{11}\left(x_i,x_j\right)& {\chi }_h^{12}\left(x_i,y_{\mathit{hj}}\right)\\ {\chi }_h^{21}\left(x_i,y_{\mathit{hj}}\right)& {\chi }_h^{22}\left(y_{\mathit{hi}},y_{\mathit{hj}}\right)\end{array}\right)$$

for any $h\in \mathrm{\mathbb{N}}$. The entries of this matrix are independent Bernoulli’s random variables. ${\chi }_h^{11}\left(x_i,x_j\right)=1$ means that an interaction between $\mathrm{\Sigma }$ in the state $\mathrm{\Sigma }$ and $\mathrm{\Sigma }$ in the state $x_j$ occurs at h-th step, and ${\chi }_h^{22}\left(y_{\mathit{hi}},y_{\mathit{hj}}\right)=1$ means that at the same step an interaction occurs between two different answers to the same question $X_h$. We have not yet defined the meaning of the word «interaction» in the present context (this point will be briefly discussed at the end of the section), so for the moment, we simply assume that ${\chi }_h^{11}\left(x_i,x_j\right)={\chi }_h^{22}\left(y_{\mathit{hl}},y_{\mathit{hm}}\right)=0$ for any $h\in \mathrm{\mathbb{N}}$ and for any $\left(i,j\right)\in {\left\{11,2,\dots ,p\right\}}^2$ and $\left(l,m\right)\in {\left\{1,2,\dots ,q_h\right\}}^2$. If ${\tau }_{i,j}^{11}=\mathbf{P}\left({\chi }_h^{11}\left(x_i,x_j\right)=1\right)$ and ${\tau }_{l,m}^{22}=\mathbf{P}{\chi }_h^{22}\left(y_{\mathit{hl}},y_{\mathit{hm}}\right)=1)$, this also means that ${\tau }_{i,j}^{11}={\tau }_{l,m}^{22}=0$. On the other hand, ${\chi }_h^{12}\left(x_i,y_{\mathit{hj}}\right)=1$ and ${\chi }_h^{21}\left(x_i,y_{\mathit{hj}}\right)=1$ mean that an interaction between $\mathrm{\Sigma }$ in the state $x_i$ and the question $X_h$ giving the answer $y_{\mathit{hj}}$ certainly occurs at h-th step; this corresponds to the fact that the interaction is decided and produced by an experimenter. We assume ${\chi }_h^{12}\left(x_i,y_{\mathit{hj}}\right)={\chi }_h^{21}\left(x_i,y_{\mathit{hj}}\right)=1$ for any $h\in \mathrm{\mathbb{N}}$ and for any $i\in \left\{11,2,\dots ,p\right\}$ so that ${\tau }_{i,j}^{12}={\tau }_{i,j}^{21}=1$. Moreover, we assume $P_{\left(x_k,y_l\right),x_i\mid {\chi }_h^{21}\left(x_i,y_{\mathit{hj}}\right)=1}^{21}=0$ for any $h\in \mathrm{\mathbb{N}}$ and for any $\left(i,j\right)\in {\left\{11,2,\dots ,p\right\}}^2$. All the other conditions assuring that we are allowed to write equations (19) are assumed to hold, These equations, with obvious meaning of the symbols, will be finally rewritten in the form

$$\begin{array}{l}{p}_{\mathit{x},h+1}^1\left(x_i\right)-p_{\mathit{x},h}^1\left(x_i\right)=\\ =\sum _{l=1}^{q_h}\left\{\sum _{k\ne i}^{1\dots p}{p}_{\mathit{x},h}^1\left(x_k\right)p_{\mathit{y},h}^2\left(y_{\mathit{hl}}\right)P_{\left(x_k,y_{\mathit{hl}}\right),x_i\mid {\chi }^{12}\left(x_i,y_{\mathit{hi}}\right)=1}^{12}+\right.\\ -\left.p_{\mathit{x},h}^1\left(x_i\right)\sum _{k\ne i}^{1\dots p}{p}_{\mathit{y},h}^2\left(y_{\mathit{hl}}\right)P_{\left(x_i,y_{\mathit{hl}}\right),x_k\mid {\chi }^{12}\left(x_i,y_{\mathit{hi}}\right)=1=1}^{12}\right\},\\ p_{\mathit{y},h+1}^2\left(y_{\left(h+1\right)i}\right)-p_{\mathit{y},h}^2\left(y_{\mathit{hi}}\right)=0\end{array}\left(i=1,2,\dots ,p\right).$$E30

To conclude the section and the chapter, we want now to expose an interpretation as complete as possible of the special model just described. To this aim, what is necessary to stress first is that system (30) is meant to describe the results of interactions between information rather than between objects. More precisely, the values $x_i$ of the variable in ${\mathrm{\Omega }}_1$ should be interpreted as labels that define, in the appropriate context, possible states of the subject; then, the interaction between the subject and the h-th experience $X_h$ on it places this latter into a state, expressed by a real value $y_{\mathit{hi}}$, that is, the outcome of a measurement of a physical parameter. Now, in virtue of the information carried by the value $y_{\mathit{hi}}$, the i-th experience interacts with the initial definition, and in the sense that it can either confirm the initial state $x_i$ (by increasing its probability) or raise doubts about it (by lowering its probability). Accordingly, the terms $p_{\mathit{x},h}^1\left(x_i\right)$ and $p_{\mathit{x},h}^1\left(x_i\right)$ in equations (30) must be interpreted respectively as the absolute probability that the definition of the state of the subject before the experience is the value $x_i$ and the absolute probability that the outcome of the h-th experience is the state $y_{\mathit{hi}}$; furthermore, the term ${\mathit{P}}_{\left(x_k,y_{\mathit{hl}}\right),x_i\mid {\chi }^{12}\left(x_i,y_{\mathit{hi}}\right)=1}^{12}$ is the probability that definition $x_k$, after the «interaction» with information $y_{\mathit{hl}}$ becomes definition $x_i$.

In [17], equations (30) were derived by discretization from the ones of kinetic theory and used to describe the way in which new experiences on a number of different parameters can modify the classification of a phenomenon, in particular the diagnosis of depression in a human subject suspected to be depressed according to the score obtained by answering a questionnaire. Of course, in an application like that, the transition probabilities must be estimated in advance by a preliminary study of the correlation of the involved random variable with the possible depression states. We have reported this application, together with the classical application of kinetic theory and its developments, to give a more complete picture of the way in which the study of joint random processes (vector random processes), in particular when one of the processes considered simultaneously is a Markov Chain, can open the way to an ever wider range of applications.