Enhancing Resilience in Cooperative Systems against Cyber-Attacks: A Defense Framework through Adaptive Network Reconfiguration and Digital Twin

Abstract

The book chapter presents a solution to enhance the resilience of multi-agent cooperative systems against cyber-attacks. Specifically, the threat involves adversaries aiming to prevent the cooperative systems from achieving consensus by launching false data injection (FDI) or denial-of-service (DoS) attacks on the communication network. First, a distributed algorithm is proposed to identify critical edges, via online estimation/learning of the network’s parameters, whose removals disconnect the network. An adaptive network reconfiguration algorithm is then presented to remove those critical edges, thus ensuring the resilient operation of the cooperative systems against arbitrary DoS attacks. Additionally, a digital twin, interconnected with the cooperative systems, is introduced to simultaneously ensure the resiliency of the cooperative systems against FDI attacks and increase the redundancy of the network. Finally, it is illustrated how the proposed defense framework also allows for detecting and identifying in real-time both attack vectors in the network. The efficacy of the proposed solution is demonstrated through numerical simulations, showcasing their effectiveness in protecting the cooperative systems against cyber threats.

1. Introduction

Cooperative systems are a collection of autonomous agents that interact/cooperate with each other to make their own decision (actions) so that the overall system achieves a common goal and/or the overall system’s performance is optimized without the need for a central entity. In practice, the individual agent can be distributed generator in a power system, an autonomous robot in a robotic swarm, or an autonomous vehicle in an intelligent transportation system, while the interaction can take place via wireless communication. The local interaction rule makes the cooperative systems appealing for real-time decision-making in large-scale and critical systems due to their scalability and low latency.

The consensus problem is a canonical issue in cooperative systems in which the agents aim to achieve agreement, via a consensus algorithm, on specific quantities of interest, such as voltage, distance, or velocity [1]. Various coordination problems in diverse domains, including power systems, transportation systems, robotic swarms, and federated learning, can be framed as consensus problems [2, 3, 4, 5]. Furthermore, the consensus algorithm also serves as a fundamental component in numerous distributed optimization algorithms [6, 7].

While the introduction of a communication network plays a crucial role in realizing cooperation among the agents, its use comes at the price of making the cooperative system vulnerable to cyber-attacks, which may prevent the systems from achieving consensus and, in the worst case, destabilize the overall systems [8, 9]. Recent history has witnessed catastrophic cyber-attacks, such as the 2015 Ukraine power grid incident, causing a 6-hour blackout [10]. Specifically, the attacker may modify the information being exchanged between the agents. Thus destroying data integrity, e.g., by launching false data injection (FDI attacks), or block the communication channels causing data unavailability, e.g., by launching denial-of-service (DoS) attacks. In addition, the attacker can also passively eavesdrop on the communication channel to acquire confidential information about the agents [11].

In order to unlock the potential economic impact of cooperative systems, it is thus imperative to ensure the resilient operation of the cooperative systems against unknown cyber-attacks. This book chapter focuses on cooperative systems under DoS or FDI attacks. The existing strategies for resilient consensus against DoS attacks impose restrictive assumptions on the frequency and duration of attacks [12, 13], which are hard to guarantee in practice. A strategy to ensure resilient consensus against FDI attacks is by removing a fixed number of extreme local values that an agent received from the other agents [14]. However, this strategy requires high network connectivity and cannot guarantee the agents to converge to the attack-free consensus value even in the absence of attacks. Another potential strategy is by identifying the compromised communication links and subsequently removing them [15]. However, this approach also requires high network connectivity, which depends on the number of compromised links, and more importantly, the system’s stability may have been destroyed during the identification process.

This book chapter presents a novel defense framework that eliminates restrictions imposed in the existing strategies discussed previously. First, to defend against DoS attacks, a distributed algorithm is presented to determine the critical edge in the network, which is an edge whose removal disconnects the network. A novel network reconfiguration algorithm is then proposed to remove the existence of the critical edge and thus ensure that the network remains connected under arbitrary DoS attacks. Furthermore, a digital twin is designed and interconnected with the cooperative systems. The digital twin not only makes the cooperative system resilient against FDI attacks but also helps to remove the critical edges in the network. The proposed defense framework also facilitates real-time and distributed detection and identification of DoS or FDI attacks in the network. Finally, a numerical example is provided to illustrate the proposed defense framework.

2. Preliminaries

Let $\mathbb{R}$ be the set of real numbers; vector ${\mathbf{1}}_n\in {\mathbb{R}}^n$ denotes the vector of all ones, ${\mathbf{0}}_n\in {\mathbb{R}}^n$ represents the vector of all zeros, and $I_n\in {\mathbb{R}}^{n\times n}$ denotes an $n\times n$ identity matrix. The cardinality of a set $\mathcal{N}$ is denoted by $\mid \mathcal{N}\mid$. Superscript ″$T$″ represents the transpose of a matrix or a vector.

Let $\mathcal{G}=\left(\mathcal{V},\mathcal{E}\right)$ be an undirected graph with a set of nodes $\mathcal{V}=\left\{1,\cdots ,n\right\}$ and a set of edges $\mathcal{E}\subset \mathcal{V}\times \mathcal{V}$. An edge $\left(i,j\right)\in \mathcal{E}$ denotes that node $j$ can receive information from node $i$. Since the graph is undirected, we have $\left(i,j\right)\in \mathcal{E}\iff \left(j,i\right)\in \mathcal{E}$. A path from node $i$ to node $j$ is a sequence of consecutive edges $\left\{\left(i,m\right),\left(m,q\right),\cdots ,\left(l,j\right)\right\}$ with distinct vertices. A path that starts from a given node and ends at the same node is called a cycle. An undirected graph $\mathcal{G}$ is connected if a path exists between every pair of nodes. The distance between two nodes in a graph is the length/number of consecutive edges connecting them. The neighbor set of node $i$ is defined as ${\mathcal{N}}_i=\left\{j|\left(j,i\right)\in \mathcal{E},j\ne i\right\}$.

3. Multi-agent cooperative system

Consider a multi-agent cooperative system (MACS) ${\mathrm{\Sigma }}_s$ consisting of $n$ agents whose individual dynamics are given by

$${\dot{x}}_i=u_i,i=\left\{1,\cdots ,n\right\}$$E1

where $x_i\in \mathbb{R}$ is the state of physical variables to be controlled and $u_i$ is the control input. The agents can exchange information with each other via a communication network whose network topology is modeled by a graph $\mathcal{G}$ whose nodes represent the agents. It is assumed that the communication between a pair of agents is bidirectional, that is graph $\mathcal{G}$ is undirected and an edge $\left(i,j\right)\in \mathcal{E}$ denotes that agents $i$ and $j$ can exchange information with each other.

In this book chapter, we focus on a consensus problem where the goal is to design a local interaction rule $u_i$ for an individual agent, which depends on its own and its neighboring agents’ states, such that the state of all the agents in the network reach a consensus, i.e.,

$$x_i\to c,\forall i\text{and}c\ne 0.$$E2

To this end, each agent executes the following consensus law [1].

$$u_i=\sum _{j\in {\mathcal{N}}_i}\left(x_j-x_i\right).$$E3

Defining a vector $x={\left[x_1,\cdots ,x_n\right]}^T$, the dynamics (1) under consensus law (3) can be written in a compact form as

$$\dot{x}=\mathit{Ax}=-\mathit{Lx},$$E4

where $L$ is called a Laplacian matrix associated with the graph $\mathcal{G}$. If the communication network topology $\mathcal{G}$ is connected, then all the agents’ states reach a consensus whose value is equal to the average of the initial conditions $x_i\left(0\right)$ of all the agents [1], as depicted in Figure 1b. In other words, we have

Figure 1.

(a) Network topology of MACS consisting of six agents; (b) under dynamics (8) and in the absence of attacks (d = 0), the states $x_i$ reach average consensus given in (5); (c) The attacker is able to prevent the agents from reaching consensus by launching FDI attacks given in (9) and modifying information sent from agents 1, 2, 3, and 6.

$$x\to x_{\mathit{ave}}{1}_n=\underset{x_{\mathit{ave}}}{\underbrace{\left(\frac{1}{n}\sum _{i=1}^n{x}_i\left(0\right)\right)}}{1}_n.$$E5

Remark 1. The work [16] presents a systematic design for general classes of nonlinear heterogeneous cooperative systems. Specifically, if the individual dynamics can become input passivity-short by a local feedback controller, the dynamic behaviors of the agents at the network level as well as their network control design can then be transformed to (3). Hence, the design presented in the book chapter has broad implications for general cooperative systems.

3.1 System identification toward defending against cyber-attacks on MACS

In practice, the communication network is vulnerable to cyber-attacks. Specifically, this book chapter considers the following attacks: (i) false data injection (FDI) attacks which destroy data integrity of the cooperative system by injecting malicious signals into the communication channels to alter exchanged data; and (ii) denial-of-service (DoS) attacks which block the communication channels causing unavailability of data.

Under DoS attacks, the system matrix $A=\left[a_{\mathit{ij}}\right]$ (i.e., Laplacian matrix $-L$) in (4) changes, and the cooperative system may lose its consensus. While agent $i$ can locally estimate the $i$-th row of the matrix $A$ (which corresponds to the information it receives from its direct neighbors) using the approach presented in Section 6, cooperative stability and performance of MACS are determined by connectivity, collective property of system parameters rather than any specific parameter. There are two ways to identify and thwart DoS attacks. One way is to identify all the system parameter changes in $a_{\mathit{ij}}$ and then attempt to maintain or recover all the original values of $a_{\mathit{ij}}$. The challenge lies in developing a distributed algorithm so that the individual agent can estimate all the parameters of the matrix $A$. To this end, each agent can perform the following consensus algorithm

$${\dot{\widehat{A}}}_i=\sum _{j\in {\mathcal{N}}_i}\left({\widehat{A}}_j-{\widehat{A}}_i\right).$$E6

Here, matrix ${\widehat{A}}_i$ denotes the agent $i$‘s a local estimate of the matrix $A$ with

$${\widehat{A}}_i\left(0\right)=n\left[\begin{array}{c}{\mathbf{0}}_n^T\\ ⋮\\ {\mathbf{0}}_n^T\\ {}^{}\left[a_{i\ast }^T\right]\\ {\mathbf{0}}_n^T\\ ⋮\end{array}\right]\begin{array}{c}\\ \\ \to i‐\mathrm{th}\mathrm{row}\\ \\ \end{array}$$

where $\left[a_{i\ast }\right]\in {\mathbb{R}}^n$ denotes the $i$-th row of the matrix $A$. The agents local estimate will then converge to the matrix $A$ given that the communication network’s topology is connected. However, this approach has several limitations. First, it is expensive as each agent has to $n^2$ exchange number of information with its neighbors. Furthermore, the network’s connectivity may have already been destroyed before each agent can estimate the overall system parameters of $A$. To overcome the limitation of the approach described previously and to quantify the connectivity property of a network, let us define the path $p_{\mathit{ij}}$ between node $i$ and node $j$ as

$$p_{\mathit{ij}}=\prod _{k_1,k_2,\cdots ,k_l}{a}_{{\mathit{ik}}_1}{a}_{k_1{k}_2}\cdots a_{k_{l}j},$$E7

where the sequence $k_1,k_2,\cdots ,k_l$ of represents the set of nodes corresponding to the sequence of edges $\left(i,k_1\right),\left(k_1,k_2\right),\cdots ,\left(k_l,j\right)$ which connect the two nodes. Since the original graph is connected, the node $i$ knows that $p_{\mathit{ij}}=1$ for any choice of $j$. Hence, the second way of distributively detecting DoS attacks and making the system resilient to such attacks is to distributively monitoring/estimating $p_{\mathit{ij}}$ in real-time and ensuring that, at any time, there are at least two distinct paths (or existence of a loop/cycle) between nodes $i$ and $j$ (such that any DoS attack will not destroy the connectivity). Accordingly, one focus of this book chapter is to develop such a distributed algorithm for node $i$ to identify/monitor $p_{\mathit{ij}}$ and maintain the existence of two distinct $p_{\mathit{ij}}=1$ for any of the node $i$‘s neighboring node $j\in {\mathcal{N}}_i$. As will be discussed in more detail later, this approach only requires each agent to exchange $n$ number of information with its neighbors.

In the following subsections, the vulnerability of the MACS under the consensus algorithm (4) against both attack vectors will be analyzed in more detail.

3.2 Vulnerability of MACS under FDI attacks

The consensus dynamics (4) under unknown FDI attacks can be written as

$$\dot{x}=-L\left(x-d\right),$$E8

where $d\left(t\right)\in {\mathbb{R}}^n$ is an unknown injection launched by the attacker. Specifically, $d_i\left(t\right)\ne 0$ means that the attacker modifies the information sent by the agent to its neighbors by injecting a malicious signal $d_i\left(t\right)$ and also corrupts the local state feedback of the agent $i$. It is assumed that the attacker has knowledge of the network topology of the MACS, that is the Laplacian matrix $L$, and also have access to the physical state $x$ which can used to launch the exogenous injections $d$. In addition, it is also assumed that the unknown injection $d$ is bounded. This assumption is a reasonable precaution for intelligent attackers to avoid detection. Furthermore, it also aligns with practical considerations, as physical signals, such as those in power systems, typically have a known range. Note that unbounded injections can then be readily rejected using a threshold check [17].

There are many possible choices of $d$ which can prevent the MACS from reaching an average consensus. For example, the attacker can generate the injection $d$ according to the following dynamics, unknown to the defender:

$$\dot{d}=F_{a}d+B_{a}x,$$E9

where matrix $F_a$ is Hurwitz, and together with matrix $B_a$ are designed by the attacker to ensure that one of the eigenvalues of the matrix $\left[\begin{array}{cc}-L& L\\ B_a& F_a\end{array}\right]$ has a positive real part as illustrated in Figure 1c.

3.3 Vulnerability of MACS under DoS attacks

Unlike FDI attacks that compromise data integrity, DoS attacks disrupt the availability of communication channels, effectively isolating agents and impeding their ability to reach consensus. During a DoS attack, malicious actors flood the network’s communication channels with overwhelming traffic, rendering them inoperable. This disruption in communication can be modeled by intermittently adjusting the Laplacian matrix $L\left(t\right)$ of the consensus algorithm to reflect the compromised links. Consequently, the consensus dynamics under DoS attacks can be expressed as:

$$\dot{x}=A\left(t\right)=-L\left(t\right)x,$$E10

where $L\left(t\right)$ dynamically changes based on the network’s connectivity at the time $t$, showcasing the impact of DoS attacks.

The resilience of MACS against DoS attacks is significantly influenced by the network’s structural properties. The initial impact of a DoS attack on a well-connected MACS might not immediately lead to a disconnection of the network. However, persistent or intense DoS attacks can gradually affect the network’s connectivity, leading to the progressive disconnection of $\mathcal{G}$. This disconnection causes the standard consensus algorithm (4) to fail in achieving a unified state among agents.

Critical edges within the communication graph $\mathcal{G}$ play a pivotal role in this context. These edges, if severed, can dramatically impact the network’s functionality by partitioning the system into disjoint segments. A critical edge, or bridge, in an undirected network, is defined as an edge whose removal leads to the disconnection of the network, thereby identifying it as a singular point of vulnerability within the network’s topology [18]. Borrowing the notation in (7), an edge $\left(i,j\right)$ is critical means that there is only one distinct path with $p_{\mathit{ij}}=1$.

Edge connectivity, on the other hand, is a fundamental metric of network robustness, quantifying the minimum number of edges that must be removed to render the network disconnected. The presence of critical edges directly impacts a network’s edge connectivity; specifically, a network with even a single critical edge has an edge connectivity of 1. A network with high-edge connectivity is more robust and capable of withstanding multiple edge failures without losing overall connectivity. In contrast, networks with critical edges are vulnerable, as the failure of just one such edge can compromise network integrity [19].

For example, in Figure 1a, the edges between agent pairs (3,4), (4,5), and (5,6) are critical, and it causes the edge connectivity of the network connecting MACS to be one. It follows that identifying and eliminating the criticality of these edges enhances the robustness of MACS and ensures its resilience to disruptions such as failures or targeted DoS attacks.

4. Resilient multi-agent cooperative system against DoS attacks

In what follows, two methods are introduced to robustify MACS against DoS attacks. The first is to make MACS robust by applying the algorithm in Ref. [20] of identifying critical edges within the network’s communication graph $\mathcal{G}$ as a result of DoS attacks so that a network reconfiguration could be done to ensure reliable consensus among agents. The second is to introduce a digital twin of the MACS to achieve robustness.

4.1 Distributed algorithms for network reconfiguration

The reconfiguration process is aided by two distributed algorithms, each fulfilling a specific role. The first algorithm is used by the individual agent $i$ to determine its neighbors, and the second is to determine if any of the edges $\left(i,j\right),j\in {\mathcal{N}}_i$, is critical. Upon identifying the critical edge(s), additional edge(s) could be added to ensure resilience against DoS attacks.

Consider two key state vectors for each agent $i\in \mathcal{N}$: ${\xi }_i\left(k\right)={\left[{\xi }_{i,1}\left(k\right)\cdots {\xi }_{i,n}\left(k\right)\right]}^T\in {\mathbb{R}}^n$ denoting the agent $i$‘s estimate of its neighbors, and ${\omega }_i\left(k\right)={\left[{\omega }_{i,1}\left(k\right)\cdots {\omega }_{i,n}\left(k\right)\right]}^T\in {\mathbb{R}}^n$ denoting agent $i$‘s estimate of its neighbor structure. First, each agent $i$ executes the following distributed maximum and minimum protocols for $n$ consecutive iterations, that is $k=0,\cdots ,\left(n-1\right)$:

$${\xi }_{i,j}\left(k+1\right)=\underset{l\in {\mathcal{N}}_i\cup i}{max}{\xi }_{l,j}\left(k\right),j\in \mathcal{N},$$E11

$${\omega }_{i,j}\left(k+1\right)=\left\{\begin{array}{ll}{\omega }_{i,j}\left(k\right)& \text{if}{\xi }_{i,j}\left(k+1\right)={\xi }_{i,j}\left(k\right),\\ \underset{l\in {\mathcal{N}}_i}{min}\left({\omega }_{l,j}\left(k\right)+1\right)& \text{if}{\xi }_{i,j}\left(k+1\right)>{\xi }_{i,j}\left(k\right)\end{array}\right.$$E12

where the initial values are set to

$${\xi }_{i,j}\left(0\right)=\left\{\begin{array}{ll}1,& \text{if}j=i\\ 0,& \text{otherwise}\end{array}\right.;{\omega }_{i,j}\left(0\right)=\left\{\begin{array}{ll}0,& \text{if}j=i\\ \infty ,& \text{otherwise}\end{array}\right.$$E13

That is, at the initialization step $k=0$, agent $i$ is aware only of itself, setting ${\xi }_{i,i}\left(0\right)=1$ and all other ${\xi }_{i,j}\left(0\right)=0$, and similarly, ${\omega }_{i,i}\left(0\right)=0$ with all other ${\omega }_{i,j}\left(0\right)$ set to infinity. In subsequent iterations, the agent $i$ updates its state based on the reachability of other agents. The state ${\xi }_{i,j}\left(k+1\right)$ turns to $1$ when agent $j$ becomes reachable within $k+1$ steps from agent $i$ through any neighbor $l$. Concurrently, the algorithm computes the shortest path ${\omega }_{i,j}\left(k+1\right)$, that is the minimum of ${\omega }_{l,j}\left(k\right)+1$ across all neighbors $l\in {\mathcal{N}}_i$ that can reach $j$ whenever a new path is found. These values are non-decreasing and only change when new paths are discovered.

Example 1.Consider the MACS inFigure 1a, specifically agent 4. The neighbor set of agent 4 is given by ${\mathcal{N}}_4=\left\{3,5\right\}$. Initially at$k=0$,

$${\xi }_4\left(0\right)={\left[\mathrm{0,0,0,1,0,0}\right]}^T;{\omega }_4\left(0\right)={\left[\infty ,\infty ,\infty ,0,\infty ,\infty \right]}^T.$$

Then, at$k=1$

$${\xi }_4\left(1\right)={\left[\mathrm{0,0,1,1,1,0}\right]}^T;{\omega }_4\left(1\right)={\left[\infty ,\infty ,\mathrm{1,0,1},\infty \right]}^T.$$

Again at $k=2$,

$${\xi }_4\left(2\right)={\left[\mathrm{1,1,1,1,1,1}\right]}^T;{\omega }_4\left(2\right)={\left[\mathrm{2,2,1,0,1,2}\right]}^T.$$

For the next 4 steps, until$k=6$,${\xi }_4$and${\omega }_4$remain the same since there are no new neighbors.

Within the next two steps, that is $k=n\cdots \left(n+2\right)$, agent $i$ aims to identify whether any of the edges $\left(i,j\right)$, $j\in {\mathcal{N}}_i$, is critical to estimate if there are at least two distinct paths $p_{\mathit{ij}}=1$ between nodes $i$ and $j$. The identification process leverages a specific measure, ${\mathrm{\Delta }}_i^{\left(\mathit{il}\right)}$ and $j$, to discern the necessity of each edge for maintaining connectivity.

Measure Definition: The criticality of an edge $\left(i,l\right)$ between agents $i$ and $l$ is assessed through:

$${\mathrm{\Delta }}_i^{\left(\mathit{il}\right)}={\left[{\mathrm{\Delta }}_{i,1}^{\left(\mathit{il}\right)}\cdots {\mathrm{\Delta }}_{i,n}^{\left(\mathit{il}\right)}\right]}^T,{\mathrm{\Delta }}_{i,j}^{\left(\mathit{il}\right)}={\omega }_{i,j}\left(n\right)-{\omega }_{l,j}\left(n\right),$$E14

where ${\mathrm{\Delta }}_{i,j}^{\left(\mathit{il}\right)}$ captures the shortest path discrepancy from any agent $j$ to $i$ and $l$, highlighting paths bypassing $\left(i,l\right)$.

Specifically, in a connected network, the value of ${\mathrm{\Delta }}_{i,j}^{\left(\mathit{il}\right)}$ for each pair of neighboring agents $i$ and $l$, with respect to any other agent $j$, can only be $-1$, $0$, or $1$. A value of $-1$ indicates that agent $j$ is relatively closer to the agent $i$ than to agent $l$, and conversely, a value of $1$ suggests $j$ agent is nearer to the agent $l$. A zero value implies that the agent $j$ is equidistant from both agents $i$ and $l$.

For every agent $j\in \mathcal{N}$ and for all adjacent agents $i^{\text{'}}\in {\mathcal{N}}_i$ and $l^{\text{'}}\in {\mathcal{N}}_l$, an edge $\left(i,l\right)$ is not a critical edge, i.e., there are at least two distinct paths $p_{\mathit{il}}=1$ (existence of cycle/loop), under these two scenarios:

• If an agent $j$ is equidistant to agents $i$ and $l$, it indicates the presence of a cycle that does not rely on $\left(i,l\right)$. Such an agent’s distance measure, ${\mathrm{\Delta }}_{i,j}^{\left(\mathit{il}\right)}$, would be zero, demonstrating that $\left(i,l\right)$ is part of a cycle and thus non-critical.

• If, for any agent $j$, there exist neighboring agents $i^{\prime }$ of $i$ and $l^{\prime }$ of $l$ such that agent $j$ is closer to both $i^{\prime }$ and $l^{\prime }$ than to $i$ and $l$, it suggests that $j$ is on a path forming a cycle with $\left(i,l\right)$. The distance measures, ${\mathrm{\Delta }}_{i,j}^{\left(i{i}^{\prime }\right)}$ and ${\mathrm{\Delta }}_{l,j}^{\left(l{l}^{\prime }\right)}$, would both be positive, confirming the existence of a cycle.

An edge is determined to be critical when no such equidistant agent or cycle patterns are found, indicating the absence of an alternative path. This means that removing this edge would result in a disconnection of the network.

Example 2.Consider the MACS inFigure 1a, specifically the edge between agents$4$and$5$. The neighbor sets are${\mathcal{N}}_4=\left\{3,5\right\}$,${\mathcal{N}}_5=\left\{4,6\right\}$. Then we have,

$$\begin{array}{c}{\mathrm{\Delta }}_4^{4,5}={\left[-1,-1,-1,-\mathrm{1,1,1}\right]}^T,\\ {\mathrm{\Delta }}_4^{4,3}={\left[\mathrm{1,1,1},-1,-1,-1\right]}^T,\\ {\mathrm{\Delta }}_5^{5,6}={\left[-1,-1,-1,-1,-1,1\right]}^T.\end{array}$$

Since neither of the scenarios for non-critical edges matches, it can be concluded that the edge$\left(4,5\right)$is a critical edge and the only path between nodes 4 and 5 is the edge (4,5).

The next step is executed only by individually each pair of agents associated with a critical edge, say $\left\{i,l\right\}$ with $i\in {\mathcal{N}}_l^c$ and $l\in {\mathcal{N}}_i^c$. The goal of this step is to identify the most remote pairs of agents adjacent to each critical edge, called augmentation nodes, $\left\{i^{\text{'}},l^{\text{'}}\right\}\in {\mathcal{N}}_i^r$. These augmentation nodes are strategically chosen as the ones farthest from the critical edge $\left(i,l\right)$. If there exist multiple agents at the same distance, the ones with the smallest index are chosen, and it reduces the number of edges added.

$$\left\{i^{\text{'}},l^{\text{'}}\right\}\in {\mathcal{N}}_i^r\text{if}\left(16\right)\text{or}\left(17\right)\text{or}\left(18\right)\text{is true},$$E15

where

$$\left\{\begin{array}{l}{\mu }_i=\mid {\mathcal{N}}_i\mid >1\\ {\mu }_l=\mid {\mathcal{N}}_l\mid >1\end{array}\right.\text{and}\left\{\begin{array}{l}i\in {\mathcal{N}}_l^c,l\in {\mathcal{N}}_i^c,\\ {\mathrm{\Delta }}_{i,i^{\text{'}}}^{\left(\mathit{il}\right)}={\mathrm{\Delta }}_{l,l^{\text{'}}}^{\left(\mathit{li}\right)}=-1\\ {\omega }_{i,i^{\text{'}}}=\underset{k}{max}{\omega }_{i,k},\\ {\omega }_{l,l^{\text{'}}}=\underset{k}{max}{\omega }_{l,k},\end{array}\right.,$$E16

$$\left\{\begin{array}{l}{\mu }_i=\mid {\mathcal{N}}_i\mid =1\\ {\mu }_l=\mid {\mathcal{N}}_l\mid >1\end{array}\right.\text{and}\left\{\begin{array}{l}i\in {\mathcal{N}}_l^c,l\in {\mathcal{N}}_i^c,\\ {\mathrm{\Delta }}_{l,l^{\text{'}}}^{\left(\mathit{li}\right)}=-1\\ i^{\text{'}}=i,\\ {\omega }_{l,l^{\text{'}}}=\underset{k}{max}{\omega }_{l,k},\end{array}\right.,$$E17

$$\left\{\begin{array}{l}{\mu }_i=\mid {\mathcal{N}}_i\mid >1\\ {\mu }_l=\mid {\mathcal{N}}_l\mid =1\end{array}\right.\text{and}\left\{\begin{array}{l}i\in {\mathcal{N}}_l^c,l\in {\mathcal{N}}_i^c,\\ {\mathrm{\Delta }}_{i,i^{\text{'}}}^{\left(\mathit{il}\right)}=-1\\ {\omega }_{i,i^{\text{'}}}=\underset{k}{max}{\omega }_{i,k},\\ l^{\text{'}}=l,\end{array}\right.,$$E18

and ${\mathcal{N}}^r={\cup }_{i\in \mathcal{N}}{\mathcal{N}}_i^r$ is the augmentation action set to be found distributively. Note that ${\mu }_i=\mid {\mathcal{N}}_i\mid ={\mu }_l=\mid {\mathcal{N}}_l\mid =1$ this is the trivial case of a two-agent network and hence is excluded from consideration.

The next $\left(n-1\right)$ steps is to propagate ${\mathcal{N}}_i^r$ to all the agents so they all have access to ${\mathcal{N}}^r$ as follows:

$${\mathcal{N}}_i^r\left(k+1\right)={\cup }_{l\in {\mathcal{N}}_i\cup \left\{i\right\}}{\mathcal{N}}_l^r\left(k\right),$$E19

where $k=\left(2n+2\right),\cdots ,\left(3n+1\right)$, $\cup$, is the union operation of sets containing non-ordering pairs (that is, if $\left\{i,j\right\}\subset {\mathcal{N}}_l^r\left(k\right)$, then $\left\{j,i\right\}\subset {\mathcal{N}}_l^r\left(k\right)$), and ${\mathcal{N}}_i^r\left(3n+2\right)={\mathcal{N}}_i^r$ is given by (15).

And, as the final step, edge addition is accomplished by the pairs of agents identified in ${\mathcal{N}}^r$ to complete their connection, thereby eliminating critical edges by ensuring there are at least two distinct paths $p_{\mathit{ij}}=1$ for any pair of nodes $\left(i,j\right)\in \mathcal{E}$. As a result, the network’s edge connectivity is increased to $2$ which enhances the MACS’s resilience against DoS attacks. This approach connects distant edges within acyclic parts of the network, specifically, those separated by critical edges, to both preserve network connectivity under attack and minimize the additional connections required. Further details can be found in Ref. [20].

4.2 Resilient multi-agent cooperative system against DoS attacks: a digital twin approach

An alternative approach to removing the critical edges is achieved by interconnecting the MACS with its digital twin ${\mathrm{\Sigma }}_{\mathit{dt}}$ as illustrated in Figure 2. Specifically, the digital twin consists of virtual agents with a similar number to the MACS and each virtual agent maintains a state $z_i$. The dynamics of the MACS interconnected with its digital twin are designed as

Figure 2.

Interconnection of MACS ${\Sigma }_s$ with its digital twin ${\Sigma }_{\mathit{dt}}$. The digital twin consists of the same number of (virtual) agents and network topology as the MACS. Furthermore, each virtual agent maintains a virtual state $z_i$.

$$\begin{array}{l}\dot{x}=-L\left(t\right)x+\mathit{Lz},\\ \dot{z}=-\mathit{Lz}-\mathit{Lx},\end{array}$$E20

where $z={\left[z_1,\cdots z_n\right]}^T$. In contrast to the physical state $x_i$, the state of the virtual agent $z_i$ does not have any physical meaning, hence it is also called a virtual state, which makes it less interesting to the attacker. Furthermore, its initial condition $z_i\left(0\right)$ can be set to any value. The digital twin can be implemented using cloud computing in combination with software-defined networking [21] to direct traffic in the network. The network of the digital twin, denoted by ${\mathcal{G}}_v=\left(\mathcal{V},{\mathcal{E}}_v\right)$, has the same structure as the MACS and we also call it a virtual network. In addition to the physical state $x_j$, agent $j$ also sends the virtual state $z_j$ and the masked physical state $x_j+z_j$ to its neighbors via the virtual network.

The interconnection of the physical network and its digital twin is a competitive interaction from a game-theoretical point of view. More on this can be found in the next section. It is worth noting that the interconnection with the digital twin does not interfere with the operation of the MACS. In other words, under this interconnection, the physical states of the MACS can still be ensured to reach average consensus in (5) as shown in Figure 3, refer [22].

Figure 3.

Physical states of all the agents converge to the average consensus under interconnected dynamics (20) and with network topology given in Figure 1a.

If a connected digital twin is introduced, the critical edges are automatically eliminated since the existence of a connected digital twin effectively introduces redundancy in the network’s connectivity. This redundancy ensures that even if critical edges in the physical network are severed due to attacks, the interconnected system can maintain operational integrity through the virtual network. This setup leverages the parallelism between the physical and virtual states, allowing the system to sidestep vulnerabilities exposed by the critical edges in the physical network.

Example 3.Consider a connected MACS with a digital twin where the adversary launches DoS attacks on the network${\mathrm{\Sigma }}_s$. Consequently, we denote the Laplacian matrix of the physical network post-attacks as${\mathcal{L}}^a$. Then, for following Lyapunov functions$V\left(x,z\right)$,

$$V=x^{T}x+z^{T}z,$$E21

it follows that under DoS attacks,

$$\begin{array}{c}\dot{V}\le -x^T{L}^{a}x+x^T{L}_{\mathit{dt}}z-z^T{L}_{\mathit{dt}}z-z^T{L}_{\mathit{dt}}x\\ =-x^T{L}^{a}x-z^T{L}_{\mathit{dt}}z,\end{array}$$E22

where ${\mathrm{L}}_{\mathrm{dt}}=\mathrm{L}$. Furthermore, by analyzing the invariant set of$\dot{V}$and since the graph${\mathcal{G}}_v$is connected, it can be concluded that the average consensus(5)is ensured under any number of DoS attacks on the physical network${\mathrm{\Sigma }}_s$, as illustrated in Figure 4.

Figure 4.

Physical states of all the agents correspond to the interconnected dynamics (20) and converge to the average consensus under DoS attacks on the physical network ${\Sigma }_s$. Specifically, the adversary disrupts the links (3,4) and (5,6) in Figure 1a, resulting in a disconnected network $\mathcal{G}$.

5. Resilient multi-agent cooperative system against FDI attacks: a competitive interaction design via digital twin

The interconnection of the MACS with its digital twin can also be extended to ensure the resilient operation of MACS under FDI attacks. To this end, we introduce a scalar gain $\beta >0$ which represents the strength of the interconnection, resulting in a competitive interaction between the MACS and its digital twin. Hence, the dynamics (20) with the gain $\beta$ and under unknown FDI attacks can then be written as:

$$\begin{array}{l}\dot{x}=-\mathit{Lx}+\mathit{\beta Lz}+\mathrm{Ld},\\ \dot{z}=-\mathit{Lz}-\mathit{\beta Lx}+{\mathrm{Ld}}^{\text{'}},\end{array}$$E23

where it is assumed that the attacker can also modify the information being exchanged via the digital twin’s network by inserting a bounded injection $d^{\text{'}}$. Here, agent $j$ sends the information of ${\beta z}_j$ and the masked physical state $\beta x_j+z_j$ via the virtual network.

The following results, illustrated in Figure 5, can then be obtained [23]:

1. In the absence of FDI attacks, i.e., $d=d^{\text{'}}=0$, the physical states of all the agents reach an average consensus (5).

2. Under unknown but bounded injections $d,d^{\text{'}}$, the physical states of all the agents will converge close to is nominal average consensus value for a sufficiently large value of $\beta$, that is

Figure 5.

Physical states of all the agents converge close to the nominal average consensus under interconnected dynamics (23) with $\beta =\mathit{150}$ and injections $d,d^{\text{'}}$ are given in (9). The network topology of MACS is given in Figure 1a.

$$\left|\underset{t\to \infty }{lim}{x}_i\left(t\right)-x_{\mathit{ave}}\right|\le \epsilon ,i=1,\cdots ,n,$$E24

where $\epsilon$ is a small non-negative constant.

Intuitively, the digital twin acts as an anchor that ensures the resilient operation of the cooperative systems ${\mathrm{\Sigma }}_s$ for a sufficiently large $\beta$. From another perspective, it can be observed that the interactions between the cooperative systems and their digital twin are competitive whose interconnected dynamics (23) correspond to the saddle-point dynamics of a zero-sum game between the two networks [24]. Specifically, the cooperative systems wish to minimize the convex-concave payoff $U=\frac{1}{2}{x}^T\mathit{Lx}-\frac{1}{2}{z}^T\mathit{Lz}-\beta x^T\mathit{Lz}$ function by choosing $x$ while its digital twin aims to maximize $U$ by choosing $z$ [24]. Note that the saddle points of $U$ for a sufficiently large $\beta$ is given by $\mathcal{S}=\text{span}\left\{\left(1_n,0_n\right),\left(0_n,1_n\right)\right\}$.

6. Cyber-attack detection and identification

The digital twin not only enhances the resilient of the MACS but also facilitates the detection and identification of cyber-attacks in a distributed manner. Let us denote the possibly corrupted information sent by agent $i$ under dynamics (23) as ${\overline{x}}_j,{\overline{z}}_j,{\overline{m}}_j$ and given by

$$\begin{array}{c}{\overline{x}}_j=x_j+d_j,\\ {\overline{z}}_j=\beta z_j+d_j^{\text{'}},\\ {\overline{m}}_j=z_j+\beta x_j+d_j^{\text{'}}.\end{array}$$E25

where ${\overline{x}}_j$ is transmitted via the network of the MACS while ${\overline{z}}_j$ and ${\overline{m}}_j$ are sent through the virtual network of the digital twin. Using this information, the agent $i$ can detect whether the information that it receives from its neighboring agent is corrupted.

First, let us focus on the detection of DoS attacks. The DoS attack on the link $\left(i,j\right)\in \mathcal{E}$ (resp. $\left(i,j\right)\in {\mathcal{E}}_v$) at a certain time results in that agent $i$ does not receive any information from the agent $j$ via the network $\mathcal{G}$ (resp. ${\mathcal{G}}_v$) and vice versa. Based on this fact, the agent $i$ can detect the DoS attack on a link $\left(i,j\right)\in \mathcal{E}$ or $\left(i,j\right)\in {\mathcal{E}}_v$ according to the following simple rules:

1. Link is $\left(i,j\right)\in \mathcal{E}$ under DoS attack if and only if ${\overline{x}}_j=0$

2. Link $\left(i,j\right)\in {\mathcal{E}}_v$ is under DoS attack if and only if ${\overline{z}}_j=0$ and ${\overline{m}}_j=0$

Note that the above condition can be distributively checked by the individual agent. Next, we describe how the individual agent can detect the FDI attack on the communication link. The idea is for agent $i$ to estimate the physical state of its neighboring agent $x_j$ using the information that it receives via the virtual network and then to compare the estimated physical state ${\widehat{x}}_j^i$ with the one received via the network of the MACS [25, 26]. To this end, the estimated physical state ${\widehat{x}}_j^i$ by the agent $i$ can be computed from the possibly corrupted information ${\overline{z}}_j$ and ${\overline{m}}_j$ in (25) as follow:

$${\widehat{x}}_j^i=\frac{1}{\beta }\left({\overline{m}}_j-\frac{{\overline{z}}_j}{\beta }\right).$$E26

Finally, agent $i$ can then detect whether the link $\left(i,j\right)\in \mathcal{E}$ or $\left(i,j\right)\in {\mathcal{E}}_v$ is under FDI attacks according to the following rule

1. If the value $\mid {\widehat{x}}_j^i-{\overline{x}}_j\mid >0$, then agent $i$ can conclude that the communication link $\left(i,j\right)\in \mathcal{E}$ and/or $\left(i,j\right)\in {\mathcal{E}}_v$ is under FDI attacks. Note that, however, agent $i$ will not be able to distinguish between the following two cases without any additional information, e.g., if the virtual network is totally secured: the case where one of the links $\left(i,j\right)\in \mathcal{E}$ and $\left(i,j\right)\in {\mathcal{E}}_v$ is being attacked and the case where both links in physical and virtual networks are being attacked.

2. If the value $\mid {\widehat{x}}_j^i-{\overline{x}}_j\mid =0$, then agent $i$ can conclude that there are no FDI attacks on links $\left(i,j\right)\in \mathcal{E}$ and/or $\left(i,j\right)\in {\mathcal{E}}_v$ or the attacker successfully launches a stealthy attack. In order to launch stealthy attacks, the attacker must know the structure of virtual states in ${\overline{z}}_j,{\overline{m}}_j$(25) and how they are related to the physical state $x_j$. However, it is challenging for the attacker to learn these correlations as the gain $\beta$ is a local information and the virtual state $z_i$ is dynamic (changes over time) and its initial value can be arbitrarily set by the agents.

Figure 6 illustrates how agent $4$ detects the FDI attacks on the link (4,3) for the network and attack shown in Figure 1. Once the agents identify the compromised communication links, they can then cooperatively remove those links [27] and create new links to the network [28] to ensure its connectivity so that the MACS can recover its nominal average consensus value.

Figure 6.

Attack identification by agent 4 for FDI attacks is shown in Figure 1. By computing the value of $\mid {\widehat{x}}_j^i-{\overline{x}}_j\mid$, agent $4$ can detect that the communication link (4, 3) is being compromised while the link (4, 5) is not compromised.

7. Conclusions

The book chapter presents a distributed strategy to enhance the resilience of multi-agent cooperative systems against FDI and DoS attacks. First, a distributed algorithm is proposed to identify critical edges, followed by introducing an adaptive network reconfiguration algorithm to remove those critical edges, thus ensuring resilient operation against arbitrary DoS attacks. A digital twin, interconnected with the cooperative systems, is further designed to increase redundancy of the network and protect the cooperative system from FDI attacks. While this book chapter considers attacks on data integrity and availability, the proposed digital twin is also promising to encounter attacks on data confidentiality [29]. Future work includes the extension of the proposed defense framework to address simultaneous FDI and DoS attacks on both the cooperative systems and its digital twin.

Acknowledgments

The work of Azwirman Gusrialdi was supported by the Research Council of Finland under Academy Project 330073. The work of Zhihua Qu was supported in part by the U.S. Department of Energy’s under Award DE-EE0007998, Award DE-EE0009028, Award DE-EE0009152, Award DE-EE0009339, and Award DE-AC05-76RL01830.