Physics-Informed Neural Network: Principles and Applications

Abstract

This chapter delves into the fascinating characteristics of physics-informed neural networks (PINNs) by outlining their fundamental principles, including their mathematical foundations and structures. PINNs are designed by incorporating governing physical equations into the loss function as constraints, which helps to ensure precise output predictions even in areas with limited or no data. This chapter presents various strategies to apply PINNs to complex systems, thereby addressing the shortcomings of conventional PINNs. Additionally, multiphysics-informed neural networks (MPINNs) are introduced, with a special emphasis on complex mechatronic systems. The effectiveness of the MPINN framework is illustrated through examples such as an electric motor and a lithium-ion battery, demonstrating accurate and efficient multidimensional predictions for mechatronic systems despite limited data availability. These applications underscore the potential of MPINNs to mitigate data scarcity challenges in various industries.

1. Introduction

Deep neural networks (DNNs) are widely used in various domains, including natural language processing [1, 2, 3] and image recognition [4, 5, 6, 7, 8]. For natural language processing, DNNs are employed in tasks like machine translation, sentiment analysis, and speech recognition, significantly enhancing the understanding and generation of human language. In the field of image recognition, DNNs are instrumental in object classification, feature detection, and image segmentation, finding applications in facial recognition systems and medical imaging diagnostics. The application of DNNs has surged in recent years across various scientific and engineering fields [9, 10, 11, 12, 13], including mechatronics, fluid dynamics, materials science, and biomedical engineering. The universal approximation theorem highlights the capability of DNNs to model any continuous function, enabling the effective estimation of complex and nonlinear behaviors in diverse systems [14]. Specifically, DNNs capture complex and nonlinear relationships between input and output through a training process that optimizes a loss function quantifying the discrepancy between predicted and actual output. Therefore, the performance of DNNs significantly depends on the volume of data available to account for system characteristics, suggesting that collecting sufficient high-quality data is crucial for ensuring robustness and generality in real-world applications. This extensive data requirement can be costly and resource-intensive, constraining the accurate prediction of system behaviors when relying on purely data-driven approaches [15]. Furthermore, DNNs are often considered “black boxes” because the decision-making processes are not easily interpretable. This lack of transparency in data-only training can be problematic in critical applications, where understanding the model’s inferences is essential for trust and validation.

To overcome these limitations, research has shifted toward hybrid approaches that combine the robustness of physics-based models with the adaptability of data-driven methods [16, 17, 18]. One notable strategy in this area involves incorporating knowledge-based features derived through feature engineering. This approach addresses domain-specific knowledge to enhance model accuracy by integrating the strengths of both fundamental physics and data-driven predictions. However, these methods still struggle with managing noisy data and outliers effectively [19]. Data noise can obscure underlying relationships between input and output, complicating the learning process for models. Outliers, on the other hand, can distort model predictions, leading to unreliable results. Meanwhile, physics-informed neural networks (PINNs) have gained attention as a powerful hybrid approach in that this method could enhance the fidelity of model predictions by incorporating physics-based regularization directly into the learning process. Specifically, PINNs minimize a loss function that combines both data-derived and physical laws to ensure robust predictions even under data scarcity or noisy situation. These networks are versatile in handling both forward and inverse problems across various time and space scales [20]. For example, Raissi et al. [21] demonstrated the application of PINNs in solving forward and inverse problems for nonlinear PDEs, showcasing their potential in accurately modeling physical systems with limited data. Another study by Tartakovsky et al. [22] applied PINNs to subsurface flow problems, achieving high accuracy in predicting hydraulic conductivity fields.

Despite these advantages, PINNs face challenges such as addressing stiff partial differential equations (PDEs) and managing multi-physics in complex systems. Issues such as spectral bias, where neural networks struggle to accurately capture high-frequency components of solutions [23], and difficulties in applying PINNs to parametric PDEs, which involve varying parameters that change the behavior of the system [24], have further been identified. These challenges can impact the accuracy and robustness of PINNs, necessitating further research and development to enhance their performance in diverse applications. Therefore, various methods have been actively suggested to overcome these challenges, including adaptively weighting each loss term [25, 26, 27, 28], domain decomposition [29, 30], improved architecture to manage interpolation and extrapolation [31, 32], and embedding input coordinate features [32, 33, 34]. These efforts extend the application of PINNs beyond solid mechanics [35], thermal dynamics [36, 37] fluid mechanics [38, 39], and electromagnetics [40], enabling practical use in multiphysics system. Furthermore, increasing insight of overcoming challenges with these hybrid approaches is driving active research to address challenges in new applications.

This chapter provides an overview of the main concepts and principles of PINNs and introduces various methods for their applications with practical examples in mechatronic systems. Specifically, the remainder of this chapter is organized as follows. Section 2 explains the fundamental principles and implementation of PINNs. Section 3 introduces the challenges and overcoming strategies in PINNs. In Section 4, the focus is on practical applications of PINNs in a complex mechatronic system. Finally, the chapter concludes with some remarks in Section 5.

2. Principles of physics-informed neural network

Physics-informed neural networks (PINNs) are designed to solve physical problems characterized by limited data availability, such as noisy measurements and time-consuming simulations [41]. These networks constrain the predicted solution to meet the laws of physics and fundamental principles to enhance predictive capabilities by overcoming a data scarcity problem. The physical laws are typically expressed through the governing equations, including parametrized and nonlinear partial differential equations (PDEs) in a generalized mathematical form [21] as

$$\frac{\partial u}{\partial t}+\mathcal{N}\left[u;\lambda \right]=0,x\in \mathrm{\Omega },t\in \left[0,T\right],$$E1

where $u$, $\mathcal{N}\left(u;\lambda \right)$, and $\mathrm{\Omega }$ denote the hidden solution $u\left(t,x\right)$ depending on time $t$ and spatial coordinates $x$, nonlinear differential operator parameterized by coefficient $\lambda$, and a subset of ${\mathbb{R}}^D$. This form can describe a variety of physics in nature and engineering system governed by dynamics, solid mechanics, thermodynamics, or fluid mechanics [21]. Some prominent examples include the Navier–Stokes equations elucidating fluid flows [36], the heat conduction equations for heat transfer [38], and Cauchy’s equations of motion which depict solid deformation [42]. Therefore, this prior knowledge can be integrated into existing neural network models due to their mathematical representation, deriving the architecture of the PINNs.

The PINN architecture closely resembles a conventional MLP structure because the architecture comprises an input layer, several hidden layers, and an output layer (Figure 1). However, PINNs differ from conventional MLPs due to the placement of the differentiation layer (the red circles with differential symbols in Figure 1), right next to the output layer. Each of these layers plays a distinct and crucial role in the training process of PINNs. The spatiotemporal information, coordinates $x$ and time $t$, are input to the neural network consisting of the hidden layers through the input layer. The neural network subsequently proceeds to predict the hidden solution $u$ of the PDE through the output layers. The outputs $u$ are directed into the subsequent differentiation layer. These layers perform automatic differentiation on neural networks with respect to both their input coordinates $x$ and time $t$ to construct each differential term of the PDE, resulting in the derivation of the loss function for network learning.

Figure 1.

A basic architecture of vanilla PINNs [43].

The loss function $\mathcal{L}$ of a PINN comprises the sum of two loss functions, ${\mathcal{L}}_{\mathit{PDE}}$ and ${\mathcal{L}}_u$ (as)

$$\mathcal{L}={\mathcal{L}}_{\mathit{PDE}}+{\mathcal{L}}_u,$$E2

where ${\mathcal{L}}_{\mathit{PDE}}$ and ${\mathcal{L}}_u$ denote the loss function for penalization of the residual of the PDE, and loss function to satisfy the model predictions for data points [39]. Specifically, the first term ${\mathcal{L}}_{\mathit{PDE}}$ of $\mathcal{L}$ enforces the predicted solution $\widehat{u}$ to obey the PDE at spatiotemporal points in a given computational domain [21] as

$${\mathcal{L}}_{\mathit{PDE}}=\frac{1}{N_f}\sum _{i=1}^{N_f}{\left|f\left(t_f^i,x_f^i\right)\right|}^2,$$E3

where $f$ denotes the residuals of the PDE expressed by the left-hand side of Eq. (1) at the collocation points $t_f^i$ and $x_f^i$ as

$$f\left(t_f^i,x_f^i\right)=\frac{\partial \widehat{u}\left(t_f^i,x_f^i\right)}{\partial t}+\mathcal{N}\left[\widehat{u}\left(t_f^i,x_f^i\right);\lambda \right].$$E4

The second term ${\mathcal{L}}_u$ represents the sum of the loss functions aimed at ensuring the adherence of model predictions $\widehat{u}$ to both initial and boundary conditions (${\mathcal{L}}_{\mathit{IC}}$ and ${\mathcal{L}}_{\mathit{BC}}$), as well as labeled data (${\mathcal{L}}_{\mathit{\text{data}}}$) as

$$\begin{array}{lll}{\mathcal{L}}_u& =& {\mathcal{L}}_{\mathit{IC}}+{\mathcal{L}}_{\mathit{BC}}+{\mathcal{L}}_{\mathit{\text{data}}}\\ & =& \frac{1}{N_{\mathit{IC}}}\sum _{i=1}^{N_{\mathit{IC}}}{\left|\widehat{u}\left(t_{\mathit{IC}}^i,x_{\mathit{IC}}^i\right)-u_{\mathit{IC}}^i\right|}^2\\ & & +\frac{1}{N_{B{C}_1}}\sum _{i=1}^{N_{B{C}_1}}{\left|\widehat{u}\left(t_{B{C}_1}^i,x_{B{C}_1}^i\right)-u_{B{C}_1}^i\right|}^2\\ & & +\frac{1}{N_{B{C}_2}}\sum _{i=1}^{N_{B{C}_2}}{\left|\mathrm{\Gamma }\left({\widehat{u}}_{B{C}_2}^i\right)-{\mathrm{\Gamma }}_{B{C}_2}^i\right|}^2\\ & & +\frac{1}{N_{\mathit{\text{data}}}}\sum _{i=1}^{N_{\mathit{\text{data}}}}{\left|\widehat{u}\left(t_{\mathit{\text{data}}}^i,x_{\mathit{\text{data}}}^i\right)-u_{\mathit{\text{data}}}^i\right|}^2,\end{array}$$E5

where $B{C}_1$, $B{C}_2$, and $\mathrm{\Gamma }$ denote Dirichlet, Neumann boundary conditions, and a function with derivatives of $\widehat{u}$ at boundary points. $N$ and the symbol $\widehat{}$ also denote the number of data points within each condition and the predicted solutions by the PINN. ${\left\{t_{\mathit{IC}}^i,x_{\mathit{IC}}^i,u_{\mathit{IC}}^i\right\}}_{i=1}^{N_{\mathit{IC}}}$, ${\left\{t_{B{C}_1}^i,x_{B{C}_1}^i,u_{B{C}_1}^i\right\}}_{i=1}^{N_{B{C}_1}}$, ${\left\{t_{B{C}_2}^i,x_{B{C}_2}^i,u_{B{C}_2}^i\right\}}_{i=1}^{N_{B{C}_2}}$, and ${\left\{t_{\mathit{\text{data}}}^i,x_{\mathit{\text{data}}}^i,u_{\mathit{\text{data}}}^i\right\}}_{i=1}^{N_{\mathit{\text{data}}}}$ denote the initial training data, boundary training data of Dirichlet and Neumann conditions, and labeled data points. Note that the PINN’s trainable parameters $\theta$ are updated via back propagation to minimize the loss function $\mathcal{L}$. During this optimization process, the ADAM optimizer [44] or L-BFGS optimizer [45] can be used.

To demonstrate the performance of PINN, the equation of motion of a 1-DOF mass-spring-damper system is used as a simple example. This equation is a linear ordinary differential equation (ODE) dependent on time $t$, describing the vibration of a mass. The governing equation is expressed as

$$\frac{d^{2}u}{d{t}^2}+2\mathrm{\zeta }{\mathrm{\omega }}_{\mathrm{n}}\frac{\mathit{du}}{\mathit{dt}}+{\mathrm{\omega }}_{\mathrm{n}}^{2}u=0,t\in \left[0,1\right]$$E6

where $u$ denotes the displacement, which is the ODE solution that PINN needs to predict. To this end, the PINN architecture uses temporal information, specifically time t, as network input and predicts the unknown solution u as output. The predicted solution $u$ undergoes automatic differentiation within the differentiation layer to construct the loss function as Eqs. (3) and (4). Therefore, the total loss function for training PINN to solve the equation of motion is formulated as

$$\begin{array}{lll}\mathcal{L}& =& {\mathcal{L}}_{\mathit{ODE}}+{\mathcal{L}}_{\mathit{\text{Data}}}=\frac{1}{N_f}\sum _{i=1}^{N_f}{\left|\left(m\frac{d^2}{d{t}^2}+c\frac{d}{\mathit{dt}}+k\right)\widehat{u}\left(t_f^i;\theta \right)\right|}^2\\ & & +\frac{1}{N_{\mathit{\text{data}}}}\sum _{i=1}^{N_{\mathit{\text{data}}}}{\left|\widehat{u}\left(t_{\mathit{\text{data}}}^i;\theta \right)-u_{\mathit{\text{data}}}^i\right|}^2,\end{array}$$E7

where ${\left\{t_f^i\right\}}_{i=1}^{N_f}$ and ${\left\{t_{\mathit{\text{data}}}^i,u_{\mathit{\text{data}}}^i\right\}}_{i=1}^{N_{\mathit{\text{data}}}}$ denote the collocation points sampled within the range $t\in \left[0,1\right]$ and the training data points, including the initial conditions.

The PINN architecture for solving Eq. (6) consists of an input layer, multiple hidden layers, and an output layer as shown in Figure 1. The input layer has a single node, because it receives one-dimensional temporal information. The hidden layer consists of three layers, each containing 35 nodes and a hyperbolic tangent activation function. The output layer also comprises a single node to predict the displacement of a single DOF under free vibration conditions. To demonstrate the superiority of PINN, an artificial neural network (ANN) with identical architecture was constructed for comparison.

The training data points for both PINN and ANN were sampled from solutions of Eq. (6), with the damping ratio $\mathrm{\zeta }$ set to 0.025 and the natural frequency ${\mathrm{\omega }}_{\mathrm{n}}$ to 20 rad/s. Specifically, the training dataset consisted of only five solutions, sampled at uniform time intervals. On the other hand, 30 collocation points were chosen for computing the physics loss in the PINN.

PINN and ANN were trained on the Colab environment with an Intel Xeon CPU and a Tesla T4. Both PINN and ANN were trained for 18,000 iterations, with identical hyper-parameters to ensure consistent comparisons except learning rates (Table 1).

	PINN	ANN
Number of input nodes	1	1
Number of hidden layers	3	3
Number of hidden nodes	35	35
Activation function	Tanh	Tanh
Number of output nodes	1	1
Learning rate	1e−4	1e−3
Epoch	18,000	18,000

Table 1.

Hyper-parameters of PINN and ANN.

Figure 2 shows the predicted solution by the trained PINN and ANN models, in comparison to the exact solution (the blue line in Figure 2(a) and (b)). Note that ANN (the red dashed line in Figure 2(a)) accurately predicted only the data points on which it had been trained (the green circular marker in Figure 2(a)), failing to generalize to any other data. This result demonstrates the common issue of poor generalization performance in ANNs, where sparse training data limits the model’s ability to learn the full spectrum of inherent physical properties, causing it to be biased toward only the observed data [43]. Therefore, the use of ANN in engineering and scientific field, where obtaining large amounts of training data is challenging [43], has significant limitations, suggesting the need for PINNs in scientific machine learning. Figure 2(b) confirms that the PINN accurately predicts the solution outside the training data region, outperforming the ANN with the same training data because the physical constraints embedded in the PINN guide the model predictions to conform to the underlying physical laws. This result suggests that PINN may exhibit superior generalization performance compared to ANN in real world applications where acquiring data is challenging.

Figure 2.

Comparisons of the exact solution with the model prediction by ANN (a) and PINN (b).

3. Challenges and overcoming strategies in physics-informed neural networks

PINNs represent a significant advancement in the integration of scientific machine learning with physics, designed to model complex systems, where traditional data-driven approaches fall short. PINNs provide predictions that adhere to known physics and fundamental principles by embedding the governing equations of physical systems into neural networks. Hence, this architecture serves as an alternative to traditional data-driven methods for solving PDEs. However, a variety of challenges along with a series of promising results are being encountered as PINNs are actively researched. This section delves into these challenges and elucidates strategies to address them, thereby enhancing the robustness and versatility of PINNs across various scientific and engineering domains.

3.1 Mitigating complexity with domain decomposition

PINNs have shown remarkable abilities in addressing high-dimensional PDEs due to their superior approximation and generalization capabilities [47]. Despite this, PINNs can encounter difficulties when dealing with discontinuities in PDE solutions, often failing to accurately capture and predict sharp changes or discontinuities, which can lead to errors and reduce the accuracy of the solution [48]. To overcome these challenges, an innovative strategy known as extended PINN (XPINN) has been developed (Figure 3), which involves decomposing the computational domain into multiple subdomains [46]. This method employs localized, robust networks specifically in regions with high-gradient solutions, leveraging known prior knowledge about the solution characteristics. In particular, the computational domain can be partitioned into $N_{\mathit{sub}}$ number of non-overlapping regular or irregular subdomains. Within the XPINN framework, the neural network’s output for the $k^{\mathit{th}}$ subdomain is defined as

Figure 3.

The schematic of the XPINN sub-net employed in a subdomain, integrating neural network and physics-informed components [46].

$$u_{{\stackrel{\sim }{\mathrm{\Theta }}}_k}\left(z\right)=N\left(z;{\stackrel{\sim }{\mathrm{\Theta }}}_k\right)\in {\mathrm{\Omega }}_k.\mathrm{k}=1,2,\dots ,N_{\mathit{sub}}$$E8

Then, the final solution is obtained as follows

$$u_{\stackrel{\sim }{\mathrm{\Theta }}}\left(z\right)=\sum _{i=1}^{N_{u_k}}{u}_{{\stackrel{\sim }{\mathrm{\Theta }}}_k}\left(z\right)·{\mathrm{I}}_{{\mathrm{\Omega }}_k}\left(\mathrm{z}\right),$$E9

where the indicator function $\mathrm{I}{\mathrm{\Omega }}_k\left(\mathrm{z}\right)$ is defined as

$${\mathrm{I}}_{{\mathrm{\Omega }}_k}\left(\mathrm{z}\right)=\left\{\begin{array}{l}0,\text{if}z\notin {\mathrm{\Omega }}_k\\ \begin{array}{l}1,\text{if}z\in {\mathrm{\Omega }}_k\end{array}\\ \begin{array}{ll}1/S,& \text{if}z\text{is}\mathrm{on}\text{the interface within}{\mathrm{\Omega }}_k\end{array}\end{array}\right.,$$E10

where $S$ denotes the number of subdomains intersecting along the interface. Each individual PINN operates independently within its assigned subdomain, thereby improving the network’s capability to accurately capture and represent complex behaviors.

The XPINN framework leverages the deployment of multiple neural networks across smaller subdomains [49]. A key advantage of XPINNs is their enhanced representation and parallelization capacity, which is essential for accurately predicting complex PDE solutions. By enabling both spatial and temporal parallelization, XPINNs can significantly reduce training costs. In each subdomain, the networks can be optimally designed with specific hyperparameters such as network depth and width of hidden layers, the number and placement of residual points, nonlinear activation functions, and optimization strategies. For subdomains with complex solution characteristics, deeper networks may be employed, whereas shallower networks can be utilized in regions with simpler, smoother solutions.

The XPINN is designed to minimize a comprehensive loss function that considers both individual subdomain losses and interface losses, effectively balancing data accuracy and adherence to physical laws across the entire system. The XPINN loss function is defined at the subdomain level, while also incorporating interface conditions to ensure continuity and coherence between subdomains. Training an XPINN involves minimizing this complete loss function, which is defined as:

$$\begin{array}{lll}\mathcal{L}\left({\stackrel{\sim }{\mathrm{\Theta }}}_k\right)& =& W_{u_k}{\mathcal{L}}_{u_k}\left({\stackrel{\sim }{\mathrm{\Theta }}}_k;{\left\{x_{u_k}^{\left(i\right)}\right\}}_{i=1}^{N_{u_k}}\right)+W_{{\mathcal{F}}_k}{\mathcal{L}}_{{\mathcal{F}}_k}\left({\stackrel{\sim }{\mathrm{\Theta }}}_k;{\left\{x_{{\mathcal{F}}_k}^{\left(i\right)}\right\}}_{i=1}^{N_{{\mathcal{F}}_k}}\right)\\ & & +W_{I_k}{\mathcal{L}}_{u_{\mathit{avg}}}\left({\stackrel{\sim }{\mathrm{\Theta }}}_k;{\left\{x_{I_k}^{\left(i\right)}\right\}}_{i=1}^{N_{I_k}}\right)+W_{{I_{\mathcal{F}}}_k}{\mathcal{L}}_{\mathcal{R}}\left({\stackrel{\sim }{\mathrm{\Theta }}}_k;{\left\{x_{I_k}^{\left(i\right)}\right\}}_{i=1}^{N_{I_k}}\right)\end{array}$$E11

where $k=1,2,\dots ,N_{\mathit{sub}}$. The weight $W_{u_k},W_{{\mathcal{F}}_k},W_{I_k}$ and $W_{{I_{\mathcal{F}}}_k}$ correspond to the loss term for discrepancies in data, residual errors, and interface conditions, including both residual and average solution continuity at the interface, respectively. Each loss function is defined as:

$${\mathcal{L}}_{u_k}\left({\stackrel{\sim }{\Theta }}_k;{\left\{x_{u_k}^{\left(i\right)}\right\}}_{i=1}^{N_{u_k}}\right)=\frac{1}{N_{u_k}}\sum _{i=1}^{N_{u_k}}{\left|u^i-u_{{\Theta }_k}\left(x_{u_k}^i\right)\right|}^2$$E12

$${\mathcal{L}}_{{\mathcal{F}}_k}\left({\stackrel{\sim }{\Theta }}_k;{\left\{x_{{\mathcal{F}}_k}^{\left(i\right)}\right\}}_{i=1}^{N_{{\mathcal{F}}_k}}\right)=\frac{1}{N_{{\mathcal{F}}_k}}\sum _{i=1}^{N_{{\mathcal{F}}_k}}{\left|{\mathcal{F}}_{{\stackrel{\sim }{\Theta }}_k}\left(x_{{\mathcal{F}}_k}^i\right)\right|}^2$$E13

$${\mathcal{L}}_{u_{\mathit{avg}}}\left({\stackrel{\sim }{\Theta }}_k;{\left\{x_{I_k}^{\left(i\right)}\right\}}_{i=1}^{N_{I_k}}\right)=\sum _{\forall k^{+}}\left(\frac{1}{N_{I_q}}\sum _{i=1}^{N_{I_k}}{\left|u_{{\stackrel{\sim }{\Theta }}_k}\left(x_{I_k}^i\right)-\left\{\left\{u\left({\mathit{x}}_{I_k}^i\right)\right\}\right\}\right|}^2\right)$$E14

$${\mathcal{L}}_{\mathcal{R}}\left({\stackrel{\sim }{\Theta }}_k;{\left\{x_{I_k}^{\left(i\right)}\right\}}_{i=1}^{N_{I_k}}\right)=\sum _{\forall k^{+}}\frac{1}{N_{I_k}}\sum _{i=1}^{N_{I_k}}{\left|{\mathcal{F}}_{{\stackrel{\sim }{\Theta }}_k}\left(x_{I_k}^i\right)-{\mathcal{F}}_{{\stackrel{\sim }{\Theta }}_{k^{+}}}\left(x_{I_k}^i\right)\right|}^2$$E15

where terms ${\mathcal{L}}_{u_k}$ and ${\mathcal{L}}_{{\mathcal{F}}_k}$ represent the data mismatch and residual losses, respectively, and ${\mathcal{F}}_{{\stackrel{\sim }{\Theta }}_k}$ denotes the PDEs residual terms of each subdomain. The residual loss term ${\mathcal{L}}_{\mathcal{R}}$ ensures residual continuity at the interface region between separate networks in subdomains $k$ and $k^{+}$, where the superscript${}^{+}$ indicates neighboring subdomain(s). Both ${\mathcal{L}}_{\mathcal{R}}$ and ${\mathcal{L}}_{u_{\mathit{avg}}}$ are defined for all neighboring subdomain $k^{+}$, as indicated by the summation over all $k^{+}$ in their respective expressions. These interface conditions enable the transfer of information between neighboring subdomains, playing a vital role in the convergence and overall accuracy of the subdomain solutions.

The XPINN architecture simplifies complex PDE solutions by decomposing them into multiple simpler components, thereby reducing the complexity required to learn each part and enhancing generalization. XPINNs also provide flexibility through time-domain decomposition for all differential equations and the ability to manage arbitrarily shaped complex sub-regions. This methodology ensures robust and accurate estimation of responses and other complex phenomena in systems modeled by PINNs.

3.2 Loss balancing with adaptive weighting

Most PINNs encounter a common convergence issue due to loss function imbalance, which arises from different optimization rates of multiple loss functions. Specifically, each of these functions contributes to the overall training objective but may have vastly different scales and sensitivities. For example, one loss function might represent the governing equation in thermodynamics, while another could enforce an initial condition. The different complexity in training these loss functions can lead to discrepancies in the magnitudes of optimizing gradients during training. This discrepancy often results in a situation where optimizing one loss function significantly overshadows the others, thereby stalling the overall convergence of the network.

To overcome this challenge, adaptive weighting methods are being actively researched, including self-adaptive [26] and learning rate annealing [28]. These methods achieve convergence by adjusting the gradient differences derived from each loss function. In particular, adaptive weighting using the neural tangent kernel (NTK) [50], a widely used method, addresses the optimization imbalance caused by multiple loss functions. This strategy uses NTK to dynamically adjust adaptive weights during training, guaranteeing that all loss terms converge equitably and thereby preserving overall balance. The standard loss function of PINNs with adaptive weights is as follows:

$$L\left(\theta ,{\lambda }_D,{\lambda }_P,{\lambda }_B,{\lambda }_I\right)={\lambda }_D{L}_D\left(\theta \right)+{\lambda }_P{L}_P\left(\theta \right)+{\lambda }_I{L}_I\left(\theta \right)+{\lambda }_B{L}_B\left(\theta \right)$$E16

where $\theta$, $L_D$, $L_P$, $L_B$, $L_I$, and ${\lambda }_{\ast }$ represent the trainable parameters, data fitting loss, physics loss, boundary condition loss, initial condition loss, and adaptive weights for each loss function, respectively. Note that the vanilla loss function of PINNs is the same as when ${\lambda }_{\ast }$ is 1. Note also that there may not be a specific loss function depending on the problem being solved using PINN. To determine the adaptive weights for each loss in Eq. (16), the NTK of PINN $H^{\mathit{NTK}}\left(\theta \right)$ is defined as follows:

$$\begin{array}{lll}{H}_{\mathit{ij}}^{\mathit{NTK}}\left({\theta }_n\right)& =& ⟨\frac{d{T}^i\left({\theta }_n\right)}{d{\theta }_n},\frac{d{T}^j\left({\theta }_n\right)}{d{\theta }_n}⟩,i,j=1,2,\cdots ,N,\\ \hfill T^k\left({\theta }_n\right)& =& \left\{\begin{array}{l}{X}^k-{\widehat{X}}^k,1\le \mathrm{k}\le N_D\\ {{\widehat{\mathit{Res}}}_P}^k,N_D<\mathrm{k}\le N_D+N_P\\ {{\widehat{\mathit{Res}}}_{\mathit{BC}}}^k,N_D+N_P<\mathrm{k}\le N_D+N_P+N_B\\ {X_0}^k-{\widehat{X_0}}^k,N_D+N_P+N_B<\mathrm{k}\le N\end{array},\right.\end{array}$$E17

where ${\theta }_n$ denotes parameters of the PINN at the $n$-th iteration, and $N=N_D+N_P+N_{\mathit{I}}+N_{\mathit{B}}$ represents the total collocation points considering data fitting, physics, initial condition, and boundary condition. Then, ${\lambda }^k$, representing the adaptive weights of the $k$-th collocation point, is obtained as follows:

$${\lambda }_{\psi }=\frac{{\sum }_{k=1}^N{H}_{\mathit{kk}}^{\mathit{NTK}}\left({\theta }_n\right)}{{\sum }_{k=1}^{N_{\psi }}{H}_{\mathit{kk}}^{\mathit{NTK}}\left({\theta }_n\right)},\psi \in \left\{D,P,B,I\right\}$$E18

where $H_{\mathit{kk}}^{\mathit{NTK}}$ represents the $k$-th diagonal entry of the NTK matrix, which indicates the backpropagation gradient of the loss function. This balancing strategy is crucial because unbalanced gradients can significantly hinder the optimization process, preventing the network from finding a solution that satisfies the imposed physical constraints.

3.3 Extending capabilities through deep neural operator

In real-world applications, engineers often face the challenge of predicting outcomes under diverse operating conditions that differ from the specific conditions in which the neural network was trained. However, conventional PINNs that use DNNs are typically designed to train and predict based on specific initial, boundary, or geometric conditions. These inherent characteristics necessitate independent training processes, such as retraining or transfer learning [51], to adapt to new operating conditions. This additional training is not only time-consuming but also computationally expensive, limiting its use as a surrogate model. Therefore, significant research efforts have been directed toward developing more capable neural network to address these challenges.

Deep Operator Network (DON) has become a state-of-the-art surrogate model due to its ability to approximate nonlinear operators through function-to-function mapping [31]. Hence, once DON is trained, it can predict results for new operational conditions, effectively addressing both interpolation and extrapolation tasks without additional retraining. Specifically, DON integrates two distinct neural networks (Figure 4): the branch and trunk networks. The branch network handles discretized inputs from the input functions, such as initial conditions, boundary conditions, or external forcing terms, represented as $\mathit{u}=\left[u\left(t_1\right),u\left(t_2\right),u\left(t_3\right),\cdots ,u\left(t_{N-1}\right),u\left(t_N\right)\right]$, whereas the trunk network uses spatiotemporal coordinates $\mathit{y}\in {\mathbb{R}}^d$ as inputs. The branch network is responsible for encoding the input function at specific points within the domain. This process is crucial for enabling the network to serve as a surrogate model under varying specific initial or boundary conditions. Simultaneously, the trunk network is designed to handle the spatial temporal coordinates for values of interest. Then, the outputs of both networks are integrated through a dot product operation, such that $G\left(\mathit{u}\right)\left(\mathit{y}\right)$ becomes a function of $\mathit{y}$ on $\mathit{u}$. Specifically, the outputs from both networks are obtained by forward propagation through an L-hidden layer DON based on a multilayer perceptron, as detailed below. The hidden layer outputs ${\mathit{H}}_u$ and ${\mathit{H}}_y$ for the branch network and the trunk network are obtained as follows:

Figure 4.

Graphical illustration of DON [31].

$${\mathit{H}}_u^{\left(1\right)}=\varnothing \left({\mathit{W}}_u^{\left(1\right)}\mathit{u}+{\mathit{b}}_u^{\left(1\right)}\right),{\mathit{H}}_y^{\left(1\right)}=\varnothing \left({\mathit{W}}_y^{\left(1\right)}\mathit{y}+{\mathit{b}}_y^{\left(1\right)}\right),$$E19

$${\mathit{H}}_u^{\left(l\right)}=\varnothing \left({\mathit{W}}_u^{\left(l\right)}{\mathit{H}}_u^{\left(l-1\right)}+{\mathit{b}}_u^{\left(l\right)}\right),{\mathit{H}}_y^{\left(l\right)}=\varnothing \left({\mathit{W}}_y^{\left(l\right)}{H}_y^{\left(l-1\right)}+{\mathit{b}}_y^{\left(l\right)}\right),l=2,\cdots ,L-1,$$E20

$${\mathit{O}}_u={\mathit{H}}_u^{\left(L\right)}=\varnothing \left({\mathit{W}}_u^{\left(L\right)}{\mathit{H}}_u^{\left(L-1\right)}+{\mathit{b}}_u^{\left(L\right)}\right),{\mathit{O}}_y={\mathit{H}}_y^{\left(L\right)}=\varnothing \left({\mathit{W}}_y^{\left(L\right)}{H}_y^{\left(L-1\right)}+{\mathit{b}}_y^{\left(L\right)}\right),$$E21

where $\varnothing$ represents an activation function and ${\left\{{\mathit{W}}_u^{\left(l\right)},{\mathit{b}}_u^{\left(l\right)}\right\}}_{l=1}^L$ and ${\left\{{\mathit{W}}_y^{\left(l\right)},{\mathit{b}}_y^{\left(l\right)}\right\}}_{l=1}^L$ refer to the weights and biases of the hidden layers for the branch network and the trunk network, respectively. In both networks, the initial hidden layers take $\mathit{u}$ and $\mathit{y}$ as inputs, whereas following hidden layers utilize the outputs of the previous hidden layer ${\mathit{H}}_u^{\left(l-1\right)}$ and ${\mathit{H}}_y^{\left(l-1\right)}$. The inputs $\mathit{u}$ and $\mathit{y}$ are converted to output vectors ${\left[O_u^1,O_u^2,O_u^3,\cdots ,O_u^{q-1},O_u^q\right]}^T\in {\mathbb{R}}^q$ and ${\left[O_y^1,O_y^2,O_y^3,\cdots ,O_y^{q-1},O_y^q\right]}^T\in {\mathbb{R}}^q$ through the branch and trunk networks, respectively. Finally, the neural network predictions are derived by integrating the output vectors from the branch and trunk networks as follows:

$$G\left(\mathit{u}\right)\left(\mathit{y}\right)\approx \widehat{G}\left(\mathit{u}\right)\left(\mathit{y}\right)=\sum _{k=1}^q\underset{\mathit{\text{branch}}}{\underbrace{O_u^k\left(u\left(t_1\right),u\left(t_2\right),\cdots ,u\left(t_N\right)\right)}}\underset{\mathit{\text{trunk}}}{\underbrace{O_y^k\left(\mathit{y}\right)}}$$E22

where $\widehat{G}\left(\mathit{u}\right)\left(\mathit{y}\right)$ denotes a predicted DON value, while $G\left(\mathit{u}\right)\left(\mathit{y}\right)$ corresponds to the ground truth value.

The capability of DON to capture operators can be leveraged as a powerful surrogate model for multiphysics systems governed by a range of ODEs and PDEs. Therefore, active research has been directed toward predicting responses under a variety of conditions within the fields of elastic-plastic stress [52], heat conduction [53], and fracture mechanics [54]. Furthermore, numerous studies are dedicated to improving the capabilities of DON through architectural advancements, integrating neural networks such as encoders [55], LSTM [56], or ResUNet [52].

3.4 Improving architecture through Fourier feature embedding

PINNs often face challenges when addressing multi-scale problems. PINNs operate with a relatively limited number of data points within the computational domain. One significant challenge is spectral bias, a phenomenon where multi-layer perceptron tends to train solutions and their PDE residuals starting with the lowest frequency components. Spectral bias is an intriguing characteristic observed in the neural networks training, where these networks tend to learn simpler or lower-frequency components before capturing more complex or higher-frequency components of the target function [57]. This phenomenon can significantly influence the training dynamics and generalization abilities of neural networks.

To address the multi-scale problem challenges in PINNs, particularly spectral bias, an innovative network architecture incorporating Fourier feature embeddings has been developed [33]. This approach involves enhancing the input space using Fourier features, which helps the network represent and learn higher frequency components more effectively.

As illustrated in Figure 5(a), the methodology involves applying Fourier feature embeddings. The inputs are embedded with the Fourier feature and then fed into a conventional multi-layer perceptron. The final outputs are concatenated using a linear function. The forward pass of Fourier feature embedding is detailed as follows:

Figure 5.

(a) Multi-scale Fourier feature network, and (b) Spatio-temporal multi-scale Fourier feature network [57].

$${\mathit{\alpha }}^{\mathit{i}}\left(\mathit{x}\right)=\left[\begin{array}{l}cos\left(2\pi {\mathit{F}}^i\mathit{x}\right)\\ sin\left(2\pi {\mathit{F}}^i\mathit{x}\right)\end{array}\right],\mathit{\text{for}}i=1,2,\dots ,M,$$E23

$${\mathit{H}}_1^i=\varphi \left({\mathit{W}}_1·{\mathit{\alpha }}^j\left(\mathit{x}\right)+{\mathit{b}}_1\right),\mathit{\text{for}}i=1,2,\dots ,M,$$E24

$${\mathit{H}}_l^i=\varphi \left({\mathit{W}}_l·{\mathit{H}}_{l-1}^j\left(\mathit{x}\right)+{\mathit{b}}_l\right),\mathit{\text{for}}l=2,\dots ,i=1,2,\dots ,M,$$E25

$$f_{\theta }\left(\mathit{x}\right)={\mathit{W}}_{L+1}\left[{\mathit{H}}_L^1,\dots ,{\mathit{H}}_L^M\right]+{\mathit{b}}_{L+1}$$E26

where ϕ and ${\mathit{\alpha }}^i$ represent nonlinear activation functions and Fourier feature mappings, respectively. Each ${\mathit{F}}^i$ is sampled from the normal distribution $\mathcal{N}\left(0,{\sigma }^i\right)$, and remains fixed during model training, meaning that ${\mathit{F}}^i$ are not trainable parameters [33]. The inputs are embedded with different frequencies, and after forward propagation, all frequency components are combined through a linear layer to ensure they are learned at the same convergence rate [58].

Time-dependent problems exhibit multi-scale behavior not only across spatial dimensions but also over temporal scales. To address these multi-scale challenges in spatial-temporal domains, an innovative multi-scale Fourier feature architecture has been proposed [59]. Specifically, the neural network’s feed-forward pass is defined as follows

$${\mathit{\alpha }}_s^i\left(\mathit{x}\right)=\left[\begin{array}{l}cos\left(2\pi {\mathit{F}}_s^i\mathit{x}\right)\\ sin\left(2\pi {\mathit{F}}_s^i\mathit{x}\right)\end{array}\right],{\mathit{H}}_{t,1}^i=\varphi \left({\mathit{W}}_1·{\mathit{\alpha }}_t^i\left(t\right)+{\mathit{b}}_1\right),\mathit{\text{for}}i=1,2,\dots ,M_s,$$E27

$${\mathit{\alpha }}_t^j\left(t\right)=\left[\begin{array}{l}cos\left(2\pi {\mathit{F}}_t^{j}t\right)\\ sin\left(2\pi {\mathit{F}}_t^{j}t\right)\end{array}\right],{\mathit{H}}_{s,1}^i=\varphi \left({\mathit{W}}_1·{\mathit{\alpha }}_t^j\left(\mathit{x}\right)+{\mathit{b}}_1\right),\mathit{\text{for}}j=1,2,\dots ,M_t,$$E28

$${\mathit{H}}_{s,l}^i=\varphi \left({\mathit{W}}_l·{\mathit{H}}_{s,l-1}^i+{\mathit{b}}_l\right),\mathit{\text{for}}i=1,2,\dots ,M_x\mathit{\text{and}}l=2,\dots ,L,$$E29

$${\mathit{H}}_{t,l}^j=\varphi \left({\mathit{W}}_l·{\mathit{H}}_{t,l-1}^j+{\mathit{b}}_l\right),\mathit{\text{for}}i=1,2,\dots ,M_t\mathit{\text{and}}l=2,\dots ,L.$$E30

$${\mathit{H}}_L^{i,j}={\mathit{H}}_{s,L}^i\odot {\mathit{H}}_{t,L}^i,$$E31

$$f_{\theta }\left(\mathit{x},t\right)={\mathit{W}}_{L+1}\left[{\mathit{H}}_L^{1,1},\dots ,{\mathit{H}}_L^{M_t,M_x}\right]+{\mathit{b}}_{L+1}.$$E32

where ${{\mathit{\alpha }}_x}^i$ and ${{\mathit{\alpha }}_t}^i$ represent spatial and temporal Fourier feature mappings, respectively, with $\odot$ denoting the point-wise multiplication. Each entry of ${\mathit{F}}_x^i$ and ${\mathit{F}}_t^j$ is sampled from $\mathcal{N}\left(0,{\sigma }_x^i\right)$ and $\mathcal{N}\left(0,{\sigma }_y^j\right)$, respectively, and these values are kept constant during the training of the model.

This structured approach ensures that the PINN can manage the natural low-frequency bias of neural networks and is capable of effectively learning and predicting high-frequency phenomena within the PDEs solutions. By embedding Fourier features directly into the input layer and merging them before producing the final output, the network utilizes the rich frequency information to achieve more precise and robust solutions for PDE. This architecture significantly improves the ability of PINNs to solve complex, multi-scale problems that are typically challenging for conventional machine learning models and some numerical methods. Note that the proposed models maintain computational efficiency by avoiding the introduction of additional trainable parameters and not requiring significantly more operations for their forward or backward pass compared to conventional PINN models.

4. Case studies in engineering applications

The potential of PINNs is actively being explored for real-world engineering applications. To address the inherent multiphysics nature of most real-world systems, PINNs should be extended to account for multiphysics in the system of interest [35]. However, incorporating multiphysics into a neural network increases the complexity, leading to convergence issues due to the more challenging training process. Additionally, having a comprehensive understanding of the target system and employing suitable strategies are crucial to ensuring optimal performance. This section presents two examples, where PINNs are applied to complex mechatronics systems, specifically involving a motor and a battery, by utilizing appropriate training strategies.

4.1 Application on an electric motor for virtual sensing and PHM

This subsection introduces the novel MPINN framework for virtual sensing and prognostics and health management (PHM) of electric motors [40]. The proposed framework is designed to accurately estimate electromagnetic responses in electric motors under various operational conditions. This MPINN utilizes novel strategies to manage complexities associated with rotational dynamics, improving both the accuracy and efficiency of network’s predictions. The MPINN specifically estimates various electromagnetic fields, including magnetic vector potentials ($A_z$), electric fields ($E_z$), magnetic flux densities ($B_x$ and $B_y$), and magnetic fields ($H_z$).

The proposed MPINN architecture integrates three crucial features to overcome the challenges of conventional networks. First, the MPINN utilizes various PDEs to describe the electromagnetic properties of motors [60], ensuring precise and reliable predictions even with limited data availability. These PDEs guide the neural network, providing supervision for accuracy. A notable innovation is the conversion of the motor’s rotor coordinates from rotational (based on angular speed) to Cartesian coordinates. This transformation overcomes the limitations of conventional PINNs, which have difficulty with rotational machinery due to their fixed coordinate systems. By aligning the rotor and stator coordinates into a unified Cartesian framework, the MPINN effectively resolves component representation mismatches and accurately depicts the electromagnetic fields influenced by rotor movement. Second, the architecture incorporates distinct networks for each domain, integrating estimation results by considering the interface loss between the two sperate domains [49]. This approach allows for more targeted training on specific components, significantly enhancing the accuracy of electromagnetic response estimations. Additionally, an interface loss function merges the outputs from these separate networks, reducing discrepancies and errors at the interface between two domains. Third, the MPINN guarantee convergence of neural networks during training by implementing a learning rate annealing method. This method adjusts adaptive weights to balance the gradient stiffness across different loss types, thereby minimizing the overall loss [25]. This enhancement not only improves the accuracy and reliability of predictions but also makes the neural network more robust in managing the complexities associated with different loss functions.

The proposed MPINN was validated under five conditions, as detailed in Table 2. These conditions include four typical operating scenarios of the PMSM and one fault condition involving rotor axis eccentricity, which is the most common mechanical defect [61]. The MPINN was trained using data generated from a Finite Element Model (FEM) that covers the entire domain of the PMSM. The performance of MPINN was verified by comparing torques measured from an experimental PMSM testbed. Additionally, comparisons with the FEM confirmed the accuracy of electromagnetic characteristics across the entire domain [62, 63].

Case	Description	Condition	Load	Speed [RPM]
1	Starting,no load	Normal	—	0 $\to$ 60
2	Variational speed, no load	Normal	—	60 $\to$ 600 $\to$ 60
3	Constant speed, linearly increasing load	Normal	0 to 0.4 Nm (linear increase)	60
4	Breaking	Normal	Brake	60 $\to$ 0
5	Starting, no load, eccentricity	Eccentricity	—	0 $\to$ 60

Table 2.

Experimental datasets under various operational conditions [40].

The accuracy of the MPINN for the electric motor was validated by comparing the experimental measurements from the those estimated by the FEM and MPINN. The torque measurements are only compared because measuring electromagnetic fields directly through the experiments is difficult, and only torque can be obtained. As illustrated in Figure 6, the torques predicted by the FEM (red line) and MPINN (blue line) are compared with those obtained from the experiments (black line) across the five different operational scenarios. Both models predicted the dynamic PMSM’s torques accurately when controlling the motor driver’s three-phase currents and rotor rotational speed. The average RMSEs for the FEM and MPINN were 0.0105 Nm and 0.0166 Nm, respectively, demonstrating the accurate estimation of the PMSM’s electromagnetic responses by both models. Furthermore, these results highlight the potential of MPINN in effectively managing complex and nonlinear behaviors in data, showcasing its applicability in various scenarios.

Figure 6.

Comparison of experimentally torque measurements with estimation results of FEM and MPINN across five different conditions [40].

The effectiveness of the proposed MPINN in predicting electromagnetic fields across the entire domain was demonstrated by comparing its results with those obtained from FEM. The prediction of electromagnetic fields from the FEM and MPINN is shown in Figure 7, along with the error of the proposed MPINN compared to the FEM under eccentricity fault conditions. Specifically, the RMSEs are 8.78e−5 for the magnetic vector potential, 4.86e−2 for the electric field, 5.64e−2 and 1.29e−1 for the magnetic flux density in the x and y directions, respectively, and 3.56e4 and 3.95e4 for the magnetic field in the x and y directions, respectively. These results show that the MPINN accurately estimates electromagnetic fields for the entire PMSM domain. Notably, errors in the electric field were observed across the entire domain due to a higher number of spatial training points compared to temporal ones, while errors in the magnetic flux density were confined to specific areas. This difference can be explained by the neural networks’ tendency to provide more accurate estimates where training data points are more abundant.

Figure 7.

Comparison of reference and estimated electromagnetic fields across eccentricity fault condition [40].

This study highlights the superior performance of MPINN, which integrates physical constraints, compared to a conventional ANN lacking such constraints. As illustrated in Figure 8, the comparison of RMSEs for the electromagnetic responses predicted by the MPINN and the conventional ANN against the values derived from FEM. These results indicate that the RMSEs for the MPINN are consistently lower than those for the ANN in tested scenarios. Compared to the ANN, the MPINN achieves notable reductions in average RMSEs for the estimated values: 21.64% for the magnetic vector potential, 80.09% for the electric field, 70.91% and 39.44% for the magnetic flux densities in the x and y directions, respectively, and 58.39% and 53.83% for the magnetic fields in the x and y directions, respectively. These comparisons highlight the advantages of incorporating physical regularizations into the neural networks, enhancing the accuracy and robustness of estimation results compared to purely data-driven methods.

Figure 8.

RMSE comparison between ANN and MPINN estimation: (a) magnetic vector potential ${\mathit{A}}_{\mathit{z}}$, magnetic flux densities (b) ${\mathit{B}}_{\mathit{x}}$ and (c) ${\mathit{B}}_{\mathit{y}}$, (d) electric field ${\mathit{E}}_{\mathit{z}}$, and magnetic fields (e) ${\mathit{H}}_{\mathit{x}}$, and (f) ${\mathit{H}}_{\mathit{y}}$ [40].

Notably, as shown in Figure 8(b), the MPINN could accurately estimate electromagnetic responses even when operational conditions changed significantly over time. In contrast, the purely data-driven approach faced difficulties in accurately capturing these variations. This highlights the effectiveness of incorporating physical principles, such as the temporal differentiation of the electric field and the magnetic vector potential via the PDE. This sharply contrasts with the ANN, which frequently violates physical laws due to having less data available compared to the network’s capacity. Additionally, the distorted magnetic fields from asymmetric rotation pose significant challenges for the ANN when estimating electromagnetic responses under eccentric conditions. Consequently, the ANN is constrained in its ability to account for all intricate phenomena occurring in the PMSM under these operational conditions. Conversely, the MPINN, guided by the governing equation that defines the relationship between inputs and outputs, achieves robustness of estimation results. These analyses indicate that incorporating prior knowledge is crucial for PINN estimation. With the same dataset size, MPINNs enhance accuracy and ensure high reliability compared to conventional data-driven methods.

This research highlights that the proposed MPINN is a powerful and reliable solution for analyzing electromagnetic responses of PMSMs. While conventional FEM are accurate, they require substantial computational resources, and physical experiments typically yield data only for partial domains. In contrast, MPINNs offer a cost-effective alternative with rapid and accurate electromagnetic characterization capabilities. The MPINN not only addresses the limitations of conventional numerical methods by offering a mesh-free solver approach but also proves more efficient and cost-effective for characterizing electromagnetic properties in both normal and fault conditions. This makes MPINNs particularly suitable for PHM applications in electric vehicles and cyber-physical systems in production facilities, where control-oriented objectives are crucial.

4.2 Application for predicting thermal runaway of a lithium-ion battery

The integration of PINN and DON, called multiphysics-informed deep operator network (MPI-DON or MPI-DeepONet) incorporating encoders, is proposed for predicting thermal runaway (TR) under a range of heating conditions, providing a rapid and accurate surrogate model for TR prediction. This neural network uses the heating curve, representing thermal abuse conditions, as input to output the chemical concentrations of the positive electrode, negative electrode, solid electrolyte interphase (SEI), and electrolyte along with the internal temperature response of the lithium-ion battery (LIB).

MPI-DON incorporating encoders have three main features that play a crucial role in attaining accurate final stage TR predictions, organized into three phases (Figure 9(a)). First, phase 1 generates virtual data by employing FEA [64] at various heating conditions (Figure 9(a), red box). Specifically, virtual data is generated to reflect the mechanism of chemical decomposition of LIB components due to temperature-dependent chain reactions involving positive electrode, negative electrode, SEI, and electrolyte. This mechanism is governed by the energy balance equation [65], Arrhenius equation [66], and heat transfer equations. MPI-DON incorporating encoders are trained with virtual data because it is challenging and expensive to conduct experiments for all operating conditions. This strategy also facilitates MPI-DON incorporating encoders in capturing hidden physics, such as four main chemical concentrations and the internal temperature response of the LIB during the training phase (Figure 9(a), purple box). Therefore, the proposed neural network can function as a virtual sensor in practical uses by acquiring this key information through FEA. Note that virtual data generated under various heating conditions are separated into a training set and a test set, with the test set validating the capability of interpolation and extrapolation. Second, DON serves as a surrogate model for predicting TR in untrained conditions, utilizing the branch network and the trunk network for operator regression (Figure 9(b), blue dashed box). As explained in Subsection 3.3, this strategy uses the branch and trunk networks to map one function (i.e., the heating condition) to another function (i.e., the responses of the cell) to approximate the solution of a nonlinear system of equations, which outperforms traditional neural networks that have to be retrained for every unseen condition. Furthermore, encoders [55] are added to the architecture of DON to resolve the issue of restricted information exchange, where branch network and trunk network interact only in the last layer. These encoders improve the ability of the DON to capture the complex response of the TR phenomenon by smoothing the input information flow to each hidden layer. Third, incorporating a novel training strategy that supervises multiple equations governing the TR of LIBs (Figure 9(b), green dashed box), along with adaptive weights allocation based on the NTK method (Figure 9(b), pink dashed box), can enhance the robustness and accuracy of the DON with encoders during the training phase. Thus, MPI-DON incorporating encoders can achieve convergence to the global minimum, thanks to the governing equations regulating the training process. Note that the input values and output values are normalized to convert the dimensionless values, addressing the mismatch between the neural network values and the physical quantities represented by the governing equations. After completing phase 2, phase 3 predicts the hidden states of a LIB, encompassing chemical concentrations and internal temperature distribution at diverse heating conditions (Figure 9(a), green box).

Figure 9.

(a) Flowchart for TR prediction, and (b) in-depth illustration of the MPI-DON incorporating encoders [37].

The effectiveness of physics constraints is demonstrated by comparing MPI-DON incorporating encoders supervised by physics constraints to DON that relies solely on data during training. Figure 10 shows an example of predicting an interpolation task in a test set, confirming the effectiveness of physics constraints for improving accuracy. Specifically, MPI-DON, achieving an RMSE of 7.6°C in temperature predictions (Figure 10(a), red line), accurately predicts temperature evolutions over all time domains, even in regions without labeled data points. In contrast, DON, achieving an RMSE of 30.5°C in temperature predictions (Figure 10(a), blue line), shows large errors in the time domain without labeled data points when TR occurs. Remarkably, DON shows little error on the trained data points labeled 0, 100, 200, and 300 min (Figure 10(a), yellow dashed line), but significant error on the untrained data points. MPI-DON predicts responses accurately across all time domains due to physics constraints. This result suggests that purely data-driven methods may be incapable of predicting the multiphysics response of LIBs, resulting in convergence to a local minimum due to insufficient training data. Figure 10(b) compares the internal temperature distribution along the center cross-section line of the LIB at the peak point, illustrating results from multiphysics FEM (black dotted line with triangles), MPI-DON (red line), and DON (blue line). This result reveals the role of physics constraints in ensuring predictions match the multiphysics characteristics of LIBs. Specifically, the symmetrical internal temperature distributions within the LIB shown by multiphysics FEM and MPI-DON, centered around the midpoint due to isotropic conduction, contrast with the asymmetric temperature distribution produced by DON, confirming the effectiveness of physics constraints. In Figure 10(c) and (d), the chemical concentrations are compared among multiphysics FEM (black dotted lines with triangles and squares), MPI-DON (red and orange lines), and DON (blue and light blue lines). The inset figure shows that MPI-DON accurately predicts the sharp variations in chemical concentration during TR, in contrast to DON, which shows significant inaccuracies. This result confirms that MPI-DON accurately predicts TR occurrence from heating conditions, predicting key chemical concentrations and temperature accurately.

Figure 10.

Comparison of MPI-DON and DON with multiphysics FEMs for TR in test set 1: (a) predicted mean temperature evolution, (b) internal LIB temperature distribution at TR peak, (c) predicted mean $c_{\mathit{SEI}}$ and $c_{\mathit{ne}}$ evolution, and (d) predicted mean $c_{\mathit{pe}}$ and $c_e$ evolution [37].

The contribution of supervision from multiphysics is highlighted by comparing predictions from MPI-DON and DON based on varying numbers of data collocation points. Data collocation intervals of 15, 20, 30, 50, 60, and 100 minutes correspond to 14,364, 10,944, 7524, 4788, 4104, and 2736 points, respectively, for training the neural networks. All other training conditions, including physics, initial, and boundary condition collocation points, remain constant. The mean and 95% confidence interval of RMSE for temperature prediction are determined for the test sets with five random initial parameters (Figure 11). Results show that MPI-DON consistently predicts TR phenomena more accurately than DON, regardless of data collocation point density, due to supervision by multiphysics characteristics. Conversely, RMSE of DON increases significantly as data become sparse, indicating a high dependence on data. The confidence interval for MPI-DON is narrower, especially with 100-minute intervals, indicating better stability even with sparse data. This observation suggests robustness of MPI-DON in industrial applications where data is often limited. While DON is cost-effective with ample data (intervals less than 30 minutes), physics constraints are generally advantageous for real-world applications.

Figure 11.

Temperature RMSE of MPI-DON and DON across various time interval of the data collocation points [37].

MPI-DON incorporating encoders offers a significantly faster inference time compared to multiphysics FEM. Specifically, MPI-DON takes just 0.5 ms for TR prediction, while multiphysics FEM takes about 7 seconds, making MPI-DON 10,000 times faster. After a 720-minute training phase, MPI-DON continues to provide instant predictions, unlike multiphysics FEM, whose prediction time increases linearly with the number of heating cases. This efficiency makes MPI-DON more versatile and effective for testing LIB reliability under various conditions through Monte Carlo analysis, a task that would be time-consuming and costly with FEM. The speed of MPI-DON also suits real-time applications, such as TR management in thermal systems.

In conclusion, this approach overcomes the limitations of traditional numerical methods and data-driven techniques, offering a solution that is both computationally efficient and highly accurate. The proposed neural network offers indispensable data for sophisticated thermal battery management systems, thereby improving the reliability and safety of LIBs. This advancement propels the development of digital twins for electric transport systems through artificial intelligence transformation. Additionally, beyond addressing the TR phenomenon, this approach can be implemented in various systems, furthering the impact of artificial intelligence transformation across various domains.

5. Conclusion

This chapter delved into the potential of artificial intelligence, particularly focusing on PINN, which are adept at handling complex and nonlinear systems in real-world engineering. PINN effectively addresses challenges such as data scarcity and imbalance by integrating physics and fundamental principles during the training phase. PINN ensures predictions adhere to physical laws, even when data is sparse, by embedding governing equations as constraints within the loss function. However, incorporating physical constraints into neural networks increases the training complexity, necessitating appropriate strategies for optimal performance. Especially, strategies to address complexity become crucial when applying PINN to multiphysics systems, where multiple physics are coupled. Therefore, this chapter also introduced various strategies for applying PINN to mechatronic systems such as motor and battery. The principles of PINN summarized and the introduced framework for applying PINN to multiphysics systems will be crucial in advancing technological innovation, ensuring system reliability, and influencing the future of engineering and applied sciences.

Acknowledgments

This work was supported by the Air Force Office of Scientific Research under award number FA2386-23-1-4094, by the National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIT) [grant number 2020R1C1C1003829], by the Korea Institute of Machinery and Materials [grant No. NK250B].

Nomenclature

DNN	deep neural network
DeepONet	deep operator network
FEM	finite element model
LIB	lithium-ion battery
MLP	multi-layer perceptron
MPI-DeepONet	multiphysics-informed deep operator network
MSE	mean squared error
NTK	neural tangent kernel
PDE	partial differential equation
PHM	prognostics and health management
PINN	physics informed neural network
RMSE	root mean squared error
SEI	solid electrolyte interphase
TR	thermal runaway

Symbols

$\mathit{B}$	magnetic flux density
${\mathit{b}}_{\ast }^{\left(l\right)}$	bias of the $l$th hidden layer
$c_e$	value of electrolyte
$c_{\mathit{ne}}$	value of negative electrode
$c_{\mathit{pe}}$	value of positive electrode
$c_{\mathit{SEI}}$	value of SEI
$\mathit{f}$	final output of the neural network
${\mathbf{F}}^{\mathbf{i}}$	Fourier feature mapping matrix for the i-th embedding
$G_{\ast }$	operator output
${\mathit{H}}_{\ast }^{\left(\mathrm{l}\right)}$	output of the $l$th hidden layer
$H^{\mathit{NTK}}$	NTK matrix
$\mathit{I}$	indicator function
$L$	total loss function
$N_{\ast }$	number of collocation points
${\mathit{O}}_{\ast }$	final output
$\mathbf{\Phi }$	Fourier feature mapping of input
${\mathit{Res}}_{\ast }$	residual of governing equation
$T$	time span
$L_{\ast }$	loss term
$\mathit{W}$	weight for each loss function
${\mathit{W}}_{\ast }^{\left(l\right)}$	weight of the $l$th hidden layer

Greek letters

${\lambda }_{\ast }$	adaptive weights
$\theta$	trainable parameters
$u$	reference solution of the PDE

Subscripts

$0$	initial value
$\mathit{BC}$	boundary condition
D	data fitting
I	initial condition
P	physics
$q$	subdomain
$u$	branch network
$y$	trunk network

Superscripts

$\widehat{}$

predicted solution of neural network