Private SVM Inference on Encrypted Data

Abstract

This tutorial chapter provides a comprehensive guide to implementing privacy-preserving Support Vector Machine (SVM) inference using Fully Homomorphic Encryption (FHE). We demonstrate a practical solution for secure and private SVM inference on encrypted data, enabling sensitive data analysis while maintaining confidentiality. Through a step-by-step implementation on a real-world dataset, we cover data preparation, SVM model training, and homomorphic inference. Our experimental results on a commodity laptop show that our approach achieves high accuracy with a reasonable latency of nearly 6 seconds for SVM inference. This chapter serves as a valuable resource for practitioners and researchers seeking to apply privacy-preserving techniques to SVM solutions, with significant implications for applications like medical diagnosis, financial prediction, and recommender systems, where data privacy is crucial. By following this tutorial, readers can gain hands-on experience with privacy-preserving SVM inference using FHE.

1. Introduction

Support Vector Machines (SVMs) have become indispensable tools in supervised machine learning, playing a crucial role in solving a wide range of problems. These algorithms are widely employed for both classification and regression tasks. Their effectiveness lies in their ability to find optimal decision boundaries by maximizing the margin between different classes making them particularly useful when dealing with complex datasets [1, 2, 3, 4, 5, 6, 7, 8, 9].

A significant challenge arises when applying SVMs to problems involving sensitive data, which poses substantial privacy risks. This is particularly concerning in scenarios where datasets contain personal and sensitive information, such as medical records, financial transactions, or user preferences. In these cases, maintaining data privacy becomes crucial to prevent unauthorized access, misuse, or exploitation. Furthermore, stringent privacy regulations like the General Data Protection Regulation (GDPR) impose strict constraints on handling sensitive data, limiting its use beyond specific purposes like storage and logging and prohibiting its use for analysis without privacy-aware methods [10, 11]. Therefore, developing machine learning algorithms, including SVMs, that effectively utilize sensitive data while ensuring privacy remains a critical and active research area [12, 13, 14, 15, 16, 17, 18, 19].

One promising solution to balancing data utility and privacy is Fully Homomorphic Encryption (FHE) [20], a cryptographic method that allows an untrusted party to perform operations directly on encrypted data without decryption. This ensures that sensitive information remains confidential throughout the analysis process, making it remarkably useful in cloud computing environments where sensitive data might be shared with the cloud provider. FHE offers a solution for utilizing sensitive data while preserving user privacy. It allows us to extract insights from the data without ever decrypting it, effectively overcoming the limitations imposed by privacy regulations and concerns.

This chapter specifically addresses the challenge of Machine Learning as a Service (MLaaS) when the analysis algorithm is an SVM [21, 22]. As shown in Figure 1, in this scenario, two main actors are involved: (1) a user with sensitive data records and (2) a public server that owns a pre-trained SVM model. The user encrypts their data using their FHE secret key and sends it, along with the FHE public material, to the server. The server can then evaluate the pre-trained SVM model on the encrypted user input using FHE evaluation methods. This generates an encrypted insight that is transmitted back to the user. Only the user who possesses the FHE secret key can decrypt the result and gain insight.

Figure 1.

Privacy-preserving MLaaS protocol with FHE.

We provide a comprehensive guide that empowers the reader to implement private SVM inference using homomorphic encryption. Our main goal is to provide readers with the knowledge and tools necessary to implement private SVM inference while preserving data confidentiality via FHE. This chapter caters to readers with varying levels of cryptographic expertise, assuming only a basic understanding of SVMs.

The remainder of this chapter is structured as follows. Section 2 reviews the state of the art on private machine and deep learning under FHE. Section 3 reviews the core methods employed in the presented solution. Section 4 details the design and implementation of the proposed solution. Section 5 presents the experimental results. Section 6 concludes the work and explores potential avenues for future research.

2. Literature review

Gentry’s seminal work [20] realized the first FHE scheme, enabling arbitrary computations on encrypted data without compromising privacy. This new cryptographic capability has seen far-reaching implications, particularly in cloud-based machine and deep learning applications. Numerous studies have investigated the potential of FHE-based private machines and deep learning.

The exploration of machine learning models under FHE began with linear classifiers [23], followed by a series of works adapting other common models, including Convolutional Neural Networks (CNN) [12], SVMs [14, 15], logistic regression [17, 24, 25], and Transformers [26, 27] and many others. These advancements have paved the way for secure and private machine learning applications, ensuring the protection of sensitive data while maintaining computational capabilities.

Notably, the majority of research in this domain has concentrated on the inference phase of the machine or deep learning process, largely due to its relative simplicity and lower computational intensity compared to the training phase. However, a growing body of recent studies has begun to explore the feasibility of machine and deep learning training under FHE, yielding promising preliminary results. These investigations encompass various aspects, including logistic regression training [28], SVM training [29], CNNs training [30], shallow neural networks training [16], and transfer learning [31, 32], which paves the way for more comprehensive advancements in secure machine learning.

In this chapter, we focus on the evaluation of the inference phase of SVM models under FHE, presenting the reader with a comprehensive overview of cutting-edge tools and expert optimization techniques to facilitate this process. Our primary objective is to provide practitioners with the knowledge and expertise necessary to develop and implement their own secure SVM inference models, enabling them to protect their sensitive data while maintaining its utility.

3. Preliminaries

In this section, we provide an overview of the basic concepts and theoretical foundations that underpin our subsequent discussion, establishing a solid basis for the developments presented in this chapter.

3.1 Notations

We adopt the following notation conventions throughout this chapter: scalar quantities are denoted by lowercase letters, e.g., $a$; vector quantities are denoted by lowercase boldface letters, e.g., $\mathbf{a}$; and matrix quantities are denoted by uppercase boldface letters, e.g., $\mathbf{A}$. The dot-product between two vectors $\mathbf{u}$ and $\mathbf{v}$ is denoted by $⟨\mathbf{u}\cdot \mathbf{v}⟩$. The point-wise addition and multiplication between two vectors are denoted by $\oplus$ and $\otimes$, respectively. We will be working with various sets of numbers such as the set of integers denoted as $\mathbb{Z}$, the set of reals $\mathbb{R}$, and the set of complex numbers $\mathbb{C}$. Encrypted quantities are distinguished by an over-bar symbol, e.g., $\overline{a}$ to denote an encryption of the scalar quantity $a$.

3.2 CKKS FHE scheme

The Cheon-Kim-Kim-Song (CKKS) scheme [33] is a FHE scheme ideally well-suited for handling encrypted real or complex quantities. Due to the wide use of real-number computations in machine learning algorithms, including SVMs, CKKS stands out as one of the most efficient FHE schemes for implementing machine learning algorithms.

Similar to other FHE schemes, CKKS operates under the honest-but-curious threat model, which assumes that all participants in a CKKS computation will follow the underlying protocol but may attempt to monitor and analyze the system to extract sensitive information. As long as participants adhere to the protocol, CKKS guarantees the confidentiality of secret keys and encrypted messages, provided that the participants’ computational power does not exceed a predefined security level. For instance, CKKS may offer a 128-bit security level, ensuring that any attempt to break the system, recover the secret key, or decrypt encrypted messages would require computational resources exceeding this threshold. The security level of CKKS is primarily determined by the selection of cryptographic parameters, which must be chosen carefully to ensure a secure instantiation of CKKS that meets a specified security requirement. The careful choice of parameters is crucial to maintain the confidentiality and integrity of encrypted data and prevent potential security breaches.

It is crucial to understand the concept of noise, also referred to as error, which is inherent in CKKS and numerous other FHE schemes. This noise is a fundamental component of the Learning With Errors (LWE) problem [34], the primary mathematical framework underlying CKKS and many other FHE schemes. The security of CKKS is predicated on the presence of this noise, which is introduced during the encryption process. As homomorphic computations are performed on encrypted data, the noise accumulates and must be managed through specific CKKS primitives. Fortunately, many of existing CKKS libraries automate this noise maintenance process and abstracts it from the user.

3.2.1 CKKS instantiation

The CKKS scheme is instantiated by specifying various user-defined parameters. Firstly, the multiplicative depth, denoted as d, must be selected, which determines the maximum number of sequential multiplications in the computation graph. It is essential to note that increasing the multiplicative depth can result in higher computational overhead, as CKKS internally manipulates larger mathematical objects. Consequently, minimizing the multiplicative depth is crucial to reduce the CKKS parameters, decrease computational overhead, and enhance the overall performance of the CKKS application.

Secondly, the user should specify the precision of the underlying CKKS computation by defining the scaling factor parameter, denoted as $\sigma$ (measured in bits). This parameter controls the precision of the computation and is typically selected within the range of 45–60 bits in most existing CKKS implementations. By adjusting $\sigma$, users can trade-off between computational efficiency and precision, with higher values resulting in more accurate computations at the cost of increased computational overhead.

Thirdly, the user can specify the CKKS security parameter, denoted as $\lambda$ (measured in bits), which determines the security level of the CKKS scheme. This parameter controls the computational overhead and overall performance of the CKKS application, with higher values of $\lambda$ providing increased security at the cost of reduced performance. Typical values for $\lambda$ include 128, 192, and 256 bits, which correspond to common security levels. It is essential to note that increasing $\lambda$ will result in higher computational overhead, leading to decreased overall performance of the CKKS application.

Upon defining the aforementioned parameters, the user can instantiate the CKKS scheme, which entails calculating the encryption keys, comprising the public key $\left(\mathit{pk}\right)$ and secret key $\left(\mathit{sk}\right)$, as well as the public evaluation keys, including the multiplication key $\left(\mathit{evk}\right)$, set of rotation keys $\left({\left\{\mathit{rk}\right\}}_{i=0}^{r-1}\right)$, and bootstrapping key $\left(\mathit{bk}\right)$. The encryption keys are utilized for data encryption and decryption operations, whereas the evaluation keys are shared with the evaluator, who will be executing the CKKS computation. The multiplication key enables homomorphic multiplication, allowing the evaluator to multiply two encrypted quantities, while the rotation keys facilitate homomorphic rotation, enabling the rotation of an encrypted vector quantity by any integral rotation index $i\in \left\{0,\dots ,r-1\right\}$. The bootstrapping key, on the other hand, is exclusively required for deep computational workloads that necessitate the refreshing of the noise in the encrypted quantities, ensuring the continued security and integrity of the computation.

3.2.2 Data encoding and encryption

CKKS provides two fundamental primitives for data encoding, defined as follows:

• ENCODE: takes as input real or complex vector $\mathbf{a}$ either $\in {\mathbb{R}}^n$ or $\in {\mathbb{C}}^n$ and returns a plaintext object $\left(a\right)$ that encodes the input vector.

• DECODE: takes as input a plaintext object $a$ and decodes it into the corresponding vector quantity $\mathbf{a}$ either $\in {\mathbb{R}}^n$ or $\in {\mathbb{C}}^n$, enabling the retrieval of the original encoded data.

For data encryption and decryption, CKKS provides two main primitives as follows:

• ENCRYPT: takes as input either $\mathit{sk}$ or $\mathit{pk}$ and a plaintext object $a$, and returns an encryption of $\overline{a}$, a.k.a., ciphertext $\overline{a}$.

• DECRYPT: takes as input $\mathit{sk}$ and a ciphertext $\overline{a}$ and returns the corresponding plaintext object $a$.

After encoding and encrypting their sensitive data, the user can transmit it to an untrusted evaluator for homomorphic computation. Upon completing the computation, the evaluator returns the encrypted results to the user, who can then decrypt and decode them to obtain the desired insights.

3.2.3 Homomorphic data processing

The CKKS homomorphic encryption scheme offers the evaluator the capability to perform specific operations on encrypted data without requiring decryption. This section details the three primary homomorphic operations supported by CKKS:

• Homomorphic Addition—$\text{EVALADD}\left({\overline{c}}_1,{\overline{c}}_2\right):$ performs homomorphic point-wise addition of the underlying encrypted vectors yielding ciphertext ${\overline{c}}_{\mathrm{add}}=\text{Encryption}\left(\text{Encoding}\left({\mathbf{v}}_1\oplus {\mathbf{v}}_2\right)\right).$ Furthermore, CKKS inherently supports homomorphic subtraction using the same principle.

• Homomorphic Multiplication—$\text{EVALMUL}\left({\overline{c}}_1,{\overline{c}}_2,\mathit{evk}\right):$ performs homomorphic point-wise multiplication of the underlying encrypted vectors yielding ciphertext ${\overline{c}}_{\mathrm{mul}}=\text{Encryption}\left(\text{Encoding}\left({\mathbf{v}}_1\otimes {\mathbf{v}}_2\right)\right).$ The multiplication key $\mathit{evk}$ is used for noise maintenance.

• EvalRotate($\overline{c}$, $i$, $\rho$, ${\mathit{rk}}_i$): performs cyclic rotation of the encrypted message vector by an amount $i\in {\mathbb{Z}}^{+}$ in direction $\rho \in \left\{0=\text{left},1=\text{right}\right\}$, generating a ciphertext that encrypts the rotated vector. This operation disrupts the the ciphertext structure and the rotation key ${\mathit{rk}}_i$ is used to restore the original structure.

It is important to note that both EVALADD and EVALMUL support a mixed-mode operation, where one input argument can be in plaintext while the output remains encrypted. Specifically, when using EVALMUL with one plaintext input argument, the computation is significantly faster compared to when both input arguments are encrypted.

3.3 SVM

SVMs are powerful supervised machine learning algorithms widely used for classification and regression tasks. While both classification and regression involve learning from a labeled dataset, this section will primarily focus on their application in classification problems. We begin by providing a brief overview of the training process employed to generate the SVM model parameters, followed by a detailed description of the SVM inference process. The latter is a computational procedure that is specifically implemented in CKKS, and it is this process that enables the homomorphic encryption and secure computation of SVM-based predictions.

3.3.1 SVM training

As mentioned previously, the presented system requires the server to have a pre-trained model. This model can be realized through various approaches, including the utilization of public domain data, which is freely available and accessible, which eliminates concerns regarding data privacy. Another common approach is the use of synthetic data, which is artificially generated to mimic real-world data, providing a viable substitute for the sensitive dataset. Furthermore, non-private segments of the dataset can be utilized, comprising data from participants who have explicitly consented to open-source their information, a common practice observed in genome studies where individuals willingly share their genetic data for research purposes (e.g., [35]). The resulting pre-trained model is used as an input to the privacy-preserving SVM inference protocol. The subsequent paragraphs describe formally the training process, which involves utilizing a subset of the evaluation dataset, designated as the training dataset, to train and fine-tune the SVM model’s performance and accuracy.

SVM training can be described as follows. Given an input dataset comprising $m$ data points, each characterized by $n$ features, which can be represented as a matrix $\mathbf{X}\in R^{m\times n}$. In addition, a label vector $\mathbf{y}\in {\left\{+1,-1\right\}}^m$ is associated with the dataset, where each entry $y_i$ indicates the class to which data point ${\mathbf{X}}_i$ belongs. Given a chosen kernel function $\mathbb{K}$, a SVM classifier can be trained to identify the optimal separating hyperplane between two classes.

Several widely utilized kernel functions exist for support vector machines, each serving distinct purposes. The identity function, which maps inputs to themselves, is primarily employed in linearly separable problems due to its simplicity. Conversely, for non-linearly separable problems, more complex kernel functions are employed. In such cases, the polynomial kernel, Radial Basis Function (RBF), also known as the Gaussian kernel, and the Sigmoid kernel, a smooth and continuously differentiable function, are commonly utilized to facilitate non-linear transformations and enhance the SVM model’s descriptive capacity.

The training process of a SVM model involves determining the optimal hyperplane that best separates data points into different classes. This is achieved by optimizing the model parameters to maximize the margin between the classes while minimizing the misclassification error. To achieve this, SVM employs quadratic programming and Lagrange multipliers. This mathematical formulation transforms the problem of finding the optimal hyperplane into a constrained optimization problem. The solution to this problem yields the model parameters, including the weights and bias of the hyperplane.

The training algorithm learns the following model parameters:

• set of support vectors ${\left\{\mathit{SV}\right\}}_0^{l-1}$ which is a subset of $\mathbf{X}$.

• the coefficients vector $\mathit{\alpha }\in {\mathbb{R}}^l$ which can be interpreted as weight factors assigned to the support vectors, influencing their contribution to the decision boundary.

• the intercept (or bias) parameter $b\in \mathbb{R}$.

We conclude our discussion on SVM training at this point, as our system assumes the availability of a trained SVM model’s parameters as input. Omitting the mathematical description of SVM training for the sake of brevity, we instead refer interested readers to consult dedicated SVM resources for a comprehensive understanding of the training process [36].

3.3.2 SVM inference

The SVM inference function is of paramount importance in our system. Specifically, the evaluator is tasked with evaluating a given SVM model on encrypted data points provided by the user. Consequently, the server performs an encrypted evaluation of the SVM inference function, which necessitates a detailed description of this function. The SVM inference function is responsible for computing the prediction output of the SVM model on the encrypted data points, and its secure evaluation is crucial for maintaining the confidentiality of the user’s data.

During SVM inference, our primary objective is to evaluate the decision function, as presented in Eq. (1). The kernel function $\mathbb{K}$ poses a challenge when it is chosen as RBF or Sigmoid. In contrast, a polynomial kernel function is inherently compatible with the CKKS computational model and does not present any difficulties. To address the challenges associated with RBF and Sigmoid kernel functions, we can employ polynomial approximations over a predefined range, which have demonstrated excellent performance in various studies [12, 13, 37, 38, 39].

$$c\left(\mathbf{x}\right)=sign\left(\sum _{i=0}^{l-1}{\alpha }_i{y}_i\mathbb{K}\left(⟨{\mathbf{x}}_i,\mathbf{x}⟩\right)+b\right),\text{where}{\mathbf{x}}_i\text{is the}i‐\mathrm{th}\text{support vector}.$$E1

The decision function presents an additional challenge in computing the $\mathit{sign}$ function, which is a non-arithmetic operation. To address this, we can draw on a similar strategy employed for handling the kernel function, approximating the sign function as a polynomial. Alternatively, the sign function can be computed on the client side upon returning the SVM insight. In our approach, we opt for the former method, approximating the sign function as a polynomial to facilitate its evaluation within the decision function. Notably, for SVM regression tasks, the sign function is not required, thereby eliminating this challenge altogether.

4. System implementation

This section outlines the implementation details of our system. We begin by introducing the dataset employed for evaluation purposes, followed by an overview of the tools utilized in our implementation.

4.1 Evaluation dataset

We evaluated our system using Dal’s Credit Card Dataset [40], a comprehensive collection of credit card transactions and user information. This dataset contains transactions made by European cardholders in September 2013, spanning two days, with a total of 284,807 transactions, of which 492 are fraudulent.

4.1.1 Data preprocessing

Due to privacy concerns, the dataset is not publicly available in its raw format. Instead, a preprocessed version is provided, which has undergone Principal Component Analysis (PCA) to extract the most informative features. The PCA transformation retains only the features that show a high correlation with the response variable, a class label indicating whether each transaction is fraudulent (Class 1) or legitimate (Class 0). This preprocessing step helps to reduce dimensionality and improve model performance.

It is in these particular situations where privacy-preserving machine learning analysis prove to be paramount. Traditional data preprocessing techniques can compromise data quality, potentially limiting the accuracy of the analysis. Furthermore, these techniques often lack cryptographic security, making it possible for unauthorized parties to invert the preprocessing and recover the original dataset [41]. To address this concern, it is essential to adopt cryptographically secure systems that can process encrypted raw data without sacrificing privacy.

4.1.2 Dataset characteristics

The dataset comprises only numerical data, making it suitable for machine learning-based fraud detection. The dataset is extremely unbalanced, with a fraud rate of approximately 0.17%. This imbalance is a common challenge in fraud detection, as fraudulent transactions are typically rare compared to legitimate ones. The dataset characteristics are summarized in Table 1. It comprises 28 predictor features, denoted as $\left\{V1,\dots ,V28\right\}$. These features represent the transformed variables resulting from the PCA preprocessing technique, with the exception of two original features:

1. Time: The timestamp of each transaction, which was not subjected to PCA transformation and is retained in its original form.

2. Amount: The transaction amount, which was also exempt from PCA preprocessing and is present in the dataset in its raw form.

Attribute	Value
Number of Features	28
Type of Classification	Binary (1: fraud, 0:legitimate)
Number of Samples	284,807
Number of Fraudulent Samples	492
Number of Legitimate Samples	284,315
Feature Type	Numerical
PCA Transformed Features	26
Original Features	2 (Time, Amount)

Table 1.

Dal’s credit card dataset characteristics [40].

The inclusion of these two original features, alongside the 26 transformed features, provides a comprehensive set of predictors for modeling and analysis.

4.2 Implementation

We used three key open-source tools to develop our privacy-preserving credit card fraud detection system. We utilized Python’s scikit-learn’s SVM package [36], which provides an efficient implementation of SVMs for both training and inference. We also employed the FHSVM framework [15], an FHE-based privacy-preserving SVM inference engine, enabling secure and private predictions on encrypted data. Lastly, we used the OpenFHE framework [42] which includes an implementation of CKKS that is required by FHSVM. Below, we describe this computational infrastructure in more detail and how they are utilized to build a privacy-preserving credit card fraud detection system.

4.2.1 Scikit-learn SVM

We employed the Python sci-kit-learn library’s SVM package to train an SVM model on the credit card dataset. The comprehensive training methodology, including data preprocessing, feature selection, model configuration, and extraction, is shown in Figure 2. Below, we detail each step in the training procedure.

Figure 2.

SVM training workflow.

4.2.1.1 Dataset processing

To prepare the dataset for model training and evaluation, we performed the following processing steps. The dataset was randomly partitioned into training and testing sets, adhering to a ratio of 80% for training and 20% for testing. This split ensures that the model is trained on a substantial portion of the data while reserving a sufficient amount for evaluation purposes. The random partitioning helps maintain the integrity of the dataset’s distribution, reducing the likelihood of bias in the training process. We also excluded samples with missing values to enhance the reliability and accuracy of our analysis.

4.2.1.2 Model training

Given the dataset’s inherent class imbalance, we adopted an undersampling approach to create a balanced training dataset. This involved maintaining the entirety of the fraudulent samples, while randomly sampling an equal number of legitimate transactions from the dataset. By doing so, we ensured that the model was trained on a dataset with an equal number of instances from both classes. It should be noted that undersampling was only applied to the training dataset. During model evaluation, the entire testing dataset is used without undersampling.

To address the critical nature of accurately detecting fraudulent transactions, we implemented a cost-sensitive approach by assigning higher penalties to false negatives than false positives. Specifically, we penalized missing a fraud more heavily by modifying the SVM’s class_weight parameter to 0:0.75, 1:0.25. This adjustment ensured that the model prioritized the accurate detection of fraudulent transactions.

Furthermore, we selected the Radial Basis Function (RBF) kernel for the SVM model due to its superior performance in capturing non-linear relationships between the predictor variables and the response variable. Given the complexity of the relationships between predictor and response variables in our dataset, the RBF kernel’s ability to model non-linear patterns made it the preferred choice for our SVM. Preliminary experiments with other kernels reinforced this decision. Detailed SVM model parameters are presented in Table 2.

Parameter	Value
Kernel	RBF
Penalty C	1.0
Gamma	scale
Probability	True
Number of Support Vectors	114
Class weight	0: 0.75, 1: 0.25
Decision function shape	OVR

Table 2.

SVM model summary.

4.2.1.3 Model evaluation

Following the training phase, we conducted a comprehensive evaluation of our model’s performance using a range of standard metrics. These metrics provide insights into the model’s ability to accurately detect fraudulent transactions while minimizing false positives. The evaluation metrics employed include:

1. Precision: Measures the proportion of true positives (correctly identified fraudulent transactions) among all positive predictions made by the model.

2. Recall: Quantifies the proportion of true positives among all actual fraudulent transactions in the dataset.

3. Classification Accuracy: Calculates the overall proportion of correctly classified transactions (both fraudulent and legitimate).

4. F1-score: Harmonizes precision and recall into a single metric, providing a balanced measure of the model’s performance.

5. Area Under the Receiver Operating Characteristic Curve (ROC-AUC): Assesses the model’s ability to distinguish between fraudulent and legitimate transactions.

4.2.1.4 Exporting the model parameters

We have successfully developed a high-performing SVM model, capable of accurately classifying transactions as fraudulent or legitimate. To facilitate the secure evaluation of this model on encrypted data, we extract its key parameters, including support vectors, gamma value, dual coefficients, and intercept, and store them in plain text files. These extracted parameters will be utilized by the FHSVM module to enable the homomorphic evaluation of our SVM model on encrypted input samples.

4.2.2 FHSVM framework

The FHSVM framework enables the secure evaluation of an SVM model on encrypted input samples, utilizing the CKKS FHE scheme to compute the decision function in Eq. (1). This framework facilitates the classification of transactions without compromising data confidentiality.

As mentioned previously, the decision function involves basic arithmetic operations on real numbers, which are inherently supported by the CKKS scheme. However, the kernel and sign functions require special handling, as they are not supported by CKKS. To address this, the FHSVM framework substitutes these functions with polynomial approximations, ensuring accurate computations.

To implement the FHSVM framework, two critical parameters must be defined:

1. The range over which the Radial Basis Function (RBF) and sign functions are evaluated, ensuring accurate polynomial approximations.

2. The degree of the polynomial approximations required to accurately represent the RBF and sign functions.

It is important to note that increasing the degree of polynomials and expanding the evaluation range can lead to more accurate polynomial approximations of the RBF and sign functions. However, this enhancement comes at a cost, as it requires the use of larger CKKS parameters, which can significantly degrade performance. To achieve a balance between accuracy and efficiency, careful optimization of these parameters is necessary to tailor them to the specific requirements of the problem at hand. In our case, we have found that evaluating the RBF and sign functions over a range of −16 to 1, with a polynomial degree of 13, provides an optimal trade-off between accuracy and performance. Through rigorous experimentation, we have validated these parameters and present them below as the optimal values for our dataset.

4.2.3 OpenFHE

OpenFHE is a comprehensive, open-source FHE library that provides a robust implementation of various FHE schemes, including the CKKS scheme. As a versatile and efficient backend, OpenFHE enables the secure evaluation of complex mathematical operations on encrypted data. In the context of our system, OpenFHE plays a crucial role as the underlying FHE library for the FHSVM framework, facilitating the secure evaluation of CKKS operations.

As a result of our system requirements, we have carefully selected the following CKKS parameters to ensure optimal performance and security. Table 3 shows the cryptographic parameters used in instantiating CKKS in OpenFHE. The polynomial ring used for encryption is set to $2^{16}$. The multiplicative depth is set to 13, indicating the maximum number of multiplications allowed during homomorphic evaluation. We have chosen a security level of 128-bit, providing robust protection against known attacks. The scale precision is set to 59-bit, ensuring accurate fixed-point arithmetic operations. As bootstrapping is not needed for our system, it is disabled to optimize performance, and the secret key distribution follows a ternary scheme. These carefully chosen parameters enable our system to achieve a balance between security, accuracy, and efficiency.

Parameter	Value
Ring Dimension	$2^{16}$
Multiplicative Depth $\left(d\right)$	13
Desired Compute Precision $\left(\sigma \right)$	59-bit
Security Level $\left(\lambda \right)$	128-bit
Bootstrapping	Disabled
Secret Key Distribution	Ternary

Table 3.

CKKS parameters used in our privacy-preserving credit card fraud detection system.

5. Results

In this section, we present the results of our experiments evaluating the performance and security of our privacy-preserving SVM inference system using the CKKS FHE scheme. We first report the results of training our SVM model in the clear. We then present the results of our homomorphic inference experiments, including the accuracy and performance overhead of encrypting and decrypting our data. Finally, we evaluate the security of our system, verifying that our encryption parameters achieve the desired level of security and that our homomorphic inference process preserves the privacy of our input data.

5.1 Training results

To establish a baseline for our privacy-preserving SVM inference, we first trained our SVM model in Python clearly. This allowed us to evaluate the model’s performance on our dataset without any encryption overhead. We report the results of this training process, including the model’s accuracy, precision, recall, F1 score, and other relevant metrics.

Table 4 characterizes the classification quality of our SVM model when evaluated on the testing dataset. The model achieves a precision of 0.9988, recall of 0.9971, and accuracy of 0.9971, demonstrating its ability to accurately detect fraudulent transactions. Moreover, the F1-score of 0.9977 and AUC of 0.9834 further validate the model’s robustness. Notably, the balanced accuracy of 0.9283 highlights the model’s effectiveness in handling class imbalance. Overall, these metrics indicate that the SVM model succeeds in distinguishing between legitimate and fraudulent transactions.

Metric	Value
Precision	0.9988
Recall	0.9971
Accuracy	0.9971
Balanced Accuracy	0.9283
F1-Score	0.9977
Area Under the Curve (AUC)	0.9834

Table 4.

Characterizing the SVM model performance.

5.2 Homomorphic inference results

With our SVM model trained, we then proceeded to evaluate its performance under homomorphic encryption using OpenFHE’s CKKS implementation. This involved encrypting our input data, performing inference on the encrypted data, and then decrypting the results. We report the accuracy and performance overhead of this homomorphic inference process, comparing it to the clear-text inference results.

Table 5 characterizes the performance of the SVM model’s homomorphic inference. The most time-consuming operation is the SVM decision function evaluation, taking approximately 5.21 seconds, which is roughly 86% of the total inference time. This suggests that optimizing the decision function evaluation could significantly improve overall performance. In contrast, sample encryption and decryption are relatively fast, taking less than 0.2 seconds combined. The sign function evaluation takes around 0.82 seconds, which is a reasonable overhead considering its importance in the inference process. The computation precision of up to 4 decimal digits (13 bits) indicates a good balance between accuracy and efficiency.

Operation	Time (sec)
Sample Encryption	0.146
SVM Decision Function Evaluation	5.214
Sign Function Evaluation	0.816
SVM Inference Total Evaluation Time	6.030
Sample Decryption	0.054
Computation Precision	Up to 4 decimal digits ($\tilde{1}$3 bits)

Table 5.

Characterizing the SVM model homomorphic inference performance.

Potential performance optimizations can significantly enhance the efficiency of our privacy-preserving SVM inference solution. One approach is to offload the sign function computation to the client, who can process it in the clear. By doing so, we can drop approximately 0.816 seconds from the overall computation time. Furthermore, it’s essential to recognize that in SVM regression tasks, the sign function evaluation is not required and can be entirely eliminated.

We emphasize here that encryption imposes a substantial performance overhead on private SVM inference, significantly amplifying runtime by four to five orders of magnitude. While plaintext SVM inference completes in mere milliseconds, the equivalent encrypted operation demands several seconds of computation. This dramatic slowdown is attributable to the inherent complexities of operating within the encrypted domain using FHE. Furthermore, the approximate nature of CKKS computations results in a notable reduction in computational precision to 13 bits, a stark contrast to the double-precision standard prevalent in plaintext calculations (52 bits). Despite this limitation, we verified that 13-bit precision is adequate for our specific use case and potentially applicable to a broader range of scenarios.

5.3 Security evaluation

Our FHE-based privacy-preserving fraud detection system assumes a semi-honest threat model. The semi-honest threat model, also known as the “honest-but-curious” model, is a security framework commonly assumed in FHE protocols. In this model, both the server and client are assumed to follow the protocol correctly, executing the agreed-upon steps without deviations. However, they are also curious and may attempt to infer additional information from the data they receive or observe during protocol execution.

Crucially, the semi-honest model assumes that the server and client do not collude with each other and do not actively attempt to cheat or manipulate the protocol. Their malicious behavior is limited to learning more information than is explicitly provided by the protocol.

Thus, the primary security goal of FHE under the semi-honest model is to prevent the server from gaining any significant knowledge about the data beyond what is intentionally revealed by the protocol. This implies that even if the server attempts to analyze or intercept the encrypted data, it should be unable to deduce sensitive information. By operating within the semi-honest threat model, FHE offers a robust security guarantee, ensuring that even curious parties cannot compromise data confidentiality.

To ensure the security and privacy of our homomorphic inference process, we conducted a thorough security evaluation. This involved analyzing the encryption parameters used in OpenFHE’s CKKS implementation to verify that they achieve the desired security level (128-bit) against known attacks. We also confirmed that the homomorphic inference process preserves the privacy of the input data, as demonstrated by the inability to recover original data from ciphertext.

It is crucial to acknowledge that current FHE schemes lack robust security guarantees, specifically Chosen-Ciphertext Attack 2 (CCA2) security and verifiable computation. This vulnerability allows a malicious server or interceptor to manipulate ciphertexts by performing computations on them, potentially disrupting the service by providing incorrect results. Furthermore, FHE inherently lacks verifiable computation security, leaving the client unable to verify whether the server has performed the intended computation. Hence, FHE operates ideally under the honest-but-curious threat model, relying on legal bindings or incentives to enforce honest behavior among participating parties.

In addition, FHE schemes are susceptible to side-channel attacks if not implemented correctly, as demonstrated in recent studies [43, 44]. These findings highlight the need for enhanced security measures and rigorous implementation to ensure the integrity of FHE-based systems.

5.4 Discussion

Our results demonstrate the feasibility of privacy-preserving SVM inference via FHE but also highlight the trade-offs between accuracy, performance, and security. In this discussion, we summarize our key findings, explore the implications of our results for practical applications, and identify avenues for future research to further improve the performance and security of homomorphic inference.

5.4.1 Predictive performance

The SVM model exhibits remarkable predictive performance, achieving high accuracy and precision in detecting fraudulent transactions. In addition, the model demonstrates robustness across different evaluation metrics, including F1-score, AUC, and balanced accuracy.

It is important to note that we view SVM training and SVM inference as essentially orthogonal processes. This independence means that any optimizations or improvements made to the training process, aimed at enhancing the classifier’s performance, will not impact the homomorphic SVM inference functionality. Advancements in training techniques, such as kernel selection, regularization, or data preprocessing, can be explored and implemented independently without affecting the encrypted inference pipeline. This decoupling of training and inference enables flexibility and future-proofing, allowing for continuous improvements in classification accuracy without compromising the security and privacy benefits of homomorphic inference.

5.4.2 The importance of privacy-preserving MLaaS

A significant challenge in our research was the unavailability of raw data due to stringent privacy regulations. Sensitive data owners often preprocess data using techniques such as PCA before sharing it, which can potentially compromise data integrity and limit the effectiveness of machine learning models. While PCA can reduce dimensionality, it also inherently discards information which impacts the model performance. Furthermore, the vulnerability of PCA-processed data to inversion attacks raises serious concerns about data privacy [41]. This highlights the importance of cryptography-based privacy-preserving technologies such as FHE which facilitates data analysis on encrypted raw data. This not only safeguards sensitive information but also enables comprehensive data utilization providing valuable insights without compromising privacy.

5.4.3 Scalability

We tested the SVM inference on a commodity laptop, which provided valuable insights into the model’s performance under typical computing conditions achieving 6 seconds overall inference latency. However, it is likely that utilizing a high-end server with advanced processing capabilities, increased memory, and optimized hardware configurations could significantly enhance the performance of the SVM inference. Such a setup could potentially reduce the computation time, especially for the most time-consuming operation, the SVM decision function evaluation, and improve the overall efficiency of the homomorphic inference process. Moreover, a high-end server could also enable the processing of multiple SVM queries, larger datasets, more complex models, and increased precision levels, making it an attractive option for applications requiring high-performance secure computations.

6. Conclusions

In conclusion, this paper has demonstrated the feasibility and effectiveness of implementing privacy-preserving SVM inference using the CKKS FHE scheme. We successfully enabled SVM inference on encrypted data samples, ensuring the confidentiality of sensitive information even when processed on untrusted servers. Our comprehensive evaluation on a large-scale credit card fraud detection dataset showcased remarkable predictive accuracy and reasonable inference latency of nearly 6 seconds, highlighting the practicality of our approach.

The significance of this work lies in its ability to bridge the gap between machine learning and privacy-preserving computing, enabling organizations to harness the power of SVM inference without compromising data privacy. As the demand for secure computing solutions continues to grow, our implementation serves as a vital step toward realizing the full potential of FHE in real-world applications.

Future research directions may focus on optimizing the performance of homomorphic SVM inference, exploring alternative FHE schemes, and expanding the scope of privacy-preserving machine learning applications.

Conflict of interest

The authors declare no conflict of interest.