Implementing Homomorphic Encryption for ML Inference on Encrypted Data

A fintech client transfers data for scoring but requires that even the cloud administrator cannot see the raw values. Homomorphic encryption (HE) allows ML inference on encrypted data—the server mathematically cannot access plaintext. For example, one of our clients, a bank, implemented HE for a scoring model, reducing audit costs by 30%. This article covers a practical HE implementation for ML using the CKKS scheme and the Microsoft SEAL library. Our experience includes 10+ projects in fintech and healthcare, guaranteeing encryption-level confidentiality. Contact us to assess HE applicability for your project.

Problems Solved by HE

Problem 1: Trust in cloud providers. MLaaS providers often cannot guarantee that administrators will not read data. HE eliminates this risk: computations on ciphertext do not require decryption.

Problem 2: Regulatory constraints. GDPR, HIPAA, and central bank regulations require personal data protection during processing. HE enables compliance without abandoning cloud computing.

Problem 3: Performance. A naive HE implementation incurs enormous overhead (up to 30,000x). Optimization via SIMD packing and choosing the CKKS scheme reduces this to approximately 600x.

Why CKKS is the Best Choice for ML

CKKS outperforms FHE by 10–50x for typical ML models. It supports approximate floating-point arithmetic and SIMD packing—a single ciphertext can hold thousands of values, accelerating batch processing. A linear layer 1024→512 takes 80 ms on encrypted data (plaintext: 0.1 ms), but parallel processing of 64 examples reduces overhead to ~600x.

Practical Implementation with Microsoft SEAL

import seal
from seal import EncryptionParameters, scheme_type, SEALContext
from seal import KeyGenerator, Encryptor, Evaluator, Decryptor
from seal import CKKSEncoder, RelinKeys, GaloisKeys

# Setup CKKS parameters
parms = EncryptionParameters(scheme_type.ckks)
poly_modulus_degree = 8192  # Security level
parms.set_poly_modulus_degree(poly_modulus_degree)
parms.set_coeff_modulus(seal.CoeffModulus.Create(poly_modulus_degree, [60, 40, 40, 60]))

context = SEALContext(parms)
keygen = KeyGenerator(context)
public_key = keygen.create_public_key()
secret_key = keygen.secret_key()
relin_keys = keygen.create_relin_keys()
galois_keys = keygen.create_galois_keys()

scale = 2.0**40
encoder = CKKSEncoder(context)

# Client encrypts input
input_data = [0.5, 0.3, 0.8, ...]  # Feature vector
plain = encoder.encode(input_data, scale)
encrypted_input = Encryptor(context, public_key).encrypt(plain)

# Server computes on encrypted data (doesn't see actual values)
evaluator = Evaluator(context)
# ... matrix multiplication, activation approximation ...
encrypted_result = evaluator.multiply_plain(encrypted_input, weight_matrix)

# Client decrypts result
result = Decryptor(context, secret_key).decrypt(encrypted_result)
output = encoder.decode(result)

How to Approximate Nonlinear Functions?

The main challenge in HE is that nonlinear functions (ReLU, sigmoid) are not directly supported—only polynomials are. Solutions:

• ReLU: Approximate with a polynomial of degree 3–7 over the working range. Degree 3 yields ~1–2% accuracy degradation but requires significantly fewer multiplications.
• Sigmoid: Use Taylor series or minimax polynomial.
• Softmax: Requires special handling due to division.

Alternative: Replace architecture with HE-friendly activations (e.g., x² instead of ReLU). This eliminates approximation but requires retraining the model.

Comparison of HE Schemes

Performance and Limitations

Practical today for logistic regression, shallow networks, and privacy-preserving inference in MLaaS. Infrastructure savings can reach 40% by eliminating dedicated HSMs.

HE-as-a-Service Pattern

The most realistic use case: a cloud MLaaS provider wants to offer inference without seeing client data.

1. Provider trains model on public/synthetic data.
2. Client encrypts data on their side.
3. Client sends ciphertext to provider.
4. Provider computes inference on ciphertext.
5. Provider returns encrypted result.
6. Client decrypts the result.

The provider never sees the input data or the result. For short computation chains (depth up to 5), bootstrapping is not needed.

Libraries and Frameworks

Our Work Process

1. Analysis: Audit the ML model, determine the HE scheme and required precision.
2. Design: Select parameters (poly_modulus_degree, scale), approximate nonlinear functions.
3. Implementation: Integrate the HE layer into the pipeline (Python/C++), write benchmarks.
4. Testing: Verify accuracy on encrypted data, optimize latency.
5. Deployment: Deploy on your infrastructure, train your team.

What's Included in the HE Implementation

• Model audit and optimal HE scheme selection.
• Development of an HE-compatible pipeline and integration with your infrastructure.
• Documentation and team training.
• Technical support during implementation.

Estimated Timelines

• Simple model (logistic regression, 2–3 layers): from 8 weeks.
• Complex model (CNN, RNN): from 12 to 16 weeks.

Timelines vary depending on architecture and accuracy requirements. Pricing is based on individual assessment.

Evaluate the possibility of implementing HE for your ML service—contact us for a consultation. Request a free engineering consultation for your project.