PQMessenger: A Web-Based Hybrid Post-Quantum Encrypted Messaging System

PQMessenger: A Web-Based Hybrid Post-Quantum Encrypted Messaging System Swechchha Adhikari∗Shrijan Poudel† Abstract Post-quantum cryptography has emerged as a critical field to address the threats posed by quantum computers to classical cryptographic algorithms. To demonstrate its practical feasibility, this paper presents PQMessenger, a proof-of-concept web-based messaging platform implementing a hybrid encryption framework that combines the CRYSTALS-Kyber key encapsulation mechanism (KEM) with AES-CBC for data confidentiality. We describe the system’s full-stack design, including its frontend, backend, and cryptographic architecture, as well as key engineering challenges related to session management, cross-platform library integration, and secure key storage. Our goal is to bridge theoretical post-quantum security with practical web deployment, and to provide an open-source foundation for future educational and experimental use. 1 Introduction and Motivation The rise of quantum computing poses a major threat to existing digital security systems. Public-key encryption algorithms, such as RSA, rely on mathematical problems that are infeasible for classical computers to solve efficiently [4]. Quantum computers, however, exploit superposition and entanglement through quantum bits (qubits), allowing them to evaluate many solutions simultaneously. Shor’s algorithm, introduced in 1994, uses these capabilities to factor large integers and compute discrete logarithms in polynomial time, compromising standard encryption methods like RSA, Diffie–Hellman, and ECC [5]. Although current quantum hardware remains limited, experimental implementations have already demonstrated successful attacks on weakened cryptographic variants [2], which underscores the urgency of developing quantum-resistant solutions. Post-quantum cryptography (PQC) aims to secure digital communication against such quantum adversaries by designing cryptographic primitives that rely on problems believed to remain hard even for quantum computers. Among the families of PQC schemes, lattice-based cryptography stands out for its strong theoretical guarantees and practical efficiency. It relies on the computational hardness of high-dimensional lattice problems such as the Shortest Vector Problem (SVP) and the Learning with Errors (LWE) problem, for which no efficient ∗Brigham Young University, Provo, Utah, USA. Email: [email protected] †St. Xavier’s College, Maitighar, Kathmandu, Nepal. Email: [email protected] 1 classical or quantum algorithms are known. Within NIST’s ongoing PQC standardization effort, the lattice-based key encapsulation mechanism (KEM) CRYSTALS–Kyber was selected for its balance of security, performance, and implementation simplicity [1]. Kyber’s security is based on the Module Learning with Errors (MLWE) problem, whose hardness is grounded in over a decade of lattice cryptography research [1]. In this work, we explore these concepts through the practical implementation of the Kyber KEM within a web-based messaging application, PQMessenger.1We begin by reviewing the theoretical foundations, then provide an overview of the application’s methodology and implementation, followed by a discussion of challenges, key takeaways, and future work. The source code for the implementation is publicly available at PQMessenger. 2 Background and Theoretical Foundations 2.1 Preliminaries We first introduce the notation, definitions, and mathematical background used throughout this work. Notation •Vectors and matrices are denoted by bold lowercase and uppercase letters, respectively (e.g., s,A). •Polynomials in the ring Rq=Zq[x]/(xn+1) are written as lowercase letters (e.g., a(x)). •Modular arithmetic modulo qis denoted by [ ·]q. •⌈·⌋ denotes rounding to the nearest integer. Lattices Alattice L ⊂ Rnis a discrete additive subgroup generated by integer combinations of a basis {b1, . . . , bn}: L=(n X i=1 zibi|zi∈Z). Key computational problems on lattices, also known as lattice hard problems include: •Shortest Vector Problem (SVP): find a nonzero lattice vector of minimal Euclidean length. •Closest Vector Problem (CVP): given a target vector, find the lattice vector closest to it. 1This work originated from a six-week virtual project-based learning program for high school and gap year students in Nepal, called Uunchai. We thank our collaborators from that program for their early contributions and discussions. 2 Learning With Errors (LWE) The LWE problem over Zqis: given a matrix A∈Zm×n qand a vector b=As +e∈Zm q (where sis secret and eis a small error vector), recover s. The hardness of LWE underpins lattice-based cryptography. Module-LWE (MLWE) Module-LWE generalizes LWE to modules over Rq. Given a public matrix A∈Rk×l qand a secret vector s∈Rl q, the problem is to distinguish As +efrom uniform, where e∈Rk qis an error vector sampled from a small distribution. Number Theoretic Transform (NTT) The NTT is a fast Fourier-like transform for polynomials over Rq, allowing efficient polynomial multiplication. Using NTT reduces the complexity from O(n2) to O(nlog n). Error Sampling Kyber samples errors from a discrete distribution (usually a centered binomial distribution) to hide secrets and ensure IND-CPA security. For x∈Z, let ψη(x) denote the probability of sampling x. Compression A compression function Compressq(·, d) reduces each coefficient modulo qto dbits while maintaining correctness with high probability. This reduces key and ciphertext sizes. 2.2 Kyber Overview Kyber is a lattice-based key encapsulation mechanism built on the Module Learning With Errors (MLWE) problem. MLWE extends the LWE problem to modules over polynomial rings Zq[x]/(xn+1), enabling efficient polynomial arithmetic while preserving security based on worst-case lattice problems such as the Shortest Vector Problem (SVP) and Closest Vector Problem (CVP). The Number Theoretic Transform (NTT) is used to accelerate polynomial operations, and modular arithmetic with noise sampling ensures that secret values remain hidden. 2.3 Key Generation Kyber generates a public-private key pair such that the public key hides the secret within a noisy linear system. Two seeds are sampled to deterministically generate the public matrix and secret vectors. The public key is then compressed for efficiency, while the secret key consists of the secret vector. 3 Algorithm 1 Kyber.CPA.KeyGen() 1: Sample seeds ρ, σ ← {0,1}256 2: Generate public matrix Afrom ρ 3: Sample secret and error vectors (s,e) from σ 4: Compute t←Compressq(As +e, dt) 5: return pk ←(t, ρ), sk ←s 2.4 Encapsulation To encapsulate a message, Kyber samples fresh polynomials for randomness, reconstructs the public matrix, and computes two ciphertext components using linear combinations of the public matrix and the secret polynomials. Compression reduces the ciphertext size while allowing correct decapsulation. Algorithm 2 Kyber.CPA.Enc(pk, m) 1: Sample fresh seed and polynomials (r, e1, e2) 2: Reconstruct Afrom pk 3: Compute u←Compressq(ATr+e1, du) 4: Compute v←Compressq(tTr+e2+⌈q/2⌉ · m, dv) 5: return c←(u, v) 2.5 Decapsulation Decapsulation reverses the encapsulation process. The ciphertext components are decompressed, and the secret key is used to recover the original message by computing a linear combination and compressing the result. Algorithm 3 Kyber.CPA.Dec(sk, c) 1: Decompress u, v from c 2: Compute m←Compressq(v−sTu, 1) 3: return m 3 System Design and Implementation We built an end-to-end encrypted messaging website using CRYSTALS-Kyber as a Key Encapsulation Mechanism (KEM). The frontend is powered by React, with framer motion as the animation library and tailwind CSS for styling. It also uses Axios as the HTTP client to make requests to the backend. The backend is powered by Flask, with Firestore as the real-time database and Redis handling session cache. Users are authenticated using hashed passwords and JWT-based session tokens, while their private keys are securely encrypted with a password-derived symmetric key. Messaging uses a hybrid cryptographic approach: 4 Kyber encrypts session-specific AES keys and AES-CBC encrypts the actual message content. The Kyber implementation used in this system was compiled from the publicly available reference code provided by the CRYSTALS project on GitHub [3]. The RESTful APIs of the site connect the frontend and backend, and social features such as friend requests, notifications, and message retrieval are all stored efficiently in Firestore. 3.1 Frontend Architecture The frontend is organized into distinct layers, built with modern JavaScript frameworks and libraries: •Routing Layer: React Router manages client-side navigation, defining routes for login, registration, dashboard, and chat views. •Authentication Layer: AuthContext provides global authentication state and ensures that only authorized users can access protected routes. This makes use of localstorage to store authentication token in the browser itself and the username for session management and API requests. (All of which are validated by the backend layer) •UI Layer: Tailwind CSS provides utility-first styling for responsive design, while Framer Motion enables smooth animations and transitions. React Icons supply standardized iconography. •State and Data Layer: Context and utility hooks manage user state and responsive behavior. Axios is used for secure communication with the Flask backend. •Component Layer: Reusable components implement key features such as messaging, friend requests, notifications, and user profiles. 3.2 Backend Architecture The backend is structured into several logical parts and layers: 3.2.1 Authentication Layer Responsible for registering new users and verifying logins, this layer calls the Security Layer to hash, encrypt, and validate sensitive data before storing them in the Firestore database. It is also responsible for responding to any POST or GET requests made to the backend server and validates the requests using JWT-encoded session tokens. Upon new user registration, this layer generates user-specific public and private keys and securely stores them in the database. It also generates a session-based AES key during login, which is used to encrypt messages and protect user data throughout the session. The generated AES key is then encrypted for the user using the CRYSTALS-Kyber KEM. A shared secret and its ciphertext are derived using kyber encapsulate with the user’s public key. The shared secret is then used to derive a symmetric wrapping key after 5 applying SHA-256. It is then used to encrypt the session AES key using AES-CBC and a newly generated Initialization Vector (IV). This ensures that we do not need to re-encrypt the AES key for the sender repeatedly during the current session. Now, the generated AES key, the encrypted AES key for the user (i.e., the sender) for the session, the ciphertext for the shared secret, the initialization vector along with the decrypted private key of the user are temporarily stored in the Redis cache for later use. During messaging operations, this encrypted AES key (for the sender) is retrieved and stored in the message document. 3.2.2 Security Layer This layer manages encryption, decryption, and user credential validation. Passwords must be at least 8 characters long and include uppercase, lowercase, numeric, and special characters. Usernames must be 3–20 characters, containing only letters, numbers, or underscores, with no spaces or other special symbols. It manages password hashing using the bcrypt library and validates user credentials during login attempts using the bcrypt.checkpw() method. Furthermore, this layer is responsible for encrypting and decrypting users’ private keys upon request from the authentication layer. Encryption and Decryption of Secret Keys: To ensure secure storage and transmission of sensitive cryptographic materials, the system employs a two-step process involving key derivation and AES encryption. 1. Key Derivation: •Utilizes the PBKDF2HMAC (Password-Based Key Derivation Function 2) algorithm with SHA-256. •Takes the user’s password, a unique salt, and a defined number of iterations (specified in the environment configuration, 100000 by default). •Produces a strong 32-byte symmetric key derived from the password. 2. AES Encryption: •Employs AES encryption in CBC (Cipher Block Chaining) mode. •The derived key and a random Initialization Vector (IV) are used to create the cipher. •The user’s secret key (in hexadecimal format) is converted to bytes and padded using PKCS7 to align with AES block size. •The padded secret key is then encrypted with AES, resulting in an encrypted secret key (stored as bytes). 3. AES Decryption: 6 •Uses the same PBKDF2HMAC parameters (password, salt, and iteration count) to regenerate the decryption key. •The AES cipher (CBC mode) decrypts the stored ciphertext using the derived key and IV. •PKCS7 padding is removed to recover the original secret key bytes. •The final output is the decrypted secret key represented as a hexadecimal string. Summary: •Encryption: secret key (hex) + password + salt + IV PBKDF2 + AES-CBC −−−−−−−−−−−−→ encrypted secret key •Decryption: encrypted secret key + password + salt + IV PBKDF2 + AES-CBC −−−−−−−−−−−−→ decrypted secret key This design ensures that even if an encrypted secret key is compromised, it remains computationally infeasible to decrypt it without the correct user password, thereby maintaining the confidentiality and integrity of stored keys. 3.2.3 Session Management Layer Uses Redis to manage short-lived session identifiers and temporarily store the decrypted private key, session-based AES key, encrypted AES key for the user along with its IV and ciphertext, ensuring rapid access while minimizing the risk of long-term exposure of sensitive information. 3.2.4 Messaging Layer Responsible for secure end-to-end message exchange between users, implementing a hybrid encryption mechanism that combines post-quantum and symmetric cryptography. It interfaces with both the Security Layer for encryption/decryption operations and the Session Management Layer (Redis) for temporary key storage and retrieval. •Message Sending Process: 1. Fetch User and Friend Data: –Retrieves the current user and friend documents from Firestore using their respective IDs. –Extracts the friend’s public key for encryption. 2. Obtain Session AES Key: –The session AES key, previously established during login, is retrieved from Redis. –This key is used for encrypting individual chat messages. 7 3. Encrypt the Message: –A random Initialization Vector (IVmessage) is generated for the message. –The plaintext message is padded using PKCS7 and encrypted with AES in CBC mode using the session AES key. 4. Wrap the AES Key for the Receiver: –Utilizes CRYSTALS-Kyber to encapsulate the AES key for the receiver. –Derives a shared secret (ssreceiver) and applies SHA-256 to obtain a symmetric wrapping key. –Encrypts the session AES key with this derived key using AES-CBC and a new Initialization Vector (IVreceiver). 5. Retrieve Sender’s Wrapped AES Key: –Obtains the sender’s ciphertext, encrypted AES key, and IV from Redis (stored during the login phase). 6. Store Encrypted Data in Firestore: –Creates a new document in the messages collection containing: ∗Encrypted message and its IV. ∗Sender and receiver ciphertexts, encrypted AES keys, and IVs. ∗Sender and receiver user IDs. ∗Timestamp of the message. 7. Session Expiry Handling: –Resets the session expiry timer in Redis to one hour to maintain active communication securely. On successful completion of these steps, the message is securely transmitted and stored, ensuring confidentiality even under potential quantum adversaries. •Message Retrieval Process: 1. Fetch User and Friend Data: –Retrieves both user and friend documents from Firestore. 2. Access User’s Private Key: –Loads the user’s decrypted private key from Redis (decrypted during login by the Security Layer). 3. Query Messages: –Queries the messages collection for all messages exchanged between the two users (both sent and received). 8 –Aggregates and sorts them chronologically. 4. Decrypt Each Message: –For each message, communicates with the Security Layer for decryption, which: ∗Identifies whether the user is the sender or receiver. ∗Decapsulates the appropriate Kyber ciphertext using the user’s private key to derive the shared secret. ∗Uses SHA-256 to obtain the AES wrapping key and decrypts the wrapped session AES key. ∗Decrypts the AES-encrypted message using the session key and message IV. –Replaces the encrypted message field with the recovered plaintext. 5. Return Decrypted Messages: –The complete set of decrypted messages is returned as JSON, ready for rendering on the frontend. 3.2.5 Database Layer Firestore manages persistent data. Two key collections are maintained: 1. Users: Stores user IDs, usernames, bcrypt password hashes, public keys, encrypted private keys, salts, and initialization vectors. 2. Messages: Stores encrypted message content, sender and receiver IDs, timestamps, and the AES session keys encrypted separately for both parties along with the required data for their decryption. 3.2.6 Kyber Layer This layer provides a Python wrapper for the kyber library. This allows our code to use the compiled native kyber functions via the use of the ctypes, a foreign function python library. The kyber library in this project has been compiled in two formats, libkyber.so for linux and libkyber.dll for windows. This layer works by providing a class, KyberWrapper, with methods which either call the functions in the kyber library after the library was successfully loaded or mock the functions when the kyber library couldn’t be loaded. The methods are: •init : Loads the compiled kyber C library into Python memory, defines argtypes and return types for all the key kyber functions and raises an exception if the library couldn’t be loaded. 9