UP | HOME
Land of Lisp

Zhao Wei

How can man die better than facing fearful odds, for the ashes of his fathers and the temples of his Gods? -- By Horatius.

JWT, JWKS in Web Development

1. Introduction

It describes the concepts and steps I learned to use tokens to secure web service, especially for:

  • Basic Authentication vs what is Bear Authentication
  • What is JWT and JWKS?
  • How JWT is used (sign and verify)?
  • How JWKS is used to verify JWT?

2. Differences between “Basic” and “Bearer” in the Authorization header

In web development, we usually need to configure “Authorization” header(I will call it auth header for short) before sending the request. If you notice, there are two kinds of auth header, “Basic” and “Bearer”.

2.1. Basic auth header

It is usually needed when we request some information from auth-provider. For example, when we want to introspect access-token from oidc-provider.

let authHeader = 'Basic ' + Buffer.from(client_id + ':' + client_secret, 'utf8').toString('base64');
let options = {
  method: 'POST'  ,
  url: `${authUrl}/token/introspection`,
  headers: {Authorization: authHeader},
  form: {
    token: access_token
  }
};
  • Its format: Basic + “ ” + base64-encoded string constructed from id(username):password(secret).
  • TODO:: Note: base64 is a reversible encoding. Does that means password(secret) is not secured? Does https make it secured?

2.2. Bearer auth header

It is needed when we want to request protected resources usually are some service’s APIs endpoints. For example, after we got access-token after login, we need to use Bearer token in request’s header.

let headersOption = {
  'Accept': 'application/json',
  'Content-Type': 'application/json',
  'Authorization': 'Bearer ' + access_token
};
  • “Bearer authorization” means “give access to the bearer of this token”.
  • No matter whether it is access-token or other kind of token, the bearer token is a encrypted string. It is generated by the auth-provider in response to a login request.

3. Basic concepts

3.1. Encryption algorithms

To make data transfered safely between network, we need to encrypt data we transfered. There are mainly two kinds of algorithms to do this: symmetric encryption and asymmetric encryption.

3.1.1. Symmetric encryption

Symmetric encryptions use the same key for doing encryption and decryption.

  • It is fast to do encryption and decryption, so it is very useful to transmit data.
  • However, we need to transmit the symmetric encryption key to the receiver side safely. In this case, this secret key becomes the content we need to encrypt. This is the moment we choose to use asymmetric encription.
  • Common symmetric encryption algorithms include:
    • HS256(HMAC + SHA256)

3.1.2. Asymmetric encryption

Instead of using the same key to do both encryption and decryption, asymmetric encryption algorithm uses a pair of key: use private key to encrypt data and use publick key to decrypt.

  • We NEVER share the private key.
  • We could share publick key safely without concern.
  • It is much slower than symmetric encryption, so we only use it to transmit curial data.
  • Common asymmetric encryption algorithms include:
    • RS256(RSASSA-PKCS1-v15 + SHA256)
    • ES256(ECDSA + P-256 + SHA256)

3.1.3. How to generate key-pair

  • To generate key-pair using RS256

    # Generate a private key
    openssl genpkey -algorithm RSA -out private_key.pem -pkeyopt rsa_keygen_bits:2048
    # Derive the public key from the private key
    openssl rsa -pubout -in private_key.pem -out public_key.pem
    
    
    # Generate a private key with key-phrase
    openssl genrsa -des3 -out private.pem 2048
    # Then generate non-phrase protected key using this command:
    openssl rsa -in private.pem -out private_unencrypted.pem -outform PEM
    
  • To generate key-pair using ES256

    # Generate a private key (prime256v1 is the name of the parameters used
    # to generate the key, this is the same as P-256 in the JWA spec).
    openssl ecparam -name prime256v1 -genkey -noout -out ecdsa_private_key.pem
    
    # Derive the public key from the private key
    openssl ec -in ecdsa_private_key.pem -pubout -out ecdsa_public_key.pem
    

3.2. JWT and JWKS

3.2.1. What is JWT?

JWT(JSON Web Token) encodes a series information(called claims). Those information typically includes:

  • iss, issuer
  • sub, subject
  • aud, audience
  • exp, expiration
  • nbf, not before
  • iat, issued at

Signing is a cryptographic operation that generates a “signature” (part of the JWT), later the recipient of the token can validate to ensure that the token has not been tampered with.

We usually use middleware in web service to verify the signed JWT passed from Authorization header. For example, access_token got from auth-provider usually contains the scopes/permissions, and middleware uses that to protect API endpoint.

3.2.2. What is JWKS?

The JSON Web Key Set (JWKS) is a set of keys containing the public keys used to verify any JSON Web Token (JWT) issued by the authorization server and signed using the RS256(RSA Signature with SHA-256) algorithm.

3.2.3. Load Keys into JWKS

We could use package node-jose to load keys (public or private) files as JWK.

function readJWKFromPEM(filename) {
  return new Promise((resolve, reject)=> {
    const key = fs.readFileSync(filename);
    const keystore = jose.JWK.createKeyStore();
    keystore.add(key, 'pem').then(()=>{
      const jwks = keystore.toJSON(true);
      resolve(jwks);
    }).catch(err=>{
      console.log("add JWK failed from PEM key:", err.message);
      reject(err);
    });
  });
}

4. How to generate JWT?

A signed JWT consists of three parts: header, payload and signature seperated by “.”:

Header specifies the algorithm used and the type

{
  "alg": "HS256",
  "typ": "JWT"
}

Payload contains the claims

{
  "sub": "1234567890",
  "name": "John Doe",
  "manager": true
}

And signature is composed from the signing of encoded header and encoded payload with a secret. See below about how to generate JWT with different algorithms.

4.1. Generate JWT signed with HS256

const encodedHeader = base64(utf8(JSON.stringify(header)));
const encodedPayload = base64(utf8(JSON.stringify(payload)));
const signature = base64(hmac(`${encodedHeader}.${encodedPayload}`,secret, sha256));
const jwt = `${encodedHeader}.${encodedPayload}.${signature}`;

If we use jsonwebtoken npm package, this could be simplified as:

var jwt = require('jsonwebtoken');

const payload = {
  sub: "1234567890",
  name: "John Doe",
  manager: true
};

const secretKey = 'secret';

const token = jwt.sign(payload, secretKey, {
    algorithm: 'HS256',
    expiresIn: '10m' // if ommited, the token will not expire
});

var decoded = jwt.verify(token, secretKey);

Warning: JSON Web Algorithms RFC 7518 states that a key of the same size as the hash output (for instance, 256 bits for “HS256”) or larger MUST be used with the HS256 algorithm. One character is 8-bits, so we need to use at least 32 character and more as secret key for signing with HS256.

4.2. Generate JWT signed with RS256

After we generate key-pair, we use private-key to sign and public-key to verify it.

4.2.1. Sign JWT with RS256

const encodedHeader = base64(utf8(JSON.stringify(header)));
const encodedPayload = base64(utf8(JSON.stringify(payload)));
const signature = base64(rsassa(`${encodedHeader}.${encodedPayload}`, privateKey, sha256));
const jwt = `${encodedHeader}.${encodedPayload}.${signature}`;

If we use jsonwebtoken to do it:

async function generateJWT(payload, privateKeyName) {
  let privateKey = fs.readFileSync(privateKeyName);
  let jwks = await readJWKFromPEM(privateKeyName);
  const signed = jwt.sign(payload, privateKey, {
    algorithm: 'RS256',
    expiresIn: '24h',
    keyid: jwks.keys[0].kid
    issuer: process.env.issuer
  });

  return signed;
}

To sign JWT with ES256, change “algorithm” to “ES256”.

5. How to verify JWT?

5.1. Verify JWT with jsonwebtoken package

5.1.1. Verify symmetric algorithm JWT

var decoded = jwt.verify(token, secretKey);

5.1.2. Verify asymmetric algorithm signed JWT

const publicRsaKey = `<YOUR-PUBLIC-RSA-KEY>`;

// For RS256
const decoded = jwt.verify(signed, publicRsaKey, {
  // Never forget to make this explicit to prevent
  // signature stripping attacks.
  algorithms: ['RS256'],
});

// For ES256
const decoded = jwt.verify(signed, publicEcdsaKey, {
    // Never forget to make this explicit to prevent
    // signature stripping attacks.
    algorithms: ['ES256'],
});

5.2. How to verify JWT using JWKS?

JWKS stores an array of public-keys in the format of JWK(See RFC 7517). So, we need to find the matching public-key using kid property from JWT’s header.

Suppose we have a priviate-key: priviate_key.pem generated with RS256. Let’s generate JWT from it.

let signedJWT = await generateJWT({
  first_name: "Zhao",
  last_name: "Wei"
}, "private_key.pem");


console.log(">> decoded JWT:");
let decodedToken = jwt.decode(signedJWT, {complete: true});
console.log(decodedToken);

Its output look like this:

>> signed JWT:
eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCIsImtpZCI6Im41SkNCY2dhUVJFeW1tb3pvcDJvYnBtalZQeUZ2aHR3UkZFY3RSdDFCeWsifQ.eyJmaXJzdF9uYW1lIjoiWmhhbyIsImxhc3RfbmFtZSI6IldlaSIsImlhdCI6MTYxOTM1NTU4MywiZXhwIjoxNjE5NDQxOTgzfQ.U06ryukF5ywMelTKYbroLnxn62bKQ683i-ty86BQeMZQuXtB5o8CzLXtOM-oW1TYv_Trpzy2EXkPhLE5YpGUwsLxPd9qCKeqMYd7Vm54v2okmfJADVndm4NL5oXHErdaSh1nL4-fkmCXL63zLsH9Sai2QkkIgOwXYAfpz4VcxatMJsLJAl3YGqgrZkKfeQvAJuOb6LXI6Uu9LZD-oGlfJR7VJCtGf4HbdTTU4DBZuIeCQ26u2EkldhW27T9pxI2_D7PVDNikkcu50Rv5s4UHql6RxCRbpHoCOVgrA9sILnlCqT6r78UkgUNrGQidTs-UOAq3DDeuJ19RZBU20xlUgw

The general steps to verify this token from JWKS:

  1. Retrieve the JWKS and filter for potential signature verification keys.
  2. Decode the JWT and grab the kid property. It is used to lookup the appropriate public-key, as it is also included in the JWT JOSE(Javascript Object Signing and Encryption) hader.
  3. Find the signature verification key in the filtered JWKS with a matching kid property.
  4. Using the x5c property build a certificate which will be used to verify the JWT signature.
  5. Ensure the JWT contains the expected audience, issuer, expiration, etc.

The potential signature verification keys means each key from “keys” member of JWKS should contains the following property:

  • kid, is used to identify a key.
  • kty, is used to identify the cryptographic algorithm family used with the key.
  • use, is used to indentify the use of the public key. Values could be: sig (signature), enc (encryption)
  • exp, is used to define the expiration time for the given JWK.
  • kid, is the unique identifier for the key.
  • n, contains the modulus value for the RSA public key. It is represented as a Base64urlUInt-encoded value. (see https://tools.ietf.org/html/rfc7518#section-6.3)
  • e, contains the exponent value for the RSA public key. It is represented as a Base64urlUInt-encoded value.

Suppose the public key stored in JWKS is something like:

{
  kty: 'RSA',
  kid: 'n5JCBcgaQREymmozop2obpmjVPyFvhtwRFEctRt1Byk',
  n: '0HWDCPjBAniQkcc0UqcMH4ZMYcrU3xnya9Bkjz4Ev6Ohj_Ff-xNnmQKvJKu3x9RzZJW6vPzVOjQRTvBqT4I3KkrUb5XVr4L_WEpXOX2JpCQlI1RdmPDUKyoMO_rGa5VoAFrj4txGLXxELw4_s2azKgjx0Wx7FbQLlMhLR0c7XzK7q1PUjBehA4_FEtUsKwFexfiXJGeryQTo3ftXmwpNf785aSZxsopuimlh8iYVvP4utI9R0c8jrIhYJ9Ijfzrv23bslf4BH-tmsEpXPxOPjdRfaJamtSrJfCtW7ZdkwDCgWt1HsrlP_p46mGMUmvqqnwG2eyDjg29eJoiu5-tKYQ',
  e: 'AQAB'
}

Usually, there are multiple such JWKs. So, we will need to use the kid property to find the exect key necessary to verify the JWT.

We could decode the JWT as it is simply base64 url encoded. Here, we decode the JWT with jwt_decode, the decoded JWT signed from above is:

{
  header: {
    alg: 'RS256',
    typ: 'JWT',
    kid: 'n5JCBcgaQREymmozop2obpmjVPyFvhtwRFEctRt1Byk'
  },
  payload: {
    first_name: 'Zhao',
    last_name: 'Wei',
    iat: 1619355693,
    exp: 1619442093
  },
  signature: 'OwijTq7lmwumcBc_srwWh53Yr5_lBHsELmKUO5m22nANoIlu-xVGD7dOw0K_JKXiLgiZV0qyJF2zU1G4fLJw7w21SO8s78430SFAmbKLYM2yTVBYKs76nRpyzAw4188DmEcXv7hb4f-58ANkdKUJ4MeFgHFD8wIB8aSA5CUxcCyEy54sNetQo_PZmrDFgfVjYHDszRG4oFmwtukxvu7ySr2SReuvboop_uw9sPvS7mcy4plLEqjHltH_QL7awdFyxEg3mC4SoIcPSDdRcs5Co1GS-Y-ZbORS65zIQlNufSD7SQyIqIVzxF60ZNCPS1SyG-QFT7kj-_PpdYg29hA2Ng'
}

Now, we have find the matching JWK using kid (assuming there are multiple public keys in JWKS). We could verify the JWT with the JWK now.

  1. Convert JWK to PEM using jwk-to-pem.
  2. Verify signed JWT token using jwt.

The simple code could look like:

let publicPem = jwkToPem(publicJWK);
jwt.verify(signedJWT, publicPem);

That is it. We verify signed JWT with matched public-key.

5.3. How to verify JWT as middleware in practise.

The above section illustrates how to verify signed JWT from multiple public keys as JWKS step by step. In practise, Auth-provider usually provide an endpoint /keys for GET requests which returns a list of keys.

curl https://<some_auth_provider>/keys |jq
  ...
{
  "keys": [
    public-key-01-in-JWK-format,
    public-key-02-in-JWK-format,
    ...
  ]
}

And the middleware which validate JWT could use jwks-rsa and express-jwt to do above operations automatically:

import express, { Request, Response } from 'express';
import { issuer } from './config';
import jwt from 'express-jwt';
import jwksRsa from 'jwks-rsa';

const checkJwt = jwt({
    secret: jwksRsa.expressJwtSecret({
        cache: true,
        rateLimit: true,
        jwksRequestsPerMinute: 5,
        jwksUri: `http://localhost:3344/keys`
    }),
    issuer,
    algorithms: ['RS256'],
    requestProperty: 'tokenData'
});

const app = express()
app.use('/some-api', checkJwt, (req: Request, res: Response) => {
    res.send(`Result: ${JSON.stringify(req.tokenData)}`);
})

app.listen(2233, () => console.log(`Backend is running on port 2233`))

6. General flow to implement JWKS

general_flow_to_implement_JWKS.png

The important steps involve with JWKS are:

  • Decode the JOSE header in order to find the JWK kid.
  • Retrieve the list of avaiable public keys.
  • Filter the key with matching kid.
  • Verify JWT with public key.

7. Summary

  • When we need to use HS256 to sign JWT, the secret length should not be shorter than 32 characters.
  • We better to use asymmetric algorithm to sign our JWT.
  • We use private key to sign JWT and use publick key to verify JWT.
  • JWKS stores array of public-key use to verify JWT.

8. References