KasmVNC SSL Certificate Failure on EL9 OpenSSL 3 - TerrenceMcGuinness-NOAA/global-workflow GitHub Wiki
Date: 2026-02-23
Severity: Critical — Parallel Works virtual desktop completely inoperable
Platform: Rocky Linux 9 (EL9) on Parallel Works AWS cluster
Affected Versions: KasmVNC 1.3.4 and 1.4.0 with OpenSSL 3.5.1 (el9_7)
Root Cause: Three compounding defects in KasmVNC when running on OpenSSL 3.5.x
Resolution Time: Significant investigation required — this document exists to prevent that next time
If you hit the lastActiveAt TypeError or 502 Bad Gateway on a Parallel Works KasmVNC desktop after an EL9 system update, apply these three fixes in order:
-
Regenerate the SSL certificate — The RPM default cert has
CA:TRUEwhich OpenSSL 3.5.x rejects - Configure VNC to disable STUN/UDP — Prevent unnecessary UDP initialization failures
- Patch the KasmVNC JavaScript client files — This is the actual fix that prevents the crash. The client sends a WebRTC UDP upgrade request that triggers a null-pointer segfault in the server.
The Parallel Works virtual desktop URL shows:
KasmVNC encountered an error:
Uncaught TypeError: Cannot read properties of undefined (reading 'lastActiveAt')
https://noaa.parallel.works/me/session/<user>/marketplace.desktop.latest_1_session/main.bundle.js:4:96353
If the user refreshes after the initial crash, the browser shows 502 Bad Gateway because the nginx proxy's backend (Xvnc) is dead.
~/.vnc/<hostname>:<display>.log
Phase 1 — SSL rejection at startup:
error:03000098:digital envelope routines:do_sigver_init:invalid digest
error:0A00018E:SSL routines:SSL_CTX_use_certificate:ca md too weak
WebUdp: Failed to create WebUDP host
The VNC server starts anyway (websocket transport still works), but the WebUDP host object is null.
Phase 2 — Segfault when client connects:
(EE) Backtrace:
(EE) 3: /usr/bin/Xvnc (WuGotHttp+0x11)
(EE) 4: /usr/bin/Xvnc (_ZN3rfb10SMsgReader16readUpgradeToUdpEv+0x128)
(EE) Segmentation fault at address 0x0
(EE) Fatal server error:
(EE) Caught signal 11 (Segmentation fault). Server aborting
The KasmVNC client JavaScript requests a WebRTC/UDP upgrade. The server's readUpgradeToUdp() calls WuGotHttp() on the null WebUDP host pointer → segfault → Xvnc dies → nginx 502.
A dnf -y update --nobest ran on the system (in our case, via SETUP/bootstrap.sh with a broken --exclude pattern that only pinned the exact kernel version instead of using a wildcard). This upgraded 970 packages including:
| Package | Before | After |
|---|---|---|
openssl |
3.2.2-6.el9_5 | 3.5.1-7.el9_7 |
openssl-libs |
3.2.2-6.el9_5 | 3.5.1-7.el9_7 |
crypto-policies |
20250128 | 20250905 |
openssl-fips-provider |
(not installed) | 3.5.1-7.el9_7 (new) |
KasmVNC 1.3.4 (and 1.4.0) were built and tested against OpenSSL 3.2.x. The jump to 3.5.1 introduced stricter cert validation that breaks KasmVNC in three places.
The kasmvncserver RPM %post scriptlet generates a self-signed cert:
openssl req -x509 -nodes -days 3650 -newkey rsa:2048 \
-keyout /etc/pki/tls/private/kasmvnc.pem \
-out /etc/pki/tls/private/kasmvnc.pem \
-subj "/C=US/ST=VA/L=None/O=None/OU=DoFu/CN=kasm/[email protected]"Missing: -addext "basicConstraints=critical,CA:FALSE". Without it, OpenSSL defaults to CA:TRUE. OpenSSL 3.5.x rejects this for TLS server auth with SSL_CTX_use_certificate:ca md too weak.
When the SSL cert is rejected, the DTLS-based WebUDP host fails to initialize (WebUdp: Failed to create WebUDP host), leaving the host pointer as NULL. However, the server-side RFB message handler for readUpgradeToUdp does not null-check the WebUDP host before calling WuGotHttp(). This is a classic null-pointer dereference bug.
In screen.bundle.js, the WebRTC setting defaults to enabled:
e.rfb.enableWebRTC = e.getSetting("enable_webrtc", !0, !1)
// ^^^ !0 = true (default value)Even though initSetting("enable_webrtc", !1) in the UI sets it to false, the screen.bundle.js overrides it with true via the second argument to getSetting(). This means every client connection sends a UDP upgrade request, triggering Defect 2.
dnf update → OpenSSL 3.2→3.5 → cert rejected → WebUDP host=NULL
↓
browser connects → JS requests UDP upgrade → server dereferences NULL → SIGSEGV
↓
Xvnc dead → nginx 502
↓
JS: "Cannot read properties of undefined (reading 'lastActiveAt')"
# Backup original
sudo cp /etc/pki/tls/private/kasmvnc.pem /etc/pki/tls/private/kasmvnc.pem.bak.$(date +%Y%m%d)
# Generate replacement with CA:FALSE and proper key usage
sudo openssl req -x509 -nodes -days 3650 -newkey rsa:4096 -sha256 \
-keyout /etc/pki/tls/private/kasmvnc.pem \
-out /etc/pki/tls/private/kasmvnc.pem \
-subj "/C=US/ST=VA/L=None/O=NOAA-EMC/OU=EIB/CN=kasm/[email protected]" \
-addext "basicConstraints=critical,CA:FALSE" \
-addext "keyUsage=digitalSignature,keyEncipherment" \
-addext "extendedKeyUsage=serverAuth"
# Set permissions (kasmvnc-cert group)
sudo chgrp kasmvnc-cert /etc/pki/tls/private/kasmvnc.pem
sudo chmod 640 /etc/pki/tls/private/kasmvnc.pemVerify:
openssl x509 -in /etc/pki/tls/private/kasmvnc.pem -text -noout | grep -A2 "Basic Constraints"
# Expected:
# X509v3 Basic Constraints: critical
# CA:FALSEImportant: If you upgrade/reinstall
kasmvncserverRPM, the%postunscript deletes the cert and%postregenerates the broken one. You must re-run this step after any RPM change.
Create/update ~/.vnc/kasmvnc.yaml:
logging:
log_writer_name: all
log_dest: logfile
level: 100
network:
udp:
public_ip: 127.0.0.1
stun_server: offThis prevents the STUN server query at startup and limits the UDP initialization scope. This alone is NOT sufficient — the server-side null-pointer crash still occurs because the client sends the upgrade request regardless.
This is the fix that actually prevents the crash. The client-side JavaScript must be patched to never request a WebRTC/UDP upgrade, preventing the server from hitting the null-pointer code path.
sudo cp /usr/share/kasmvnc/www/screen.bundle.js /usr/share/kasmvnc/www/screen.bundle.js.bak
# Force enableWebRTC to false instead of reading the setting
sudo sed -i \
's/e\.rfb\.enableWebRTC=e\.getSetting("enable_webrtc",!0,!1)/e.rfb.enableWebRTC=!1/' \
/usr/share/kasmvnc/www/screen.bundle.jsThe UI JS filename includes a hash that varies by version. Find it first:
UI_JS=$(find /usr/share/kasmvnc/www/assets -name "ui-*.js" | head -1)
sudo cp "$UI_JS" "${UI_JS}.bak"
# 1. Force the rfb.enableWebRTC assignment to false
sudo sed -i \
's/o\.rfb\.enableWebRTC=o\.getSetting("enable_webrtc")/o.rfb.enableWebRTC=!1/g' \
"$UI_JS"
# 2. Neuter the enableWebRTC setter so nothing can re-enable it
sudo sed -i \
's/set enableWebRTC(e){this\._useUdp=e/set enableWebRTC(e){this._useUdp=!1/' \
"$UI_JS"echo "=== screen.bundle.js ==="
grep -o '.\{20\}enableWebRTC.\{30\}' /usr/share/kasmvnc/www/screen.bundle.js
echo "=== ui JS ==="
grep -o '.\{20\}enableWebRTC.\{30\}' "$UI_JS" | head -5You should see enableWebRTC=!1 everywhere instead of getSetting("enable_webrtc"...).
Important: These patches must be re-applied after any
kasmvncserverRPM upgrade/reinstall.
# Kill existing VNC (may error if already dead — that's OK)
vncserver -kill :<display> 2>/dev/null
# Clean stale X display locks
sudo rm -f /tmp/.X<display>-lock /tmp/.X11-unix/X<display>
# Start VNC
/usr/bin/vncserver :<display> -disableBasicAuth \
-xstartup ~/.vnc/kasm-xstartup \
-websocketPort <websocket_port> \
-rfbport <rfb_port>
# Restart the PW nginx proxy to clear cached 502 state
docker restart nginx-<proxy_port># VNC process should be running
ps aux | grep Xvnc | grep -v grep
# Websocket port should be listening
ss -tlnp | grep <websocket_port>
# Web UI should respond
curl -sk https://localhost:<websocket_port>/ | grep -o "<title>[^<]*</title>"
# Expected: <title>KasmVNC</title>
# Proxy should also respond
curl -sk http://localhost:<proxy_port>/ | grep -o "<title>[^<]*</title>"
# Expected: <title>KasmVNC</title>Then open the Parallel Works desktop URL in your browser. If you see the old error, do a hard refresh (Ctrl+Shift+R) to clear the browser's cached main.bundle.js.
During investigation, we tried adding -udpPort 0 to the vncserver command line. This sets the UDP listening port to 0 but does not prevent:
- The SSL cert from being loaded for DTLS (still triggers the "ca md too weak" error)
- The WebUDP host creation attempt (still fails → null pointer)
- The client from sending an
upgradeToUdpRFB message (still crashes the server)
The -udpPort 0 flag only affects which port the server binds for UDP traffic. The WebUDP code path in the RFB message handler has no guard for a null host object. The JavaScript client patch is the only fix that prevents the crash.
The marketplace.desktop.latest workflow launches KasmVNC via ~/pw/jobs/marketplace.desktop.latest/00001/run.sh. Key parameters:
- Display:
:84(dynamically assigned) - Websocket port:
37797(allocated viapw agent open-port) - RFB port:
5984 - Nginx proxy: Docker container
nginx-<port>(e.g.,nginx-39251) proxying HTTP tohttps://127.0.0.1:<websocket_port> - Desktop: xfce4 via
~/.vnc/kasm-xstartup
The run script uses whatever KasmVNC is installed on the system — it does not install, update, or manage the KasmVNC RPM. The RPM is installed during the initial image provisioning (transaction 86 in dnf history: localinstall -y ./kasmvncserver_oracle_9_1.3.4_x86_64.rpm).
The PW run script replaces /usr/lib/kasmvncserver/select-de.sh with a no-op (exit 0). This is necessary because KasmVNC's DE selection conflicts with PW's xstartup approach. After an RPM reinstall, this override must be re-applied:
sudo mv /usr/lib/kasmvncserver/select-de.sh /usr/lib/kasmvncserver/select-de.sh.bak
sudo tee /usr/lib/kasmvncserver/select-de.sh >/dev/null <<'EOF'
#!/bin/sh
exit 0
EOF
sudo chmod +x /usr/lib/kasmvncserver/select-de.shThe system update that caused this was triggered by SETUP/bootstrap.sh using a version-pinned kernel exclude:
# BROKEN — only excludes the exact version, not newer kernels
--exclude=kernel-5.14.0-570.28.1.el9_6.x86_64,...This was fixed to use a wildcard (commit aa277d9):
# FIXED — excludes all kernel packages
sudo dnf -y update --nobest --exclude='kernel*'To additionally prevent KasmVNC-breaking updates from recurring, add OpenSSL to the exclude list or use dnf versionlock:
# Option A: Add to bootstrap exclude
sudo dnf -y update --nobest --exclude='kernel*' --exclude='openssl*'
# Option B: Version lock (permanent)
sudo dnf versionlock add openssl openssl-libs openssl-develIssue: %post scriptlet generates cert without basicConstraints=critical,CA:FALSE
Impact: Cert rejected by OpenSSL >= 3.5.x
Fix: Add -addext flags to the openssl req command
GitHub: https://github.com/kasmtech/KasmVNC
Issue: SMsgReader::readUpgradeToUdp() calls WuGotHttp() without checking if the WebUDP host was successfully created
Impact: Segfault (SIGSEGV at address 0x0) when any client sends a UDP upgrade request and WebUDP init failed
Fix: Add null-check before WuGotHttp() call, or gracefully reject the upgrade
Affected: Both v1.3.4 and v1.4.0
Issue: screen.bundle.js passes !0 (true) as default to getSetting("enable_webrtc"), overriding the UI's initSetting("enable_webrtc", !1) (false)
Impact: Client always sends UDP upgrade request even when user hasn't enabled WebRTC
Fix: Change default to !1 (false) in screen.bundle.js
| Component | Version |
|---|---|
| OS | Rocky Linux 9 (EL9) |
| Kernel | 5.14.0-570.28.1.el9_6.x86_64 |
| OpenSSL (post-update) | 3.5.1-7.el9_7 |
| OpenSSL (pre-update, worked) | 3.2.2-6.el9_5 |
| Crypto Policy | DEFAULT |
| KasmVNC | 1.3.4 -> upgraded to 1.4.0 (same bug in both) |
| Platform | Parallel Works AWS (emcmcpawsrocky9functionalii) |
| Desktop Workflow | marketplace.desktop.latest |
| Nginx Proxy |
nginx-unprivileged:1.25.3 (Docker container) |
| File | Action | Description |
|---|---|---|
/etc/pki/tls/private/kasmvnc.pem |
Regenerated | SSL cert with CA:FALSE, 4096-bit RSA, SHA256 |
~/.vnc/kasmvnc.yaml |
Created/Modified | Disabled STUN and set UDP public IP to localhost |
/usr/share/kasmvnc/www/screen.bundle.js |
Patched | Hardcoded enableWebRTC=!1
|
/usr/share/kasmvnc/www/assets/ui-*.js |
Patched | Neutered enableWebRTC setter + assignment |
/usr/lib/kasmvncserver/select-de.sh |
Replaced | PW desktop compatibility (exit 0) |
Backups of all patched files exist with .bak suffix in the same directories.
If kasmvncserver RPM is ever reinstalled or upgraded, all three fixes must be re-applied:
- Regenerate SSL certificate (Step 1)
- Verify
~/.vnc/kasmvnc.yamlstill has STUN/UDP disabled (Step 2) - Re-patch
screen.bundle.jsandui-*.js(Step 3 — filenames may change) - Re-apply
select-de.shoverride for Parallel Works - Test desktop connection end-to-end before declaring success