Skip to content

VOR Stream v26.1.2 Release

VOR Stream v26.1.2 is the second patch release on the v26.1 line and is primarily a security and operational reliability release. A broad sweep of CVE patches has been applied across the Go runtime, Django, and dozens of transitive Python and Node dependencies. Seven bugs reported from the field have been fixed in the engine, Django, and CLI, including two engine issues that affected correctness or reliability under load. A single new deployment capability rounds out the release: a custom Caddy imports directory that survives Ansible redeploys. There are no breaking changes; v26.1.2 is a drop-in upgrade from v26.1.1.

Highlights

Long-running database nodes no longer fail with 403 after a wait

A report came in from a v26.1.1 deployment: output nodes configured with db=mssql and an upstream getdyn= factor were failing with 403 permission denied from the engine's database-credential helper. The same process worked when the wait was short, and failed when the upstream factor took 15 minutes or so to land.

The cause turned out to be ordering inside the generated worker code for the database I/O templates. The engine acquired its Vault-issued database credential after WaitSignals(...) returned, not before. Vault credentials are issued per node with a short time-to-live; a long wait outlives that TTL, so the post-wait credential fetch is rejected.

In v26.1.2, the database connection is acquired at the top of the worker function, before any wait. The credential is consumed while it is still valid, and the resulting *sql.DB is held open through the wait. Both the db=mssql and db=psql paths are fixed; the fix applies to the input-node read path as well as the output-node write path.

Who is affected

Processes with db=mssql or db=psql I/O nodes that also use getsig= or getdyn=. Processes where the awaited signal arrives in well under the credential TTL are unlikely to have seen this. Database nodes without an explicit wait, and waits on non-database nodes, were never affected.

No more phantom zero-rows from multi-threaded nodes

A separate deployment reported unexpected zero-valued rows appearing in the output of a multi-threaded Go model node: 100 phantom rows in a single run of an automotive LGD model, with the underlying data otherwise correct. The phantom rows were non-deterministic: they appeared under broker load and disappeared on quieter runs.

The cause was a race condition inside the engine's multi-threading shutdown logic. Under heavy load on the message broker, the race allowed a stray "termination" signal from one worker thread to land in a sibling thread's input queue as if it were a data row, surfacing as a zero-valued row in output.

In v26.1.2, the shutdown logic is no longer susceptible to this race. Multi-threaded nodes produce output that matches their single-threaded equivalent exactly, regardless of broker load.

Who is affected

Processes with multi-threaded Go, model, or Python nodes (anywhere threads > 1 is configured). Single-threaded nodes, SQL nodes, CSV I/O nodes, and S3 nodes are not affected. If you have ever observed unexpected zero-row data in output from a multi-threaded node, v26.1.2 is the upgrade that removes the cause.

Vault credential recovery on web nodes

Two related Django reliability issues have been fixed in this release.

The first: when Vault revoked dynamic Postgres credentials earlier than vault12factor's cached lease expected (which happens during VM suspend/resume, clock skew, Vault restart, or manual revocation), the cached credentials were never refreshed and every subsequent Django query failed indefinitely with password authentication failed. A new pre-reconnect receiver clears the cached lease before the retry path runs, so the existing refresh hook fetches fresh credentials from Vault and the query succeeds.

The second: when Consul was healthy at uWSGI startup but had not yet registered Vault Agent, the Django Vault bootstrap loop's narrow except clause did not catch the bare Exception raised by the "no healthy nodes" branch. The exception propagated out of settings.py, uWSGI dropped into "no python application found" mode, and supervisord could not restart it because uWSGI itself was still serving 500s. The exception type has been changed so the existing retry loop handles this case the same way it handles any other Consul transient.

Together with the existing token-file retry loop from v26.1.0 and v26.1.1, this closes the remaining gaps in Django's Vault bootstrap and credential-refresh paths.

Spurious "Job Cancelled" popups on RHEL-style hosts

On hosts where the parent shell exports functions (for example, RHEL hosts with a which wrapper from /etc/profile.d/which2.sh), opening the Process Viewer for any run in Waterways showed a "Job Cancelled" popup containing shell error text. The generated run script was trying to export an environment variable whose name contained %% (bash's encoding for exported functions), which bash rejected as not a valid identifier. The engine captured the stderr and surfaced it as a cancellation reason.

In v26.1.2, environment variable names are filtered against a POSIX-compliant shell identifier regex before being re-exported, so bash-function variables are skipped and the generated script runs cleanly.

Who is affected

Installations on RHEL and similar hosts where /etc/profile.d/ scripts export shell functions into the parent environment. The quickest check is env | grep ^BASH_FUNC_ on a super host: any matches mean this fix applies.

Python nodes can import packages from the playpen venv

vor create process previously failed with import errors when a Python node depended on a package that had been installed only into a playpen-local virtual environment at <playpen>/venv. The build checked imports against the shared /opt/vor/venv runtime, so playpen-local packages (including editable setup.py develop installs) were invisible at build time even though they would have imported fine at run time.

In v26.1.2, the build auto-detects <playpen>/venv and runs its import checks against that interpreter when it exists. Playpen-local packages are visible, and vor create process succeeds without modifying the shared venv. Behavior is unchanged when no playpen-local venv exists.

The Python venv setup recipe in the Getting Started guide has also been rewritten around this: the shared venv is presented as the documented default runtime, the setup recipe is Python-version-agnostic (no longer hardcoding python3.9), and each shell step is its own copy-paste block.

Custom Caddy configuration that survives redeploys

A new admin-facing capability lands in this release. Caddy's main Caddyfile is generated from Ansible templates during deployment, which means any manual changes to add custom routes or proxy targets get overwritten on the next deploy.

In v26.1.2, the main Caddyfile imports every file from etc/caddy.d/ under the VOR root. Administrators can drop site-specific configuration there and it will persist across deployments.

# Add a custom route
sudo -u vrisk vi /opt/vor/etc/caddy.d/my-route.conf

# Apply
sudo -u vrisk supervisorctl restart caddy

See the "Extending Caddy Web Server Configuration" page in the admin guide for the full recipe and the conflict-avoidance notes.

Security hardening

v26.1.2 includes a broad sweep of security updates across the runtime, server image, and application dependencies:

  • The Go runtime has been upgraded to 1.26.3, picking up six runtime fixes.
  • Django has been upgraded to address vulnerabilities in Django core and its packaged contrib applications.
  • CVE patches have been applied across the Python and Node dependency trees, including updates to urllib3, authlib, cryptography, idna, pip, pytest, gitpython, langchain-core, langchain-openai, langsmith, dompurify, hono, postcss, qs, fast-uri, ip-address, brace-expansion, and ws.
  • Dependency scanning has been migrated to Google's OSV-Scanner for recursive coverage across Python, npm, and Go manifests. New manifests are picked up without further CI configuration.

Did you see one of these symptoms?

If you experienced any of the following on v26.1.0 or v26.1.1, the underlying defect is fixed in v26.1.2:

  • Phantom zero-valued rows in output from multi-threaded nodes.
  • db=mssql or db=psql nodes failing with Vault 403 after a long getsig= or getdyn= wait.
  • Django stuck in password authentication failed after Vault revokes Postgres credentials.
  • uWSGI stuck in --- no python application found --- when Consul boots ahead of Vault Agent.
  • "Job Cancelled" popup with BASH_FUNC_...%%: not a valid identifier on RHEL hosts.
  • vor create process failing with spurious import errors for packages installed into a playpen-local Python venv.
  • vor create secret --ci silently accepting an invalid --ssl mode and failing later at connect.

Upgrade Notes

Drop-in upgrade

v26.1.2 is a drop-in upgrade from v26.1.1 with no breaking changes and no required migration steps. Installations running v26.1.0 or v26.1.1 can upgrade directly.

If you have been working around any of the fixed bugs (phantom zero-rows from multi-threaded nodes, 403s on long-wait database nodes, "Job Cancelled" popups on RHEL hosts, or import errors when building Python nodes against a playpen-local venv), v26.1.2 is the upgrade that removes the cause.