We’re excited to present the latest updates and improvements in VOR. Our development team has been hard at work to bring you an enhanced experience with this new release. In these release notes, you’ll find information about the changes, upgrades, and fixes we’ve made to make your user journey even better. We value your feedback and strive to continuously improve VOR to meet your needs and exceed your expectations. Whether you’re a new user or a long-time supporter, we’re grateful for your ongoing support and trust in our product. Have an idea or suggestion? Send us a note with your thoughts to our dedicated inbox. Thank you for choosing VOR for all your risk management needs. Let’s dive into what’s new! -FRG VOR Development Team

Release

This version, 2024.05.06.01, was released on May 6, 2024.

🎆 Key Highlights

✏️ Management of tables.csv in the UI

With this release, VOR Stream implementations can now import their table metadata for management in the VOR Stream UI:

This makes visualizing and editing inheritance relationships between tables easier as it renders the final table after inheritance and drop/keep directives are applied. To use this functionality, first import your existing dictionary.csv and tables.csv definition from the CLI using these commands from the playpen root:

vor update dictionary –file src/dictionary.csv
vor update tables –file src/tables.csv

In this release, the playpen still uses the tables.csv and dictionary.csv definitions in the playpen, so changes from the UI need to be manually synced down to the playpen. To sync changes made from the UI, run the following commands from the playpen root:

vor show dictionary -f src/dictionary.csv
vor show tables -f src/tables.csv

[!TIP] To see effective queue definitions after accounting for inheritance, execute vor show queue.

🧪 vor test enhancements

The vor test command is useful for benchmarking and testing process flows. A number of enhancements were made to the command in this release: Add a test summary report Provide a clear summary of all tests that were executed and their results:

Also provide a summary of the total number of tests executed and pass, failure, difference statuses:

Add a “stream file” field so that you can quickly scan for a particular log:

If a process has errors or an error occurs when comparing benchmarks, always print the command output and record an execution failure. This example of csvdiff not being available was never relayed to the user previously:

Missing input file and the node logs are automatically printed:

Previously you could run the entire test suite or subset down to only one test. Allow for multiple stream files to be specified via Cobra’s StringSliceP(). On the CLI, this would look like: bash vor test -s alias1,alias2,alias3 When benchmark differences are detected, pretty format the command to copy and clearly report the difference summary in the log.

🖥️ Process CLI Changes

In addition to refactoring the process API behind the scenes, a number of process CLI commands were added or enhanced:

Use the vor show process command:

shell-session

$ vor show process

NAME │ TYPE │ RUNNABLE │ PROCESSID

────────────────────┼──────┼──────────┼────────────

NullPythonProcess │ │ true │ 4

firstprocess │ │ true │ 1

testpy │ │ true │ 2

testpy2 │ │ true │ 3

With this change, CLI users can recall the process definition that was used to create the process. Use the vor show process <process-name> command:

NOTE: The replayed process definition may differ slightly in formatting and syntax from the definition used.

shell-session

$ vor show process firstprocess

INFO Process code:

name firstprocess

in first -> input name=input_input_firstprocess

out output -> output name=output_output_firstprocess

node usernode (input)(output)

The –getmap and –setmap flags for vor craete process were deprecated. This functionality is still available via a strm file.

Support for piping of process definitions to vor create process added. All of the following are valid ways to create a process:

bash

# Create a process from a file

vor create process first_process.strm

# create a process from output piped from another command

cat first_process.strm | vor create process

# read from standard input to create a process (terminate with Ctrl+D)

vor create process –

🏎️ vor create queue performance enhancements

vor create queue should be significantly faster. Deployments with a large amount of tables will notice this change when setting up playpens for the first time and running vor create queue or vor create process (creates queues automatically) for the first time.

🌈 UI theme selection

On first login, a user’s theme setting will default to system, which means it will follow the system setting of light or dark mode. Existing accounts will not have their theme setting changed.

📖 Documentation Highlights

The reference architecture page was updated for the following: [!NOTE] Some of this content is still a work-in-progress.

Add a section for supported databases for Django.
Add a System Requirements section. Specifically, add the following:

Service mappings to nodes (cloud instances/VMs).

Initial sizing recommendations for deployments. This includes mappings to cloud-provider instance types.
Further details on the rationale for the sizing recommendations, e.g., if a node is typically compute or memory intensive.

Additionally, an Authentication page that details configuration options and has sample steps to register a Microsoft Entra ID app registration for use with VOR Stream was added.

Details

✨ Features

Specify job options for End-to-End tests [VS-473] (#463)
(process) Display stats + drill down for processes with repeated subprocesses and queue loops [VS-304] (#256)
(tables) Managing tables.csv on the UI [VS-25] (#361)
(engine) Add the ability to limit queue sizes to reduce resource consumption [VS-459] (#446)
(scenarioweight) Add feature flag to disable weights for scenario selection [VS-432] (#378)
(configuration) Add parameter (column) delete functionality [VS-402] (#352)

🐛 Bug Fixes

(waterways) Add _ to job_cancelled in enumeration [VS-365] (#537)
(engine) Address panics from sending on a closed channel encountered when canceling a process [VS-501] (#540)
(engine) Job labeled as canceled when it finished with errors [VS-430], [VS-365] (#536)
Resolve data race when flushing node logs [VS-502] (#531)
Missing error message when job fails to initialize [VS-500] (#528)
(tables) Add loading animation at start of import [VS-25] (#514)
(waterways) Make sure both refresh buttons on waterways demonstrate the same behavior [VS-365] (#513)
Group key information missing from table import [VS-489] (#505)
(engine) Wait for node stat updates to complete before writing node summary stats [VS-472] (#455)
(sql) Restrict the number of into keywords in a SQL query [VS-142] (#460)
(sql) Group By clause with an aliased/computed group key assigned to old value [VS-421] (#376)
(logging) Fix reversed logic on waterways status check [VS-365] (#386)
(sql) Enable expressions in SQL ON clause [VS-55] (#213)
(engine) Drop and Keeps in new table format cause errors [VS-463] (#451)
(sql) Remove extra argument in parsedate function [VS-426] (#447)
(ui) Ensure all Django API requests include a trailing slash [VS-449] (#413)
(engine) Resolve panic on healthcheck GET and playpen list, GET, and DELETE calls [VS-451] (#420)
(study) Fix missing author on run study tooltip [VS-422] (#377)
(modelrun) Refresh model run history when model run completes + add refresh button [VS-396] (#342)
Ensure correct version tag is embedded in vor version [VS-281] (#375)
(sas) Use nosource option to make SAS log bench-able [VS-303]
(engine) Avoid race condition in accessing dynamic facts [VS-387] (#364)
(runstudy) Fix [object Object] on comments and blank methodology label on run study tooltip [VS-406] (#356)
Add permission to view on screen help for PCF group [PCF-90] (#357)
(configuration) Improved scroll area and UI for config tables (#348)
(sas) Force output formats of numeric variables returned from SAS [VS-386] (#346)
(theme) Fix theming issues with ag-grid menus in dark mode [VS-401] (#350)
Correct permissions of generated files [VS-371] (#344)
(sas) Unconditionaly delete Pipe before using (#335)
Reset consul server age to rejoin cluster during deployment (#301)
(engine) Reading excel files with formatted columns failures (#262)

📖 Documentation

Expand documentation on architectural considerations [VS-15] (#445)
Fix typo in reference_architecture.md (#369)
Add documentation for setting up SSO [VS-409] (#358)
Add documentation on publishing of releases (#296)

✏️ Refactor

(tables) Fix initial loading performance of tables.csv on UI [VS-25] (#542)
(tables) Enhance UX of tables.csv [VS-25] (#499)
(engine) Speed up execution time for vor create queue [VS-474] (#459)
(engine) Move storage of tables and data dictionary to Django [VS-383] (#298)
(sql) Use constants and enumeration for sql functions [VS-428] (#442)
(sql) Use cast package for type conversions [VS-427] (#432)
(sql) Move sql code to a separate package [VS-426] (#393)
(engine) Create process API [VS-219],[VS-220],[VS-34] (#155)
Vor test: add summary report, add structured logging, and improve error handling and feedback [VS-450] (#394)
(engine) Remove –getmap and –setmap flags from vor create process, support piping [VS-52] (#354)
(reporting) Support CRUD operations on /run-study/report-types/ endpoint [VS-397] (#336)
(pcf) Update permissions for VOR PCF Users group (#337)

↩️ Revert

“Ci: reorganize the sequence of engine builds to run playwright execution before goveralls [VS-274] (#387)”

💅 Styling

(lint) Enable auto-fixable gocritic checks [VS-371] (#373)
(lint) Add gocritic linting [VS-371] (#372)

🧪 Testing

(ui) Add conditional playpen selection on test reruns [VS-481] (#480)
Add end-to-end test suite to automation [VS-450] (#448)
(sql) Increase test coverage for sql package [VS-429] (#415)
(engine) Add missing test files [VS-412]
(engine) Include test suites in the repository [VS-412] (#363)
(engine) Ensure unit tests are run on all pushes [VS-435] (#384)
(engine) Add coveralls Go test coverage reporting [VS-435] (#383)
(ui) Temporarily back out Playwright automation [VS-274] (#379)
(ui) Implementation of Automated Playwright Tests for VOR Functionalities [VS-274] (#271)

VOR Release 2024.05.01