VOR Stream Features¶
VOR Stream is a system for creating and maintaining processes designed to solve financial risk problems. Key features of the system are:
- Batch-Stream oriented execution of a process - i.e., partial results available early - before the job finishes
- Parallel processing across threads, across machines, and across the workflow
- Each step (node) in the process is run independently of and in parallel with the other steps
- Each step can be written in a different language (Python, Golang, SAS, and SQL languages are currently supported)
- Documentation is autogenerated
Playpens¶
VOR Stream is playpen-oriented, meaning that a node named bar in a playpen called mine is considered distinct from a node named bar in a different playpen, such as mine2. If you create a new playpen—whether by pulling source code from a repository or copying it from another playpen—you must register the new playpen and its artifacts with the mid-tier.
To do this:
- Run
vor create queueonce to register the queue. - Then run
vor create processfor each process in the new playpen.
Documentation¶
Queues, nodes, and processes automatically generate documentation when they are created. Process pages include a diagram with links to the nodes and queues, followed by a list of the nodes or sub-processes that run. Node pages show the node type and its input and output queues. They also include any SQL query or SAS configuration as written. Queue pages list fields and inherited queues.
Once the documentation is created for computational nodes and processes, the
documentation is not updated if one of these objects is updated. If you want
the documentation to be updated, delete the specific document located in the
<playpen-dir>/src/docs directory and recreate the node or process.
Documentation is created in Markdown format that can be easily modified,
extended, and themed. Documentation is stored in the <playpen-dir>/src/docs
directory so it is under source control. The documentation is then converted to
static web pages to be referenced from the UI. The static web pages are
generated using Zensical when a playpen is created and
whenever you run the vor create doc command. The static pages are not
updated automatically.