Performance Considerations¶

Buffer length¶

VOR Stream does not, by default, send one observation at a time to a queue. Instead, observations are grouped together and sent as a buffer. The number of observations in a group is called the buffer length. This grouping reduces the load on the queuing service and significantly improves the overall run time of a process. Observation grouping is also what enables grouping keys. The buffer length is adjusted to fit an entire grouping so that rewinds can occur.

The buffer length can either be implemented site-wide using the config file for super or on a playpen by playpen basis. For a site-wide setting use the following entry in the config.yaml file for super:

bufferlen - Number of observations to be grouped together.

The buffer length is set in the joboptions.json file as follows:

{
    "system": {
     "bufferlen":5000
    }
}

The default buffer length is 50.

Choosing the proper buffer length is balance. There is a limit on the maximum buffer length. The size of sum of all the observations in a buffer needs to be less than 512 MiB. Smaller buffer lengths lead to better memory management for the queue server. Larger buffer lengths also lead to less parallelization. Each thread on a node consumes an entire buffer at a time. If the buffer length is too large, some of the threads will starve for data. Additionally, next node in a process doesn't receive data until a whole buffers worth of data is output from a node. Historically, buffer lengths have been set between 50 and 10,000.

Queue length limits¶

The functioning of a VOR Stream process is analogous to an assembly line with one key difference; the rate at which a nodes (steps in an assembly line) produce output and consume input vary wildly. In a true assembly line, the rate at which output is created is limited by the slowest steps as there is no storage between steps. This option adds the ability to limit the number of observations waiting in a queue. Limiting the queue lengths will reduce memory consumption issues for large processes.

Limits on queues can either be implemented site-wide using the config file for super or on a playpen by playpen basis. For a site-wide setting use the following entries in the config.yaml file for super:

queuelimit - Number of observations allowed to be waiting on a queue. Defaults to 0 which implies no limit.
queuelimitwt - Number of milliseconds to wait between checks of the queue length for polling purposes. Defaults to 10.

For playpen specific settings use the queuelimit tag in the joboptions.json file:

{
    "system": {
        "queuelimit": 1000
    }
}

This queue limit specification will override the site settings for the queue limit for the current playpen. A negative or 0 queue limit will disable limits on queues. The queuelimitwt setting is not available on a playpen by playpen basis.

The queue limit is automatically adjusted up to be at least as big as the buffer length. The queue length should be set to a number that is a multiple of the buffer length. The ideal multiple has not been determined.

The results of limiting the size of queues doesn't appear to have an impact on run times except if you use value that is too small for the queue limit. Even it that case, the effect seems to be minimal.

Setting a queue limit, in rare cases, can cause a process to hang. If your process has a node that is consumming multiple queues from another queue and consumming all the data from one queue before consuming data from the others, then you will get a hang when queue limits are used. Here is a simple example:

node outTwo(in)(out1,out2)
sql select * from out1 inner join out2 using(id) into out3

SQL reads all data from queue out2 before it starts reading from queue out1. The node outTwo will get paused when the queue queue out1 reaches its limit and will stop putting data on out2 and the process will hang.