Computation

3.5.11

The CLI can also be used to trigger the computation of a Project once it has been set up properly.

3.5.11.1

Validating the Project Before Computation

There are two utility commands that can be used to validate a Project before computation: “utility validateNames” and “utility validateForComputation”.

The GUI does not currently enforce unique names for individual objects because there is an internal accounting process other than naming that keeps entities distinct from each other. In the CLI, however, all objects are created and manipulated using their names, so non-unique names can be a problem. Some degree of name duplication can be tolerated for Amplicon and Variant objects if the Reference Sequences for those objects can be combined with the object names to unambiguously identify individual objects (using the “‑ofRef” parameter available in various commands). Any name duplication that cannot be disambiguated in this manner must be fixed. Additionally, the GUI can allow objects to have empty names, which also would cause problems for the CLI.

The “utility validateNames” command can be used to detect and correct naming problems in Projects (see section 3.4.17.1 for the usage statement). Running the command without any arguments will report an error if any problem names are encountered (irresolvable duplicates or empty names). The command does nothing if there are no errors to report.

You can also use the “‑fix” flag with the “utility validateNames” command to enable it to correct the naming problems it finds (rather than using it just as a problem detection tool as above). The default fix is to put a “FIX_” prefix and an underscore (“_”) suffix followed by a unique number. For example, two Reference Sequences with the same name “MyRef” would be converted to FIX_MyRef_1 and FIX_MyRef_2. You can specify alternate prefixes and suffix separators using the “‑fixPrefix” and “‑fixSuffix” options. The common prefix makes it easy to find these “fixed” names in the sorted Project Tab tables of the GUI and manually adjust them, if desired, to different unique names.

Even if the names are perfectly fine, there can be other problems with a Project that might impact its computation. The command “utility validateForComputation” checks for these problems (see section 3.4.17.2 for the usage statement). Specifically, the command verifies the following:

1.	all Reference Sequences contain a sequence that is at least 1 base in length;

2.	all Amplicons are associated to valid Reference Sequences and have target start and end coordinates that are contained within that Reference Sequence;

3.	all Read Data files that are associated with at least one Sample and one or more valid Amplicons are available; and

4.	if Variants are defined in the Project that are associated to valid Reference Sequences, they have non-empty patterns that are valid with respect to that Reference Sequence.

If any of these criteria aren’t met, warnings are reported and the command throws an error; otherwise it does nothing. The warnings can be silenced by setting the “‑silent” option to “true”.

3.5.11.2

Managing the Computation

The “computation” command allows you to start or stop a computation, or check its status (see section 3.4.3 for the usage statement). You are only allowed to use the “computation start” and “computation stop” commands on Projects on which you have full control. If you were able to successfully do a simple “open” on a Project, or if you were able to open it with the “‑control preempt” option, you have the appropriate level of control to start or stop a computation.

If you have read-only access to the Project, you cannot influence the course of the computation, but you can run the “computation status” command which will report “running” or “stopped”, as appropriate.

Unlike with the GUI, a computation started by the CLI is not run as a separate background process, but rather as part of the CLI process itself. This means that if the CLI instance that started a computation is terminated (via a control-C, for instance) the Project computation will abruptly stop as well. Additionally, it means that the next step of the current CLI script (or the display of the next command prompt, if in interactive mode) will not be executed until the computation finishes. Thus, the “computation status” command for checking the status of a computation is mostly useful for checking on the status of computations that were initiated either from the GUI or from other CLI instances, but not for computations that a particular CLI instance started itself. Unlike the GUI, therefore, the CLI does not currently provide the means to obtain status reporting to determine the current stage of computation or the dispositions of the processed and queued computation steps. However, if a computation were started from a CLI instance, you could open the project in the GUI (perhaps in read only mode, so as not to seize control from the CLI instance that may have additional steps to perform after the Project computation is finished) and track its detailed progress there.

Likewise, because computations run as part of the CLI process, you can’t execute a clean “computation stop” within the same instance of the CLI that you used to initiate the “computation start.” The safest way to halt a computation is to start another instance of the CLI (or open the GUI), open the Project in preempt mode, and type “computation stop” (in the CLI) or press the stop button on the main Computation tab (in the GUI).

3.5.11.3

Loading Automatically Detected Variants

Once a computation is complete, any Variants that were part of the Project prior to the computation will have their frequency statistics updated. As part of the computation, the application also attempts to automatically detect potential variations in the data. In the GUI, you have the option of importing the automatically detected variants using a “Load” button on the main Variants Tab, with the option to narrow down the set to import using a variety of filters. In the current version of the CLI, by contrast, no filters are provided and the import is an all or nothing proposition.

You can import the pool of automatically detected variants in the CLI by using the “computation loadDetectedVariants” command (see section 3.4.3.4 for the usage statement). Note that this command loads the automatically detected Variants into memory, but just like in the GUI, you must save the Project if you want the load to be permanent. Since the load is currently all or nothing, a good strategy may be to load the Variants but not save them, so you can run a report command (see section 3.5.12) on all the potential Variants for a Project without cluttering the GUI view of the Project with too many marginal Variants.

If you choose not to load the detected Variants after a computation triggered from the CLI (or you choose to load them but not save the Project), the Auto-Detected Variants will not be lost to you even if you exit the CLI: they will remain in the Project, in a queue to be loaded in a subsequent session. For instance, you could start a new instance of the CLI, reopen the Project and run the “computation loadDetectedVariants” command, and thus recover the Auto-Detected Variants you chose not to load or save the first time. Similarly, you could finish the computation in the CLI without loading (or with loading and not saving) the Auto-Detected Variants. Later, you can open the Project in the GUI, and you would be able to access the Auto-Detected Variants via the Load button on the main Variants Tab.