Creating References

3.5.3

The next step is to add Reference Sequences since they are necessary for the full specification of Amplicons. This is done using the “create reference” command (see section 3.4.4.7 for the usage statement). Multiple Reference Sequences can be created in a single invocation of the “create reference” command by saving to a file a table containing all the specific Reference Sequence features, and calling the “create reference” command on that file using the “‑file” option. The file containing the reference features may be in either tab-separated value (tsv) or comma-separated value (csv) formats, but the file is assumed to be in the tsv format unless the “‑format” option is set to “csv” or the file name ends in the suffix “.csv”. See section 3.3.2.3 for more details on using tabular files as input.

Generally speaking, any of the commands that support tabular input work by combining the given command line option values with the parameters specified in the table headers and associated values found in the table, to synthesize a set of command options that are the union of both sets of values. This allows all the parameters of the table to be nested within a constant set of options specified on the command line, reducing the chance of error that might occur when manually creating a table with the repeated constant values within the table itself. The example below is the simplest example of using a file as input, in which all the parameters to the “create reference” command are, in fact, coming from the supplied file. See section 3.5.7 for an example of a command that combines options on the command lines with values specified in a file.

Below is an example tsv-formatted table that could be used to create the 5 Reference Sequences for the EGFR Project. Note that the double quotes around the field names and values aren’t strictly necessary for this particular example, but quotes would be necessary if the values contained the value delimiter character of the format being used. As explained in section 3.3.2.3, the syntax of the tabular input files follows the standard tsv and csv formatting conventions, not the syntax rules of the CLI itself.

Due to the limitations of this printed document, the “Sequence” entry for each Reference Sequence appears on a separate group of lines, when they are in fact on a single line and are separated from the “Annotation” entry only by a tab character. A similar situation occurs in various other file listings, throughout this Manual section.

"Name"     "Annotation"     "Sequence"

"EGFR_Exon_18"     "EGFR_Exon_18" "GACCCTTGTCTCTGTGTTCTTGTCCCCCCCAGCTTGTGGAGCCTCTTACAC
CCAGTGGAGAAGCTCCCAACCAAGCTCTCTTGAGGATCTTGAAGGAAACTGAATTCAAAAAGATCAAAGTGCTGGGCTCCGGTGCG
TTCGGCACGGTGTATAAGGTAAGGTCCCTGGCACAGGCCTCTGGGCTGGGCCGCAGGGCCTCTCATGGTCTGGTGGGG"

"EGFR_Exon_19"     "EGFR_Exon_19"

"TCACAATTGCCAGTTAACGTCTTCCTTCTCTCTCTGTCATAGGGACTCTGGATCCCAGAAGGTGAGAAAGTTAAAATTCCCGTCG
CTATCAAGGAATTAAGAGAAGCAACATCTCCGAAAGCCAACAAGGAAATCCTCGATGTGAGTTTCTGCTTTGCTGTGTGGGGGTCC
ATGGCTCTGAACCTCAGGCCCACCTTTTCTC"

"EGFR_Exon_20"     "EGFR_Exon_20"

"CCACACTGACGTGCCTCTCCCTCCCTCCAGGAAGCCTACGTGATGGCCAGCGTGGACAACCCCCACGTGTGCCGCCTGCTGGGCA
TCTGCCTCACCTCCACCGTGCAGCTCATCACGCAGCTCATGCCCTTCGGCTGCCTCCTGGACTATGTCCGGGAACACAAAGACAAT
ATTGGCTCCCAGTACCTGCTCAACTGGTGTGTGCAGATCGCAAAGGTAATCAGGGAAGGGAGATACGGGGAGGGGAGATAAGGAGC
CAGGATC"

"EGFR_Exon_21"     "EGFR_Exon_21"

"TCTTCCCATGATGATCTGTCCCTCACAGCAGGGTCTTCTCTGTTTCAGGGCATGAACTACTTGGAGGACCGTCGCTTGGTGCACC
GCGACCTGGCAGCCAGGAACGTACTGGTGAAAACACCGCAGCATGTCAAGATCACAGATTTTGGGCTGGCCAAACTGCTGGGTGCG
GAAGAGAAAGAATACCATGCAGAAGGAGGCAAAGTAAGGAGGTGGCTTTAGGTCAGCCAGCAT"

"EGFR_Exon_22"     "EGFR_Exon_22"

"CACTGCCTCATCTCTCACCATCCCAAGGTGCCTATCAAGTGGATGGCATTGGAATCAATTTTACACAGAATCTATACCCACCAGA
GTGATGTCTGGAGCTACGGTGAGTCATAATCCTGATGCTAATGAGTTTGTACTGAGGCCAAGCTGG"

The header of the table shows the names of the parameters you want to supply to the “create reference command”. In order to use the table to create the Reference Sequences in the Project, it can be saved as a file (e.g. “EGFR_CLI_references.txt”) in the directory from which you plan to run the script, and cited as argument under the ‑file option of the “create reference” command:

create reference –file EGFR_CLI_references.txt

When you use a file as input to a command, the AVA-CLI uses the command and any of its other arguments (besides the “-file” argument) as a prefix command and applies them to each non-header line in the file, such that the contents of each row are converted into their command line option form. Thus, the first Reference Sequence line in our example file gets converted into the following command:

create reference EGFR_Exon_18 –annotation EGFR_Exon_18 –sequence 
GACCCTTGTCTCTGTGTTCTTGTCCCCCCCAGCTTGTGGAGCCTCTTACACCCAGTGGAGAAGCTCCCAACCAAGCTCTCTTGAGG
ATCTTGAAGGAAACTGAATTCAAAAAGATCAAAGTGCTGGGCTCCGGTGCGTTCGGCACGGTGTATAAGGTAAGGTCCCTGGCACA
GGCCTCTGGGCTGGGCCGCAGGGCCTCTCATGGTCTGGTGGGG

Tabular input to commands can also be used without saving the table contents to a separate file: you can include the tables directly in your script using the “here” format (which is discussed in section 3.3.2.3). The symbols “‑ <<” after “‑file” indicate that a table in “here” format follows, starting and ending with its “terminator” (in this case, “HERE_TERMINATOR”). The terminator text used to indicate the end of the table should obviously not be found in the table contents. The AVA-CLI will treat the lines following the command as if they were read from a separate file, until it encounters the “HERE_TERMINATOR” text at the beginning of a line.

create reference -file - << HERE_TERMINATOR

"Name"     "Annotation"     "Sequence"

"EGFR_Exon_18"     "EGFR_Exon_18"

"GACCCTTGTCTCTGTGTTCTTGTCCCCCCCAGCTTGTGGAGCCTCTTACACCCAGTGGAGAAGCTCCCAACCAAGCTCTCTTGAG
GATCTTGAAGGAAACTGAATTCAAAAAGATCAAAGTGCTGGGCTCCGGTGCGTTCGGCACGGTGTATAAGGTAAGGTCCCTGGCAC
AGGCCTCTGGGCTGGGCCGCAGGGCCTCTCATGGTCTGGTGGGG"

"EGFR_Exon_19"     "EGFR_Exon_19"

"TCACAATTGCCAGTTAACGTCTTCCTTCTCTCTCTGTCATAGGGACTCTGGATCCCAGAAGGTGAGAAAGTTAAAATTCCCGTCG
CTATCAAGGAATTAAGAGAAGCAACATCTCCGAAAGCCAACAAGGAAATCCTCGATGTGAGTTTCTGCTTTGCTGTGTGGGGGTCC
ATGGCTCTGAACCTCAGGCCCACCTTTTCTC"

"EGFR_Exon_20"     "EGFR_Exon_20"

"CCACACTGACGTGCCTCTCCCTCCCTCCAGGAAGCCTACGTGATGGCCAGCGTGGACAACCCCCACGTGTGCCGCCTGCTGGGCA
TCTGCCTCACCTCCACCGTGCAGCTCATCACGCAGCTCATGCCCTTCGGCTGCCTCCTGGACTATGTCCGGGAACACAAAGACAAT
ATTGGCTCCCAGTACCTGCTCAACTGGTGTGTGCAGATCGCAAAGGTAATCAGGGAAGGGAGATACGGGGAGGGGAGATAAGGAGC
CAGGATC"

"EGFR_Exon_21"     "EGFR_Exon_21"

"TCTTCCCATGATGATCTGTCCCTCACAGCAGGGTCTTCTCTGTTTCAGGGCATGAACTACTTGGAGGACCGTCGCTTGGTGCACC
GCGACCTGGCAGCCAGGAACGTACTGGTGAAAACACCGCAGCATGTCAAGATCACAGATTTTGGGCTGGCCAAACTGCTGGGTGCG
GAAGAGAAAGAATACCATGCAGAAGGAGGCAAAGTAAGGAGGTGGCTTTAGGTCAGCCAGCAT"

"EGFR_Exon_22"     "EGFR_Exon_22"

"CACTGCCTCATCTCTCACCATCCCAAGGTGCCTATCAAGTGGATGGCATTGGAATCAATTTTACACAGAATCTATACCCACCAGA
GTGATGTCTGGAGCTACGGTGAGTCATAATCCTGATGCTAATGAGTTTGTACTGAGGCCAAGCTGG"

HERE_TERMINATOR

The use of regular files or “here” files to input data is equivalent, and the choice is up to the user. Regular files can prove especially useful if the elements of the Project definition are supplied to you from a third-party (such as from the client of a sequencing center). On the other hand, “here” files allow you to encapsulate all the data except for the actual Read Data files into a single, relatively portable script. High-throughput environments with a workflow system might generate such scripts, thereby automating the project creation process.

The “create reference” command will normally fail if you try to create a Reference Sequence with a name that already exists. However, there are situations where it may be legitimate to attempt to do so. For example, a script may have correctly created Reference Sequences and then reached a save point before terminating early due to some error in the script; in such a case, it would be useful to be able to fix the problem in the script and just run it again without it throwing errors due to the Reference Sequences previously saved in the system. The optional flag “‑orUpdate” allows this: if you try to create a Reference Sequence under a name that already exists in the Project, the “‑orUpdate” flag converts the operation into an update, and the annotation and/or sequence provided in the “create” command are used to update the existing Reference Sequence. Without the flag, the name collision would throw an error.

This “‑orUpdate” flag is available for most of the create commands, including those for Amplicons, Samples, and Variants. The flag is discussed in more detail for the Amplicon creation example (section 3.5.4).