Data Acquisition, Data Processing and Data Analysis

1.1	Data Acquisition, Data Processing and Data Analysis

Data handling in the 454 Sequencing System occurs in three main phases, each governed by one or more specific applications:

•	Data Acquisition

•	Data Processing

•	Data Analysis

The data acquisition phase is controlled by the GS Sequencer software application on the GS FLX+ Instrument or by GS Junior Sequencer on the GS Junior Instrument (this is the only application that has two different implementations under the two systems). The raw data consists of a series of digital images captured by the camera. The images are a representation of the surface of the PicoTiterPlate device over which the sequencing reactions are taking place; and each image corresponds to one nucleotide flow over that surface. If the sample DNA fragment present in a given well of the PicoTiterPlate device is extended during a nucleotide flow, light is emitted from the well and captured on the image corresponding to that flow. Furthermore, the amount of light emitted is proportional to the number of nucleotides extended.

The data processing phase is carried out by the GS Run Processor application and converts raw image data to base-called results suitable for use by downstream data analysis applications. Data processing is done in two main steps, image processing and signal processing. The software first identifies well locations and measures the amount of light emitted in each well location during each flow. It then uses this information to determine the sequence of DNA fragments located in each well.

The data acquisition and data processing steps are as follows:

1.	The GS Junior Sequencer or GS Sequencer application records a set of raw digital images representing the light detected over the surface of the PicoTiterPlate device, during each reagent flow of the sequencing Run (data acquisition).

2.	The first step of the GS Run Processor application, image processing, performs initial pixel-level calculations, and then groups pixels from the image set into a representation of the PicoTiterPlate wells where sequencing reactions were detected.

3.	The second step of the GS Run Processor application, signal processing, performs well-level calculations across the whole series of images to generate well “flowgrams” (and the basecalls of the DNA fragments being sequenced in all the active wells of the PicoTiterPlate device; “reads”).

Table 1 lists the inputs and outputs of the three main early components of data handling, from data acquisition through data processing, as well as the individual functions carried out by each application.

Application

Input

Output

Main processing steps

GS Junior Sequencer or GS Sequencer

Light

Raw images

•	Image acquisition and storage to disk

GS Run Processor (image processing step)

Raw images

Composite Wells Format (CWF) Files

•	Subtract background and normalize the images (at the pixel level)

•	Find the active wells on the PicoTiterPlate device

•	Extract the raw signals for each flow, in each active well

•	Write the resulting flow signals into “composite wells format” (CWF) files

GS Run Processor (signal processing step)

Composite Wells Format (CWF) Files

Corrected CWF files, and SFF files containing read basecalls and per-base quality scores

•	Filter out lower signal ghost wells (Amplicon sequencing pipeline only)

•	Correct for crosstalk between neighboring wells

•	Correct for known “out-of phase” errors (incomplete extension and carry-forward)

•	Correct for signal droop and perform residual background subtraction

•	Filter out any residual ghost wells (Amplicon sequencing pipeline only)

•	Filter (pass or fail) the processed reads based on signal quality

•	Trim read ends for low quality and primer sequence

•	Update the CWF files with the fully processed data

•	Generate Standard Flowgram Format (SFF) files containing the basecalled read sequences and per-base quality scores

Table 1: The 3 main early components of data handling, from data acquisition through data processing, in the 454 Sequencing System, with their inputs, outputs, and main processing steps. They are performed in succession, in the order indicated; the SFF files output by the signal processing step of the GS Run Processor application are used as input to the data analysis applications (see Table 2). For a description of the data processing pipeline options, see section 1.2. For a full description of the GS Junior Sequencer or GS Sequencer application, see Part A of this manual (Section 2 in the GS Junior System version or Section 3 in the GS FLX+ System version); and for the GS Run Processor application, see Part B, Section 1.

•	With the GS Junior System, all data acquisition and data processing can be handled by the Attendant PC, as part of the sequencing Run.

•	With the GS FLX+ System, the computationally-intensive image and signal processing may be configured to take place on an external DataRig.

The data analysis phase offers a choice of several downstream analysis paths to generate the desired final output: a consensus sequence of the DNA sample generated by the assembly of reads into contigs and scaffolds (GS De Novo Assembler); a consensus sequence along with a list of high-confidence differences obtained by mapping the reads to a known reference sequence (GS Reference Mapper); or the identification and quantitation of sequence variants by the ultra deep sequencing of amplicons (GS Amplicon Variant Analyzer). All data analysis outputs also include base-per-base quality scores (Phred-equivalent) and other specific metric files.

The data analysis steps are as follows:

The GS De Novo Assembler application generates a consensus sequence of the whole DNA sample, by assembling the reads into contigs (de novo shotgun assembly). An option allows the use of one or more sequencing Runs performed on a Paired End library (any type, or even a combination of Paired End library types) prepared from the same DNA sample, to be analyzed together with Shotgun sequencing Run(s) and help order and orient the resulting contigs into scaffolds. (Paired End reads do not necessarily need to be analyzed together with Shotgun reads.)

The GS Reference Mapper application generates the consensus DNA sequence by mapping, or aligning, the reads to a reference sequence; as well as a list of high-confidence differences (individual bases or blocks of bases that differ between the consensus DNA sequence of the sample and the reference sequence). Robust cDNA analysis is also available.

3.	The GS Amplicon Variant Analyzer application compares reads from an Amplicon library to corresponding reference sequences, and allows the user to detect, identify and quantitate the prevalence of sequence variants.

The data analysis applications use the fully processed and “trimmed” read basecalls of a sequencing Run, or of a pool of Runs, to produce initial alignments to the reference sequence (or read-to-read overlaps for the GS De Novo Assembler). They then use a combination of nucleotide and flowgram information for consensus-calling of the contigs and determination of quality values for the contig sequences. Contig consensus-calling is carried out in “flowspace” (i.e. it operates directly on the processed signals measured from the wells), followed by basecalling to produce a consensus sequence for the sample. Table 2 lists the specific outputs of the 3 data analysis applications as well as the individual functions carried out by each one.

The final output of the 454 Sequencing System thus varies depending on what kind of analysis is performed: Assembly, Mapping or Amplicon Variant Analysis (or no analysis of any kind). In all cases, however, the output DNA sequence is supplied as a set of FASTA files, with associated “Quality Scores” and other Run and data metrics files useful for troubleshooting and determining the overall quality of the sequencing Run. ACE-formatted files are also produced by each of the data analysis applications to allow users to view alignment results using third-party software tools.

Application	Input	Output	Main processing steps
GS De Novo Assembler	SFF files, from one or multiple sequencing Runs, containing read flowgrams and basecalls, and per-base quality scores	Sample consensus sequence, assembled de novo (and scaffold information, with Paired End option)	Identify pairwise overlaps between reads, in nucleotide space Construct multiple alignments of reads that tile together (i.e. form contigs), based on the pairwise overlaps Generate consensus basecalls of the contigs by averaging the processed flow signals for each nucleotide flow included in the alignment, in flowspace Output the contig consensus sequences and corresponding quality scores, along with an ACE file of the multiple alignments and assembly metrics files Additional steps with Paired End option: Identify pairwise overlaps between Paired End tags and the shotgun contigs Organize the contigs into scaffolds (order, orientation, and approximate distance) Output the scaffolded consensus sequences and corresponding quality scores, along with an AGP file of the scaffolds and specific metrics Tables
GS Reference Mapper		Sample consensus sequence, mapped to a reference sequence; and list of differences	For each read, search for alignment(s) to the reference sequence, in nucleotide space Construct contigs and compute a consensus basecall sequence from the signals of the aligned reads (flowspace) Identify the positions where the consensus or subsets of the reads that comprise it differ from the reference sequence (or reads from one another); these are the “putative differences” Evaluate the putative differences to identify high-confidence differences Output contig consensus sequence(s) and corresponding quality scores, an ACE file of the multiple alignments of the reads and contigs to the reference, the list of identified differences, and mapping metrics files
GS Amplicon Variant Analyzer		Identity and quantitation of sequence variants	Trim reads (remove primer sequences) Assign reads to “Samples” (demultiplex data sets) Align Sample reads to their reference sequences Quantitate variant frequency for each Sample

Table 2: The 3 applications of the data analysis phase of the 454 Sequencing System, with their inputs, outputs, and main processing steps. Note that all data analysis applications use as input the reads and flowgrams output in SFF format by the data processing (GS Run Processor application). For a full description of the various data analysis applications, see Parts C and D in this manual.

The software package described in this manual also includes a variety of applications that are used primarily or exclusively off-instrument (on a DataRig or GS Junior Attendant PC). The GS Reporter and the GS Run Browser applications are used to view and troubleshoot the results of a completed sequencing Run; the GS Support Tool is used to package sequencing Run data to send to Roche Customer Support for further help and troubleshooting; and the SFF Tools are a set of commands used to create, manipulate and access sequencing trace data from SFF files. However, these applications and commands are not required steps of data processing and analysis.