GS Run Processor

1. GS Run Processor

1.	GS Run Processor

1.1 Data Processing Pipelines

1.2 Image Processing

1.3 Signal Processing

1.4 Signal Processing Filters

The GS Run Processor application is identical for both the GS Junior and GS FLX+ Instruments. However, due to differences in Instrument hardware, some references in this manual will be specific to either the GS Junior or the GS FLX+ Instruments. There are three main differences: the on-instrument computing resources, the available data processing pipelines and the PicoTiterPlate (PTP) device.

The GS Junior Instrument does not have on-instrument computing capability, but instead includes an Attendant PC with all of the 454 Sequencing Software components installed. It is designed to perform image and signal processing, as well as data analysis. The 454 Sequencing Software can optionally be installed on a DataRig or cluster.

Therefore, references to on-instrument software or computation is specific to the GS FLX+ Instrument, while references to a DataRig or cluster can apply to either instrument.

The GS FLX+ Instrument supports five data processing pipelines. On the GS Junior Instrument, there are three available data processing pipelines. The supported pipelines are detailed in section 1.1.

The PTP device on the GS FLX+ Instrument supports division into multiple (2, 4, 8 or 16) regions. The GS Junior Instrument PTP device only supports a single region. Therefore, any references to multiple regions are specific to the GS FLX+ Instrument.

The GS Run Processor application performs the data processing of a sequencing Run to convert raw images into signal intensity values. This is done in two steps: image processing and signal processing. During a sequencing Run, the CCD camera on the GS Junior or GS FLX+ Instrument takes an image of the PicoTiterPlate Device for each nucleotide flow of the sequencing Run protocol. This image capturing step generates the image files (.pif for GS Junior Instrument; .png for GS FLX+ Instrument) for all the flows which can then be processed. The image analysis step finds raw wells (a well containing a DNA fragment that produced light due to base incorporations during the sequencing Run) across the entire PTP Device, and implements algorithms to normalize the background. This is followed by signal processing which filters, corrects and trims the raw flow signals to produce high quality sequence information.

The core of the GS Run Processor application is the gsRunProcessor executable, which contains algorithms for determining location of the loading regions of the PTP Device, for image processing, signal correction, basecalling and Run metrics generation. The components of the GS Run Processor application are described in Table 1 below:

Component	Description
gsRunProcessor executable	Contains the core algorithms of the data processing application
startGsProcessor script	Calls the gsRunProcessor executable
Launching scripts (runImagePipe, runAnalysisPipe, runAnalysisFilter, etc.)	Provides a command line interface to the gsRunProcessor executable
Processing scripts (imageProcessing.xml, signalProcessing.xml, etc.)	A set of XML files passed to the gsRunProcessor executable containing default commands and parameters for various processing pipeline options
gsRunProcessorManager executable	Provides a simple job batching system for the gsRunProcessor executable in shared user environments
gsReporter executable	Extracts reports and other output data from the processed CWF files

Table 1: GS Run Processor components

There are some environmental variables that control the gsRunProcessorManager application. One key variable is the GS_LAUNCH_MODE, which can operate in one of the modes listed in Table 2:

Value	Description
SINGLE	Starts a single copy of the gsRunProcessor (Default)
MULTI	Starts multiple copies of the gsRunProcessor, equal to the number of processors in the current workstation (Non-cluster)
MPI	Uses ‘mpiexec’ for launching jobs on a compute cluster. Refer to the System Administrator’s Guide for details on configuring the gsRunProcessor suite for use on a non-Titanium cluster.
GSRPM	Starts the job using the gsRunProcessorManager. Will submit jobs to the same job queue as users who use gsRunBrowser to submit processing jobs. Recommended as a multi-user job queuing system.

Table 2: gsRunProcessorManager Environmental Variable GS_LAUNCH_MODE

If GS_LAUNCH_MODE is set to GSRPM, the jobs are launched via the gsRunProcessorManager and the queued jobs can be monitored and aborted via the gsRunProcessorManagerCtrl command. This is the recommended configuration and is also the default.

The gsRunProcessorManager manages the gsRunProcessor application and launches and queues all processing jobs. The gsRunProcessorManager daemon is implemented as a system V script. If the Data Processing software was installed with system-level privileges, the gsRunProcessorManager will automatically start when the system is restarted. The following command (run as root) can be used to start the manager manually if necessary:

/etc/init.d/gsRunProcessorManager start

The following command can be used to view the job queue:

gsRunProcessorManagerCtrl queue

This command will return the job _ids of currently queued processing jobs. The following command can be used to abort a job that is in the job queue:

gsRunProcessorManagerCtrl abort job_id