Adjustable Signal Processing Filter Parameters

1.4.4

Signal processing filters may be turned off or adjusted to control the stringency of the output by editing an XML filter template file. A fragment of the default XML filter template file is shown in Figure 6.

    <!-- Quality filter.-->
    <qualityFilter>
      <!-- Enable or disable the trim back filter.  Normally, enabling
	   this filter is recommended. 
	   default=true -->
      <doValleyFilterTrimBack>true</doValleyFilterTrimBack>
      
      <!-- This parameter controls the number of "bad" flows counted 
	   before the read is discarded.  Increase this number to
	   increase the number of wells while reducing quality.
	   default=4 -->
      <vfBadFlowThreshold>4</vfBadFlowThreshold>
      
      <!-- This is the last flow to test for the valley filter.  Set it
	   to a larger number to make the filter more stringent. 
	   default=320 -->
      <vfLastFlowToTest>320</vfLastFlowToTest>
      
      <!-- Use this parameter to control the percentage of reads that
	   can fail before the trimback filter discards the read.
	   Setting it to 100 disables the trimback filter's error
	   checking component (recommended). 
	   default=100 -->
      <vfMaxFailedPercent>100</vfMaxFailedPercent>
      
      <!-- Use this parameter to control the minimum number of good
	   flows that must be present before the trim-back filter discards it. 
	   default=84 -->
      <vfTrimBackMinimumLength>84</vfTrimBackMinimumLength>
      
      <!-- This parameter controls the stringency of the 
	   valley filter.  A higher number rates ambiguous
	   signals more strictly and more likely to throw
	   out the read.
	   
	   default=0.7 for FLX, 1.6 for Junior, 2.2 for 400 cycle runs -->
      <vfTrimBackScaleFactor>0.7</vfTrimBackScaleFactor>
      <if test="starts-with(InstrumentModel,'GSJUNIOR_A')">
        <!-- This is for Junior runs only. -->
        <vfTrimBackScaleFactor>1.6</vfTrimBackScaleFactor>
      </if>
      <if test="Run.Flow.CycleCount &gt; 299">
        <vfTrimBackScaleFactor>2.2</vfTrimBackScaleFactor>
      </if>
      
      <!-- This parameter controls the last flow over
	   which the valley filter scans.  For longer
	   reads, lower quality signals at higher
	   flows may penalize otherwise good data.  Set
	   this limit to a number lower than the number of
	   flows to enable this limit.
	   
	   default=4096 to disable -->
      <vfScanLimit>4096</vfScanLimit>
    </qualityFilter>

Figure 6: User-editable XML filter template file fragment, with default signal processing quality filter settings used for a shotgun sequencing run (emphasis added).

This section provides instructions for customizing the XML filter template parameters for signal processing.

1.	Open a command line terminal and change directory to the Data Processing folder (‘D_’) of the sequencing Run whose reads are to be re-filtered.

cd /Path/to/D_folder

2.	Generate a template file in the D_ folder by typing one of the following two commands (the XML output template filename, to the right of the “>”, can be any valid string ending with ‘.xml’):

gsRunProcessor --template=filterOnly > myShotgunTemplate.xml
gsRunProcessor --template=filterOnlyAmplicons > myAmpliconsTemplate.xml

3.	Open the template file with a text editor, for example “nedit” (on the GS FLX+ Instrument) or “gedit” (on the GS Junior Attendant PC). Type one of the following commands, specifying the template file you just created:

nedit myShotgunTemplate.xml   or   gedit myShotgunTemplate.xml

Within the template, scroll down to the <qualityFilter> section (Figure 6) or <baseCaller> section (located just below <qualityFilter>). User-editable filter parameters that govern the selection of high quality reads are adjusted in these two sections. Edit filter parameters to either increase stringency (fewer, higher quality reads) or decrease stringency (more, lower quality reads). A short description and the default values for each filter in the template are shown in Table 10 (Valley Filter/Valley TrimBack Filter Parameters) and in Table 11 (Basecall Quality Score Filter Parameters).

5.	If desired, add other filter parameters and values. Use with care.

6.	After the edits have been made, save and exit the text editor.

7.	Change directory, to the parent Run folder directory:

cd ..

8.	Launch the processing job, specifying the modified filter parameter script.

runAnalysisFilter --pipe=/Path/to/myShotgunTemplate.xml /Path/to/D_folder

If the job is successfully launched, a new Data Processing directory with the name convention D_..myShotgunTemplate will be created in the Run directory.

Concentrate on the use of two valley filter parameters: reduce vfScanLimit to include more reads with low quality at the 3’ end; and increase or decrease vfTrimBackScaleFactor to increase or decrease the number of rejected reads with ambiguous flows.

While it is possible to turn off the read rejecting and signal intensity filters (Key Pass, Dot, Mixed and Short Signal ) this will result in output of sub-standard quality data that is NOT recommended for use in subsequent data analysis. However, to turn off these filters, add the following lines under the <qualityFilter> section of the Shotgun Processing template.

<doClassifierCheck>false</doClassifierCheck>

<doDotCheck>false</doDotCheck>

<doMixedCheck>false</doMixedCheck>

<doShortSignalCheck>false</doShortSignalCheck>

For more information regarding Amplicon filter parameters, please refer to Application Note APN-10002: Amplicon Sequencing; Experimental Designs, Guidelines and Tips, available at www.454.com/my454.