Read Trimming Filters

1.4.3

Several chemical- and system-related effects can gradually degrade the signal over the course of a sequencing run, resulting in lower quality signal towards the 3’ end. After read rejecting filters have been applied, low quality reads are trimmed from the 3’ end until they either (a) exceed the filter quality threshold or (b) are discarded as too short (numTrimmedTooShortQuality metric).

Any trimmed read that is less than the minLength parameter is rejected as too short by the Dots or Primer Filters. Other filters have their own minimums, e.g. vfTrimBackMinimumLength for the Trimback Valley Filter or QScoreTrimMinLength for the Basecall Quality Score Filter. The number of reads rejected due to a particular trimming filter is output in the metrics file generated by the gsReporter application (Section 6.5).

Read quality information can be visualized in a Signal Flowgram (see example in Figure 4). The x-axis corresponds to the individual nucleotide flows of the sequencing Run and the y-axis is a measure of signal intensity. The processed signal flowgrams have the following characteristics;

•	Each nucleotide type is plotted in a different color allowing for easy ‘reading’ of the sequence trace data from visual inspection of the flowgram.

•	Nucleotide incorporations are normalized such that a signal intensity near 1.0 corresponds to a single nucleotide, a signal intensity near 2.0 corresponds to a dinucleotide homopolymer, etc.

Figure 4: Normalized signal flowgram

Read trimming filters are applied in the following order.

Signal Intensity Filter – trims reads that lose signal ‘crispness’ near the 3’ end. Some flows have an ambiguous signal intensity (0.5-0.7 on a 0-1.0 scale), possibly from overall signal droop and/or CAFIE error accumulation that leads to a low signal-to-noise ratio (as shown in Figure 5). The 3’ end of a read is trimmed such that <3% of the remaining flows have ambiguous signal intensity for incorporation (numTrimmedTooShortQuality metric).

Figure 5: Signal Intensity Filter

Signal Intensity Filter parameters are described in Table 8.

Signal Intensity Parameter	Description	Values	Shotgun/PE	Amplicon
doShortSignalCheck	Enable/disable Signal Intensity Filter calculation.	true/false	true	true
minLength	Minimum acceptable read length.	# flows	83	83

Table 8: Signal Intensity Filter Parameters. A “false” parameter disables the filter, which may increase the length (and possibly the number) of reads, but will likely reduce overall read quality. Filter stringency can be increased by using larger green values or lower red values.

2.	Primer Filter – trims the end of a read when it matches a 454 Sequencing System Adaptor sequence (numTrimmedTooShortPrimer metric).

Primer Filter parameters are described in Table 9.

Primer Filter Parameter	Description	Values	Shotgun/PE	Amplicon
doPrimerTrimming	Enable/disable Primer filter.	true/false	true	true
minLength	Minimum acceptable read length.	# flows	83	83
minPrimerMatchScore	Primer sequence match score.	fraction	0.5	0.5
useAmpliconPrimers	Use only sequencing primers appropriate for Amplicons experiments, or specify each primer explicitly inside a <primer></primer> element. eg: <primers> <primer>...</primer> <primer>...</primer> </primers>	true/false	false (n/a)	true

Table 9: Primer Filter Parameters. A “false” parameter disables the filter, which leaves non-sample primer sequences attached to each read. Filter stringency can be increased by using larger green values or lower red values.

Valley Filter/TrimBack Valley Filter – Filters or trims reads with many off-peak signal intensities. A Valley flow is defined as an intermediate signal intensity, i.e., a signal intensity occurring in the valley between the peaks for 0-mer and 1-mer incorporations, 1-mer and 2-mer incorporations or between 2-mer and 3-mer incorporations. The signal distribution of all reads of the Run is used to define the peaks of the homopolymer incorporations relative to the valleys. One of the following three valley filters is applied when doValleyFilter = ‘true’.

•

Counting Valley Filter [read rejecting] is applied when both vfScanAllFlows and doValleyFilterTrimBack are ‘false’. The Counting Valley Filter evaluates each flow up to a limit specified by vfLastFlowToTest (default = 320 flows). For each read, the number of borderline valley flows is compared to a threshold specified by vfBadFlowThreshold (default = 4 “bad” flows with intermediate signal intensity). Reads that exceed the threshold are discarded (numTrimmedTooShortQuality metric). The Counting Valley Filter has been largely superseded by the Scoring and TrimBack Valley Filters.

•

Scoring Valley Filter [read rejecting] is applied when vfScanAllFlows is enabled, and is the default for amplicon processing. The Scoring Valley Filter calculates a valley score for each flow up to a limit specified by vfScanLimit (default = 700 flows for amplicons). For each read, the sum of the flow valley scores is scaled by a factor (vfTrimBackScaleFactor) and compared to a threshold ratio calculated from vfBadFlowThreshold and vfLastFlowToTest (default = 4 bad flows per 320 flows). Reads with a scaled, summed score that exceeds the threshold are discarded.

•

TrimBack Valley Filter [read trimming] is applied when doValleyFilterTrimBack is ‘true’, and is the default for shotgun processing. The TrimBack Valley Filter utilizes the same mechanism as the Scoring Valley Filter, but with the additional feature that it scans reads backwards from the last flow of the read and trims flows until the scaled, summed valley score for the read no longer exceeds the scoring valley filter threshold. This trimming is used to retain the higher quality portion of a read (numTrimmedTooShortQuality metric).

Counting Valley Filter [Default Value(s)]

Score = # borderline valley flows

Threshold = vfBadFlowThreshold [4 flows]

Last Flow Evaluated = vfLastFlowToTest [320 flows]

Scoring Valley Filter and TrimBack Valley Filter [Default Value(s)]

Score = Sum (Flow Valley Scores) * vfTrimBackScaleFactor

Threshold = vfBadFlowThreshold / vfLastFlowToTest [4 / 320 = 0.0125]

Last Flow Evaluated = vfScanLimit [700 flows for amplicons, 4096 flows for shotgun]

Valley Filter parameters are described in Table 10.

Valley Filter Parameter	Description	Values	Shotgun/PE	Amplicon
doValleyFilter	Enable/disable all Valley Filters.	true/false	true	true
vfBadFlowThreshold	The maximum number of "bad" flows allowed before a read is discarded by the Counting Valley Filter.	# flows 2-6	4	4
vfLastFlowToTest	The last flow evaluated by the Counting Valley Filter.	flow # 168-400	320	320
vfScanAllFlows	Enable/disable the read rejecting Scoring Valley Filter (default amplicon processing).	true/false tiOnly/flxOnly	false	tiOnly
vfTrimBackScaleFactor	Scale factor (stringency) of the Scoring and TrimBack Valley Filters.	arbitrary number	Junior Kit: 1.6 XLR70 Kit: 0.7 XL+Kit: 2.2	Junior Kit: 3.0 XLR70 Kit: 2.0 XL+Kit: n/a
vfScanLimit	Last flow evaluated by the Scoring and TrimBack Valley Filters (4096 evaluates all flows).	flow #	4096	700
doValleyFilterTrimBack	Enable/disable the read trimming TrimBack Valley Filter (default shotgun processing).	true/false	true	false (n/a)
vfTrimBackMinimumLength	Minimum acceptable read length for the TrimBack Valley Filter.	# flows	84	84
vfMaxFailedPercent	The minimum percentage of “good” flows required before a read is retained by the TrimBack Valley Filter (100 disables this error check).	percentage 90-100	100	100
vfUseRollingWindows	Enable/disable a dynamic estimate of the valley thresholds between the 0, 1, 2, and 3-mers across the run.	true/false	true	true

Table 10: Valley Filter Parameters. A “false” parameter disables the filter, which may increase the length (and possibly the number) of reads, but will likely reduce overall read quality. Filter stringency can be increased by using larger green values or lower red values.

Basecall Quality Score Trimming Filter - Technically, this filter is executed as part of basecalling, but functionally it belongs with the other quality filters. The quality score trimming filter trims back from the 3’ end of reads based on estimated quality scores (not the final quality scores) derived from an internal calibrated signal histogram. The error rate in a sliding window (default size of 40 bp) is calculated form the estimated quality scores, and multiplied by an empirical scaling factor (default of 1.1). The window is slid leftwards until the estimated error rate in the window is less than 1.0% (by default). If the resulting read is less than 40 bp (default) the read is discarded (numTrimmedTooShortQuality metric).

Basecall Quality Score Trimming Filter parameters are described in Table 11.

Quality Score Parameter	Description	Values	Shotgun/PE	Amplicon
doQScoreTrim	Enable/disable Basecall Quality Score Filter.	true/false	true	false
QScoreTrimMinLength	Minimum acceptable read length for Basecall Quality Filter.	# bp	40	(n/a)
QScoreTrimBackScaleFactor	Scale factor (stringency) of the Basecall Quality Score Filter valley calculation.	arbitrary number	Junior Kit: 1.2 XLR70 Kit: 0.9 XL+Kit: 1.5	(n/a)
QScoreTrimNukeWindowSize	Trimming window size.	# bp	40	(n/a)
errorQscoreWindowTrim	Stringency of the Basecall Quality Score Filter trimback window.	fraction	0.010	(n/a)
carryOverPeaks	Apply flowgram signal distribution statistics from earlier flows to later flows.	true/false	false (n/a)	true
flowRadius	Titanium Amplicons experiments only. Flow radius for computing smoothed signal strength distributions.	# flows	(n/a)	32
useCorrectionGlobalLimit	Titanium Amplicons experiments only. Enable/disable a global limit on the SFF flowgram correction limit.	true/false	false (n/a)	true

Table 11: Basecall Quality Score Filter Parameters. A “false” parameter disables the filter, which may increase the length (and possibly the number) of reads, but will likely reduce overall read quality. Filter stringency can be increased by using larger green values or lower red values.