Title: Oct4

Report generated at Thu Jun 7 04:42:10 PDT 2018

Command line arguments

-species mm10 -filt_bam0 /srv/scratch/shared/surya/raunaq/chip-nexus/data-stowers/semi_processed/Oct4/Oct4_1234_1/out/align/rep1/Oct4_1b_matched_barcode.filt_filtered.bam -filt_bam1 /srv/scratch/shared/surya/raunaq/chip-nexus/data-stowers/semi_processed/Oct4/Oct4_1234_1/out/align/rep2/Oct4_2c_matched_barcode.filt_filtered.bam -filt_bam2 /srv/scratch/shared/surya/raunaq/chip-nexus/data-stowers/semi_processed/Oct4/Oct4_1234_1/out/align/rep3/Oct4_3b_matched_barcode.filt_filtered.bam -filt_bam3 /srv/scratch/shared/surya/raunaq/chip-nexus/data-stowers/semi_processed/Oct4/Oct4_1234_1/out/align/rep4/Oct4_4b_matched_barcode.filt_filtered.bam -ctl_filt_bam0 /srv/scratch/shared/surya/raunaq/chip-nexus/data-stowers/semi_processed/Oct4/Oct4_1234_1/out/align/ctl1/mESC_WCE_1.filt.bam -peak_caller macs2 -shift_macs2 -75 -extsize_macs2 150 -nth 8 -idr_thresh 0.05 -no_dup_removal -out_dir data/Oct4 -title Oct4

Pipeline version

Latest git commit SHA1: 57cb2ff2e9cedfcf961680c36b37fe219ccf9c98 (Tue May 29 09:47:38 2018)


Directories and files
Expand all   Collapse all
FilesPath
Alignment
Replicate 1
Filtered & deduped bam /srv/scratch/shared/surya/raunaq/chip-nexus/data-stowers/semi_processed/Oct4/Oct4_1234_1/out/align/rep2/Oct4_2c_matched_barcode.filt_filtered.bam
Tag-align ./align/rep1/Oct4_2c_matched_barcode.filt_filtered.tagAlign.gz
Replicate 2
Filtered & deduped bam /srv/scratch/shared/surya/raunaq/chip-nexus/data-stowers/semi_processed/Oct4/Oct4_1234_1/out/align/rep3/Oct4_3b_matched_barcode.filt_filtered.bam
Tag-align ./align/rep2/Oct4_3b_matched_barcode.filt_filtered.tagAlign.gz
Replicate 3
Filtered & deduped bam /srv/scratch/shared/surya/raunaq/chip-nexus/data-stowers/semi_processed/Oct4/Oct4_1234_1/out/align/rep4/Oct4_4b_matched_barcode.filt_filtered.bam
Tag-align ./align/rep3/Oct4_4b_matched_barcode.filt_filtered.tagAlign.gz
Pooled replicate
Tag-align ./align/pooled_rep/Oct4_2c_matched_barcode.filt_filtered_pooled.tagAlign.gz
Pseudo-replicates
Replicate 1
Pseudo-replicate 1
Tag-align ./align/pseudo_reps/rep1/pr1/Oct4_2c_matched_barcode.filt_filtered.pr1.tagAlign.gz
Pseudo-replicate 2
Tag-align ./align/pseudo_reps/rep1/pr2/Oct4_2c_matched_barcode.filt_filtered.pr2.tagAlign.gz
Replicate 2
Pseudo-replicate 1
Tag-align ./align/pseudo_reps/rep2/pr1/Oct4_3b_matched_barcode.filt_filtered.pr1.tagAlign.gz
Pseudo-replicate 2
Tag-align ./align/pseudo_reps/rep2/pr2/Oct4_3b_matched_barcode.filt_filtered.pr2.tagAlign.gz
Replicate 3
Pseudo-replicate 1
Tag-align ./align/pseudo_reps/rep3/pr1/Oct4_4b_matched_barcode.filt_filtered.pr1.tagAlign.gz
Pseudo-replicate 2
Tag-align ./align/pseudo_reps/rep3/pr2/Oct4_4b_matched_barcode.filt_filtered.pr2.tagAlign.gz
Pooled pseudo-replicates
Pooled pseudo-replicate 1
Tag-align ./align/pooled_pseudo_reps/ppr1/Oct4_2c_matched_barcode.filt_filtered.pr1_pooled.tagAlign.gz
Pooled pseudo-replicate 2
Tag-align ./align/pooled_pseudo_reps/ppr2/Oct4_2c_matched_barcode.filt_filtered.pr2_pooled.tagAlign.gz
Signal tracks
MACS2
Replicate 1
P-value ./signal/macs2/rep1/Oct4_2c_matched_barcode.filt_filtered.pval.signal.bw
Fold enrichment ./signal/macs2/rep1/Oct4_2c_matched_barcode.filt_filtered.fc.signal.bw
Replicate 2
P-value ./signal/macs2/rep2/Oct4_3b_matched_barcode.filt_filtered.pval.signal.bw
Fold enrichment ./signal/macs2/rep2/Oct4_3b_matched_barcode.filt_filtered.fc.signal.bw
Replicate 3
P-value ./signal/macs2/rep3/Oct4_4b_matched_barcode.filt_filtered.pval.signal.bw
Fold enrichment ./signal/macs2/rep3/Oct4_4b_matched_barcode.filt_filtered.fc.signal.bw
Pooled replicate
P-value ./signal/macs2/pooled_rep/Oct4_2c_matched_barcode.filt_filtered_pooled.pval.signal.bw
Fold enrichment ./signal/macs2/pooled_rep/Oct4_2c_matched_barcode.filt_filtered_pooled.fc.signal.bw
Peaks
MACS2
Replicate 1
Narrow peak ./peak/macs2/rep1/Oct4_2c_matched_barcode.filt_filtered.pval0.01.500K.narrowPeak.gz
Replicate 2
Narrow peak ./peak/macs2/rep2/Oct4_3b_matched_barcode.filt_filtered.pval0.01.500K.narrowPeak.gz
Replicate 3
Narrow peak ./peak/macs2/rep3/Oct4_4b_matched_barcode.filt_filtered.pval0.01.500K.narrowPeak.gz
Pooled replicate
Narrow peak ./peak/macs2/pooled_rep/Oct4_2c_matched_barcode.filt_filtered_pooled.pval0.01.500K.narrowPeak.gz
Pseudo-replicates
Replicate 1
Pseudo-replicate 1
Narrow peak ./peak/macs2/pseudo_reps/rep1/pr1/Oct4_2c_matched_barcode.filt_filtered.pr1.pval0.01.500K.narrowPeak.gz
Pseudo-replicate 2
Narrow peak ./peak/macs2/pseudo_reps/rep1/pr2/Oct4_2c_matched_barcode.filt_filtered.pr2.pval0.01.500K.narrowPeak.gz
Replicate 2
Pseudo-replicate 1
Narrow peak ./peak/macs2/pseudo_reps/rep2/pr1/Oct4_3b_matched_barcode.filt_filtered.pr1.pval0.01.500K.narrowPeak.gz
Pseudo-replicate 2
Narrow peak ./peak/macs2/pseudo_reps/rep2/pr2/Oct4_3b_matched_barcode.filt_filtered.pr2.pval0.01.500K.narrowPeak.gz
Replicate 3
Pseudo-replicate 1
Narrow peak ./peak/macs2/pseudo_reps/rep3/pr1/Oct4_4b_matched_barcode.filt_filtered.pr1.pval0.01.500K.narrowPeak.gz
Pseudo-replicate 2
Narrow peak ./peak/macs2/pseudo_reps/rep3/pr2/Oct4_4b_matched_barcode.filt_filtered.pr2.pval0.01.500K.narrowPeak.gz
Pooled pseudo-replicate
Pooled pseudo-replicate 1
Narrow peak ./peak/macs2/pooled_pseudo_reps/ppr1/Oct4_2c_matched_barcode.filt_filtered.pr1_pooled.pval0.01.500K.narrowPeak.gz
Pooled pseudo-replicate 2
Narrow peak ./peak/macs2/pooled_pseudo_reps/ppr2/Oct4_2c_matched_barcode.filt_filtered.pr2_pooled.pval0.01.500K.narrowPeak.gz
Naive overlap
True replicates
Rep. 1 vs. Rep. 2
Overlapping peak ./peak/macs2/overlap/true_reps/rep1-rep2/Oct4_rep1-rep2.naive_overlap.filt.narrowPeak.gz
Rep. 1 vs. Rep. 3
Overlapping peak ./peak/macs2/overlap/true_reps/rep1-rep3/Oct4_rep1-rep3.naive_overlap.filt.narrowPeak.gz
Rep. 2 vs. Rep. 3
Overlapping peak ./peak/macs2/overlap/true_reps/rep2-rep3/Oct4_rep2-rep3.naive_overlap.filt.narrowPeak.gz
Pseudo-replicates
Replicate 1
Overlapping peak ./peak/macs2/overlap/pseudo_reps/rep1/Oct4_rep1-pr.naive_overlap.filt.narrowPeak.gz
Replicate 2
Overlapping peak ./peak/macs2/overlap/pseudo_reps/rep2/Oct4_rep2-pr.naive_overlap.filt.narrowPeak.gz
Replicate 3
Overlapping peak ./peak/macs2/overlap/pseudo_reps/rep3/Oct4_rep3-pr.naive_overlap.filt.narrowPeak.gz
Pooled pseudo-replicates
Overlapping peak ./peak/macs2/overlap/pooled_pseudo_reps/Oct4_ppr.naive_overlap.filt.narrowPeak.gz
Optimal set
Overlapping peak ./peak/macs2/overlap/optimal_set/Oct4_ppr.naive_overlap.filt.narrowPeak.gz
Conservative set
Overlapping peak ./peak/macs2/overlap/conservative_set/Oct4_rep1-rep2.naive_overlap.filt.narrowPeak.gz
IDR
True replicates
Rep. 1 vs. Rep. 2
IDR peak ./peak/macs2/idr/true_reps/rep1-rep2/Oct4_rep1-rep2.IDR0.05.filt.narrowPeak.gz
Rep. 1 vs. Rep. 3
IDR peak ./peak/macs2/idr/true_reps/rep1-rep3/Oct4_rep1-rep3.IDR0.05.filt.narrowPeak.gz
Rep. 2 vs. Rep. 3
IDR peak ./peak/macs2/idr/true_reps/rep2-rep3/Oct4_rep2-rep3.IDR0.05.filt.narrowPeak.gz
Pseudo-replicates
Replicate 1
IDR peak ./peak/macs2/idr/pseudo_reps/rep1/Oct4_rep1-pr.IDR0.05.filt.narrowPeak.gz
Replicate 2
IDR peak ./peak/macs2/idr/pseudo_reps/rep2/Oct4_rep2-pr.IDR0.05.filt.narrowPeak.gz
Replicate 3
IDR peak ./peak/macs2/idr/pseudo_reps/rep3/Oct4_rep3-pr.IDR0.05.filt.narrowPeak.gz
Pooled pseudo-replicates
IDR peak ./peak/macs2/idr/pooled_pseudo_reps/Oct4_ppr.IDR0.05.filt.narrowPeak.gz
Optimal set
IDR peak ./peak/macs2/idr/optimal_set/Oct4_ppr.IDR0.05.filt.narrowPeak.gz
Conservative set
IDR peak ./peak/macs2/idr/conservative_set/Oct4_rep2-rep3.IDR0.05.filt.narrowPeak.gz
QC and logs
Replicate 1
Cross-corr. log ./qc/rep1/Oct4_2c_matched_barcode.filt_filtered.no_chrM.15M.cc.qc
Cross-corr. plot ./qc/rep1/Oct4_2c_matched_barcode.filt_filtered.no_chrM.15M.cc.plot.pdf
Replicate 2
Cross-corr. log ./qc/rep2/Oct4_3b_matched_barcode.filt_filtered.no_chrM.15M.cc.qc
Cross-corr. plot ./qc/rep2/Oct4_3b_matched_barcode.filt_filtered.no_chrM.15M.cc.plot.pdf
Replicate 3
Cross-corr. log ./qc/rep3/Oct4_4b_matched_barcode.filt_filtered.no_chrM.15M.cc.qc
Cross-corr. plot ./qc/rep3/Oct4_4b_matched_barcode.filt_filtered.no_chrM.15M.cc.plot.pdf
Naive overlap
Overlap QC log ./qc/Oct4_peak_overlap_final.qc
True replicates
Rep. 1 vs. Rep. 2
FRiP ./peak/macs2/overlap/true_reps/rep1-rep2/Oct4_rep1-rep2.naive_overlap.filt.narrowPeak.FRiP.qc
Rep. 1 vs. Rep. 3
FRiP ./peak/macs2/overlap/true_reps/rep1-rep3/Oct4_rep1-rep3.naive_overlap.filt.narrowPeak.FRiP.qc
Rep. 2 vs. Rep. 3
FRiP ./peak/macs2/overlap/true_reps/rep2-rep3/Oct4_rep2-rep3.naive_overlap.filt.narrowPeak.FRiP.qc
Pseudo-replicates
Replicate 1
FRiP ./peak/macs2/overlap/pseudo_reps/rep1/Oct4_rep1-pr.naive_overlap.filt.narrowPeak.FRiP.qc
Replicate 2
FRiP ./peak/macs2/overlap/pseudo_reps/rep2/Oct4_rep2-pr.naive_overlap.filt.narrowPeak.FRiP.qc
Replicate 3
FRiP ./peak/macs2/overlap/pseudo_reps/rep3/Oct4_rep3-pr.naive_overlap.filt.narrowPeak.FRiP.qc
IDR
IDR QC log ./qc/Oct4_IDR_final.qc
True replicates
Rep. 1 vs. Rep. 2
FRiP ./peak/macs2/idr/true_reps/rep1-rep2/Oct4_rep1-rep2.IDR0.05.filt.narrowPeak.FRiP.qc
IDR plot ./peak/macs2/idr/true_reps/rep1-rep2/Oct4_rep1-rep2.unthresholded-peaks.txt.png
Rep. 1 vs. Rep. 3
FRiP ./peak/macs2/idr/true_reps/rep1-rep3/Oct4_rep1-rep3.IDR0.05.filt.narrowPeak.FRiP.qc
IDR plot ./peak/macs2/idr/true_reps/rep1-rep3/Oct4_rep1-rep3.unthresholded-peaks.txt.png
Rep. 2 vs. Rep. 3
FRiP ./peak/macs2/idr/true_reps/rep2-rep3/Oct4_rep2-rep3.IDR0.05.filt.narrowPeak.FRiP.qc
IDR plot ./peak/macs2/idr/true_reps/rep2-rep3/Oct4_rep2-rep3.unthresholded-peaks.txt.png
Pseudo-replicates
Replicate 1
FRiP ./peak/macs2/idr/pseudo_reps/rep1/Oct4_rep1-pr.IDR0.05.filt.narrowPeak.FRiP.qc
IDR plot ./peak/macs2/idr/pseudo_reps/rep1/Oct4_rep1-pr.unthresholded-peaks.txt.png
Replicate 2
FRiP ./peak/macs2/idr/pseudo_reps/rep2/Oct4_rep2-pr.IDR0.05.filt.narrowPeak.FRiP.qc
IDR plot ./peak/macs2/idr/pseudo_reps/rep2/Oct4_rep2-pr.unthresholded-peaks.txt.png
Replicate 3
FRiP ./peak/macs2/idr/pseudo_reps/rep3/Oct4_rep3-pr.IDR0.05.filt.narrowPeak.FRiP.qc
IDR plot ./peak/macs2/idr/pseudo_reps/rep3/Oct4_rep3-pr.unthresholded-peaks.txt.png
Pooled pseudo-replicates
IDR plot ./peak/macs2/idr/pooled_pseudo_reps/Oct4_ppr.unthresholded-peaks.txt.png

Visualization (add -url_base [URL_ROOT_DIR_FOR_OUT_DIR] to the command line.)
Visualize  JSON (datahub)



Workflow diagram (g./b. peak : gapped/broad peak from MACS2, n. peak : narrow peak from MACS2, r. peak : relaxed peak from SPP)


Enrichment QC (strand cross-correlation measures)
 rep1
rep2
rep3
Reads150000001500000015000000
Est. Fragment Len.-5-15-10
Corr. Est. Fragment Len.0.2008077424100670.1985169094931710.198844662539728
Phantom Peak606060
Corr. Phantom Peak0.20347760.19905070.1995663
Argmin. Corr.150015001500
Min. Corr.0.19156630.1923430.1918844
NSC1.0482411.0320981.036273
RSC0.77585680.92041580.9060609

NOTE: Reads from replicates are subsampled to a max of 15M

  • Normalized strand cross-correlation coefficient (NSC) = col9 in outFile
  • Relative strand cross-correlation coefficient (RSC) = col10 in outFile
  • Estimated fragment length = col3 in outFile, take the top value


rep1
 
rep2
 
rep3
 
Number of peaks
rep177437
rep1-pr171024
rep1-pr269513
rep281228
rep2-pr161678
rep2-pr261415
rep373493
rep3-pr168031
rep3-pr268938
pooled112693
ppr186618
ppr286136

  • ppr1: Raw peaks called on the first pooled pseudoreplicates
  • ppr2: Raw peaks called on the second pooled pseudoreplicates
  • repi: Raw peaks called on true replicate i
  • repi-pr1 : Raw peaks called on the first pseudoreplicate from replicate i
  • repi-pr2 : Raw peaks called on the second pseudoreplicates from replicate i
  • overlap : Overlapped peaks (filtered if blacklist exists)


Enrichment QC (Fraction of reads in raw peaks)
 pooled_rep
rep1
rep2
rep3
Fraction of Reads in Peak0.07341920.06790080.05411690.0543243
Enrichment QC (Fraction of reads in overlapping peaks)
 rep1-pr
rep1-rep2
rep1-rep3
rep2-pr
rep2-rep3
rep3-pr
Fraction of Reads in Peak0.04579790.04957450.04926360.03476840.04952760.0382074

  • ppr: Overlapping peaks comparing pooled pseudo replicates
  • rep1-pr: Overlapping peaks comparing pseudoreplicates from replicate 1
  • rep2-pr: Overlapping peaks comparing pseudoreplicates from replicate 2
  • repi-repj: Overlapping peaks comparing true replicates (rep i vs. rep j)


Reproducibility QC and Peak Detection Statistics (Overlapping peaks)
Nt40785
N128973
N226275
Np51097
N optimal51097
N conservative40785
Optimal Setpooled_pseudo_rep
Conservative Setrep1-rep2
Rescue Ratio1.2528380532058354
Self Consistency Ratio1.1026831588962893
Reproducibility Testpass

  • N1: Replicate 1 self-consistent overlapping peaks (comparing two pseudoreplicates generated by subsampling Rep1 reads)
  • N2: Replicate 2 self-consistent overlapping peaks (comparing two pseudoreplicates generated by subsampling Rep2 reads)
  • Nt: True Replicate consisten overlapping peaks (comparing true replicates Rep1 vs Rep2 )
  • Np: Pooled-pseudoreplicate consistent overlapping peaks (comparing two pseudoreplicates generated by subsampling pooled reads from Rep1 and Rep2 )
  • Self-consistency Ratio: max(N1,N2) / min (N1,N2)
  • Rescue Ratio: max(Np,Nt) / min (Np,Nt)
  • Reproducibility Test: If Self-consistency Ratio >2 AND Rescue Ratio > 2, then 'Fail' else 'Pass'


Enrichment QC (Fraction of reads in IDR peaks)
 rep1-pr
rep1-rep2
rep1-rep3
rep2-pr
rep2-rep3
rep3-pr
Fraction of Reads in Peak0.02952250.03317190.03262530.02360130.03349760.0268596

  • ppr: IDR peaks comparing pooled pseudo replicates
  • rep1-pr: IDR peaks comparing pseudoreplicates from replicate 1
  • rep2-pr: IDR peaks comparing pseudoreplicates from replicate 2
  • repi-repj: IDR peaks comparing true replicates (rep i vs. rep j)


Reproducibility QC and Peak Detection Statistics (Irreproducible Discovery Rate)
Nt17257
N111146
N210947
Np21841
N optimal21841
N conservative17257
Optimal Setpooled_pseudo_rep
Conservative Setrep2-rep3
Rescue Ratio1.265631338007765
Self Consistency Ratio1.1160135196857586
Reproducibility Testpass

  • N1: Replicate 1 self-consistent IDR 0.05 peaks (comparing two pseudoreplicates generated by subsampling Rep1 reads)
  • N2: Replicate 2 self-consistent IDR 0.05 peaks (comparing two pseudoreplicates generated by subsampling Rep2 reads)
  • Nt: True Replicate consistent IDR 0.05 peaks (comparing true replicates Rep1 vs Rep2 )
  • Np: Pooled-pseudoreplicate consistent IDR 0.05 peaks (comparing two pseudoreplicates generated by subsampling pooled reads from Rep1 and Rep2 )
  • Self-consistency Ratio: max(N1,N2) / min (N1,N2)
  • Rescue Ratio: max(Np,Nt) / min (Np,Nt)
  • Reproducibility Test: If Self-consistency Ratio >2 AND Rescue Ratio > 2, then 'Fail' else 'Pass'


true reps (rep1-rep2)
 
true reps (rep1-rep3)
 
true reps (rep2-rep3)
 
pooled pseudo-reps
 
rep1 pseudo-reps
 
rep2 pseudo-reps
 
rep3 pseudo-reps