Title: Pstat3

Report generated at Thu Dec 20 19:49:09 PST 2018

Command line arguments

-out_dir data/Pstat3 -title Pstat3 -species mm10 -nth 8 -no_dup_removal -peak_caller macs2 -extsize_macs2 150 -shift_macs2 -75 -idr_thresh 0.05 -filt_bam0 /home/users/avsec/oak/basepair-workflow/data/raw/deduplicated_bam/mesc_stat3_nexus_1_id3679_filtered.bam -filt_bam1 /home/users/avsec/oak/basepair-workflow/data/raw/deduplicated_bam/mesc_pstat3_nexus_2_id3583_filtered.bam -filt_bam2 /home/users/avsec/oak/basepair-workflow/data/raw/deduplicated_bam/mesc_stat3_nexus_3_id3685_filtered.bam

Pipeline version

Latest git commit SHA1: c44dcbf74eb0935a3a663db32cedc8ce35a4eb79 (Mon Dec 10 14:17:30 2018)


Directories and files
Expand all   Collapse all
FilesPath
Alignment
Replicate 1
Filtered & deduped bam /home/users/avsec/oak/basepair-workflow/data/raw/deduplicated_bam/mesc_pstat3_nexus_2_id3583_filtered.bam
Tag-align ./align/rep1/mesc_pstat3_nexus_2_id3583_filtered.tagAlign.gz
Replicate 2
Filtered & deduped bam /home/users/avsec/oak/basepair-workflow/data/raw/deduplicated_bam/mesc_stat3_nexus_3_id3685_filtered.bam
Tag-align ./align/rep2/mesc_stat3_nexus_3_id3685_filtered.tagAlign.gz
Pooled replicate
Tag-align ./align/pooled_rep/mesc_pstat3_nexus_2_id3583_filtered_mesc_stat3_nexus_3_id3685_filtered.tagAlign.gz
Pseudo-replicates
Replicate 1
Pseudo-replicate 1
Tag-align ./align/pseudo_reps/rep1/pr1/mesc_pstat3_nexus_2_id3583_filtered.pr1.tagAlign.gz
Pseudo-replicate 2
Tag-align ./align/pseudo_reps/rep1/pr2/mesc_pstat3_nexus_2_id3583_filtered.pr2.tagAlign.gz
Replicate 2
Pseudo-replicate 1
Tag-align ./align/pseudo_reps/rep2/pr1/mesc_stat3_nexus_3_id3685_filtered.pr1.tagAlign.gz
Pseudo-replicate 2
Tag-align ./align/pseudo_reps/rep2/pr2/mesc_stat3_nexus_3_id3685_filtered.pr2.tagAlign.gz
Pooled pseudo-replicates
Pooled pseudo-replicate 1
Tag-align ./align/pooled_pseudo_reps/ppr1/mesc_pstat3_nexus_2_id3583_filtered.pr1_pooled.tagAlign.gz
Pooled pseudo-replicate 2
Tag-align ./align/pooled_pseudo_reps/ppr2/mesc_pstat3_nexus_2_id3583_filtered.pr2_pooled.tagAlign.gz
Signal tracks
MACS2
Replicate 1
P-value ./signal/macs2/rep1/mesc_pstat3_nexus_2_id3583_filtered.pval.signal.bw
Fold enrichment ./signal/macs2/rep1/mesc_pstat3_nexus_2_id3583_filtered.fc.signal.bw
Replicate 2
P-value ./signal/macs2/rep2/mesc_stat3_nexus_3_id3685_filtered.pval.signal.bw
Fold enrichment ./signal/macs2/rep2/mesc_stat3_nexus_3_id3685_filtered.fc.signal.bw
Pooled replicate
P-value ./signal/macs2/pooled_rep/mesc_pstat3_nexus_2_id3583_filtered_mesc_stat3_nexus_3_id3685_filtered.pval.signal.bw
Fold enrichment ./signal/macs2/pooled_rep/mesc_pstat3_nexus_2_id3583_filtered_mesc_stat3_nexus_3_id3685_filtered.fc.signal.bw
Peaks
MACS2
Replicate 1
Narrow peak ./peak/macs2/rep1/mesc_pstat3_nexus_2_id3583_filtered.pval0.01.500K.narrowPeak.gz
Replicate 2
Narrow peak ./peak/macs2/rep2/mesc_stat3_nexus_3_id3685_filtered.pval0.01.500K.narrowPeak.gz
Pooled replicate
Narrow peak ./peak/macs2/pooled_rep/mesc_pstat3_nexus_2_id3583_filtered_mesc_stat3_nexus_3_id3685_filtered.pval0.01.500K.narrowPeak.gz
Pseudo-replicates
Replicate 1
Pseudo-replicate 1
Narrow peak ./peak/macs2/pseudo_reps/rep1/pr1/mesc_pstat3_nexus_2_id3583_filtered.pr1.pval0.01.500K.narrowPeak.gz
Pseudo-replicate 2
Narrow peak ./peak/macs2/pseudo_reps/rep1/pr2/mesc_pstat3_nexus_2_id3583_filtered.pr2.pval0.01.500K.narrowPeak.gz
Replicate 2
Pseudo-replicate 1
Narrow peak ./peak/macs2/pseudo_reps/rep2/pr1/mesc_stat3_nexus_3_id3685_filtered.pr1.pval0.01.500K.narrowPeak.gz
Pseudo-replicate 2
Narrow peak ./peak/macs2/pseudo_reps/rep2/pr2/mesc_stat3_nexus_3_id3685_filtered.pr2.pval0.01.500K.narrowPeak.gz
Pooled pseudo-replicate
Pooled pseudo-replicate 1
Narrow peak ./peak/macs2/pooled_pseudo_reps/ppr1/mesc_pstat3_nexus_2_id3583_filtered.pr1_pooled.pval0.01.500K.narrowPeak.gz
Pooled pseudo-replicate 2
Narrow peak ./peak/macs2/pooled_pseudo_reps/ppr2/mesc_pstat3_nexus_2_id3583_filtered.pr2_pooled.pval0.01.500K.narrowPeak.gz
Naive overlap
True replicates
Rep. 1 vs. Rep. 2
Overlapping peak ./peak/macs2/overlap/true_reps/rep1-rep2/Pstat3_rep1-rep2.naive_overlap.filt.narrowPeak.gz
Pseudo-replicates
Replicate 1
Overlapping peak ./peak/macs2/overlap/pseudo_reps/rep1/Pstat3_rep1-pr.naive_overlap.filt.narrowPeak.gz
Replicate 2
Overlapping peak ./peak/macs2/overlap/pseudo_reps/rep2/Pstat3_rep2-pr.naive_overlap.filt.narrowPeak.gz
Pooled pseudo-replicates
Overlapping peak ./peak/macs2/overlap/pooled_pseudo_reps/Pstat3_ppr.naive_overlap.filt.narrowPeak.gz
Optimal set
Overlapping peak ./peak/macs2/overlap/optimal_set/Pstat3_ppr.naive_overlap.filt.narrowPeak.gz
Conservative set
Overlapping peak ./peak/macs2/overlap/conservative_set/Pstat3_rep1-rep2.naive_overlap.filt.narrowPeak.gz
IDR
True replicates
Rep. 1 vs. Rep. 2
IDR peak ./peak/macs2/idr/true_reps/rep1-rep2/Pstat3_rep1-rep2.IDR0.05.filt.narrowPeak.gz
Pseudo-replicates
Replicate 1
IDR peak ./peak/macs2/idr/pseudo_reps/rep1/Pstat3_rep1-pr.IDR0.05.filt.narrowPeak.gz
Replicate 2
IDR peak ./peak/macs2/idr/pseudo_reps/rep2/Pstat3_rep2-pr.IDR0.05.filt.narrowPeak.gz
Pooled pseudo-replicates
IDR peak ./peak/macs2/idr/pooled_pseudo_reps/Pstat3_ppr.IDR0.05.filt.narrowPeak.gz
Optimal set
IDR peak ./peak/macs2/idr/optimal_set/Pstat3_ppr.IDR0.05.filt.narrowPeak.gz
Conservative set
IDR peak ./peak/macs2/idr/conservative_set/Pstat3_rep1-rep2.IDR0.05.filt.narrowPeak.gz
QC and logs
Replicate 1
Cross-corr. log ./qc/rep1/mesc_pstat3_nexus_2_id3583_filtered.no_chrM.15M.cc.qc
Cross-corr. plot ./qc/rep1/mesc_pstat3_nexus_2_id3583_filtered.no_chrM.15M.cc.plot.pdf
Replicate 2
Cross-corr. log ./qc/rep2/mesc_stat3_nexus_3_id3685_filtered.no_chrM.15M.cc.qc
Cross-corr. plot ./qc/rep2/mesc_stat3_nexus_3_id3685_filtered.no_chrM.15M.cc.plot.pdf
Naive overlap
Overlap QC log ./qc/Pstat3_peak_overlap_final.qc
True replicates
Rep. 1 vs. Rep. 2
FRiP ./peak/macs2/overlap/true_reps/rep1-rep2/Pstat3_rep1-rep2.naive_overlap.filt.narrowPeak.FRiP.qc
Pseudo-replicates
Replicate 1
FRiP ./peak/macs2/overlap/pseudo_reps/rep1/Pstat3_rep1-pr.naive_overlap.filt.narrowPeak.FRiP.qc
Replicate 2
FRiP ./peak/macs2/overlap/pseudo_reps/rep2/Pstat3_rep2-pr.naive_overlap.filt.narrowPeak.FRiP.qc
IDR
IDR QC log ./qc/Pstat3_IDR_final.qc
True replicates
Rep. 1 vs. Rep. 2
FRiP ./peak/macs2/idr/true_reps/rep1-rep2/Pstat3_rep1-rep2.IDR0.05.filt.narrowPeak.FRiP.qc
IDR plot ./peak/macs2/idr/true_reps/rep1-rep2/Pstat3_rep1-rep2.unthresholded-peaks.txt.png
Pseudo-replicates
Replicate 1
FRiP ./peak/macs2/idr/pseudo_reps/rep1/Pstat3_rep1-pr.IDR0.05.filt.narrowPeak.FRiP.qc
IDR plot ./peak/macs2/idr/pseudo_reps/rep1/Pstat3_rep1-pr.unthresholded-peaks.txt.png
Replicate 2
FRiP ./peak/macs2/idr/pseudo_reps/rep2/Pstat3_rep2-pr.IDR0.05.filt.narrowPeak.FRiP.qc
IDR plot ./peak/macs2/idr/pseudo_reps/rep2/Pstat3_rep2-pr.unthresholded-peaks.txt.png
Pooled pseudo-replicates
IDR plot ./peak/macs2/idr/pooled_pseudo_reps/Pstat3_ppr.unthresholded-peaks.txt.png

Visualization (add -url_base [URL_ROOT_DIR_FOR_OUT_DIR] to the command line.)
Visualize  JSON (datahub)



Workflow diagram (g./b. peak : gapped/broad peak from MACS2, n. peak : narrow peak from MACS2, r. peak : relaxed peak from SPP)


Enrichment QC (strand cross-correlation measures)
 rep1
rep2
Reads1500000015000000
Est. Fragment Len.-5-20
Corr. Est. Fragment Len.0.2121489904290310.220127191885773
Phantom Peak4040
Corr. Phantom Peak0.21739740.2247196
Argmin. Corr.15001500
Min. Corr.0.20003820.2039115
NSC1.0605421.079523
RSC0.69765610.7792955

NOTE: Reads from replicates are subsampled to a max of 15M

  • Normalized strand cross-correlation coefficient (NSC) = col9 in outFile
  • Relative strand cross-correlation coefficient (RSC) = col10 in outFile
  • Estimated fragment length = col3 in outFile, take the top value


rep1
 
rep2
 
Number of peaks
rep147782
rep1-pr135531
rep1-pr235548
rep250769
rep2-pr137777
rep2-pr237656
pooled67051
ppr147954
ppr248600

  • ppr1: Raw peaks called on the first pooled pseudoreplicates
  • ppr2: Raw peaks called on the second pooled pseudoreplicates
  • repi: Raw peaks called on true replicate i
  • repi-pr1 : Raw peaks called on the first pseudoreplicate from replicate i
  • repi-pr2 : Raw peaks called on the second pseudoreplicates from replicate i
  • overlap : Overlapped peaks (filtered if blacklist exists)


Enrichment QC (Fraction of reads in raw peaks)
 pooled_rep
rep1
rep2
Fraction of Reads in Peak0.08714270.0659060.0795672
Enrichment QC (Fraction of reads in overlapping peaks)
 rep1-pr
rep1-rep2
rep2-pr
Fraction of Reads in Peak0.04760260.06634460.0589734

  • ppr: Overlapping peaks comparing pooled pseudo replicates
  • rep1-pr: Overlapping peaks comparing pseudoreplicates from replicate 1
  • rep2-pr: Overlapping peaks comparing pseudoreplicates from replicate 2
  • repi-repj: Overlapping peaks comparing true replicates (rep i vs. rep j)


Reproducibility QC and Peak Detection Statistics (Overlapping peaks)
Nt23397
N115086
N216632
Np24149
N optimal24149
N conservative23397
Optimal Setpooled_pseudo_rep
Conservative Setrep1-rep2
Rescue Ratio1.0321408727614652
Self Consistency Ratio1.1024791197136419
Reproducibility Testpass

  • N1: Replicate 1 self-consistent overlapping peaks (comparing two pseudoreplicates generated by subsampling Rep1 reads)
  • N2: Replicate 2 self-consistent overlapping peaks (comparing two pseudoreplicates generated by subsampling Rep2 reads)
  • Nt: True Replicate consisten overlapping peaks (comparing true replicates Rep1 vs Rep2 )
  • Np: Pooled-pseudoreplicate consistent overlapping peaks (comparing two pseudoreplicates generated by subsampling pooled reads from Rep1 and Rep2 )
  • Self-consistency Ratio: max(N1,N2) / min (N1,N2)
  • Rescue Ratio: max(Np,Nt) / min (Np,Nt)
  • Reproducibility Test: If Self-consistency Ratio >2 AND Rescue Ratio > 2, then 'Fail' else 'Pass'


Enrichment QC (Fraction of reads in IDR peaks)
 rep1-pr
rep1-rep2
rep2-pr
Fraction of Reads in Peak0.03169660.04647180.0391315

  • ppr: IDR peaks comparing pooled pseudo replicates
  • rep1-pr: IDR peaks comparing pseudoreplicates from replicate 1
  • rep2-pr: IDR peaks comparing pseudoreplicates from replicate 2
  • repi-repj: IDR peaks comparing true replicates (rep i vs. rep j)


Reproducibility QC and Peak Detection Statistics (Irreproducible Discovery Rate)
Nt8877
N15213
N25642
Np9264
N optimal9264
N conservative8877
Optimal Setpooled_pseudo_rep
Conservative Setrep1-rep2
Rescue Ratio1.043595809395066
Self Consistency Ratio1.0822942643391522
Reproducibility Testpass

  • N1: Replicate 1 self-consistent IDR 0.05 peaks (comparing two pseudoreplicates generated by subsampling Rep1 reads)
  • N2: Replicate 2 self-consistent IDR 0.05 peaks (comparing two pseudoreplicates generated by subsampling Rep2 reads)
  • Nt: True Replicate consistent IDR 0.05 peaks (comparing true replicates Rep1 vs Rep2 )
  • Np: Pooled-pseudoreplicate consistent IDR 0.05 peaks (comparing two pseudoreplicates generated by subsampling pooled reads from Rep1 and Rep2 )
  • Self-consistency Ratio: max(N1,N2) / min (N1,N2)
  • Rescue Ratio: max(Np,Nt) / min (Np,Nt)
  • Reproducibility Test: If Self-consistency Ratio >2 AND Rescue Ratio > 2, then 'Fail' else 'Pass'


true reps (rep1-rep2)
 
pooled pseudo-reps
 
rep1 pseudo-reps
 
rep2 pseudo-reps