Script file	/users/pangwei/bds_atac/atac.bds
Program ID	atac.bds.20160517_165228_260
Start time	2016-05-17 16:52:28
Run time	04:51:45.512
Tasks executed	16
Tasks failed	0
Tasks failed names
Arguments*	[-out_dir, /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10, -num_rep, 2, -fastq1_1, /srv/scratch/pangwei/mesoderm/data/ATACseq/ATAC9/ATAC9-trimmed-pair1.fastq.gz, -fastq1_2, /srv/scratch/pangwei/mesoderm/data/ATACseq/ATAC9/ATAC9-trimmed-pair2.fastq.gz, -fastq2_1, /srv/scratch/pangwei/mesoderm/data/ATACseq/ATAC10/ATAC10-trimmed-pair1.fastq.gz, -fastq2_2, /srv/scratch/pangwei/mesoderm/data/ATACseq/ATAC10/ATAC10-trimmed-pair2.fastq.gz, -subsample, 17500000, -true_rep, -species, hg19]
System*	local
Cpus*	1
Exit value	0

Thread ID	Parent	Tasks
thread_Root	None	atac.bds.20160517_165228_260/task.callpeak_etc.naive_overlap_thresh_peak.line_136.id_6 atac.bds.20160517_165228_260/task.callpeak_etc.naive_overlap_thresh_gpeak.line_136.id_7 atac.bds.20160517_165228_260/task.idr.idr2_rep1_rep2.line_72.id_8 atac.bds.20160517_165228_260/task.base.line_414.id_11 atac.bds.20160517_165228_260/task.base.line_414.id_12 atac.bds.20160517_165228_260/task.base.line_414.id_13 atac.bds.20160517_165228_260/task.base.line_414.id_14 atac.bds.20160517_165228_260/task.idr.idr_final_qc.line_210.id_15 atac.bds.20160517_165228_260/task.report.line_417.id_20 atac.bds.20160517_165228_260/task.report.line_417.id_21 atac.bds.20160517_165228_260/task.report.line_417.id_22 atac.bds.20160517_165228_260/task.report_graphviz.line_91.id_23
thread_20	thread_Root
thread_40	thread_Root	atac.bds.20160517_165228_260_parallel_40/task.postalign_bam.srt_bam_rep1.line_345.id_16 atac.bds.20160517_165228_260_parallel_40/task.atac.ataqc_rep1.line_846.id_18
thread_21	thread_Root
thread_41	thread_Root	atac.bds.20160517_165228_260_parallel_41/task.postalign_bam.srt_bam_rep2.line_345.id_17 atac.bds.20160517_165228_260_parallel_41/task.atac.ataqc_rep2.line_846.id_19

Task details

Task

Execution

Time

Dependencies

Task program, Errors, StdOut / StdErr

Num	1
ID	task.callpeak_etc.naive_overlap_thresh_peak.line_136.id_6
Name	naive_overlap_thresh peak
Thread	thread_Root
PID	69537

OK	true
Exit Code	0
Retries
State	FINISHED
Dep.	OK
Cpus	1
Mem

Start	2016-05-17 16:52:29
End	2016-05-17 16:52:50
Elapsed	00:00:21
Timeout	00:00:-1
Wall Timeout	100 days

Input files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/pooled_rep/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.narrowPeak.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/rep1/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.pval0.1.500000.narrowPeak.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/rep2/ATAC10-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.pval0.1.500000.narrowPeak.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata
Output files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.narrowPeak.gz
Dependencies

 
# SYS command. line 138

 if [[ -f $(which activate) ]]; then source activate bds_atac; fi;  export PATH=/users/pangwei/bds_atac/.:/users/pangwei/bds_atac/modules:/users/pangwei/bds_atac/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds;

# SYS command. line 141

 intersectBed -wo -a /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/pooled_rep/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.narrowPeak.gz -b /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/rep1/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.pval0.1.500000.narrowPeak.gz | awk 'BEGIN{FS="	";OFS="	"} {s1=$3-$2; s2=$13-$12; if (($21/s1 >= 0.5) || ($21/s2 >= 0.5)) {print $0}}' | cut -f 1-10 | sort | uniq | intersectBed -wo -a stdin -b /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/rep2/ATAC10-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.pval0.1.500000.narrowPeak.gz | awk 'BEGIN{FS="	";OFS="	"} {s1=$3-$2; s2=$13-$12; if (($21/s1 >= 0.5) || ($21/s2 >= 0.5)) {print $0}}' | cut -f 1-10 | sort | uniq | gzip -c > /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.PooledInReps.narrowPeak.gz

# SYS command. line 144

 echo

# SYS command. line 147

 zcat /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.PooledInReps.narrowPeak.gz  | sort | uniq | gzip -c > /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.narrowPeak.gz

# SYS command. line 149

 rm -f /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.PooledInReps.narrowPeak.gz

   
--------------------Stdout--------------------


 
--------------------Stderr--------------------
discarding /users/pangwei/anaconda3/bin from PATH
prepending /users/pangwei/anaconda3/envs/bds_atac/bin to PATH

Num	2
ID	task.callpeak_etc.naive_overlap_thresh_gpeak.line_136.id_7
Name	naive_overlap_thresh gpeak
Thread	thread_Root
PID	69550

OK	true
Exit Code	0
Retries
State	FINISHED
Dep.	OK
Cpus	1
Mem

Start	2016-05-17 16:52:29
End	2016-05-17 16:52:52
Elapsed	00:00:23
Timeout	00:00:-1
Wall Timeout	100 days

Input files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/pooled_rep/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.gappedPeak.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/rep1/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.pval0.1.500000.gappedPeak.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/rep2/ATAC10-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.pval0.1.500000.gappedPeak.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata
Output files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.gappedPeak.gz
Dependencies

 
# SYS command. line 138

 if [[ -f $(which activate) ]]; then source activate bds_atac; fi;  export PATH=/users/pangwei/bds_atac/.:/users/pangwei/bds_atac/modules:/users/pangwei/bds_atac/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds;

# SYS command. line 141

 intersectBed -wo -a /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/pooled_rep/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.gappedPeak.gz -b /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/rep1/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.pval0.1.500000.gappedPeak.gz | awk 'BEGIN{FS="	";OFS="	"} {s1=$3-$2; s2=$18-$17; if (($31/s1 >= 0.5) || ($31/s2 >= 0.5)) {print $0}}' | cut -f 1-15 | sort | uniq | intersectBed -wo -a stdin -b /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/rep2/ATAC10-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.pval0.1.500000.gappedPeak.gz | awk 'BEGIN{FS="	";OFS="	"} {s1=$3-$2; s2=$18-$17; if (($31/s1 >= 0.5) || ($31/s2 >= 0.5)) {print $0}}' | cut -f 1-15 | sort | uniq | gzip -c > /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.PooledInReps.gappedPeak.gz

# SYS command. line 144

 echo

# SYS command. line 147

 zcat /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.PooledInReps.gappedPeak.gz  | sort | uniq | gzip -c > /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.gappedPeak.gz

# SYS command. line 149

 rm -f /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.PooledInReps.gappedPeak.gz

   
--------------------Stdout--------------------


 
--------------------Stderr--------------------
discarding /users/pangwei/anaconda3/bin from PATH
prepending /users/pangwei/anaconda3/envs/bds_atac/bin to PATH

Num	3
ID	task.idr.idr2_rep1_rep2.line_72.id_8
Name	idr2 rep1-rep2
Thread	thread_Root
PID	69575

OK	true
Exit Code	0
Retries
State	FINISHED
Dep.	OK
Cpus	1
Mem

Start	2016-05-17 16:52:29
End	2016-05-17 16:56:24
Elapsed	00:03:54
Timeout	00:00:-1
Wall Timeout	100 days

Input files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/rep1/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.pval0.1.500000.narrowPeak.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/rep2/ATAC10-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.pval0.1.500000.narrowPeak.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/pooled_rep/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.narrowPeak.gz
Output files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.filt.narrowPeak.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.unthresholded-peaks.txt.png /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.unthresholded-peaks.txt.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.filt.12-col.bed.gz
Dependencies

 
# SYS command. line 74

 if [[ -f $(which activate) ]]; then source activate bds_atac_py3; fi;  export PATH=/users/pangwei/bds_atac/.:/users/pangwei/bds_atac/modules:/users/pangwei/bds_atac/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds;

# SYS command. line 76

 idr --samples /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/rep1/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.pval0.1.500000.narrowPeak.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/rep2/ATAC10-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.pval0.1.500000.narrowPeak.gz --peak-list /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/pooled_rep/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.narrowPeak.gz --input-file-type narrowPeak \
			--output-file /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.unthresholded-peaks.txt --rank p.value --soft-idr-threshold 0.1 \
			--plot --use-best-multisummit-IDR

# SYS command. line 80

 idr_thresh_transformed=$(awk -v p=0.1 'BEGIN{print -log(p)/log(10)}')

# SYS command. line 83

 awk 'BEGIN{OFS="\t"} $12>='"${idr_thresh_transformed}"' {print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,"0"}' /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.unthresholded-peaks.txt \
			| sort | uniq | sort -k7n,7n | gzip -c > /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.13-col.bed.gz

# SYS command. line 86

 zcat /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.13-col.bed.gz | awk 'BEGIN{OFS="\t"} {print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10}' | gzip -c > /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.narrowPeak.gz

# SYS command. line 87

 zcat /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.13-col.bed.gz | awk 'BEGIN{OFS="\t"} {print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12}' | gzip -c > /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.12-col.bed.gz

# SYS command. line 89

 bedtools intersect -v -a /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.13-col.bed.gz -b /mnt/data/ENCODE/blacklists/wgEncodeDacMapabilityConsensusExcludable.bed.gz | gzip -c > /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.filt.13-col.bed.gz

# SYS command. line 90

 zcat /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.filt.13-col.bed.gz | awk 'BEGIN{OFS="	"} {print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10}' | gzip -c > /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.filt.narrowPeak.gz

# SYS command. line 91

 zcat /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.filt.13-col.bed.gz | awk 'BEGIN{OFS="	"} {print $1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12}' | gzip -c > /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.filt.12-col.bed.gz

# SYS command. line 93

 gzip -f /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.unthresholded-peaks.txt

# SYS command. line 94

 rm -f /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.13-col.bed.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.filt.13-col.bed.gz

    
--------------------Stderr--------------------
discarding /users/pangwei/anaconda3/bin from PATH
prepending /users/pangwei/anaconda3/envs/bds_atac_py3/bin to PATH
Initial parameter values: [0.10 1.00 0.20 0.50]
Final parameter values: [1.76 0.75 0.78 0.31]
Number of reported peaks - 210642/210642 (100.0%)

Number of peaks passing IDR cutoff of 0.1 - 83055/210642 (39.4%)

Num	4
ID	task.base.line_414.id_11
Name	task.base.line_414.id_11
Thread	thread_Root
PID	71989

OK	true
Exit Code	0
Retries
State	FINISHED
Dep.	OK
Cpus	1
Mem

Start	2016-05-17 16:56:24
End	2016-05-17 16:56:25
Elapsed	00:00:00
Timeout	00:00:-1
Wall Timeout	100 days

Input files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.filt.narrowPeak.gz
Output files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/optimal_set/rep1-rep2.IDR0.1.filt.narrowPeak.gz
Dependencies

 
# SYS command. line 416

 cp /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.filt.narrowPeak.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/optimal_set/rep1-rep2.IDR0.1.filt.narrowPeak.gz

# SYS command. line 417

 while [ ! -f /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/optimal_set/rep1-rep2.IDR0.1.filt.narrowPeak.gz ]; do echo FOUND DELAYED WRITE, WAITING...; sleep 0.1; done

Num	5
ID	task.base.line_414.id_12
Name	task.base.line_414.id_12
Thread	thread_Root
PID	71994

OK	true
Exit Code	0
Retries
State	FINISHED
Dep.	OK
Cpus	1
Mem

Start	2016-05-17 16:56:24
End	2016-05-17 16:56:25
Elapsed	00:00:00
Timeout	00:00:-1
Wall Timeout	100 days

Input files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.filt.narrowPeak.gz
Output files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/conservative_set/rep1-rep2.IDR0.1.filt.narrowPeak.gz
Dependencies

 
# SYS command. line 416

 cp /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.filt.narrowPeak.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/conservative_set/rep1-rep2.IDR0.1.filt.narrowPeak.gz

# SYS command. line 417

 while [ ! -f /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/conservative_set/rep1-rep2.IDR0.1.filt.narrowPeak.gz ]; do echo FOUND DELAYED WRITE, WAITING...; sleep 0.1; done

Num	6
ID	task.base.line_414.id_13
Name	task.base.line_414.id_13
Thread	thread_Root
PID	72005

OK	true
Exit Code	0
Retries
State	FINISHED
Dep.	OK
Cpus	1
Mem

Start	2016-05-17 16:56:24
End	2016-05-17 16:56:25
Elapsed	00:00:00
Timeout	00:00:-1
Wall Timeout	100 days

Input files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.filt.12-col.bed.gz
Output files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/optimal_set/rep1-rep2.IDR0.1.filt.12-col.bed.gz
Dependencies

 
# SYS command. line 416

 cp /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.filt.12-col.bed.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/optimal_set/rep1-rep2.IDR0.1.filt.12-col.bed.gz

# SYS command. line 417

 while [ ! -f /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/optimal_set/rep1-rep2.IDR0.1.filt.12-col.bed.gz ]; do echo FOUND DELAYED WRITE, WAITING...; sleep 0.1; done

Num	7
ID	task.base.line_414.id_14
Name	task.base.line_414.id_14
Thread	thread_Root
PID	72019

OK	true
Exit Code	0
Retries
State	FINISHED
Dep.	OK
Cpus	1
Mem

Start	2016-05-17 16:56:25
End	2016-05-17 16:56:25
Elapsed	00:00:00
Timeout	00:00:-1
Wall Timeout	100 days

Input files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.filt.12-col.bed.gz
Output files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/conservative_set/rep1-rep2.IDR0.1.filt.12-col.bed.gz
Dependencies

 
# SYS command. line 416

 cp /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.filt.12-col.bed.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/conservative_set/rep1-rep2.IDR0.1.filt.12-col.bed.gz

# SYS command. line 417

 while [ ! -f /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/conservative_set/rep1-rep2.IDR0.1.filt.12-col.bed.gz ]; do echo FOUND DELAYED WRITE, WAITING...; sleep 0.1; done

Num	8
ID	task.idr.idr_final_qc.line_210.id_15
Name	idr final qc
Thread	thread_Root
PID	72039

OK	true
Exit Code	0
Retries
State	FINISHED
Dep.	OK
Cpus	1
Mem

Start	2016-05-17 16:56:25
End	2016-05-17 16:56:26
Elapsed	00:00:01
Timeout	00:00:-1
Wall Timeout	100 days

Input files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/true_reps/rep1-rep2/rep1-rep2.IDR0.1.filt.narrowPeak.gz
Output files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/IDR_final.qc
Dependencies

 
# SYS command. line 212

 if [[ -f $(which activate) ]]; then source activate bds_atac; fi;  export PATH=/users/pangwei/bds_atac/.:/users/pangwei/bds_atac/modules:/users/pangwei/bds_atac/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds;

# SYS command. line 214

 echo -e "Nt\t""Np\tconservative_set\toptimal_set\trescue_ratio\tself_consistency_ratio\treproducibility" > /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/IDR_final.qc

# SYS command. line 215

 echo -e "83109\t""0\trep1-rep2\trep1-rep2\tInfinity\tNaN\t0" >> /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/IDR_final.qc

    
--------------------Stderr--------------------
discarding /users/pangwei/anaconda3/bin from PATH
prepending /users/pangwei/anaconda3/envs/bds_atac/bin to PATH

Num	9
ID	task.postalign_bam.srt_bam_rep1.line_345.id_16
Name	srt_bam rep1
Thread	thread_40
PID	72044

OK	true
Exit Code	0
Retries
State	FINISHED
Dep.	OK
Cpus	1
Mem

Start	2016-05-17 16:56:26
End	2016-05-17 17:36:38
Elapsed	00:40:11
Timeout	00:00:-1
Wall Timeout	100 days

Input files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep1/ATAC9-trimmed-pair1.trim.PE2SE.bam
Output files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep1/ATAC9-trimmed-pair1.trim.PE2SE.srt.bam
Dependencies

 
# SYS command. line 347

 if [[ -f $(which activate) ]]; then source activate bds_atac; fi;  export PATH=/users/pangwei/bds_atac/.:/users/pangwei/bds_atac/modules:/users/pangwei/bds_atac/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds;

# SYS command. line 353

 samtools sort /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep1/ATAC9-trimmed-pair1.trim.PE2SE.bam /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep1/ATAC9-trimmed-pair1.trim.PE2SE.srt

# SYS command. line 354

 samtools index /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep1/ATAC9-trimmed-pair1.trim.PE2SE.srt.bam

# SYS command. line 356

 if [ "/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep1/ATAC9-trimmed-pair1.trim.PE2SE.srt.bam.bai" != "/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep1/ATAC9-trimmed-pair1.trim.PE2SE.srt.bai" ]; then \
			cp /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep1/ATAC9-trimmed-pair1.trim.PE2SE.srt.bam.bai /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep1/ATAC9-trimmed-pair1.trim.PE2SE.srt.bai; \
			fi

    
--------------------Stderr--------------------
discarding /users/pangwei/anaconda3/bin from PATH
prepending /users/pangwei/anaconda3/envs/bds_atac/bin to PATH
[bam_sort_core] merging from 39 files...

Num	10
ID	task.postalign_bam.srt_bam_rep2.line_345.id_17
Name	srt_bam rep2
Thread	thread_41
PID	72054

OK	true
Exit Code	0
Retries
State	FINISHED
Dep.	OK
Cpus	1
Mem

Start	2016-05-17 16:56:26
End	2016-05-17 18:12:41
Elapsed	01:16:15
Timeout	00:00:-1
Wall Timeout	100 days

Input files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep2/ATAC10-trimmed-pair1.trim.PE2SE.bam
Output files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep2/ATAC10-trimmed-pair1.trim.PE2SE.srt.bam
Dependencies

 
# SYS command. line 347

 if [[ -f $(which activate) ]]; then source activate bds_atac; fi;  export PATH=/users/pangwei/bds_atac/.:/users/pangwei/bds_atac/modules:/users/pangwei/bds_atac/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds;

# SYS command. line 353

 samtools sort /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep2/ATAC10-trimmed-pair1.trim.PE2SE.bam /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep2/ATAC10-trimmed-pair1.trim.PE2SE.srt

# SYS command. line 354

 samtools index /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep2/ATAC10-trimmed-pair1.trim.PE2SE.srt.bam

# SYS command. line 356

 if [ "/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep2/ATAC10-trimmed-pair1.trim.PE2SE.srt.bam.bai" != "/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep2/ATAC10-trimmed-pair1.trim.PE2SE.srt.bai" ]; then \
			cp /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep2/ATAC10-trimmed-pair1.trim.PE2SE.srt.bam.bai /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep2/ATAC10-trimmed-pair1.trim.PE2SE.srt.bai; \
			fi

    
--------------------Stderr--------------------
discarding /users/pangwei/anaconda3/bin from PATH
prepending /users/pangwei/anaconda3/envs/bds_atac/bin to PATH
[bam_sort_core] merging from 79 files...

Num	11
ID	task.atac.ataqc_rep1.line_846.id_18
Name	ataqc rep1
Thread	thread_40
PID	76056

OK	true
Exit Code	0
Retries
State	FINISHED
Dep.	OK
Cpus	1
Mem

Start	2016-05-17 17:36:39
End	2016-05-17 19:38:31
Elapsed	02:01:52
Timeout	00:00:-1
Wall Timeout	100 days

Input files	/srv/scratch/pangwei/mesoderm/data/ATACseq/ATAC9/ATAC9-trimmed-pair1.fastq.gz /srv/scratch/pangwei/mesoderm/data/ATACseq/ATAC9/ATAC9-trimmed-pair2.fastq.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep1/ATAC9-trimmed-pair1.trim.PE2SE.bam /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep1/ATAC9-trimmed-pair1.trim.PE2SE.align.log /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep1/ATAC9-trimmed-pair1.trim.PE2SE.nodup.pbc.qc /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep1/ATAC9-trimmed-pair1.trim.PE2SE.dup.qc /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep1/ATAC9-trimmed-pair1.trim.PE2SE.nodup.bam /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep1/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5.tagAlign.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/signal/macs2/rep1/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.pval.signal.bigwig /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/rep1/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.narrowPeak.gz
Output files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep1/ATAC9-trimmed-pair1.PE2SE_qc.html /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep1/ATAC9-trimmed-pair1.PE2SE_qc.txt
Dependencies

 
# SYS command. line 848

 if [[ -f $(which activate) ]]; then source activate bds_atac; fi;  export PATH=/users/pangwei/bds_atac/.:/users/pangwei/bds_atac/modules:/users/pangwei/bds_atac/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds;

# SYS command. line 852

 export _JAVA_OPTIONS="-Xms256M -Xmx4096M -XX:ParallelGCThreads=1"

# SYS command. line 854

 cd /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep1

# SYS command. line 856

 /users/pangwei/bds_atac/ataqc/run_ataqc.py \
			    --workdir /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep1 \
			    --outdir /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep1 \
			    --outprefix ATAC9-trimmed-pair1.PE2SE \
			    --genome hg19 \
			    --ref /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/encodeHg19Male.fa \
			    --tss /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/hg19_RefSeq_stranded.bed.gz \
			    --dnase /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/reg2map_honeybadger2_dnase_all_p10_ucsc.bed.gz \
			    --blacklist /mnt/data/ENCODE/blacklists/wgEncodeDacMapabilityConsensusExcludable.bed.gz \
			    --prom /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/reg2map_honeybadger2_dnase_prom_p2.bed.gz \
			    --enh /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/reg2map_honeybadger2_dnase_enh_p2.bed.gz \
			    --reg2map /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/dnase_avgs_reg2map_p10_merged_named.pvals.gz \
			    --meta /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/eid_to_mnemonic.txt \
			    --pbc /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep1/ATAC9-trimmed-pair1.trim.PE2SE.nodup.pbc.qc\
			    --fastq1 /srv/scratch/pangwei/mesoderm/data/ATACseq/ATAC9/ATAC9-trimmed-pair1.fastq.gz --fastq2 /srv/scratch/pangwei/mesoderm/data/ATACseq/ATAC9/ATAC9-trimmed-pair2.fastq.gz \
			    --alignedbam /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep1/ATAC9-trimmed-pair1.trim.PE2SE.bam \
			    --alignmentlog /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep1/ATAC9-trimmed-pair1.trim.PE2SE.align.log \
			    --coordsortbam /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep1/ATAC9-trimmed-pair1.trim.PE2SE.srt.bam \
			    --duplog /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep1/ATAC9-trimmed-pair1.trim.PE2SE.dup.qc \
			    --finalbam /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep1/ATAC9-trimmed-pair1.trim.PE2SE.nodup.bam \
			    --finalbed /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep1/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5.tagAlign.gz \
			    --bigwig /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/signal/macs2/rep1/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.pval.signal.bigwig \
			    --peaks /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/rep1/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.narrowPeak.gz \
			    --naive_overlap_peaks /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.narrowPeak.gz \
			    --idr_peaks /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/optimal_set/rep1-rep2.IDR0.1.filt.narrowPeak.gz

   
--------------------Stdout--------------------
Run time: 2:01:50

 
--------------------Stderr--------------------
discarding /users/pangwei/anaconda3/bin from PATH
prepending /users/pangwei/anaconda3/envs/bds_atac/bin to PATH
Picked up _JAVA_OPTIONS: -Xms256M -Xmx4096M -XX:ParallelGCThreads=1
ERROR	2016-05-17 17:39:33	ProcessExecutor	Warning messages:
ERROR	2016-05-17 17:39:33	ProcessExecutor	1: In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH,  :
ERROR	2016-05-17 17:39:33	ProcessExecutor	  zero-length arrow is of indeterminate angle and so skipped
ERROR	2016-05-17 17:39:33	ProcessExecutor	2: In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH,  :
ERROR	2016-05-17 17:39:33	ProcessExecutor	  zero-length arrow is of indeterminate angle and so skipped
ERROR	2016-05-17 17:39:33	ProcessExecutor	3: In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH,  :
ERROR	2016-05-17 17:39:33	ProcessExecutor	  zero-length arrow is of indeterminate angle and so skipped
ERROR	2016-05-17 17:39:33	ProcessExecutor	4: In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH,  :
ERROR	2016-05-17 17:39:33	ProcessExecutor	  zero-length arrow is of indeterminate angle and so skipped
Picked up _JAVA_OPTIONS: -Xms256M -Xmx4096M -XX:ParallelGCThreads=1
[bam_sort_core] merging from 40 files...
Picked up _JAVA_OPTIONS: -Xms256M -Xmx4096M -XX:ParallelGCThreads=1
Picked up _JAVA_OPTIONS: -Xms256M -Xmx4096M -XX:ParallelGCThreads=1
processing chromosomes
/users/pangwei/bds_atac/ataqc/run_ataqc.py:851: FutureWarning: sort(columns=....) is deprecated, use sort_values(by=.....)
  sorted_results = merged.sort('corr', ascending=True)

Num	12
ID	task.atac.ataqc_rep2.line_846.id_19
Name	ataqc rep2
Thread	thread_41
PID	78428

OK	true
Exit Code	0
Retries
State	FINISHED
Dep.	OK
Cpus	1
Mem

Start	2016-05-17 18:12:42
End	2016-05-17 21:43:56
Elapsed	03:31:14
Timeout	00:00:-1
Wall Timeout	100 days

Input files	/srv/scratch/pangwei/mesoderm/data/ATACseq/ATAC10/ATAC10-trimmed-pair1.fastq.gz /srv/scratch/pangwei/mesoderm/data/ATACseq/ATAC10/ATAC10-trimmed-pair2.fastq.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep2/ATAC10-trimmed-pair1.trim.PE2SE.bam /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep2/ATAC10-trimmed-pair1.trim.PE2SE.align.log /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep2/ATAC10-trimmed-pair1.trim.PE2SE.nodup.pbc.qc /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep2/ATAC10-trimmed-pair1.trim.PE2SE.dup.qc /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep2/ATAC10-trimmed-pair1.trim.PE2SE.nodup.bam /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep2/ATAC10-trimmed-pair1.trim.PE2SE.nodup.17.tn5.tagAlign.gz /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/signal/macs2/rep2/ATAC10-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.pval.signal.bigwig /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/rep2/ATAC10-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.narrowPeak.gz
Output files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep2/ATAC10-trimmed-pair1.PE2SE_qc.html /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep2/ATAC10-trimmed-pair1.PE2SE_qc.txt
Dependencies

 
# SYS command. line 848

 if [[ -f $(which activate) ]]; then source activate bds_atac; fi;  export PATH=/users/pangwei/bds_atac/.:/users/pangwei/bds_atac/modules:/users/pangwei/bds_atac/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds;

# SYS command. line 852

 export _JAVA_OPTIONS="-Xms256M -Xmx4096M -XX:ParallelGCThreads=1"

# SYS command. line 854

 cd /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep2

# SYS command. line 856

 /users/pangwei/bds_atac/ataqc/run_ataqc.py \
			    --workdir /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep2 \
			    --outdir /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep2 \
			    --outprefix ATAC10-trimmed-pair1.PE2SE \
			    --genome hg19 \
			    --ref /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/encodeHg19Male.fa \
			    --tss /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/hg19_RefSeq_stranded.bed.gz \
			    --dnase /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/reg2map_honeybadger2_dnase_all_p10_ucsc.bed.gz \
			    --blacklist /mnt/data/ENCODE/blacklists/wgEncodeDacMapabilityConsensusExcludable.bed.gz \
			    --prom /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/reg2map_honeybadger2_dnase_prom_p2.bed.gz \
			    --enh /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/reg2map_honeybadger2_dnase_enh_p2.bed.gz \
			    --reg2map /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/dnase_avgs_reg2map_p10_merged_named.pvals.gz \
			    --meta /mnt/lab_data/kundaje/users/dskim89/ataqc/annotations/hg19/eid_to_mnemonic.txt \
			    --pbc /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep2/ATAC10-trimmed-pair1.trim.PE2SE.nodup.pbc.qc\
			    --fastq1 /srv/scratch/pangwei/mesoderm/data/ATACseq/ATAC10/ATAC10-trimmed-pair1.fastq.gz --fastq2 /srv/scratch/pangwei/mesoderm/data/ATACseq/ATAC10/ATAC10-trimmed-pair2.fastq.gz \
			    --alignedbam /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep2/ATAC10-trimmed-pair1.trim.PE2SE.bam \
			    --alignmentlog /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep2/ATAC10-trimmed-pair1.trim.PE2SE.align.log \
			    --coordsortbam /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep2/ATAC10-trimmed-pair1.trim.PE2SE.srt.bam \
			    --duplog /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/qc/rep2/ATAC10-trimmed-pair1.trim.PE2SE.dup.qc \
			    --finalbam /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep2/ATAC10-trimmed-pair1.trim.PE2SE.nodup.bam \
			    --finalbed /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/align/rep2/ATAC10-trimmed-pair1.trim.PE2SE.nodup.17.tn5.tagAlign.gz \
			    --bigwig /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/signal/macs2/rep2/ATAC10-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.pval.signal.bigwig \
			    --peaks /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/rep2/ATAC10-trimmed-pair1.trim.PE2SE.nodup.17.tn5.pf.narrowPeak.gz \
			    --naive_overlap_peaks /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.narrowPeak.gz \
			    --idr_peaks /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/optimal_set/rep1-rep2.IDR0.1.filt.narrowPeak.gz

   
--------------------Stdout--------------------
Run time: 3:31:11

 
--------------------Stderr--------------------
discarding /users/pangwei/anaconda3/bin from PATH
prepending /users/pangwei/anaconda3/envs/bds_atac/bin to PATH
Picked up _JAVA_OPTIONS: -Xms256M -Xmx4096M -XX:ParallelGCThreads=1
ERROR	2016-05-17 18:16:51	ProcessExecutor	Warning messages:
ERROR	2016-05-17 18:16:51	ProcessExecutor	1: In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH,  :
ERROR	2016-05-17 18:16:51	ProcessExecutor	  zero-length arrow is of indeterminate angle and so skipped
ERROR	2016-05-17 18:16:51	ProcessExecutor	2: In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH,  :
ERROR	2016-05-17 18:16:51	ProcessExecutor	  zero-length arrow is of indeterminate angle and so skipped
Picked up _JAVA_OPTIONS: -Xms256M -Xmx4096M -XX:ParallelGCThreads=1
[bam_sort_core] merging from 80 files...
Picked up _JAVA_OPTIONS: -Xms256M -Xmx4096M -XX:ParallelGCThreads=1
Could not delete file /tmp/pangwei/sortingcollection.7977846028064485274.tmp
Could not delete file /tmp/pangwei/sortingcollection.6585011905118053813.tmp
Picked up _JAVA_OPTIONS: -Xms256M -Xmx4096M -XX:ParallelGCThreads=1
processing chromosomes
/users/pangwei/bds_atac/ataqc/run_ataqc.py:851: FutureWarning: sort(columns=....) is deprecated, use sort_values(by=.....)
  sorted_results = merged.sort('corr', ascending=True)

Num	13
ID	task.report.line_417.id_20
Name	task.report.line_417.id_20
Thread	thread_Root
PID	105475

OK	true
Exit Code	0
Retries
State	FINISHED
Dep.	OK
Cpus	1
Mem

Start	2016-05-17 21:43:59
End	2016-05-17 21:44:03
Elapsed	00:00:03
Timeout	00:00:-1
Wall Timeout	100 days

Input files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.narrowPeak.gz
Output files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.narrowPeak.hammock.gz
Dependencies

 
# SYS command. line 419

 if [[ -f $(which activate) ]]; then source activate bds_atac; fi;  export PATH=/users/pangwei/bds_atac/.:/users/pangwei/bds_atac/modules:/users/pangwei/bds_atac/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds;

# SYS command. line 420

 zcat /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.narrowPeak.gz | sed '/^\(chr\)/!d' | sort -k1,1V -k2,2n > /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.narrowPeak.tmp

# SYS command. line 422

 /users/pangwei/bds_atac/utils/narrowpeak.py /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.narrowPeak.tmp /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.narrowPeak.hammock

# SYS command. line 423

 rm -f /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.narrowPeak.tmp

    
--------------------Stderr--------------------
discarding /users/pangwei/anaconda3/bin from PATH
prepending /users/pangwei/anaconda3/envs/bds_atac/bin to PATH

Num	14
ID	task.report.line_417.id_21
Name	task.report.line_417.id_21
Thread	thread_Root
PID	105544

OK	true
Exit Code	0
Retries
State	FINISHED
Dep.	OK
Cpus	1
Mem

Start	2016-05-17 21:44:04
End	2016-05-17 21:44:09
Elapsed	00:00:05
Timeout	00:00:-1
Wall Timeout	100 days

Input files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.gappedPeak.gz
Output files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.gappedPeak.hammock.gz
Dependencies

 
# SYS command. line 419

 if [[ -f $(which activate) ]]; then source activate bds_atac; fi;  export PATH=/users/pangwei/bds_atac/.:/users/pangwei/bds_atac/modules:/users/pangwei/bds_atac/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds;

# SYS command. line 420

 zcat /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.gappedPeak.gz | sed '/^\(chr\)/!d' | sort -k1,1V -k2,2n > /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.gappedPeak.tmp

# SYS command. line 422

 /users/pangwei/bds_atac/utils/gappedpeak.py /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.gappedPeak.tmp /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.gappedPeak.hammock

# SYS command. line 423

 rm -f /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/macs2/overlap/ATAC9-trimmed-pair1.trim.PE2SE.nodup.17.tn5_pooled.pf.pval0.1.500000.naive_overlap.gappedPeak.tmp

    
--------------------Stderr--------------------
discarding /users/pangwei/anaconda3/bin from PATH
prepending /users/pangwei/anaconda3/envs/bds_atac/bin to PATH

Num	15
ID	task.report.line_417.id_22
Name	task.report.line_417.id_22
Thread	thread_Root
PID	105618

OK	true
Exit Code	0
Retries
State	FINISHED
Dep.	OK
Cpus	1
Mem

Start	2016-05-17 21:44:10
End	2016-05-17 21:44:12
Elapsed	00:00:02
Timeout	00:00:-1
Wall Timeout	100 days

Input files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/optimal_set/rep1-rep2.IDR0.1.filt.12-col.bed.gz
Output files	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/optimal_set/rep1-rep2.IDR0.1.filt.12-col.bed.hammock.gz
Dependencies

 
# SYS command. line 419

 if [[ -f $(which activate) ]]; then source activate bds_atac; fi;  export PATH=/users/pangwei/bds_atac/.:/users/pangwei/bds_atac/modules:/users/pangwei/bds_atac/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds;

# SYS command. line 420

 zcat /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/optimal_set/rep1-rep2.IDR0.1.filt.12-col.bed.gz | sed '/^\(chr\)/!d' | sort -k1,1V -k2,2n > /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/optimal_set/rep1-rep2.IDR0.1.filt.12-col.bed.tmp

# SYS command. line 422

 /users/pangwei/bds_atac/utils/narrowpeak.py /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/optimal_set/rep1-rep2.IDR0.1.filt.12-col.bed.tmp /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/optimal_set/rep1-rep2.IDR0.1.filt.12-col.bed.hammock

# SYS command. line 423

 rm -f /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/peak/idr/optimal_set/rep1-rep2.IDR0.1.filt.12-col.bed.tmp

    
--------------------Stderr--------------------
discarding /users/pangwei/anaconda3/bin from PATH
prepending /users/pangwei/anaconda3/envs/bds_atac/bin to PATH

Num	16
ID	task.report_graphviz.line_91.id_23
Name	task.report_graphviz.line_91.id_23
Thread	thread_Root
PID	105705

OK	true
Exit Code	0
Retries
State	FINISHED
Dep.	OK
Cpus	1
Mem

Start	2016-05-17 21:44:13
End	2016-05-17 21:44:13
Elapsed	00:00:00
Timeout	00:00:-1
Wall Timeout	100 days

Input files
Output files
Dependencies

 
# SYS command. line 92

 if [[ -f $(which activate) ]]; then source activate bds_atac; fi;  export PATH=/users/pangwei/bds_atac/.:/users/pangwei/bds_atac/modules:/users/pangwei/bds_atac/utils:${PATH}:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds;

# SYS command. line 93

 dot -Tsvg /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/report/workflow.dot > /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10/report/workflow.svg

    
--------------------Stderr--------------------
discarding /users/pangwei/anaconda3/bin from PATH
prepending /users/pangwei/anaconda3/envs/bds_atac/bin to PATH

Type	Name	Value
string	_	/usr/bin/bds
string	_LMFILES_	/software/modulefiles/modsappsdir/modsappsdir:/software/modulefiles/samtools/1.2:/software/modulefiles/bedtools/2.23.0:/software/modulefiles/rsem/1.2.21
string	_ZSH_TMUX_FIXED_CONFIG	/users/pangwei/.oh-my-zsh/plugins/tmux/tmux.extra.conf
bool	allowEmpty	false
string[]	args	[-out_dir, /srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata/ATAC9+10, -num_rep, 2, -fastq1_1, /srv/scratch/pangwei/mesoderm/data/ATACseq/ATAC9/ATAC9-trimmed-pair1.fastq.gz, -fastq1_2, /srv/scratch/pangwei/mesoderm/data/ATACseq/ATAC9/ATAC9-trimmed-pair2.fastq.gz, -fastq2_1, /srv/scratch/pangwei/mesoderm/data/ATACseq/ATAC10/ATAC10-trimmed-pair1.fastq.gz, -fastq2_2, /srv/scratch/pangwei/mesoderm/data/ATACseq/ATAC10/ATAC10-trimmed-pair2.fastq.gz, -subsample, 17500000, -true_rep, -species, hg19]
string	C_INCLUDE_PATH	/software/samtools/samtools-1.2/include/htslib
bool	canFail	false
int	cpus	1
int	cpusLocal	48
int	day	86400
real	E	2.718281828459045
int	G	1073741824
string	HOME	/users/pangwei/
int	hour	3600
int	K	1024
string	LANG	en_US.UTF-8
string	LC_CTYPE	en_US.UTF-8
string	LD_LIBRARY_PATH	/software/samtools/samtools-1.2/lib:/usr/local/cudnn-3/lib64/:/usr/local/cudnn-3/lib64/
string	LESS	-R
string	LOADEDMODULES	modsappsdir/modsappsdir:samtools/1.2:bedtools/2.23.0:rsem/1.2.21
string	LOGNAME	pangwei
string	LSCOLORS	Gxfxcxdxbxegedabagacad
int	M	1048576
string	MAIL	/var/mail/pangwei
string	MANPATH	/software/samtools/samtools-1.2/share/man:/usr/share/man
int	mem	-1
int	minute	60
string	MODULE_VERSION	3.2.10
string	MODULE_VERSION_STACK	3.2.10
string	MODULEPATH	/usr/local/Modules/versions\t\t\t\t:/usr/local/Modules/$MODULE_VERSION/modulefiles\t:/modules/\t\t\t\t:/software/modulefiles
string	MODULESAPPSDIR	/software
string	MODULESHOME	/software/env_module/3.2.10
string	node
int	P	1125899906842624
string	PAGER	less
string	PATH	/software/RSEM/rsem-1.2.21/:/software/bedtools/2.23.0/bin:/software/samtools/samtools-1.2/bin:/users/pangwei/anaconda3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
real	PI	3.141592653589793
string	ppwd	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata
string	programName	atac.bds
string	programPath	/users/pangwei/bds_atac/atac.bds
string	PWD	/srv/scratch/pangwei/mesoderm/bulk-atac/output-hg19-scidata
string	PYTHONPATH	/home/jisraeli/src/caffe_atac/python/:/users/pangwei/stemcells:/home/jisraeli/src/caffe_atac/python/:/users/pangwei/stemcells:
string	queue
int	retry	0
string	SAMTOOLS_HOME	/software/samtools/samtools-1.2/bin
string	SHELL	/usr/bin/zsh
string	SHLVL	3
string	SSH_AUTH_SOCK	/tmp/ssh-5gr8qSU2WF/agent.74700
string	SSH_CLIENT	171.65.76.244 36245 22
string	SSH_CONNECTION	171.65.76.244 49273 171.65.76.236 22
string	SSH_TTY	/dev/pts/0
string	system	local
int	T	1099511627776
string	TERM	screen
string	THEANO_FLAGS	device=gpu6,floatX=float32,warn_float64=warn,optimizer_excluding=cudnn
int	timeout	-1
string	TMUX	/tmp/tmux-1077/default,106699,3
string	TMUX_PANE	%3
string	USER	pangwei
string	UTIL_SCRIPTS_DIR	/users/pangwei/av_scripts
int	walltimeout	8640000
int	week	604800
string	WWW	/srv/www/kundaje/pangwei
string	XDG_RUNTIME_DIR	/run/user/1077
string	XDG_SESSION_ID	1907
string	ZSH	/users/pangwei//.oh-my-zsh
string	ZSH_TMUX_TERM	screen

BigDataScript report: atac.bds

Timeline

Task Graph

Parallel (threads) details

Task details

Global scope