BigDataScript report: chipseq.bds

Script file	/users/pangwei/ENCODE_chipseq_pipeline/chipseq.bds
Program ID	chipseq.bds.20151215_200717_622
Start time	2015-12-15 20:07:17
Run time	00:38:36.260
Tasks executed	2
Tasks failed	0
Tasks failed names
Arguments*	[-out_dir, /srv/scratch/pangwei/deepimpute/data/peaks, -num_rep, 1, -tag1, /srv/scratch/pangwei/deepimpute/data/merged/ULI-H3K27ME3_merged.tagAlign.gz, -peakcall, macs2, -species, hg19]
System*	local
Cpus*	1
Exit value	0

* Values in global scope when program finished execution.

Parallel (threads) details

Thread ID	Parent	Tasks
thread_Root	None
thread_16	thread_Root	chipseq.bds.20151215_200717_622_parallel_16/task.tagalign.xcor_rep1.line_96.id_4 chipseq.bds.20151215_200717_622_parallel_16/task.tagalign.spr_rep1.line_125.id_5

Task details

Task

Execution

Time

Dependencies

Task program, Errors, StdOut / StdErr

Num	1
ID	task.tagalign.xcor_rep1.line_96.id_4
Name	xcor rep1
Thread	thread_16
PID	9265

OK	true
Exit Code	0
Retries
State	FINISHED
Dep.	OK
Cpus	1
Mem

Start	2015-12-15 20:07:18
End	2015-12-15 20:44:28
Elapsed	00:37:09
Timeout	00:00:-1
Wall Timeout	100 days

Input files	/srv/scratch/pangwei/deepimpute/data/merged/ULI-H3K27ME3_merged.tagAlign.gz
Output files	/srv/scratch/pangwei/deepimpute/data/peaks/align_rep1/ULI-H3K27ME3_merged.cc.qc /srv/scratch/pangwei/deepimpute/data/peaks/align_rep1/ULI-H3K27ME3_merged.cc.plot.pdf
Dependencies

 
# SYS command. line 100

 export PATH=/users/pangwei/ENCODE_chipseq_pipeline/.:/users/pangwei/ENCODE_chipseq_pipeline/..:/users/pangwei/ENCODE_chipseq_pipeline/modules:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds:${PATH};  . /etc/profile.d/modules.sh; module add bwa/0.7.3 gem/2.6 r/2.15.1 deepTools/latest align2rawsignal/2.0 phantompeakqualtools/default idr/latest; export PATH=${PATH}:/software/idrCode; module add samtools/0.1.19 bedtools/2.19.1 ucsc_tools/3.0.9 picard-tools/1.92 MACS2/2.1.0 java/latest; export _JAVA_OPTIONS='-Xms256M -Xmx512M -XX:ParallelGCThreads=1'; export MAX_JAVA_MEM='8G'; export MALLOC_ARENA_MAX=4

# SYS command. line 104

 Rscript $(which run_spp_nodups.R) -rf \
			-c=/srv/scratch/pangwei/deepimpute/data/merged/ULI-H3K27ME3_merged.tagAlign.gz -p=1 \
			-filtchr=chrM -savp=/srv/scratch/pangwei/deepimpute/data/peaks/align_rep1/ULI-H3K27ME3_merged.cc.plot.pdf -out=/srv/scratch/pangwei/deepimpute/data/peaks/align_rep1/ULI-H3K27ME3_merged.cc.qc 

# SYS command. line 107

 sed -r 's/,[^\t]+//g' /srv/scratch/pangwei/deepimpute/data/peaks/align_rep1/ULI-H3K27ME3_merged.cc.qc > /srv/scratch/pangwei/deepimpute/data/peaks/align_rep1/ULI-H3K27ME3_merged.cc.qc.tmp

# SYS command. line 108

 mv /srv/scratch/pangwei/deepimpute/data/peaks/align_rep1/ULI-H3K27ME3_merged.cc.qc.tmp /srv/scratch/pangwei/deepimpute/data/peaks/align_rep1/ULI-H3K27ME3_merged.cc.qc

   
--------------------Stdout--------------------
################
ChIP data: /srv/scratch/pangwei/deepimpute/data/merged/ULI-H3K27ME3_merged.tagAlign.gz 
Control data: NA 
strandshift(min): -500 
strandshift(step): 5 
strandshift(max) 1500 
user-defined peak shift NA 
exclusion(min): 10 
exclusion(max): NaN 
num parallel nodes: 1 
FDR threshold: 0.01 
NumPeaks Threshold: NA 
Output Directory: /srv/scratch/pangwei/deepimpute/data/merged 
narrowPeak output file name: NA 
regionPeak output file name: NA 
Rdata filename: NA 
plot pdf filename: /srv/scratch/pangwei/deepimpute/data/peaks/align_rep1/ULI-H3K27ME3_merged.cc.plot.pdf 
result filename: /srv/scratch/pangwei/deepimpute/data/peaks/align_rep1/ULI-H3K27ME3_merged.cc.qc 
Overwrite files?: TRUE

Decompressing ChIP file
Reading ChIP tagAlign/BAM file /srv/scratch/pangwei/deepimpute/data/merged/ULI-H3K27ME3_merged.tagAlign.gz 
opened /tmp/RtmpLs7tfC/ULI-H3K27ME3_merged.tagAlign243f41d69b33
done. read 30797416 fragments
ChIP data read length 100 
[1] TRUE
Calculating peak characteristics
Minimum cross-correlation value 0.3234678 
Minimum cross-correlation shift 1500 
Top 3 cross-correlation values 0.406201080684826 
Top 3 estimates for fragment length 130 
Window half size 155 
Phantom peak location 105 
Phantom peak Correlation 0.3349602 
Normalized Strand cross-correlation coefficient (NSC) 1.25577 
Relative Strand Cross correlation Coefficient (RSC) 7.198974 
Phantom Peak Quality Tag 2 
null device 
          1 

 
--------------------Stderr--------------------
Loading required package: caTools

Num	2
ID	task.tagalign.spr_rep1.line_125.id_5
Name	spr rep1
Thread	thread_16
PID	10665

OK	true
Exit Code	0
Retries
State	FINISHED
Dep.	OK
Cpus	1
Mem

Start	2015-12-15 20:44:29
End	2015-12-15 20:45:48
Elapsed	00:01:19
Timeout	00:00:-1
Wall Timeout	100 days

Input files	/srv/scratch/pangwei/deepimpute/data/merged/ULI-H3K27ME3_merged.tagAlign.gz
Output files	/srv/scratch/pangwei/deepimpute/data/peaks/align_rep1/ULI-H3K27ME3_merged.pr1.tagAlign.gz /srv/scratch/pangwei/deepimpute/data/peaks/align_rep1/ULI-H3K27ME3_merged.pr2.tagAlign.gz
Dependencies

 
# SYS command. line 129

 export PATH=/users/pangwei/ENCODE_chipseq_pipeline/.:/users/pangwei/ENCODE_chipseq_pipeline/..:/users/pangwei/ENCODE_chipseq_pipeline/modules:/bin:/usr/bin:/usr/local/bin:${HOME}/.bds:${PATH};  . /etc/profile.d/modules.sh; module add bwa/0.7.3 gem/2.6 r/2.15.1 deepTools/latest align2rawsignal/2.0 phantompeakqualtools/default idr/latest; export PATH=${PATH}:/software/idrCode; module add samtools/0.1.19 bedtools/2.19.1 ucsc_tools/3.0.9 picard-tools/1.92 MACS2/2.1.0 java/latest; export _JAVA_OPTIONS='-Xms256M -Xmx512M -XX:ParallelGCThreads=1'; export MAX_JAVA_MEM='8G'; export MALLOC_ARENA_MAX=4

# SYS command. line 132

 nlines=$( zcat /srv/scratch/pangwei/deepimpute/data/merged/ULI-H3K27ME3_merged.tagAlign.gz | wc -l )

# SYS command. line 133

 nlines=$(( (nlines + 1) / 2 ))

# SYS command. line 137

 zcat /srv/scratch/pangwei/deepimpute/data/merged/ULI-H3K27ME3_merged.tagAlign.gz | shuf | split -d -l $((nlines)) - /srv/scratch/pangwei/deepimpute/data/peaks/align_rep1/ULI-H3K27ME3_merged.

# SYS command. line 140

 gzip -c /srv/scratch/pangwei/deepimpute/data/peaks/align_rep1/ULI-H3K27ME3_merged.00 > /srv/scratch/pangwei/deepimpute/data/peaks/align_rep1/ULI-H3K27ME3_merged.pr1.tagAlign.gz

# SYS command. line 141

 rm -f /srv/scratch/pangwei/deepimpute/data/peaks/align_rep1/ULI-H3K27ME3_merged.00

# SYS command. line 142

 gzip -c /srv/scratch/pangwei/deepimpute/data/peaks/align_rep1/ULI-H3K27ME3_merged.01 > /srv/scratch/pangwei/deepimpute/data/peaks/align_rep1/ULI-H3K27ME3_merged.pr2.tagAlign.gz

# SYS command. line 143

 rm -f /srv/scratch/pangwei/deepimpute/data/peaks/align_rep1/ULI-H3K27ME3_merged.01

Global scope

Note: Global scope when program finished execution.

Type	Name	Value
string	_	/usr/bin/bds
string	_LMFILES_	/software/modulefiles/modsappsdir/modsappsdir:/software/modulefiles/samtools/1.2:/software/modulefiles/bedtools/2.23.0:/software/modulefiles/rsem/1.2.21
string	_ZSH_TMUX_FIXED_CONFIG	/users/pangwei/.oh-my-zsh/plugins/tmux/tmux.extra.conf
bool	allowEmpty	false
string[]	args	[-out_dir, /srv/scratch/pangwei/deepimpute/data/peaks, -num_rep, 1, -tag1, /srv/scratch/pangwei/deepimpute/data/merged/ULI-H3K27ME3_merged.tagAlign.gz, -peakcall, macs2, -species, hg19]
string	C_INCLUDE_PATH	/software/samtools/samtools-1.2/include/htslib
bool	canFail	false
int	cpus	1
int	cpusLocal	48
int	day	86400
real	E	2.718281828459045
int	G	1073741824
string	HOME	/users/pangwei/
int	hour	3600
int	K	1024
string	LANG	en_US.UTF-8
string	LC_CTYPE	en_US.UTF-8
string	LD_LIBRARY_PATH	/software/samtools/samtools-1.2/lib
string	LESS	-R
string	LOADEDMODULES	modsappsdir/modsappsdir:samtools/1.2:bedtools/2.23.0:rsem/1.2.21
string	LOGNAME	pangwei
string	LSCOLORS	Gxfxcxdxbxegedabagacad
int	M	1048576
string	MAIL	/var/mail/pangwei
string	MANPATH	/software/samtools/samtools-1.2/share/man:/usr/share/man
int	mem	-1
int	minute	60
string	MODULE_VERSION	3.2.10
string	MODULE_VERSION_STACK	3.2.10
string	MODULEPATH	/usr/local/Modules/versions\t\t\t\t:/usr/local/Modules/$MODULE_VERSION/modulefiles\t:/modules/\t\t\t\t:/software/modulefiles
string	MODULESAPPSDIR	/software
string	MODULESHOME	/software/env_module/3.2.10
string	node
int	P	1125899906842624
string	PAGER	less
string	PATH	/software/RSEM/rsem-1.2.21/:/software/bedtools/2.23.0/bin:/software/samtools/samtools-1.2/bin:/users/pangwei/anaconda3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
real	PI	3.141592653589793
string	ppwd	/users/pangwei/deepimpute
string	programName	chipseq.bds
string	programPath	/users/pangwei/ENCODE_chipseq_pipeline/chipseq.bds
string	PWD	/users/pangwei/deepimpute
string	PYTHONPATH	/home/jisraeli/src/caffe_atac/python/:/users/pangwei/stemcells:/home/jisraeli/src/caffe_atac/python/:/users/pangwei/stemcells:/home/jisraeli/src/caffe_atac/python/:/users/pangwei/stemcells:
string	queue
int	retry	0
string	SAMTOOLS_HOME	/software/samtools/samtools-1.2/bin
string	SHELL	/usr/bin/zsh
string	SHLVL	3
string	SSH_AUTH_SOCK	/tmp/ssh-R6m54r7qsT/agent.37345
string	SSH_CLIENT	171.65.76.105 57099 22
string	SSH_CONNECTION	171.65.76.105 49430 171.65.76.162 22
string	SSH_TTY	/dev/pts/110
string	system	local
int	T	1099511627776
string	TERM	screen
string	THEANO_FLAGS	device=gpu1,floatX=float32,warn_float64=warn
int	timeout	86400
string	TMUX	/tmp/tmux-1077/default,42799,28
string	TMUX_PANE	%28
string	USER	pangwei
string	UTIL_SCRIPTS_DIR	/users/pangwei/av_scripts
int	walltimeout	86400
int	week	604800
string	WWW	/srv/www/kundaje/pangwei
string	XDG_RUNTIME_DIR	/run/user/1077
string	XDG_SESSION_ID	1268
string	ZSH	/users/pangwei//.oh-my-zsh
string	ZSH_TMUX_TERM	screen

BigDataScript report: chipseq.bds

Timeline

Task Graph

Parallel (threads) details

Task details

Global scope