Command line: /oak/stanford/groups/akundaje/marinovg/programs/SPAdes-3.15.4-Linux/bin/spades.py --pe1-1 /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/Illumina_gDNA_of_Prymnesium_parvum_12B1_read1.paired.fastq.gz --pe1-2 /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/Illumina_gDNA_of_Prymnesium_parvum_12B1_read2.paired.fastq.gz --nanopore /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/ONT_gDNA_of_Prymnesium_parvum_12B1.fastq.gz -t 40 -o /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1 System information: SPAdes version: 3.15.4 Python version: 2.7.16 OS: Linux-3.10.0-1160.108.1.el7.x86_64-x86_64-with-centos-7.9.2009-Core Output dir: /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1 Mode: read error correction and assembling Debug mode is turned OFF Dataset parameters: Standard mode For multi-cell/isolate data we recommend to use '--isolate' option; for single-cell MDA data use '--sc'; for metagenomic data use '--meta'; for RNA-Seq use '--rna'. Reads: Library number: 1, library type: nanopore left reads: not specified right reads: not specified interlaced reads: not specified single reads: ['/oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/ONT_gDNA_of_Prymnesium_parvum_12B1.fastq.gz'] merged reads: not specified Library number: 2, library type: paired-end orientation: fr left reads: ['/oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/Illumina_gDNA_of_Prymnesium_parvum_12B1_read1.paired.fastq.gz'] right reads: ['/oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/Illumina_gDNA_of_Prymnesium_parvum_12B1_read2.paired.fastq.gz'] interlaced reads: not specified single reads: not specified merged reads: not specified Read error correction parameters: Iterations: 1 PHRED offset will be auto-detected Corrected reads will be compressed Assembly parameters: k: automatic selection based on read length Repeat resolution is enabled Mismatch careful mode is turned OFF MismatchCorrector will be SKIPPED Coverage cutoff is turned OFF Other parameters: Dir for temp files: /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/tmp Threads: 40 Memory limit (in Gb): 250 ======= SPAdes pipeline started. Log can be found here: /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/spades.log /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/Illumina_gDNA_of_Prymnesium_parvum_12B1_read2.paired.fastq.gz: max reads length: 151 /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/Illumina_gDNA_of_Prymnesium_parvum_12B1_read1.paired.fastq.gz: max reads length: 151 Reads length: 151 Default k-mer sizes were set to [21, 33, 55, 77] because estimated read length (151) is equal to or greater than 150 ===== Before start started. ===== Read error correction started. ===== Read error correction started. == Running: /oak/stanford/groups/akundaje/marinovg/programs/SPAdes-3.15.4-Linux/bin/spades-hammer /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/corrected/configs/config.info 0:00:00.000 1M / 13M INFO General (main.cpp : 75) Starting BayesHammer, built from refs/heads/spades_3.15.4, git revision 1a5332e5608a073a9d28a6d0e30836f38dcc5613 0:00:00.000 1M / 13M INFO General (main.cpp : 76) Loading config from /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/corrected/configs/config.info 0:00:00.003 1M / 13M INFO General (main.cpp : 78) Maximum # of threads to use (adjusted due to OMP capabilities): 8 0:00:00.003 1M / 13M INFO General (memory_limit.cpp : 54) Memory limit set to 250 Gb 0:00:00.004 1M / 13M INFO General (main.cpp : 86) Trying to determine PHRED offset 0:00:00.005 1M / 13M INFO General (main.cpp : 92) Determined value is 33 0:00:00.005 1M / 13M INFO General (hammer_tools.cpp : 38) Hamming graph threshold tau=1, k=21, subkmer positions = [ 0 10 ] 0:00:00.005 1M / 13M INFO General (main.cpp : 113) Size of aux. kmer data 24 bytes === ITERATION 0 begins === 0:00:00.006 1M / 13M INFO General (kmer_index_builder.hpp : 243) Splitting kmer instances into 16 files using 8 threads. This might take a while. 0:00:00.007 1M / 13M INFO General (file_limit.hpp : 42) Open file limit set to 131072 0:00:00.007 1M / 13M INFO General (kmer_splitter.hpp : 93) Memory available for splitting buffers: 10.4167 Gb 0:00:00.007 1M / 13M INFO General (kmer_splitter.hpp : 101) Using cell size of 4194304 0:00:00.008 4609M / 4609M INFO K-mer Splitting (kmer_data.cpp : 97) Processing /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/Illumina_gDNA_of_Prymnesium_parvum_12B1_read1.paired.fastq.gz 0:00:18.680 4609M / 4613M INFO K-mer Splitting (kmer_data.cpp : 107) Processed 1583316 reads 0:00:36.271 4609M / 4997M INFO K-mer Splitting (kmer_data.cpp : 107) Processed 3258627 reads 0:00:54.161 4609M / 5142M INFO K-mer Splitting (kmer_data.cpp : 107) Processed 4962905 reads 0:01:12.575 4609M / 5202M INFO K-mer Splitting (kmer_data.cpp : 107) Processed 6696633 reads 0:01:30.128 4609M / 5202M INFO K-mer Splitting (kmer_data.cpp : 107) Processed 8387728 reads 0:01:48.511 4609M / 5204M INFO K-mer Splitting (kmer_data.cpp : 107) Processed 10105041 reads 0:02:07.192 4609M / 5295M INFO K-mer Splitting (kmer_data.cpp : 107) Processed 11884732 reads 0:02:25.624 4609M / 5295M INFO K-mer Splitting (kmer_data.cpp : 107) Processed 13651988 reads 0:02:43.311 4609M / 5295M INFO K-mer Splitting (kmer_data.cpp : 107) Processed 15353013 reads 0:03:01.283 4609M / 5295M INFO K-mer Splitting (kmer_data.cpp : 107) Processed 17047598 reads 0:06:18.767 4609M / 5295M INFO K-mer Splitting (kmer_data.cpp : 107) Processed 35226558 reads 0:12:18.202 4609M / 5295M INFO K-mer Splitting (kmer_data.cpp : 107) Processed 68601992 reads 0:15:49.993 4609M / 5295M INFO K-mer Splitting (kmer_data.cpp : 97) Processing /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/Illumina_gDNA_of_Prymnesium_parvum_12B1_read2.paired.fastq.gz 0:24:13.933 4609M / 5295M INFO K-mer Splitting (kmer_data.cpp : 107) Processed 135173102 reads 0:31:38.446 4609M / 5295M INFO K-mer Splitting (kmer_data.cpp : 112) Total 176392594 reads processed 0:31:38.447 1M / 5295M INFO General (kmer_index_builder.hpp : 249) Starting k-mer counting. 0:34:46.478 1M / 57G INFO General (kmer_index_builder.hpp : 260) K-mer counting done. There are 1352976208 kmers in total. 0:34:46.480 1M / 57G INFO K-mer Index Building (kmer_index_builder.hpp : 395) Building perfect hash indices 0:35:50.503 999M / 57G INFO K-mer Index Building (kmer_index_builder.hpp : 431) Index built. Total 1352976208 kmers, 977217792 bytes occupied (5.77818 bits per kmer). 0:35:50.554 999M / 57G INFO K-mer Counting (kmer_data.cpp : 354) Arranging kmers in hash map order 0:37:11.685 22G / 57G INFO General (main.cpp : 148) Clustering Hamming graph. 1:19:36.444 22G / 57G INFO General (main.cpp : 155) Extracting clusters: 1:19:36.444 22G / 57G INFO General (concurrent_dsu.cpp : 18) Connecting to root 1:19:47.465 22G / 57G INFO General (concurrent_dsu.cpp : 34) Calculating counts 1:24:38.704 30G / 57G INFO General (concurrent_dsu.cpp : 63) Writing down entries 1:39:19.762 22G / 57G INFO General (main.cpp : 167) Clustering done. Total clusters: 278443344 1:39:21.689 12G / 57G INFO K-mer Counting (kmer_data.cpp : 371) Collecting K-mer information, this takes a while. 1:39:38.180 42G / 57G INFO K-mer Counting (kmer_data.cpp : 377) Processing /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/Illumina_gDNA_of_Prymnesium_parvum_12B1_read1.paired.fastq.gz 2:27:23.585 42G / 57G INFO K-mer Counting (kmer_data.cpp : 377) Processing /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/Illumina_gDNA_of_Prymnesium_parvum_12B1_read2.paired.fastq.gz 3:13:52.862 42G / 57G INFO K-mer Counting (kmer_data.cpp : 384) Collection done, postprocessing. 3:13:58.357 42G / 57G INFO K-mer Counting (kmer_data.cpp : 397) There are 1352976208 kmers in total. Among them 977343306 (72.2365%) are singletons. 3:13:58.357 42G / 57G INFO General (main.cpp : 173) Subclustering Hamming graph 5:21:22.090 42G / 57G INFO Hamming Subclustering (kmer_cluster.cpp : 650) Subclustering done. Total 300503 non-read kmers were generated. 5:21:22.090 42G / 57G INFO Hamming Subclustering (kmer_cluster.cpp : 651) Subclustering statistics: 5:21:22.090 42G / 57G INFO Hamming Subclustering (kmer_cluster.cpp : 652) Total singleton hamming clusters: 167170692. Among them 5013498 (2.99903%) are good 5:21:22.090 42G / 57G INFO Hamming Subclustering (kmer_cluster.cpp : 653) Total singleton subclusters: 340080. Among them 297835 (87.5779%) are good 5:21:22.090 42G / 57G INFO Hamming Subclustering (kmer_cluster.cpp : 654) Total non-singleton subcluster centers: 137803421. Among them 124474787 (90.3278%) are good 5:21:22.090 42G / 57G INFO Hamming Subclustering (kmer_cluster.cpp : 655) Average size of non-trivial subcluster: 8.60866 kmers 5:21:22.090 42G / 57G INFO Hamming Subclustering (kmer_cluster.cpp : 656) Average number of sub-clusters per non-singleton cluster: 1.24149 5:21:22.090 42G / 57G INFO Hamming Subclustering (kmer_cluster.cpp : 657) Total solid k-mers: 129786120 5:21:22.090 42G / 57G INFO Hamming Subclustering (kmer_cluster.cpp : 658) Substitution probabilities: [4,4]((0.887423,0.0602007,0.014036,0.0383398),(0.0318458,0.936985,0.0189,0.0122697),(0.012169,0.0188173,0.936775,0.0322384),(0.0380694,0.0137847,0.0600364,0.88811)) 5:21:22.680 42G / 57G INFO General (main.cpp : 178) Finished clustering. 5:21:22.699 42G / 57G INFO General (main.cpp : 197) Starting solid k-mers expansion in 8 threads. 5:55:18.745 42G / 57G INFO General (main.cpp : 218) Solid k-mers iteration 0 produced 65787198 new k-mers. 6:28:48.980 42G / 57G INFO General (main.cpp : 218) Solid k-mers iteration 1 produced 8031250 new k-mers. 7:01:43.465 42G / 57G INFO General (main.cpp : 218) Solid k-mers iteration 2 produced 1096763 new k-mers. 7:35:24.153 42G / 57G INFO General (main.cpp : 218) Solid k-mers iteration 3 produced 160087 new k-mers. 8:08:36.725 42G / 57G INFO General (main.cpp : 218) Solid k-mers iteration 4 produced 24112 new k-mers. 8:41:32.119 42G / 57G INFO General (main.cpp : 218) Solid k-mers iteration 5 produced 4615 new k-mers. 9:14:54.544 42G / 57G INFO General (main.cpp : 218) Solid k-mers iteration 6 produced 898 new k-mers. 9:47:59.677 42G / 57G INFO General (main.cpp : 218) Solid k-mers iteration 7 produced 312 new k-mers. 10:21:34.741 42G / 57G INFO General (main.cpp : 218) Solid k-mers iteration 8 produced 189 new k-mers. 10:54:41.859 42G / 57G INFO General (main.cpp : 218) Solid k-mers iteration 9 produced 83 new k-mers. 11:27:39.517 42G / 57G INFO General (main.cpp : 218) Solid k-mers iteration 10 produced 41 new k-mers. 12:01:23.451 42G / 57G INFO General (main.cpp : 218) Solid k-mers iteration 11 produced 0 new k-mers. 12:01:23.451 42G / 57G INFO General (main.cpp : 222) Solid k-mers finalized 12:01:23.477 42G / 57G INFO General (hammer_tools.cpp : 222) Starting read correction in 8 threads. 12:01:23.494 42G / 57G INFO General (hammer_tools.cpp : 235) Correcting pair of reads: /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/Illumina_gDNA_of_Prymnesium_parvum_12B1_read1.paired.fastq.gz and /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/Illumina_gDNA_of_Prymnesium_parvum_12B1_read2.paired.fastq.gz 12:01:27.689 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 0 of 800000 reads. 12:01:46.856 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 0 12:01:50.568 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 0 12:01:55.370 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 1 of 800000 reads. 12:02:14.607 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 1 12:02:17.297 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 1 12:02:21.182 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 2 of 800000 reads. 12:02:41.181 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 2 12:02:43.844 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 2 12:02:47.805 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 3 of 800000 reads. 12:03:06.933 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 3 12:03:09.590 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 3 12:03:13.535 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 4 of 800000 reads. 12:03:33.727 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 4 12:03:37.236 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 4 12:03:41.087 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 5 of 800000 reads. 12:03:59.947 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 5 12:04:02.838 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 5 12:04:06.851 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 6 of 800000 reads. 12:04:27.026 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 6 12:04:30.263 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 6 12:04:34.138 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 7 of 800000 reads. 12:04:52.985 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 7 12:04:55.614 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 7 12:04:59.463 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 8 of 800000 reads. 12:05:19.594 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 8 12:05:22.142 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 8 12:05:25.999 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 9 of 800000 reads. 12:05:45.584 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 9 12:05:49.241 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 9 12:05:54.186 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 10 of 800000 reads. 12:06:14.922 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 10 12:06:17.951 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 10 12:06:21.933 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 11 of 800000 reads. 12:06:40.806 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 11 12:06:43.658 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 11 12:06:47.552 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 12 of 800000 reads. 12:07:06.835 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 12 12:07:09.772 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 12 12:07:13.778 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 13 of 800000 reads. 12:07:33.254 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 13 12:07:35.933 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 13 12:07:39.780 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 14 of 800000 reads. 12:07:59.735 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 14 12:08:02.420 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 14 12:08:06.286 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 15 of 800000 reads. 12:08:25.507 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 15 12:08:28.186 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 15 12:08:32.049 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 16 of 800000 reads. 12:08:51.334 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 16 12:08:54.077 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 16 12:08:57.922 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 17 of 800000 reads. 12:09:17.463 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 17 12:09:21.007 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 17 12:09:24.851 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 18 of 800000 reads. 12:09:43.776 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 18 12:09:46.626 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 18 12:09:50.596 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 19 of 800000 reads. 12:10:10.083 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 19 12:10:13.276 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 19 12:10:17.122 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 20 of 800000 reads. 12:10:36.364 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 20 12:10:39.485 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 20 12:10:43.336 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 21 of 800000 reads. 12:11:02.014 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 21 12:11:05.208 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 21 12:11:09.049 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 22 of 800000 reads. 12:11:28.841 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 22 12:11:34.214 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 22 12:11:38.189 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 23 of 800000 reads. 12:11:57.553 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 23 12:12:00.152 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 23 12:12:04.037 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 24 of 800000 reads. 12:12:23.819 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 24 12:12:27.067 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 24 12:12:30.918 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 25 of 800000 reads. 12:12:51.142 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 25 12:12:53.772 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 25 12:12:57.610 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 26 of 800000 reads. 12:13:18.203 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 26 12:13:21.290 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 26 12:13:25.213 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 27 of 800000 reads. 12:13:47.139 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 27 12:13:50.466 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 27 12:13:54.453 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 28 of 800000 reads. 12:14:15.444 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 28 12:14:18.668 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 28 12:14:22.610 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 29 of 800000 reads. 12:14:44.488 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 29 12:14:47.776 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 29 12:14:51.681 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 30 of 800000 reads. 12:15:13.912 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 30 12:15:17.268 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 30 12:15:21.205 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 31 of 800000 reads. 12:15:43.491 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 31 12:15:46.922 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 31 12:15:50.910 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 32 of 800000 reads. 12:16:12.108 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 32 12:16:15.354 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 32 12:16:19.244 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 33 of 800000 reads. 12:16:40.641 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 33 12:16:44.177 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 33 12:16:48.083 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 34 of 800000 reads. 12:17:08.630 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 34 12:17:11.910 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 34 12:17:15.816 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 35 of 800000 reads. 12:17:37.008 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 35 12:17:40.229 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 35 12:17:44.056 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 36 of 800000 reads. 12:18:02.906 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 36 12:18:05.633 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 36 12:18:09.560 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 37 of 800000 reads. 12:18:28.387 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 37 12:18:31.555 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 37 12:18:35.377 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 38 of 800000 reads. 12:18:55.063 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 38 12:18:58.757 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 38 12:19:02.745 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 39 of 800000 reads. 12:19:21.385 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 39 12:19:24.553 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 39 12:19:28.490 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 40 of 800000 reads. 12:19:47.523 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 40 12:19:50.741 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 40 12:19:54.597 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 41 of 800000 reads. 12:20:13.769 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 41 12:20:17.305 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 41 12:20:21.146 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 42 of 800000 reads. 12:20:40.059 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 42 12:20:43.186 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 42 12:20:47.101 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 43 of 800000 reads. 12:21:06.203 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 43 12:21:08.976 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 43 12:21:12.790 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 44 of 800000 reads. 12:21:31.872 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 44 12:21:35.139 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 44 12:21:39.089 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 45 of 800000 reads. 12:21:58.203 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 45 12:22:01.469 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 45 12:22:05.482 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 46 of 800000 reads. 12:22:24.449 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 46 12:22:27.294 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 46 12:22:31.155 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 47 of 800000 reads. 12:22:51.127 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 47 12:22:54.492 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 47 12:22:58.379 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 48 of 800000 reads. 12:23:17.722 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 48 12:23:20.478 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 48 12:23:24.354 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 49 of 800000 reads. 12:23:43.195 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 49 12:23:46.617 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 49 12:23:50.698 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 50 of 800000 reads. 12:24:11.623 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 50 12:24:14.931 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 50 12:24:18.883 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 51 of 800000 reads. 12:24:39.247 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 51 12:24:42.615 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 51 12:24:46.590 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 52 of 800000 reads. 12:25:07.150 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 52 12:25:10.510 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 52 12:25:14.428 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 53 of 800000 reads. 12:25:35.162 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 53 12:25:38.050 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 53 12:25:41.971 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 54 of 800000 reads. 12:26:02.941 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 54 12:26:06.413 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 54 12:26:10.614 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 55 of 800000 reads. 12:26:33.384 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 55 12:26:36.785 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 55 12:26:40.748 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 56 of 800000 reads. 12:27:01.534 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 56 12:27:04.876 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 56 12:27:09.876 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 57 of 800000 reads. 12:27:31.133 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 57 12:27:34.047 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 57 12:27:37.936 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 58 of 800000 reads. 12:27:58.755 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 58 12:28:02.064 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 58 12:28:06.007 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 59 of 800000 reads. 12:28:28.301 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 59 12:28:31.337 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 59 12:28:35.667 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 60 of 800000 reads. 12:28:58.916 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 60 12:29:02.495 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 60 12:29:06.429 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 61 of 800000 reads. 12:29:27.289 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 61 12:29:30.405 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 61 12:29:34.396 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 62 of 800000 reads. 12:29:54.870 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 62 12:29:57.594 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 62 12:30:01.511 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 63 of 800000 reads. 12:30:21.996 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 63 12:30:25.017 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 63 12:30:28.888 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 64 of 800000 reads. 12:30:49.813 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 64 12:30:52.716 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 64 12:30:56.978 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 65 of 800000 reads. 12:31:17.313 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 65 12:31:20.192 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 65 12:31:23.994 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 66 of 800000 reads. 12:31:44.731 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 66 12:31:47.208 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 66 12:31:51.050 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 67 of 800000 reads. 12:32:10.406 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 67 12:32:13.663 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 67 12:32:17.563 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 68 of 800000 reads. 12:32:37.959 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 68 12:32:41.335 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 68 12:32:45.258 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 69 of 800000 reads. 12:33:05.767 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 69 12:33:09.111 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 69 12:33:12.992 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 70 of 800000 reads. 12:33:34.045 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 70 12:33:37.206 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 70 12:33:41.095 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 71 of 800000 reads. 12:34:01.541 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 71 12:34:04.747 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 71 12:34:08.627 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 72 of 800000 reads. 12:34:29.391 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 72 12:34:32.613 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 72 12:34:36.496 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 73 of 800000 reads. 12:34:57.349 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 73 12:34:59.857 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 73 12:35:03.729 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 74 of 800000 reads. 12:35:25.324 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 74 12:35:28.728 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 74 12:35:32.605 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 75 of 800000 reads. 12:35:53.048 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 75 12:35:55.543 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 75 12:35:59.408 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 76 of 800000 reads. 12:36:19.959 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 76 12:36:23.192 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 76 12:36:27.082 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 77 of 800000 reads. 12:36:47.434 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 77 12:36:49.992 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 77 12:36:53.862 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 78 of 800000 reads. 12:37:14.916 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 78 12:37:18.174 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 78 12:37:22.048 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 79 of 800000 reads. 12:37:42.690 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 79 12:37:45.233 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 79 12:37:49.115 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 80 of 800000 reads. 12:38:09.283 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 80 12:38:12.163 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 80 12:38:16.018 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 81 of 800000 reads. 12:38:36.438 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 81 12:38:39.705 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 81 12:38:43.558 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 82 of 800000 reads. 12:39:02.510 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 82 12:39:05.154 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 82 12:39:08.981 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 83 of 800000 reads. 12:39:28.617 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 83 12:39:31.917 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 83 12:39:35.776 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 84 of 800000 reads. 12:39:54.199 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 84 12:39:56.878 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 84 12:40:00.742 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 85 of 800000 reads. 12:40:20.279 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 85 12:40:23.061 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 85 12:40:26.896 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 86 of 800000 reads. 12:40:45.658 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 86 12:40:48.307 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 86 12:40:52.154 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 87 of 800000 reads. 12:41:11.585 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 87 12:41:14.776 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 87 12:41:18.637 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 88 of 800000 reads. 12:41:37.372 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 88 12:41:40.166 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 88 12:41:44.081 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 89 of 800000 reads. 12:42:03.202 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 89 12:42:05.875 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 89 12:42:09.751 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 90 of 800000 reads. 12:42:28.849 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 90 12:42:31.459 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 90 12:42:35.299 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 91 of 800000 reads. 12:42:53.991 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 91 12:42:56.652 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 91 12:43:00.604 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 92 of 800000 reads. 12:43:19.495 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 92 12:43:22.113 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 92 12:43:27.090 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 93 of 800000 reads. 12:43:46.010 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 93 12:43:48.719 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 93 12:43:52.740 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 94 of 800000 reads. 12:44:12.392 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 94 12:44:15.710 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 94 12:44:19.639 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 95 of 800000 reads. 12:44:39.752 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 95 12:44:42.720 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 95 12:44:46.639 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 96 of 800000 reads. 12:45:06.872 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 96 12:45:10.212 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 96 12:45:14.333 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 97 of 800000 reads. 12:45:36.218 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 97 12:45:39.556 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 97 12:45:43.524 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 98 of 800000 reads. 12:46:03.664 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 98 12:46:06.308 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 98 12:46:10.204 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 99 of 800000 reads. 12:46:33.625 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 99 12:46:37.069 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 99 12:46:40.979 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 100 of 800000 reads. 12:47:02.431 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 100 12:47:05.846 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 100 12:47:09.773 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 101 of 800000 reads. 12:47:31.946 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 101 12:47:35.259 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 101 12:47:39.306 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 102 of 800000 reads. 12:48:00.437 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 102 12:48:03.735 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 102 12:48:07.690 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 103 of 800000 reads. 12:48:30.573 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 103 12:48:33.769 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 103 12:48:37.699 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 104 of 800000 reads. 12:48:59.731 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 104 12:49:02.866 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 104 12:49:06.875 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 105 of 800000 reads. 12:49:27.854 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 105 12:49:31.124 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 105 12:49:35.062 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 106 of 800000 reads. 12:49:58.173 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 106 12:50:01.718 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 106 12:50:05.614 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 107 of 800000 reads. 12:50:27.089 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 107 12:50:30.464 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 107 12:50:34.408 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 108 of 800000 reads. 12:50:55.685 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 108 12:50:58.658 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 108 12:51:02.509 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 109 of 800000 reads. 12:51:23.540 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 109 12:51:26.062 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 109 12:51:27.001 43G / 57G INFO General (hammer_tools.cpp : 170) Prepared batch 110 of 196297 reads. 12:51:32.661 43G / 57G INFO General (hammer_tools.cpp : 177) Processed batch 110 12:51:33.402 43G / 57G INFO General (hammer_tools.cpp : 187) Written batch 110 12:51:33.859 42G / 57G INFO General (hammer_tools.cpp : 276) Correction done. Changed 47320674 bases in 24966984 reads. 12:51:33.930 42G / 57G INFO General (hammer_tools.cpp : 277) Failed to correct 14503 bases out of 25837403448. 12:51:41.926 1M / 57G INFO General (main.cpp : 255) Saving corrected dataset description to /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/corrected/corrected.yaml 12:51:42.181 1M / 57G INFO General (main.cpp : 262) All done. Exiting. ===== Read error correction finished. ===== corrected reads compression started. == Running: /oak/stanford/groups/akundaje/marinovg/programs/Anaconda2/bin/python /oak/stanford/groups/akundaje/marinovg/programs/SPAdes-3.15.4-Linux/share/spades/spades_pipeline/scripts/compress_all.py --input_file /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/corrected/corrected.yaml --ext_python_modules_home /oak/stanford/groups/akundaje/marinovg/programs/SPAdes-3.15.4-Linux/share/spades --max_threads 40 --output_dir /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/corrected --not_used_yaml_file /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/corrected/dont_correct.yaml --gzip_output == Compressing corrected reads (with pigz) == Files to compress: ['/oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/corrected/Illumina_gDNA_of_Prymnesium_parvum_12B1_read2.paired.fastq.00.0_0.cor.fastq', '/oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/corrected/Illumina_gDNA_of_Prymnesium_parvum_12B1_read_unpaired.00.0_0.cor.fastq', '/oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/corrected/Illumina_gDNA_of_Prymnesium_parvum_12B1_read1.paired.fastq.00.0_0.cor.fastq'] == Files compression is finished == Info about datasets not used in error correction stage is loaded == Dataset yaml file is updated ===== corrected reads compression finished. ===== Read error correction finished. ===== Assembling started. ===== K21 started. == Running: /oak/stanford/groups/akundaje/marinovg/programs/SPAdes-3.15.4-Linux/bin/spades-core /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K21/configs/config.info 0:00:00.000 1M / 13M INFO General (main.cpp : 99) Loaded config from /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K21/configs/config.info 0:00:00.000 1M / 13M INFO General (memory_limit.cpp : 54) Memory limit set to 250 Gb 0:00:00.000 1M / 13M INFO General (main.cpp : 107) Starting SPAdes, built from refs/heads/spades_3.15.4, git revision 1a5332e5608a073a9d28a6d0e30836f38dcc5613 0:00:00.001 1M / 13M INFO General (main.cpp : 108) Maximum k-mer length: 128 0:00:00.001 1M / 13M INFO General (main.cpp : 109) Assembling dataset (/oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/dataset.info) with K=21 0:00:00.001 1M / 13M INFO General (main.cpp : 110) Maximum # of threads to use (adjusted due to OMP capabilities): 8 0:00:00.001 1M / 13M INFO General (pipeline.cpp : 212) SPAdes started 0:00:00.001 1M / 13M INFO General (pipeline.cpp : 225) Starting from stage: read_conversion 0:00:00.001 1M / 13M INFO General (pipeline.cpp : 231) Two-step repeat resolution disabled 0:00:00.001 1M / 13M INFO GraphCore (graph_core.hpp : 672) Graph created, vertex min_id: 3, edge min_id: 3 0:00:00.001 1M / 13M INFO GraphCore (graph_core.hpp : 673) Vertex size: 48, edge size: 40 0:00:00.001 1M / 13M INFO General (edge_index.hpp : 113) Size of edge index entries: 12/8 0:00:00.001 1M / 13M INFO StageManager (stage.cpp : 185) STAGE == Binary Read Conversion (id: read_conversion) 0:00:00.004 1M / 13M INFO General (read_converter.cpp : 72) Converting reads to binary format for library #0 (takes a while) 0:00:00.004 1M / 13M INFO General (read_converter.cpp : 73) Converting paired reads 0:00:01.306 21M / 68M INFO General (binary_converter.cpp : 96) 16384 reads processed 0:00:01.340 21M / 70M INFO General (binary_converter.cpp : 96) 32768 reads processed 0:00:01.396 21M / 74M INFO General (binary_converter.cpp : 96) 65536 reads processed 0:00:02.106 21M / 114M INFO General (binary_converter.cpp : 96) 131072 reads processed 0:00:02.925 21M / 149M INFO General (binary_converter.cpp : 96) 262144 reads processed 0:00:04.620 12M / 201M INFO General (binary_converter.cpp : 96) 524288 reads processed 0:00:08.647 0M / 224M INFO General (binary_converter.cpp : 96) 1048576 reads processed 0:00:14.115 0M / 250M INFO General (binary_converter.cpp : 96) 2097152 reads processed 0:00:28.667 0M / 330M INFO General (binary_converter.cpp : 96) 4194304 reads processed 0:00:56.457 0M / 334M INFO General (binary_converter.cpp : 96) 8388608 reads processed 0:01:49.014 0M / 334M INFO General (binary_converter.cpp : 96) 16777216 reads processed 0:03:35.526 0M / 335M INFO General (binary_converter.cpp : 96) 33554432 reads processed 0:07:12.166 0M / 335M INFO General (binary_converter.cpp : 96) 67108864 reads processed 0:09:27.505 0M / 343M INFO General (binary_converter.cpp : 111) 87973211 reads written 0:09:27.505 0M / 343M INFO General (read_converter.cpp : 86) Converting single reads 0:09:28.902 0M / 343M INFO General (binary_converter.cpp : 96) 16384 reads processed 0:09:28.912 0M / 343M INFO General (binary_converter.cpp : 96) 32768 reads processed 0:09:28.934 0M / 343M INFO General (binary_converter.cpp : 96) 65536 reads processed 0:09:29.950 0M / 343M INFO General (binary_converter.cpp : 96) 131072 reads processed 0:09:30.382 0M / 343M INFO General (binary_converter.cpp : 111) 221245 reads written 0:09:30.382 0M / 343M INFO General (read_converter.cpp : 92) Converting merged reads 0:09:30.385 0M / 343M INFO General (binary_converter.cpp : 111) 0 reads written 0:09:30.391 0M / 343M INFO General (read_converter.cpp : 72) Converting reads to binary format for library #1 (takes a while) 0:09:30.391 0M / 343M INFO General (read_converter.cpp : 73) Converting paired reads 0:09:30.392 0M / 343M INFO General (binary_converter.cpp : 111) 0 reads written 0:09:30.392 0M / 343M INFO General (read_converter.cpp : 86) Converting single reads 0:09:40.232 0M / 839M INFO General (binary_converter.cpp : 96) 16384 reads processed 0:09:40.586 0M / 843M INFO General (binary_converter.cpp : 96) 32768 reads processed 0:09:41.015 0M / 858M INFO General (binary_converter.cpp : 96) 65536 reads processed 0:09:50.924 0M / 1452M INFO General (binary_converter.cpp : 96) 131072 reads processed 0:10:13.461 0M / 2835M INFO General (binary_converter.cpp : 96) 262144 reads processed 0:10:34.085 0M / 3143M INFO General (binary_converter.cpp : 96) 524288 reads processed 0:11:05.756 0M / 3754M INFO General (binary_converter.cpp : 111) 849424 reads written 0:11:05.756 0M / 3754M INFO General (read_converter.cpp : 92) Converting merged reads 0:11:05.759 0M / 3754M INFO General (binary_converter.cpp : 111) 0 reads written 0:11:05.949 1M / 3754M INFO StageManager (stage.cpp : 185) STAGE == de Bruijn graph construction (id: construction) 0:11:06.711 1M / 3754M INFO General (construction.cpp : 153) Max read length 151 0:11:06.711 1M / 3754M INFO General (construction.cpp : 159) Average read length 146.594 0:11:06.711 1M / 3754M INFO General (stage.cpp : 117) PROCEDURE == k+1-mer counting (id: construction:kpomer_counting) 0:11:06.711 1M / 3754M INFO General (kmer_index_builder.hpp : 243) Splitting kmer instances into 80 files using 8 threads. This might take a while. 0:11:06.713 1M / 3754M INFO General (file_limit.hpp : 42) Open file limit set to 131072 0:11:06.713 1M / 3754M INFO General (kmer_splitter.hpp : 93) Memory available for splitting buffers: 10.4167 Gb 0:11:06.713 1M / 3754M INFO General (kmer_splitter.hpp : 101) Using cell size of 838860 0:11:24.455 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 6719196 reads 0:11:37.783 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 13317095 reads 0:11:51.597 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 19932381 reads 0:12:05.357 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 26508337 reads 0:12:20.370 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 33014824 reads 0:12:35.740 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 39449606 reads 0:12:50.276 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 45920297 reads 0:13:04.605 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 52453773 reads 0:13:18.767 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 58986706 reads 0:13:33.273 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 65476917 reads 0:13:47.880 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 71934980 reads 0:14:01.677 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 78399733 reads 0:16:16.770 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 136780187 reads 0:21:25.226 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 273856805 reads 0:24:49.599 1M / 4209M INFO General (kmer_splitters.hpp : 131) Used 352335334 reads 0:24:50.127 1M / 4209M INFO General (kmer_index_builder.hpp : 249) Starting k-mer counting. 0:25:22.156 1M / 4209M INFO General (kmer_index_builder.hpp : 260) K-mer counting done. There are 185438076 kmers in total. 0:25:22.156 1M / 4209M INFO General (stage.cpp : 117) PROCEDURE == Extension index construction (id: construction:extension_index_construction) 0:25:22.186 1M / 4209M INFO K-mer Index Building (kmer_index_builder.hpp : 438) Building kmer index 0:25:22.186 1M / 4209M INFO General (kmer_index_builder.hpp : 243) Splitting kmer instances into 80 files using 8 threads. This might take a while. 0:25:22.187 1M / 4209M INFO General (file_limit.hpp : 42) Open file limit set to 131072 0:25:22.187 1M / 4209M INFO General (kmer_splitter.hpp : 93) Memory available for splitting buffers: 10.4166 Gb 0:25:22.187 1M / 4209M INFO General (kmer_splitter.hpp : 101) Using cell size of 838860 0:25:35.014 4641M / 4641M INFO General (kmer_splitters.hpp : 194) Processed 129419751 kmers 0:25:40.904 4641M / 4641M INFO General (kmer_splitters.hpp : 194) Processed 185438107 kmers 0:25:40.905 4641M / 4641M INFO General (kmer_splitters.hpp : 199) Used 185438107 kmers. 0:25:40.905 1M / 4209M INFO General (kmer_index_builder.hpp : 249) Starting k-mer counting. 0:25:43.350 1M / 4209M INFO General (kmer_index_builder.hpp : 260) K-mer counting done. There are 176365680 kmers in total. 0:25:43.350 1M / 4209M INFO K-mer Index Building (kmer_index_builder.hpp : 395) Building perfect hash indices 0:25:47.729 142M / 4209M INFO K-mer Index Building (kmer_index_builder.hpp : 431) Index built. Total 176365680 kmers, 127441816 bytes occupied (5.7808 bits per kmer). 0:25:47.730 142M / 4209M INFO General (kmer_index_builder.hpp : 169) Merging final buckets. 0:25:51.705 315M / 4209M INFO DeBruijnExtensionIndexBu (kmer_extension_index_build: 100) Building k-mer extensions from k+1-mers 0:26:11.762 315M / 4209M INFO DeBruijnExtensionIndexBu (kmer_extension_index_build: 105) Building k-mer extensions from k+1-mers finished. 0:26:11.896 315M / 4209M INFO General (stage.cpp : 117) PROCEDURE == Early tip clipping (id: construction:early_tip_clipper) 0:26:11.896 315M / 4209M INFO General (construction.cpp : 301) Early tip clipper length bound set as (RL - K) 0:26:11.896 315M / 4209M INFO Early tip clipping (early_simplification.hpp : 47) Early tip clipping 0:26:33.267 633M / 4209M INFO Early tip clipping (early_simplification.hpp : 82) #tipped junctions: 2902818 0:26:33.716 633M / 4209M INFO Early tip clipping (early_simplification.hpp : 93) Clipped tips: 2950689 0:26:33.842 315M / 4209M INFO Early tip clipping (early_simplification.hpp : 49) 26865670 22-mers were removed by early tip clipper 0:26:33.842 315M / 4209M INFO General (stage.cpp : 117) PROCEDURE == Condensing graph (id: construction:graph_condensing) 0:26:33.856 315M / 4209M INFO UnbranchingPathExtractor (debruijn_graph_constructor: 354) Extracting unbranching paths 0:26:59.090 1022M / 4209M INFO UnbranchingPathExtractor (debruijn_graph_constructor: 373) Extracting unbranching paths finished. 21036628 sequences extracted 0:27:10.351 1022M / 4209M INFO UnbranchingPathExtractor (debruijn_graph_constructor: 309) Collecting perfect loops 0:27:13.307 1022M / 4209M INFO UnbranchingPathExtractor (debruijn_graph_constructor: 342) Collecting perfect loops finished. 24 loops collected 0:27:15.004 1022M / 4209M INFO General (debruijn_graph_constructor: 487) Total 42073304 edges to create 0:27:15.006 2643M / 4209M INFO General (debruijn_graph_constructor: 489) Collecting link records 0:27:37.150 3288M / 4209M INFO General (debruijn_graph_constructor: 491) Ordering link records 0:27:37.956 3288M / 4586M INFO General (debruijn_graph_constructor: 493) Sorting done 0:27:38.240 3416M / 4586M INFO General (debruijn_graph_constructor: 503) Total 11964256 vertices to create 0:27:38.241 4523M / 4586M INFO General (debruijn_graph_constructor: 506) Connecting the graph 0:27:53.053 3452M / 5156M INFO General (stage.cpp : 117) PROCEDURE == Filling coverage indices (PHM) (id: construction:coverage_filling_phm) 0:27:53.054 3452M / 5156M INFO K-mer Index Building (kmer_index_builder.hpp : 395) Building perfect hash indices 0:27:59.309 3594M / 5156M INFO K-mer Index Building (kmer_index_builder.hpp : 431) Index built. Total 185438076 kmers, 133993400 bytes occupied (5.78062 bits per kmer). 0:28:00.066 4302M / 5156M INFO General (coverage_hash_map_builder.: 47) Collecting k-mer coverage information from reads, this takes a while. 0:47:38.086 4302M / 5156M INFO General (construction.cpp : 430) Filling coverage and flanking coverage from PHM 0:47:43.053 4302M / 5156M INFO General (coverage_filling.hpp : 82) Processed 8000000 edges 0:47:46.878 4302M / 5156M INFO General (coverage_filling.hpp : 82) Processed 16000000 edges 0:47:50.731 4302M / 5156M INFO General (coverage_filling.hpp : 82) Processed 24000000 edges 0:47:54.560 4302M / 5156M INFO General (coverage_filling.hpp : 82) Processed 32000000 edges 0:47:58.411 4302M / 5156M INFO General (coverage_filling.hpp : 82) Processed 40000000 edges 0:47:59.377 4302M / 5156M INFO General (coverage_filling.hpp : 82) Processed 42071773 edges 0:48:00.756 3799M / 6181M INFO StageManager (stage.cpp : 185) STAGE == EC Threshold Finding (id: ec_threshold_finder) 0:48:04.856 4459M / 6624M INFO General (kmer_coverage_model.cpp : 181) Kmer coverage valley at: 45 0:48:04.992 3799M / 6624M INFO General (kmer_coverage_model.cpp : 201) K-mer histogram maximum: 223 0:48:04.993 3799M / 6624M INFO General (kmer_coverage_model.cpp : 237) Estimated median coverage: 223. Coverage mad: 42.9954 0:48:05.101 3799M / 6624M INFO General (kmer_coverage_model.cpp : 259) Fitting coverage model 0:48:05.819 4459M / 6624M INFO General (kmer_coverage_model.cpp : 295) ... iteration 2 0:48:06.485 4459M / 6624M INFO General (kmer_coverage_model.cpp : 295) ... iteration 4 0:48:08.763 4459M / 6624M INFO General (kmer_coverage_model.cpp : 295) ... iteration 8 0:48:13.671 4459M / 6624M INFO General (kmer_coverage_model.cpp : 295) ... iteration 16 0:48:22.996 4459M / 6624M INFO General (kmer_coverage_model.cpp : 295) ... iteration 32 0:48:24.830 4459M / 6624M INFO General (kmer_coverage_model.cpp : 309) Fitted mean coverage: 204.459. Fitted coverage std. dev: 49.8666 0:48:24.835 4459M / 6624M INFO General (kmer_coverage_model.cpp : 334) Probability of erroneous kmer at valley: 0.755067 0:48:24.835 4459M / 6624M INFO General (kmer_coverage_model.cpp : 358) Preliminary threshold calculated as: 123 0:48:24.836 4459M / 6624M INFO General (kmer_coverage_model.cpp : 362) Threshold adjusted to: 123 0:48:24.836 4459M / 6624M INFO General (kmer_coverage_model.cpp : 375) Estimated genome size (ignoring repeats): 68055509 0:48:24.889 3799M / 6624M INFO General (genomic_info_filler.cpp : 55) Mean coverage was calculated as 204.459 0:48:24.890 3799M / 6624M INFO General (genomic_info_filler.cpp : 70) EC coverage threshold value was calculated as 123 0:48:24.890 3799M / 6624M INFO General (genomic_info_filler.cpp : 71) Trusted kmer low bound: 0 0:48:24.890 3799M / 6624M INFO StageManager (stage.cpp : 185) STAGE == Raw Simplification (id: raw_simplification) 0:48:24.890 3799M / 6624M INFO General (simplification.cpp : 127) PROCEDURE == Initial cleaning 0:48:24.890 3799M / 6624M INFO General (graph_simplification.hpp : 662) Flanking coverage based disconnection disabled 0:48:24.890 3799M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Self conjugate edge remover 0:48:26.243 3799M / 6624M INFO Simplification (parallel_processing.hpp : 170) Self conjugate edge remover triggered 0 times 0:48:26.243 3799M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Initial tip clipper 0:48:28.913 3799M / 6624M INFO Simplification (parallel_processing.hpp : 170) Initial tip clipper triggered 88739 times 0:48:28.917 3799M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Initial ec remover 0:49:29.589 4182M / 6624M INFO Simplification (parallel_processing.hpp : 170) Initial ec remover triggered 5663232 times 0:49:29.716 4182M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Initial isolated edge remover 0:49:30.361 4182M / 6624M INFO Simplification (parallel_processing.hpp : 170) Initial isolated edge remover triggered 5697 times 0:49:30.412 3565M / 6624M INFO StageManager (stage.cpp : 185) STAGE == Simplification (id: simplification) 0:49:30.412 3565M / 6624M INFO General (simplification.cpp : 368) Graph simplification started 0:49:30.412 3565M / 6624M INFO General (graph_simplification.hpp : 634) Creating parallel br instance 0:49:30.412 3565M / 6624M INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 1 0:49:30.412 3565M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:49:31.500 3564M / 6624M INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 48339 times 0:49:31.506 3564M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:51:34.097 3595M / 6624M INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 491560 times 0:51:34.126 3595M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:51:58.893 3694M / 6624M INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 2106283 times 0:51:58.986 3694M / 6624M INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 2 0:51:58.986 3694M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:52:00.166 3658M / 6624M INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 1870 times 0:52:00.167 3658M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:52:23.799 3560M / 6624M INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 80878 times 0:52:23.826 3560M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:52:25.645 3565M / 6624M INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 187687 times 0:52:25.667 3565M / 6624M INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 3 0:52:25.667 3565M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:52:25.952 3556M / 6624M INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 63 times 0:52:25.952 3556M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:52:30.500 3532M / 6624M INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 11094 times 0:52:30.503 3532M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:52:31.092 3531M / 6624M INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 56770 times 0:52:31.101 3531M / 6624M INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 4 0:52:31.101 3531M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:52:31.194 3529M / 6624M INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 12 times 0:52:31.194 3529M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:52:33.128 3526M / 6624M INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 3129 times 0:52:33.129 3526M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:52:33.456 3526M / 6624M INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 34195 times 0:52:33.462 3526M / 6624M INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 5 0:52:33.462 3526M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:52:33.502 3525M / 6624M INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 1 times 0:52:33.502 3525M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:52:34.821 3524M / 6624M INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 1373 times 0:52:34.822 3524M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:52:35.098 3526M / 6624M INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 26423 times 0:52:35.103 3526M / 6624M INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 6 0:52:35.103 3526M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:52:35.130 3524M / 6624M INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 1 times 0:52:35.130 3524M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:52:36.486 3523M / 6624M INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 866 times 0:52:36.487 3523M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:52:36.727 3524M / 6624M INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 22594 times 0:52:36.731 3524M / 6624M INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 7 0:52:36.731 3524M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:52:36.754 3522M / 6624M INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 2 times 0:52:36.754 3522M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:52:37.818 3521M / 6624M INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 636 times 0:52:37.818 3521M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:52:38.038 3522M / 6624M INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 20618 times 0:52:38.042 3522M / 6624M INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 8 0:52:38.042 3522M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:52:38.062 3522M / 6624M INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 3 times 0:52:38.062 3522M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:52:38.875 3520M / 6624M INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 562 times 0:52:38.875 3520M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:52:39.076 3521M / 6624M INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 18517 times 0:52:39.079 3521M / 6624M INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 9 0:52:39.079 3521M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:52:39.096 3520M / 6624M INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 2 times 0:52:39.096 3520M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:52:39.910 3520M / 6624M INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 451 times 0:52:39.910 3520M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:52:40.102 3520M / 6624M INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 16442 times 0:52:40.105 3520M / 6624M INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 10 0:52:40.105 3520M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:52:40.121 3520M / 6624M INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 3 times 0:52:40.121 3520M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:52:40.757 3509M / 6624M INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 415 times 0:52:40.758 3509M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:52:40.938 3509M / 6624M INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 15750 times 0:52:40.941 3509M / 6624M INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 11 0:52:40.941 3509M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:52:41.137 3509M / 6624M INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 16 times 0:52:41.137 3509M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:52:46.515 3512M / 6624M INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 1033 times 0:52:46.515 3512M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:52:46.657 3443M / 6624M INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 0 times 0:52:46.658 3443M / 6624M INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 12 0:52:46.658 3443M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:52:46.664 3443M / 6624M INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 0 times 0:52:46.664 3443M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:52:46.664 3443M / 6624M INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 0 times 0:52:46.664 3443M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:52:46.664 3443M / 6624M INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 0 times 0:52:46.771 3422M / 6624M INFO StageManager (stage.cpp : 185) STAGE == Simplification Cleanup (id: simplification_cleanup) 0:52:46.771 3422M / 6624M INFO General (simplification.cpp : 176) PROCEDURE == Post simplification 0:52:46.771 3422M / 6624M INFO General (graph_simplification.hpp : 448) Disconnection of relatively low covered edges disabled 0:52:46.771 3422M / 6624M INFO General (graph_simplification.hpp : 485) Complex tip clipping disabled 0:52:46.771 3422M / 6624M INFO General (graph_simplification.hpp : 634) Creating parallel br instance 0:52:46.771 3422M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:52:46.876 3422M / 6624M INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 0 times 0:52:46.877 3422M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:52:51.908 3430M / 6624M INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 5 times 0:52:51.908 3430M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:52:52.010 3430M / 6624M INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 0 times 0:52:52.010 3430M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:52:56.878 3430M / 6624M INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 1 times 0:52:56.878 3430M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:52:56.878 3430M / 6624M INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 0 times 0:52:56.878 3430M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:52:56.878 3430M / 6624M INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 0 times 0:52:56.878 3430M / 6624M INFO General (simplification.cpp : 327) Disrupting self-conjugate edges 0:52:57.536 3430M / 6624M INFO Simplification (parallel_processing.hpp : 167) Running Removing isolated edges 0:52:57.631 3430M / 6624M INFO Simplification (parallel_processing.hpp : 170) Removing isolated edges triggered 8 times 0:52:57.771 3422M / 6624M INFO General (simplification.cpp : 495) After simplification: 0:52:57.771 3422M / 6624M INFO General (simplification.cpp : 496) Average coverage = 305.815 0:52:57.903 3422M / 6624M INFO General (simplification.cpp : 497) Total length = 70781969 0:52:58.084 3422M / 6624M INFO General (simplification.cpp : 498) Median edge length: 885 0:52:58.085 3422M / 6624M INFO General (simplification.cpp : 499) Edges: 1591053 0:52:58.085 3422M / 6624M INFO General (simplification.cpp : 500) Vertices: 956440 0:52:58.085 3422M / 6624M INFO StageManager (stage.cpp : 185) STAGE == Contig Output (id: contig_output) 0:52:58.085 3422M / 6624M INFO General (read_converter.cpp : 107) Outputting contigs to /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K21/simplified_contigs 0:52:58.102 3425M / 6624M INFO General (binary_converter.cpp : 96) 16384 reads processed 0:52:58.114 3425M / 6624M INFO General (binary_converter.cpp : 96) 32768 reads processed 0:52:58.138 3425M / 6624M INFO General (binary_converter.cpp : 96) 65536 reads processed 0:52:58.184 3425M / 6624M INFO General (binary_converter.cpp : 96) 131072 reads processed 0:52:58.262 3425M / 6624M INFO General (binary_converter.cpp : 96) 262144 reads processed 0:52:58.420 3425M / 6624M INFO General (binary_converter.cpp : 96) 524288 reads processed 0:52:58.634 3425M / 6624M INFO General (binary_converter.cpp : 111) 795801 reads written 0:52:58.638 3422M / 6624M INFO General (pipeline.cpp : 287) SPAdes finished 0:53:00.016 1M / 6624M INFO General (main.cpp : 136) Assembling time: 0 hours 53 minutes 0 seconds ===== K21 finished. ===== K33 started. == Running: /oak/stanford/groups/akundaje/marinovg/programs/SPAdes-3.15.4-Linux/bin/spades-core /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K33/configs/config.info 0:00:00.000 1M / 13M INFO General (main.cpp : 99) Loaded config from /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K33/configs/config.info 0:00:00.000 1M / 13M INFO General (memory_limit.cpp : 54) Memory limit set to 250 Gb 0:00:00.000 1M / 13M INFO General (main.cpp : 107) Starting SPAdes, built from refs/heads/spades_3.15.4, git revision 1a5332e5608a073a9d28a6d0e30836f38dcc5613 0:00:00.000 1M / 13M INFO General (main.cpp : 108) Maximum k-mer length: 128 0:00:00.000 1M / 13M INFO General (main.cpp : 109) Assembling dataset (/oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/dataset.info) with K=33 0:00:00.000 1M / 13M INFO General (main.cpp : 110) Maximum # of threads to use (adjusted due to OMP capabilities): 8 0:00:00.000 1M / 13M INFO General (pipeline.cpp : 212) SPAdes started 0:00:00.000 1M / 13M INFO General (pipeline.cpp : 225) Starting from stage: read_conversion 0:00:00.000 1M / 13M INFO General (pipeline.cpp : 231) Two-step repeat resolution disabled 0:00:00.000 1M / 13M INFO GraphCore (graph_core.hpp : 672) Graph created, vertex min_id: 3, edge min_id: 3 0:00:00.000 1M / 13M INFO GraphCore (graph_core.hpp : 673) Vertex size: 48, edge size: 40 0:00:00.000 1M / 13M INFO General (edge_index.hpp : 113) Size of edge index entries: 12/8 0:00:00.000 1M / 13M INFO StageManager (stage.cpp : 185) STAGE == Binary Read Conversion (id: read_conversion) 0:00:00.002 1M / 13M INFO General (read_converter.cpp : 53) Binary reads detected 0:00:00.004 1M / 13M INFO General (read_converter.cpp : 53) Binary reads detected 0:00:00.005 1M / 13M INFO StageManager (stage.cpp : 185) STAGE == de Bruijn graph construction (id: construction) 0:00:00.005 1M / 13M INFO General (construction.cpp : 118) Contigs from previous K will be used: /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K21/simplified_contigs 0:00:00.052 1M / 13M INFO General (construction.cpp : 153) Max read length 151 0:00:00.052 1M / 13M INFO General (construction.cpp : 159) Average read length 146.594 0:00:00.052 1M / 13M INFO General (stage.cpp : 117) PROCEDURE == k+1-mer counting (id: construction:kpomer_counting) 0:00:00.052 1M / 13M INFO General (kmer_index_builder.hpp : 243) Splitting kmer instances into 80 files using 8 threads. This might take a while. 0:00:00.054 1M / 13M INFO General (file_limit.hpp : 42) Open file limit set to 131072 0:00:00.054 1M / 13M INFO General (kmer_splitter.hpp : 93) Memory available for splitting buffers: 10.4166 Gb 0:00:00.054 1M / 13M INFO General (kmer_splitter.hpp : 101) Using cell size of 419430 0:00:10.495 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 3768534 reads 0:00:20.776 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 7413743 reads 0:00:30.696 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 11056248 reads 0:00:42.095 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 14713645 reads 0:00:51.880 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 18377242 reads 0:01:01.532 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 22024253 reads 0:01:11.422 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 25656261 reads 0:01:20.829 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 29257088 reads 0:01:29.835 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 32850435 reads 0:01:40.073 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 36406312 reads 0:01:49.904 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 39973766 reads 0:03:06.700 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 68747598 reads 0:06:11.277 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 136882609 reads 0:12:07.487 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 270497302 reads 0:16:21.559 1M / 4252M INFO General (kmer_splitters.hpp : 131) Used 353926936 reads 0:16:22.060 1M / 4252M INFO General (kmer_index_builder.hpp : 249) Starting k-mer counting. 0:17:48.834 1M / 12G INFO General (kmer_index_builder.hpp : 260) K-mer counting done. There are 272656109 kmers in total. 0:17:48.895 1M / 12G INFO General (stage.cpp : 117) PROCEDURE == Extension index construction (id: construction:extension_index_construction) 0:17:48.995 1M / 12G INFO K-mer Index Building (kmer_index_builder.hpp : 438) Building kmer index 0:17:49.005 1M / 12G INFO General (kmer_index_builder.hpp : 243) Splitting kmer instances into 80 files using 8 threads. This might take a while. 0:17:49.006 1M / 12G INFO General (file_limit.hpp : 42) Open file limit set to 131072 0:17:49.060 1M / 12G INFO General (kmer_splitter.hpp : 93) Memory available for splitting buffers: 10.4166 Gb 0:17:49.060 1M / 12G INFO General (kmer_splitter.hpp : 101) Using cell size of 419430 0:17:51.513 4641M / 12G INFO General (kmer_splitters.hpp : 194) Processed 14529098 kmers 0:17:54.612 4641M / 12G INFO General (kmer_splitters.hpp : 194) Processed 40509988 kmers 0:17:57.830 4641M / 12G INFO General (kmer_splitters.hpp : 194) Processed 67651388 kmers 0:18:01.070 4641M / 12G INFO General (kmer_splitters.hpp : 194) Processed 94880357 kmers 0:18:04.709 4641M / 12G INFO General (kmer_splitters.hpp : 194) Processed 122151822 kmers 0:18:07.957 4641M / 12G INFO General (kmer_splitters.hpp : 194) Processed 149427761 kmers 0:18:11.734 4641M / 12G INFO General (kmer_splitters.hpp : 194) Processed 176694037 kmers 0:18:15.508 4641M / 12G INFO General (kmer_splitters.hpp : 194) Processed 203972039 kmers 0:18:19.003 4641M / 12G INFO General (kmer_splitters.hpp : 194) Processed 231213775 kmers 0:18:22.410 4641M / 12G INFO General (kmer_splitters.hpp : 194) Processed 258489814 kmers 0:18:24.513 4641M / 12G INFO General (kmer_splitters.hpp : 194) Processed 272656549 kmers 0:18:24.514 4641M / 12G INFO General (kmer_splitters.hpp : 199) Used 272656549 kmers. 0:18:24.514 1M / 12G INFO General (kmer_index_builder.hpp : 249) Starting k-mer counting. 0:18:31.293 1M / 12G INFO General (kmer_index_builder.hpp : 260) K-mer counting done. There are 267524969 kmers in total. 0:18:31.294 1M / 12G INFO K-mer Index Building (kmer_index_builder.hpp : 395) Building perfect hash indices 0:18:38.496 202M / 12G INFO K-mer Index Building (kmer_index_builder.hpp : 431) Index built. Total 267524969 kmers, 193282096 bytes occupied (5.77986 bits per kmer). 0:18:38.497 202M / 12G INFO General (kmer_index_builder.hpp : 169) Merging final buckets. 0:18:50.060 458M / 12G INFO DeBruijnExtensionIndexBu (kmer_extension_index_build: 100) Building k-mer extensions from k+1-mers 0:19:22.470 458M / 12G INFO DeBruijnExtensionIndexBu (kmer_extension_index_build: 105) Building k-mer extensions from k+1-mers finished. 0:19:22.817 458M / 12G INFO General (stage.cpp : 117) PROCEDURE == Early tip clipping (id: construction:early_tip_clipper) 0:19:22.817 458M / 12G INFO General (construction.cpp : 301) Early tip clipper length bound set as (RL - K) 0:19:22.818 458M / 12G INFO Early tip clipping (early_simplification.hpp : 47) Early tip clipping 0:19:56.558 778M / 12G INFO Early tip clipping (early_simplification.hpp : 82) #tipped junctions: 4649827 0:19:57.303 778M / 12G INFO Early tip clipping (early_simplification.hpp : 93) Clipped tips: 4765972 0:19:57.650 458M / 12G INFO Early tip clipping (early_simplification.hpp : 49) 63169392 34-mers were removed by early tip clipper 0:19:57.651 458M / 12G INFO General (stage.cpp : 117) PROCEDURE == Condensing graph (id: construction:graph_condensing) 0:19:57.671 458M / 12G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 354) Extracting unbranching paths 0:20:33.879 1175M / 12G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 373) Extracting unbranching paths finished. 15396055 sequences extracted 0:20:48.482 1175M / 12G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 309) Collecting perfect loops 0:20:53.751 1175M / 12G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 342) Collecting perfect loops finished. 31 loops collected 0:20:54.970 1175M / 12G INFO General (debruijn_graph_constructor: 487) Total 30792172 edges to create 0:20:55.006 2363M / 12G INFO General (debruijn_graph_constructor: 489) Collecting link records 0:21:04.102 2835M / 12G INFO General (debruijn_graph_constructor: 491) Ordering link records 0:21:05.129 2835M / 12G INFO General (debruijn_graph_constructor: 493) Sorting done 0:21:05.400 2964M / 12G INFO General (debruijn_graph_constructor: 503) Total 10264946 vertices to create 0:21:05.401 3914M / 12G INFO General (debruijn_graph_constructor: 506) Connecting the graph 0:21:19.354 3084M / 12G INFO General (stage.cpp : 117) PROCEDURE == Filling coverage indices (PHM) (id: construction:coverage_filling_phm) 0:21:19.355 3084M / 12G INFO K-mer Index Building (kmer_index_builder.hpp : 395) Building perfect hash indices 0:21:26.697 3286M / 12G INFO K-mer Index Building (kmer_index_builder.hpp : 431) Index built. Total 272656109 kmers, 196988192 bytes occupied (5.77983 bits per kmer). 0:21:27.466 4330M / 12G INFO General (coverage_hash_map_builder.: 47) Collecting k-mer coverage information from reads, this takes a while. 0:40:07.457 4330M / 12G INFO General (construction.cpp : 430) Filling coverage and flanking coverage from PHM 0:40:14.993 4330M / 12G INFO General (coverage_filling.hpp : 82) Processed 8000000 edges 0:40:21.345 4330M / 12G INFO General (coverage_filling.hpp : 82) Processed 16000000 edges 0:40:27.680 4330M / 12G INFO General (coverage_filling.hpp : 82) Processed 24000000 edges 0:40:33.093 4330M / 12G INFO General (coverage_filling.hpp : 82) Processed 30791879 edges 0:40:34.761 3195M / 12G INFO StageManager (stage.cpp : 185) STAGE == EC Threshold Finding (id: ec_threshold_finder) 0:40:38.793 3763M / 12G INFO General (kmer_coverage_model.cpp : 181) Kmer coverage valley at: 42 0:40:38.983 3195M / 12G INFO General (kmer_coverage_model.cpp : 201) K-mer histogram maximum: 201 0:40:39.031 3195M / 12G INFO General (kmer_coverage_model.cpp : 237) Estimated median coverage: 201. Coverage mad: 40.0302 0:40:39.125 3195M / 12G INFO General (kmer_coverage_model.cpp : 259) Fitting coverage model 0:40:39.995 3763M / 12G INFO General (kmer_coverage_model.cpp : 295) ... iteration 2 0:40:40.786 3763M / 12G INFO General (kmer_coverage_model.cpp : 295) ... iteration 4 0:40:43.066 3763M / 12G INFO General (kmer_coverage_model.cpp : 295) ... iteration 8 0:40:47.373 3763M / 12G INFO General (kmer_coverage_model.cpp : 295) ... iteration 16 0:40:55.984 3763M / 12G INFO General (kmer_coverage_model.cpp : 295) ... iteration 32 0:40:57.004 3763M / 12G INFO General (kmer_coverage_model.cpp : 309) Fitted mean coverage: 183.382. Fitted coverage std. dev: 47.2799 0:40:57.009 3763M / 12G INFO General (kmer_coverage_model.cpp : 334) Probability of erroneous kmer at valley: 0.638567 0:40:57.010 3763M / 12G INFO General (kmer_coverage_model.cpp : 358) Preliminary threshold calculated as: 101 0:40:57.010 3763M / 12G INFO General (kmer_coverage_model.cpp : 362) Threshold adjusted to: 101 0:40:57.010 3763M / 12G INFO General (kmer_coverage_model.cpp : 375) Estimated genome size (ignoring repeats): 71452523 0:40:57.068 3195M / 12G INFO General (genomic_info_filler.cpp : 55) Mean coverage was calculated as 183.382 0:40:57.089 3195M / 12G INFO General (genomic_info_filler.cpp : 70) EC coverage threshold value was calculated as 101 0:40:57.089 3195M / 12G INFO General (genomic_info_filler.cpp : 71) Trusted kmer low bound: 0 0:40:57.089 3195M / 12G INFO StageManager (stage.cpp : 185) STAGE == Raw Simplification (id: raw_simplification) 0:40:57.165 3195M / 12G INFO General (simplification.cpp : 127) PROCEDURE == Initial cleaning 0:40:57.571 3195M / 12G INFO General (graph_simplification.hpp : 662) Flanking coverage based disconnection disabled 0:40:57.591 3195M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Self conjugate edge remover 0:40:59.161 3195M / 12G INFO Simplification (parallel_processing.hpp : 170) Self conjugate edge remover triggered 43 times 0:40:59.161 3195M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Initial tip clipper 0:41:03.272 3189M / 12G INFO Simplification (parallel_processing.hpp : 170) Initial tip clipper triggered 374755 times 0:41:03.306 3189M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Initial ec remover 0:41:52.523 3318M / 12G INFO Simplification (parallel_processing.hpp : 170) Initial ec remover triggered 3689490 times 0:41:52.684 3318M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Initial isolated edge remover 0:41:53.975 3313M / 12G INFO Simplification (parallel_processing.hpp : 170) Initial isolated edge remover triggered 297753 times 0:41:54.030 2872M / 12G INFO StageManager (stage.cpp : 185) STAGE == Simplification (id: simplification) 0:41:54.040 2872M / 12G INFO General (simplification.cpp : 368) Graph simplification started 0:41:54.052 2872M / 12G INFO General (graph_simplification.hpp : 634) Creating parallel br instance 0:41:54.052 2872M / 12G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 1 0:41:54.053 2872M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:41:55.464 2867M / 12G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 121474 times 0:41:55.481 2867M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:43:01.782 2888M / 12G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 629906 times 0:43:01.838 2888M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:43:15.127 2892M / 12G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 810865 times 0:43:15.209 2892M / 12G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 2 0:43:15.209 2892M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:43:15.925 2855M / 12G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 13028 times 0:43:15.928 2855M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:43:23.287 2805M / 12G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 74400 times 0:43:23.312 2805M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:43:23.900 2806M / 12G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 52238 times 0:43:23.911 2806M / 12G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 3 0:43:23.911 2806M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:43:24.052 2802M / 12G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 319 times 0:43:24.052 2802M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:43:25.077 2788M / 12G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 8320 times 0:43:25.079 2788M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:43:25.228 2788M / 12G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 13305 times 0:43:25.231 2788M / 12G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 4 0:43:25.231 2788M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:43:25.268 2787M / 12G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 65 times 0:43:25.268 2787M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:43:25.748 2784M / 12G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 2295 times 0:43:25.749 2784M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:43:25.818 2784M / 12G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 6415 times 0:43:25.819 2784M / 12G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 5 0:43:25.819 2784M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:43:25.831 2783M / 12G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 19 times 0:43:25.832 2783M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:43:26.041 2783M / 12G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 1001 times 0:43:26.041 2783M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:43:26.095 2783M / 12G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 4438 times 0:43:26.095 2783M / 12G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 6 0:43:26.095 2783M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:43:26.101 2782M / 12G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 8 times 0:43:26.101 2782M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:43:26.231 2782M / 12G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 596 times 0:43:26.232 2782M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:43:26.270 2782M / 12G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 3207 times 0:43:26.271 2782M / 12G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 7 0:43:26.271 2782M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:43:26.274 2782M / 12G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 3 times 0:43:26.275 2782M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:43:26.348 2782M / 12G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 361 times 0:43:26.348 2782M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:43:26.378 2782M / 12G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 2513 times 0:43:26.379 2782M / 12G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 8 0:43:26.379 2782M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:43:26.381 2781M / 12G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 4 times 0:43:26.381 2781M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:43:26.438 2781M / 12G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 287 times 0:43:26.439 2781M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:43:26.466 2781M / 12G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 2210 times 0:43:26.467 2781M / 12G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 9 0:43:26.467 2781M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:43:26.469 2781M / 12G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 2 times 0:43:26.469 2781M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:43:26.512 2781M / 12G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 227 times 0:43:26.512 2781M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:43:26.538 2781M / 12G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 2026 times 0:43:26.538 2781M / 12G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 10 0:43:26.538 2781M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:43:26.540 2781M / 12G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 4 times 0:43:26.540 2781M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:43:26.588 2781M / 12G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 222 times 0:43:26.589 2781M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:43:26.615 2781M / 12G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 2035 times 0:43:26.615 2781M / 12G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 11 0:43:26.615 2781M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:43:26.706 2781M / 12G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 18 times 0:43:26.706 2781M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:43:27.283 2783M / 12G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 328 times 0:43:27.284 2783M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:43:27.353 2747M / 12G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 0 times 0:43:27.353 2747M / 12G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 12 0:43:27.354 2747M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:43:27.355 2747M / 12G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 0 times 0:43:27.355 2747M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:43:27.355 2747M / 12G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 0 times 0:43:27.355 2747M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 0:43:27.355 2747M / 12G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 0 times 0:43:27.434 2736M / 12G INFO StageManager (stage.cpp : 185) STAGE == Simplification Cleanup (id: simplification_cleanup) 0:43:27.480 2736M / 12G INFO General (simplification.cpp : 176) PROCEDURE == Post simplification 0:43:27.484 2736M / 12G INFO General (graph_simplification.hpp : 448) Disconnection of relatively low covered edges disabled 0:43:27.484 2736M / 12G INFO General (graph_simplification.hpp : 485) Complex tip clipping disabled 0:43:27.484 2736M / 12G INFO General (graph_simplification.hpp : 634) Creating parallel br instance 0:43:27.505 2736M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:43:27.570 2736M / 12G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 0 times 0:43:27.570 2736M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:43:28.089 2738M / 12G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 5 times 0:43:28.089 2738M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 0:43:28.160 2738M / 12G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 0 times 0:43:28.160 2738M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 0:43:28.644 2738M / 12G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 0 times 0:43:28.644 2738M / 12G INFO General (simplification.cpp : 327) Disrupting self-conjugate edges 0:43:28.967 2738M / 12G INFO Simplification (parallel_processing.hpp : 167) Running Removing isolated edges 0:43:29.027 2738M / 12G INFO Simplification (parallel_processing.hpp : 170) Removing isolated edges triggered 23 times 0:43:29.101 2736M / 12G INFO General (simplification.cpp : 495) After simplification: 0:43:29.124 2736M / 12G INFO General (simplification.cpp : 496) Average coverage = 256.473 0:43:29.195 2736M / 12G INFO General (simplification.cpp : 497) Total length = 76860844 0:43:29.296 2736M / 12G INFO General (simplification.cpp : 498) Median edge length: 3416 0:43:29.296 2736M / 12G INFO General (simplification.cpp : 499) Edges: 679257 0:43:29.296 2736M / 12G INFO General (simplification.cpp : 500) Vertices: 434948 0:43:29.296 2736M / 12G INFO StageManager (stage.cpp : 185) STAGE == Contig Output (id: contig_output) 0:43:29.366 2736M / 12G INFO General (read_converter.cpp : 107) Outputting contigs to /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K33/simplified_contigs 0:43:29.524 2739M / 12G INFO General (binary_converter.cpp : 96) 16384 reads processed 0:43:29.545 2739M / 12G INFO General (binary_converter.cpp : 96) 32768 reads processed 0:43:29.639 2739M / 12G INFO General (binary_converter.cpp : 96) 65536 reads processed 0:43:29.784 2739M / 12G INFO General (binary_converter.cpp : 96) 131072 reads processed 0:43:29.928 2739M / 12G INFO General (binary_converter.cpp : 96) 262144 reads processed 0:43:30.114 2739M / 12G INFO General (binary_converter.cpp : 111) 339656 reads written 0:43:30.470 2736M / 12G INFO General (pipeline.cpp : 287) SPAdes finished 0:43:31.590 1M / 12G INFO General (main.cpp : 136) Assembling time: 0 hours 43 minutes 31 seconds ===== K33 finished. ===== K55 started. == Running: /oak/stanford/groups/akundaje/marinovg/programs/SPAdes-3.15.4-Linux/bin/spades-core /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K55/configs/config.info 0:00:00.000 1M / 13M INFO General (main.cpp : 99) Loaded config from /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K55/configs/config.info 0:00:00.000 1M / 13M INFO General (memory_limit.cpp : 54) Memory limit set to 250 Gb 0:00:00.000 1M / 13M INFO General (main.cpp : 107) Starting SPAdes, built from refs/heads/spades_3.15.4, git revision 1a5332e5608a073a9d28a6d0e30836f38dcc5613 0:00:00.000 1M / 13M INFO General (main.cpp : 108) Maximum k-mer length: 128 0:00:00.000 1M / 13M INFO General (main.cpp : 109) Assembling dataset (/oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/dataset.info) with K=55 0:00:00.000 1M / 13M INFO General (main.cpp : 110) Maximum # of threads to use (adjusted due to OMP capabilities): 8 0:00:00.000 1M / 13M INFO General (pipeline.cpp : 212) SPAdes started 0:00:00.000 1M / 13M INFO General (pipeline.cpp : 225) Starting from stage: read_conversion 0:00:00.000 1M / 13M INFO General (pipeline.cpp : 231) Two-step repeat resolution disabled 0:00:00.000 1M / 13M INFO GraphCore (graph_core.hpp : 672) Graph created, vertex min_id: 3, edge min_id: 3 0:00:00.000 1M / 13M INFO GraphCore (graph_core.hpp : 673) Vertex size: 48, edge size: 40 0:00:00.001 1M / 13M INFO General (edge_index.hpp : 113) Size of edge index entries: 12/8 0:00:00.001 1M / 13M INFO General (pipeline.cpp : 242) Will need read mapping, kmer mapper will be attached 0:00:00.001 1M / 13M INFO StageManager (stage.cpp : 185) STAGE == Binary Read Conversion (id: read_conversion) 0:00:00.002 1M / 13M INFO General (read_converter.cpp : 53) Binary reads detected 0:00:00.004 1M / 13M INFO General (read_converter.cpp : 53) Binary reads detected 0:00:00.005 1M / 13M INFO StageManager (stage.cpp : 185) STAGE == de Bruijn graph construction (id: construction) 0:00:00.005 1M / 13M INFO General (construction.cpp : 118) Contigs from previous K will be used: /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K33/simplified_contigs 0:00:00.128 1M / 13M INFO General (construction.cpp : 153) Max read length 151 0:00:00.128 1M / 13M INFO General (construction.cpp : 159) Average read length 146.594 0:00:00.128 1M / 13M INFO General (stage.cpp : 117) PROCEDURE == k+1-mer counting (id: construction:kpomer_counting) 0:00:00.128 1M / 13M INFO General (kmer_index_builder.hpp : 243) Splitting kmer instances into 80 files using 8 threads. This might take a while. 0:00:00.129 1M / 13M INFO General (file_limit.hpp : 42) Open file limit set to 131072 0:00:00.129 1M / 13M INFO General (kmer_splitter.hpp : 93) Memory available for splitting buffers: 10.4166 Gb 0:00:00.130 1M / 13M INFO General (kmer_splitter.hpp : 101) Using cell size of 419430 0:00:11.956 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 4758017 reads 0:00:21.621 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 9363769 reads 0:00:31.358 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 13996683 reads 0:00:41.666 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 18633592 reads 0:00:51.906 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 23253381 reads 0:01:01.302 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 27829326 reads 0:01:10.440 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 32378284 reads 0:01:19.507 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 36884690 reads 0:01:30.563 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 41416506 reads 0:01:40.018 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 45954682 reads 0:01:49.777 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 50543642 reads 0:02:28.537 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 68783698 reads 0:04:56.485 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 136955510 reads 0:09:36.848 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 269760742 reads 0:13:01.230 1M / 4478M INFO General (kmer_splitters.hpp : 131) Used 353014646 reads 0:13:01.848 1M / 4478M INFO General (kmer_index_builder.hpp : 249) Starting k-mer counting. 0:14:12.274 1M / 10G INFO General (kmer_index_builder.hpp : 260) K-mer counting done. There are 343015708 kmers in total. 0:14:12.274 1M / 10G INFO General (stage.cpp : 117) PROCEDURE == Extension index construction (id: construction:extension_index_construction) 0:14:12.314 1M / 10G INFO K-mer Index Building (kmer_index_builder.hpp : 438) Building kmer index 0:14:12.315 1M / 10G INFO General (kmer_index_builder.hpp : 243) Splitting kmer instances into 80 files using 8 threads. This might take a while. 0:14:12.316 1M / 10G INFO General (file_limit.hpp : 42) Open file limit set to 131072 0:14:12.316 1M / 10G INFO General (kmer_splitter.hpp : 93) Memory available for splitting buffers: 10.4166 Gb 0:14:12.317 1M / 10G INFO General (kmer_splitter.hpp : 101) Using cell size of 419430 0:14:14.870 4641M / 10G INFO General (kmer_splitters.hpp : 194) Processed 14545587 kmers 0:14:17.230 4641M / 10G INFO General (kmer_splitters.hpp : 194) Processed 30220898 kmers 0:14:19.995 4641M / 10G INFO General (kmer_splitters.hpp : 194) Processed 48216978 kmers 0:14:22.199 4641M / 10G INFO General (kmer_splitters.hpp : 194) Processed 64047265 kmers 0:14:24.802 4641M / 10G INFO General (kmer_splitters.hpp : 194) Processed 82485857 kmers 0:14:27.233 4641M / 10G INFO General (kmer_splitters.hpp : 194) Processed 98318207 kmers 0:14:29.879 4641M / 10G INFO General (kmer_splitters.hpp : 194) Processed 116768484 kmers 0:14:32.259 4641M / 10G INFO General (kmer_splitters.hpp : 194) Processed 132629032 kmers 0:14:34.816 4641M / 10G INFO General (kmer_splitters.hpp : 194) Processed 151074687 kmers 0:14:37.206 4641M / 10G INFO General (kmer_splitters.hpp : 194) Processed 166922970 kmers 0:14:39.842 4641M / 10G INFO General (kmer_splitters.hpp : 194) Processed 185389092 kmers 0:14:42.292 4641M / 10G INFO General (kmer_splitters.hpp : 194) Processed 201243170 kmers 0:14:45.160 4641M / 10G INFO General (kmer_splitters.hpp : 194) Processed 219682077 kmers 0:14:47.311 4641M / 10G INFO General (kmer_splitters.hpp : 194) Processed 235531340 kmers 0:14:49.898 4641M / 10G INFO General (kmer_splitters.hpp : 194) Processed 253984392 kmers 0:14:52.321 4641M / 10G INFO General (kmer_splitters.hpp : 194) Processed 269842392 kmers 0:14:54.987 4641M / 10G INFO General (kmer_splitters.hpp : 194) Processed 288286261 kmers 0:14:57.336 4641M / 10G INFO General (kmer_splitters.hpp : 194) Processed 304131862 kmers 0:15:00.164 4641M / 10G INFO General (kmer_splitters.hpp : 194) Processed 322577782 kmers 0:15:03.197 4641M / 10G INFO General (kmer_splitters.hpp : 199) Used 343016588 kmers. 0:15:03.198 1M / 10G INFO General (kmer_index_builder.hpp : 249) Starting k-mer counting. 0:15:12.237 1M / 10G INFO General (kmer_index_builder.hpp : 260) K-mer counting done. There are 341106642 kmers in total. 0:15:12.237 1M / 10G INFO K-mer Index Building (kmer_index_builder.hpp : 395) Building perfect hash indices 0:15:21.542 264M / 10G INFO K-mer Index Building (kmer_index_builder.hpp : 431) Index built. Total 341106642 kmers, 246427504 bytes occupied (5.77948 bits per kmer). 0:15:21.543 264M / 10G INFO General (kmer_index_builder.hpp : 169) Merging final buckets. 0:15:36.222 592M / 10G INFO DeBruijnExtensionIndexBu (kmer_extension_index_build: 100) Building k-mer extensions from k+1-mers 0:16:16.365 592M / 10G INFO DeBruijnExtensionIndexBu (kmer_extension_index_build: 105) Building k-mer extensions from k+1-mers finished. 0:16:16.375 592M / 10G INFO General (stage.cpp : 117) PROCEDURE == Condensing graph (id: construction:graph_condensing) 0:16:16.379 592M / 10G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 354) Extracting unbranching paths 0:17:12.840 1562M / 10G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 373) Extracting unbranching paths finished. 20243965 sequences extracted 0:17:36.207 1562M / 10G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 309) Collecting perfect loops 0:17:44.236 1562M / 10G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 342) Collecting perfect loops finished. 1 loops collected 0:17:46.409 1562M / 10G INFO General (debruijn_graph_constructor: 487) Total 40487932 edges to create 0:17:46.411 3124M / 10G INFO General (debruijn_graph_constructor: 489) Collecting link records 0:17:57.449 3744M / 10G INFO General (debruijn_graph_constructor: 491) Ordering link records 0:17:58.273 3744M / 10G INFO General (debruijn_graph_constructor: 493) Sorting done 0:17:58.707 4000M / 10G INFO General (debruijn_graph_constructor: 503) Total 18334900 vertices to create 0:17:58.709 5693M / 10G INFO General (debruijn_graph_constructor: 506) Connecting the graph 0:18:21.306 4508M / 10G INFO General (stage.cpp : 117) PROCEDURE == Filling coverage indices (PHM) (id: construction:coverage_filling_phm) 0:18:21.306 4508M / 10G INFO K-mer Index Building (kmer_index_builder.hpp : 395) Building perfect hash indices 0:18:38.993 4770M / 10G INFO K-mer Index Building (kmer_index_builder.hpp : 431) Index built. Total 343015708 kmers, 247805992 bytes occupied (5.77947 bits per kmer). 0:18:39.873 6083M / 10G INFO General (coverage_hash_map_builder.: 47) Collecting k-mer coverage information from reads, this takes a while. 0:34:15.674 6083M / 10G INFO General (construction.cpp : 430) Filling coverage and flanking coverage from PHM 0:34:26.120 6083M / 10G INFO General (coverage_filling.hpp : 82) Processed 8000000 edges 0:34:35.457 6083M / 10G INFO General (coverage_filling.hpp : 82) Processed 16000000 edges 0:34:44.117 6083M / 10G INFO General (coverage_filling.hpp : 82) Processed 24000000 edges 0:34:52.967 6083M / 10G INFO General (coverage_filling.hpp : 82) Processed 32000000 edges 0:35:01.729 6083M / 10G INFO General (coverage_filling.hpp : 82) Processed 40000000 edges 0:35:02.405 6083M / 10G INFO General (coverage_filling.hpp : 82) Processed 40487889 edges 0:35:05.157 4353M / 10G INFO StageManager (stage.cpp : 185) STAGE == EC Threshold Finding (id: ec_threshold_finder) 0:35:07.684 4789M / 10G INFO General (kmer_coverage_model.cpp : 181) Kmer coverage valley at: 33 0:35:07.828 4353M / 10G INFO General (kmer_coverage_model.cpp : 201) K-mer histogram maximum: 163 0:35:07.828 4353M / 10G INFO General (kmer_coverage_model.cpp : 237) Estimated median coverage: 163. Coverage mad: 35.5824 0:35:07.900 4353M / 10G INFO General (kmer_coverage_model.cpp : 259) Fitting coverage model 0:35:08.402 4789M / 10G INFO General (kmer_coverage_model.cpp : 295) ... iteration 2 0:35:08.897 4789M / 10G INFO General (kmer_coverage_model.cpp : 295) ... iteration 4 0:35:10.637 4789M / 10G INFO General (kmer_coverage_model.cpp : 295) ... iteration 8 0:35:14.489 4789M / 10G INFO General (kmer_coverage_model.cpp : 295) ... iteration 16 0:35:21.467 4789M / 10G INFO General (kmer_coverage_model.cpp : 295) ... iteration 32 0:35:21.923 4789M / 10G INFO General (kmer_coverage_model.cpp : 309) Fitted mean coverage: 147.56. Fitted coverage std. dev: 40.3741 0:35:21.929 4789M / 10G INFO General (kmer_coverage_model.cpp : 334) Probability of erroneous kmer at valley: 0.572312 0:35:21.929 4789M / 10G INFO General (kmer_coverage_model.cpp : 358) Preliminary threshold calculated as: 75 0:35:21.929 4789M / 10G INFO General (kmer_coverage_model.cpp : 362) Threshold adjusted to: 75 0:35:21.929 4789M / 10G INFO General (kmer_coverage_model.cpp : 375) Estimated genome size (ignoring repeats): 74716758 0:35:21.992 4353M / 10G INFO General (genomic_info_filler.cpp : 55) Mean coverage was calculated as 147.56 0:35:21.992 4353M / 10G INFO General (genomic_info_filler.cpp : 70) EC coverage threshold value was calculated as 75 0:35:21.992 4353M / 10G INFO General (genomic_info_filler.cpp : 71) Trusted kmer low bound: 0 0:35:21.992 4353M / 10G INFO StageManager (stage.cpp : 185) STAGE == Gap Closer (id: early_gapcloser) 0:35:21.999 4362M / 10G INFO General (edge_index.hpp : 113) Size of edge index entries: 12/8 0:35:25.441 4634M / 10G INFO General (gap_closer.cpp : 101) Total edges in tip neighborhood: 8257546 out of 40487889, length: 178119890 0:35:26.950 4750M / 10G INFO General (edge_index.hpp : 167) Using small index (max_id = 40690374) 0:35:29.476 4750M / 10G INFO K-mer Index Building (kmer_index_builder.hpp : 395) Building perfect hash indices 0:35:51.876 4861M / 10G INFO K-mer Index Building (kmer_index_builder.hpp : 431) Index built. Total 154002822 kmers, 111231488 bytes occupied (5.77815 bits per kmer). 0:35:52.656 6037M / 10G INFO General (edge_index_builders.hpp : 265) Collecting edge information from graph, this takes a while. 0:36:32.723 6037M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 200000 reads 0:36:32.852 6037M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 400000 reads 0:36:32.983 6037M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 600000 reads 0:36:32.991 5921M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 800000 reads 0:36:33.039 5922M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 1000000 reads 0:36:33.078 5922M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 1200000 reads 0:37:01.549 5922M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 2200000 reads 0:37:30.691 5922M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 4200000 reads 0:38:52.669 5923M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 8400000 reads 0:41:13.209 5924M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 16800000 reads 0:46:03.360 5926M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 33600000 reads 0:55:45.661 5932M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 67200000 reads 1:01:56.787 6016M / 10G INFO General (sequence_mapper_notifier.h: 95) Total 87973211 reads processed 1:01:59.173 4517M / 10G INFO General (gap_closer.cpp : 452) Initializing gap closer 1:01:59.224 4517M / 10G INFO GapCloser (gap_closer.cpp : 359) Closing short gaps 1:02:54.465 4517M / 10G INFO GapCloser (gap_closer.cpp : 391) Closing short gaps complete: filled 269 gaps after checking 52814 candidates 1:02:55.693 4517M / 10G INFO General (gap_closer.cpp : 456) Gap closer done 1:02:56.132 4353M / 10G INFO StageManager (stage.cpp : 185) STAGE == Raw Simplification (id: raw_simplification) 1:02:56.319 4353M / 10G INFO General (simplification.cpp : 127) PROCEDURE == Initial cleaning 1:02:56.720 4353M / 10G INFO General (graph_simplification.hpp : 662) Flanking coverage based disconnection disabled 1:02:56.750 4353M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Self conjugate edge remover 1:02:58.605 4353M / 10G INFO Simplification (parallel_processing.hpp : 170) Self conjugate edge remover triggered 4 times 1:02:58.605 4353M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Initial tip clipper 1:03:52.998 4084M / 10G INFO Simplification (parallel_processing.hpp : 170) Initial tip clipper triggered 6206926 times 1:03:53.103 4084M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Initial ec remover 1:04:20.465 4091M / 10G INFO Simplification (parallel_processing.hpp : 170) Initial ec remover triggered 1864203 times 1:04:20.600 4091M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Initial isolated edge remover 1:04:23.041 4070M / 10G INFO Simplification (parallel_processing.hpp : 170) Initial isolated edge remover triggered 851713 times 1:04:23.093 3802M / 10G INFO StageManager (stage.cpp : 185) STAGE == Simplification (id: simplification) 1:04:23.100 3802M / 10G INFO General (simplification.cpp : 368) Graph simplification started 1:04:23.121 3802M / 10G INFO General (graph_simplification.hpp : 634) Creating parallel br instance 1:04:23.121 3802M / 10G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 1 1:04:23.121 3802M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 1:04:24.714 3796M / 10G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 129820 times 1:04:24.738 3796M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 1:06:30.951 6352M / 10G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 449704 times 1:06:30.997 6352M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 1:06:35.445 6363M / 10G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 285811 times 1:06:35.459 6363M / 10G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 2 1:06:35.461 6363M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 1:06:36.004 6310M / 10G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 12161 times 1:06:36.011 6310M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 1:06:50.347 6547M / 10G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 43201 times 1:06:50.357 6547M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 1:06:50.747 6546M / 10G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 26489 times 1:06:50.755 6546M / 10G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 3 1:06:50.758 6546M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 1:06:50.844 6542M / 10G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 423 times 1:06:50.847 6542M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 1:06:52.585 6563M / 10G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 4671 times 1:06:52.594 6563M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 1:06:52.684 6564M / 10G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 6411 times 1:06:52.688 6564M / 10G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 4 1:06:52.690 6564M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 1:06:52.712 6563M / 10G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 67 times 1:06:52.714 6563M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 1:06:53.212 6569M / 10G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 1288 times 1:06:53.217 6569M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 1:06:53.265 6569M / 10G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 3041 times 1:06:53.268 6569M / 10G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 5 1:06:53.270 6569M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 1:06:53.279 6568M / 10G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 30 times 1:06:53.281 6568M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 1:06:53.489 6571M / 10G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 536 times 1:06:53.493 6571M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 1:06:53.524 6571M / 10G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 1911 times 1:06:53.526 6571M / 10G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 6 1:06:53.529 6571M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 1:06:53.534 6570M / 10G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 12 times 1:06:53.536 6570M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 1:06:53.633 6572M / 10G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 262 times 1:06:53.637 6572M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 1:06:53.661 6572M / 10G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 1419 times 1:06:53.664 6572M / 10G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 7 1:06:53.666 6572M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 1:06:53.670 6572M / 10G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 4 times 1:06:53.673 6572M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 1:06:53.746 6573M / 10G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 183 times 1:06:53.749 6573M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 1:06:53.770 6572M / 10G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 1113 times 1:06:53.773 6572M / 10G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 8 1:06:53.775 6572M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 1:06:53.779 6572M / 10G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 5 times 1:06:53.781 6572M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 1:06:53.825 6573M / 10G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 115 times 1:06:53.828 6573M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 1:06:53.846 6573M / 10G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 1027 times 1:06:53.849 6573M / 10G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 9 1:06:53.851 6573M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 1:06:53.854 6573M / 10G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 3 times 1:06:53.857 6573M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 1:06:53.908 6574M / 10G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 124 times 1:06:53.911 6574M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 1:06:53.931 6574M / 10G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 980 times 1:06:53.934 6574M / 10G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 10 1:06:53.936 6574M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 1:06:53.939 6574M / 10G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 1 times 1:06:53.942 6574M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 1:06:53.978 6574M / 10G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 87 times 1:06:53.981 6574M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 1:06:53.999 6574M / 10G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 895 times 1:06:54.002 6574M / 10G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 11 1:06:54.004 6574M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 1:06:54.081 6574M / 10G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 8 times 1:06:54.085 6574M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 1:06:54.377 6576M / 10G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 152 times 1:06:54.381 6576M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 1:06:54.456 6559M / 10G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 1 times 1:06:54.460 6559M / 10G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 12 1:06:54.463 6559M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 1:06:54.465 6559M / 10G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 0 times 1:06:54.468 6559M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 1:06:54.470 6559M / 10G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 0 times 1:06:54.473 6559M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 1:06:54.475 6559M / 10G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 0 times 1:06:54.535 6554M / 10G INFO StageManager (stage.cpp : 185) STAGE == Gap Closer (id: late_gapcloser) 1:06:54.542 6563M / 10G INFO General (edge_index.hpp : 113) Size of edge index entries: 12/8 1:06:54.610 6563M / 10G INFO General (gap_closer.cpp : 101) Total edges in tip neighborhood: 3031 out of 391198, length: 3013803 1:06:54.614 6563M / 10G INFO General (edge_index.hpp : 167) Using small index (max_id = 40690374) 1:06:54.715 6563M / 10G INFO K-mer Index Building (kmer_index_builder.hpp : 395) Building perfect hash indices 1:06:55.090 6565M / 10G INFO K-mer Index Building (kmer_index_builder.hpp : 431) Index built. Total 2703726 kmers, 1953520 bytes occupied (5.78023 bits per kmer). 1:06:55.123 6586M / 10G INFO General (edge_index_builders.hpp : 265) Collecting edge information from graph, this takes a while. 1:07:23.108 6586M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 200000 reads 1:07:23.150 6586M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 400000 reads 1:07:23.169 6586M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 600000 reads 1:07:23.498 6586M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 800000 reads 1:07:28.249 6586M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 1000000 reads 1:07:28.367 6586M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 1200000 reads 1:07:52.569 6586M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 2200000 reads 1:08:29.144 6586M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 4200000 reads 1:09:51.740 6586M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 8400000 reads 1:12:21.257 6586M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 16800000 reads 1:17:29.620 6586M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 33600000 reads 1:28:11.073 6586M / 10G INFO General (sequence_mapper_notifier.h: 77) Processed 67200000 reads 1:34:40.662 6586M / 10G INFO General (sequence_mapper_notifier.h: 95) Total 87973211 reads processed 1:34:40.684 6563M / 10G INFO General (gap_closer.cpp : 452) Initializing gap closer 1:34:40.695 6563M / 10G INFO GapCloser (gap_closer.cpp : 359) Closing short gaps 1:34:41.310 6563M / 10G INFO GapCloser (gap_closer.cpp : 391) Closing short gaps complete: filled 6 gaps after checking 366 candidates 1:34:41.406 6563M / 10G INFO General (gap_closer.cpp : 456) Gap closer done 1:34:42.757 6554M / 10G INFO StageManager (stage.cpp : 185) STAGE == Simplification Cleanup (id: simplification_cleanup) 1:34:42.764 6554M / 10G INFO General (simplification.cpp : 176) PROCEDURE == Post simplification 1:34:42.770 6554M / 10G INFO General (graph_simplification.hpp : 448) Disconnection of relatively low covered edges disabled 1:34:42.776 6554M / 10G INFO General (graph_simplification.hpp : 485) Complex tip clipping disabled 1:34:42.781 6554M / 10G INFO General (graph_simplification.hpp : 634) Creating parallel br instance 1:34:42.787 6554M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 1:34:42.978 6554M / 10G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 0 times 1:34:42.986 6554M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 1:34:43.357 6555M / 10G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 1 times 1:34:43.365 6555M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 1:34:43.494 6555M / 10G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 0 times 1:34:43.501 6555M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 1:34:43.822 6555M / 10G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 0 times 1:34:43.827 6555M / 10G INFO General (simplification.cpp : 327) Disrupting self-conjugate edges 1:34:44.161 6555M / 10G INFO Simplification (parallel_processing.hpp : 167) Running Removing isolated edges 1:34:44.262 6555M / 10G INFO Simplification (parallel_processing.hpp : 170) Removing isolated edges triggered 10 times 1:34:44.358 6554M / 10G INFO General (simplification.cpp : 495) After simplification: 1:34:44.364 6554M / 10G INFO General (simplification.cpp : 496) Average coverage = 197.152 1:34:44.457 6554M / 10G INFO General (simplification.cpp : 497) Total length = 80228821 1:34:44.575 6554M / 10G INFO General (simplification.cpp : 498) Median edge length: 5000 1:34:44.580 6554M / 10G INFO General (simplification.cpp : 499) Edges: 391162 1:34:44.584 6554M / 10G INFO General (simplification.cpp : 500) Vertices: 256306 1:34:44.587 6554M / 10G INFO StageManager (stage.cpp : 185) STAGE == Contig Output (id: contig_output) 1:34:44.621 6554M / 10G INFO General (read_converter.cpp : 107) Outputting contigs to /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K55/simplified_contigs 1:34:44.758 6556M / 10G INFO General (binary_converter.cpp : 96) 16384 reads processed 1:34:44.786 6556M / 10G INFO General (binary_converter.cpp : 96) 32768 reads processed 1:34:44.832 6556M / 10G INFO General (binary_converter.cpp : 96) 65536 reads processed 1:34:44.977 6556M / 10G INFO General (binary_converter.cpp : 96) 131072 reads processed 1:34:45.245 6556M / 10G INFO General (binary_converter.cpp : 111) 195585 reads written 1:34:45.622 6554M / 10G INFO General (pipeline.cpp : 287) SPAdes finished 1:35:02.732 1M / 10G INFO General (main.cpp : 136) Assembling time: 1 hours 35 minutes 3 seconds ===== K55 finished. ===== K77 started. == Running: /oak/stanford/groups/akundaje/marinovg/programs/SPAdes-3.15.4-Linux/bin/spades-core /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/configs/config.info 0:00:00.000 1M / 13M INFO General (main.cpp : 99) Loaded config from /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/configs/config.info 0:00:00.000 1M / 13M INFO General (memory_limit.cpp : 54) Memory limit set to 250 Gb 0:00:00.000 1M / 13M INFO General (main.cpp : 107) Starting SPAdes, built from refs/heads/spades_3.15.4, git revision 1a5332e5608a073a9d28a6d0e30836f38dcc5613 0:00:00.000 1M / 13M INFO General (main.cpp : 108) Maximum k-mer length: 128 0:00:00.000 1M / 13M INFO General (main.cpp : 109) Assembling dataset (/oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/dataset.info) with K=77 0:00:00.000 1M / 13M INFO General (main.cpp : 110) Maximum # of threads to use (adjusted due to OMP capabilities): 8 0:00:00.000 1M / 13M INFO General (pipeline.cpp : 212) SPAdes started 0:00:00.000 1M / 13M INFO General (pipeline.cpp : 225) Starting from stage: read_conversion 0:00:00.000 1M / 13M INFO General (pipeline.cpp : 231) Two-step repeat resolution disabled 0:00:00.000 1M / 13M INFO GraphCore (graph_core.hpp : 672) Graph created, vertex min_id: 3, edge min_id: 3 0:00:00.001 1M / 13M INFO GraphCore (graph_core.hpp : 673) Vertex size: 48, edge size: 40 0:00:00.001 1M / 13M INFO General (edge_index.hpp : 113) Size of edge index entries: 12/8 0:00:00.001 1M / 13M INFO General (pipeline.cpp : 242) Will need read mapping, kmer mapper will be attached 0:00:00.001 1M / 13M INFO StageManager (stage.cpp : 185) STAGE == Binary Read Conversion (id: read_conversion) 0:00:00.003 1M / 13M INFO General (read_converter.cpp : 53) Binary reads detected 0:00:00.005 1M / 13M INFO General (read_converter.cpp : 53) Binary reads detected 0:00:00.006 1M / 13M INFO StageManager (stage.cpp : 185) STAGE == de Bruijn graph construction (id: construction) 0:00:00.006 1M / 13M INFO General (construction.cpp : 118) Contigs from previous K will be used: /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K55/simplified_contigs 0:00:00.076 1M / 13M INFO General (construction.cpp : 153) Max read length 151 0:00:00.076 1M / 13M INFO General (construction.cpp : 159) Average read length 146.594 0:00:00.076 1M / 13M INFO General (stage.cpp : 117) PROCEDURE == k+1-mer counting (id: construction:kpomer_counting) 0:00:00.076 1M / 13M INFO General (kmer_index_builder.hpp : 243) Splitting kmer instances into 80 files using 8 threads. This might take a while. 0:00:00.078 1M / 13M INFO General (file_limit.hpp : 42) Open file limit set to 131072 0:00:00.078 1M / 13M INFO General (kmer_splitter.hpp : 93) Memory available for splitting buffers: 10.4166 Gb 0:00:00.078 1M / 13M INFO General (kmer_splitter.hpp : 101) Using cell size of 279620 0:00:10.637 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 4161634 reads 0:00:18.104 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 8183621 reads 0:00:26.153 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 12197642 reads 0:00:34.435 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 16238177 reads 0:00:42.540 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 20265619 reads 0:00:50.617 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 24291795 reads 0:00:58.048 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 28279822 reads 0:01:05.599 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 32231441 reads 0:01:13.861 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 36176033 reads 0:01:21.936 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 40116997 reads 0:01:30.200 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 44065680 reads 0:02:16.911 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 67950579 reads 0:04:28.799 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 135306881 reads 0:08:56.466 4641M / 4641M INFO General (kmer_splitters.hpp : 125) Processed 270981866 reads 0:12:03.043 1M / 4409M INFO General (kmer_splitters.hpp : 131) Used 352726504 reads 0:12:03.719 1M / 4409M INFO General (kmer_index_builder.hpp : 249) Starting k-mer counting. 1:22:17.058 1M / 13G INFO General (kmer_index_builder.hpp : 260) K-mer counting done. There are 358797809 kmers in total. 1:22:17.468 1M / 13G INFO General (stage.cpp : 117) PROCEDURE == Extension index construction (id: construction:extension_index_construction) 1:22:18.034 1M / 13G INFO K-mer Index Building (kmer_index_builder.hpp : 438) Building kmer index 1:22:18.060 1M / 13G INFO General (kmer_index_builder.hpp : 243) Splitting kmer instances into 80 files using 8 threads. This might take a while. 1:22:18.197 1M / 13G INFO General (file_limit.hpp : 42) Open file limit set to 131072 1:22:18.229 1M / 13G INFO General (kmer_splitter.hpp : 93) Memory available for splitting buffers: 10.4166 Gb 1:22:18.229 1M / 13G INFO General (kmer_splitter.hpp : 101) Using cell size of 279620 1:22:21.496 4641M / 13G INFO General (kmer_splitters.hpp : 194) Processed 8669392 kmers 1:22:23.186 4641M / 13G INFO General (kmer_splitters.hpp : 194) Processed 19116359 kmers 1:22:24.785 4641M / 13G INFO General (kmer_splitters.hpp : 194) Processed 29111841 kmers 1:22:29.355 4641M / 13G INFO General (kmer_splitters.hpp : 194) Processed 43817493 kmers 1:22:31.117 4641M / 13G INFO General (kmer_splitters.hpp : 194) Processed 54443717 kmers 1:22:32.884 4641M / 13G INFO General (kmer_splitters.hpp : 194) Processed 64478712 kmers 1:22:37.034 4641M / 13G INFO General (kmer_splitters.hpp : 194) Processed 79651413 kmers 1:22:38.817 4641M / 13G INFO General (kmer_splitters.hpp : 194) Processed 90280461 kmers 1:22:40.537 4641M / 13G INFO General (kmer_splitters.hpp : 194) Processed 100316161 kmers 1:22:44.873 4641M / 13G INFO General (kmer_splitters.hpp : 194) Processed 115529602 kmers 1:22:46.607 4641M / 13G INFO General (kmer_splitters.hpp : 194) Processed 126153730 kmers 1:22:48.384 4641M / 13G INFO General (kmer_splitters.hpp : 194) Processed 136197112 kmers 1:22:53.103 4641M / 13G INFO General (kmer_splitters.hpp : 194) Processed 151405826 kmers 1:22:54.906 4641M / 13G INFO General (kmer_splitters.hpp : 194) Processed 162034093 kmers 1:22:56.682 4641M / 13G INFO General (kmer_splitters.hpp : 194) Processed 172067403 kmers 1:23:00.289 4641M / 13G INFO General (kmer_splitters.hpp : 194) Processed 187295169 kmers 1:23:01.980 4641M / 13G INFO General (kmer_splitters.hpp : 194) Processed 197920441 kmers 1:23:03.817 4641M / 13G INFO General (kmer_splitters.hpp : 194) Processed 207960674 kmers 1:23:18.125 4641M / 13G INFO General (kmer_splitters.hpp : 194) Processed 269679762 kmers 1:23:37.341 4641M / 13G INFO General (kmer_splitters.hpp : 199) Used 358799129 kmers. 1:23:37.342 1M / 13G INFO General (kmer_index_builder.hpp : 249) Starting k-mer counting. 1:23:48.849 1M / 13G INFO General (kmer_index_builder.hpp : 260) K-mer counting done. There are 359001275 kmers in total. 1:23:48.849 1M / 13G INFO K-mer Index Building (kmer_index_builder.hpp : 395) Building perfect hash indices 1:24:00.824 264M / 13G INFO K-mer Index Building (kmer_index_builder.hpp : 431) Index built. Total 359001275 kmers, 259352960 bytes occupied (5.77943 bits per kmer). 1:24:00.832 264M / 13G INFO General (kmer_index_builder.hpp : 169) Merging final buckets. 1:24:17.948 608M / 13G INFO DeBruijnExtensionIndexBu (kmer_extension_index_build: 100) Building k-mer extensions from k+1-mers 1:25:01.577 608M / 13G INFO DeBruijnExtensionIndexBu (kmer_extension_index_build: 105) Building k-mer extensions from k+1-mers finished. 1:25:01.896 608M / 13G INFO General (stage.cpp : 117) PROCEDURE == Condensing graph (id: construction:graph_condensing) 1:25:01.940 608M / 13G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 354) Extracting unbranching paths 1:26:03.446 1485M / 13G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 373) Extracting unbranching paths finished. 16906444 sequences extracted 1:26:29.276 1485M / 13G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 309) Collecting perfect loops 1:26:39.670 1485M / 13G INFO UnbranchingPathExtractor (debruijn_graph_constructor: 342) Collecting perfect loops finished. 2 loops collected 1:26:41.710 1485M / 13G INFO General (debruijn_graph_constructor: 487) Total 33812892 edges to create 1:26:41.793 2789M / 13G INFO General (debruijn_graph_constructor: 489) Collecting link records 1:26:51.297 3305M / 13G INFO General (debruijn_graph_constructor: 491) Ordering link records 1:26:52.058 3305M / 13G INFO General (debruijn_graph_constructor: 493) Sorting done 1:26:52.500 3562M / 13G INFO General (debruijn_graph_constructor: 503) Total 17109912 vertices to create 1:26:52.504 5142M / 13G INFO General (debruijn_graph_constructor: 506) Connecting the graph 1:27:12.007 4111M / 13G INFO General (stage.cpp : 117) PROCEDURE == Filling coverage indices (PHM) (id: construction:coverage_filling_phm) 1:27:12.007 4111M / 13G INFO K-mer Index Building (kmer_index_builder.hpp : 395) Building perfect hash indices 1:27:23.061 4374M / 13G INFO K-mer Index Building (kmer_index_builder.hpp : 431) Index built. Total 358797809 kmers, 259205944 bytes occupied (5.77943 bits per kmer). 1:27:24.061 5746M / 13G INFO General (coverage_hash_map_builder.: 47) Collecting k-mer coverage information from reads, this takes a while. 1:39:47.915 5746M / 13G INFO General (construction.cpp : 430) Filling coverage and flanking coverage from PHM 1:39:59.741 5746M / 13G INFO General (coverage_filling.hpp : 82) Processed 8000000 edges 1:40:10.259 5746M / 13G INFO General (coverage_filling.hpp : 82) Processed 16000000 edges 1:40:20.763 5746M / 13G INFO General (coverage_filling.hpp : 82) Processed 24000000 edges 1:40:31.357 5746M / 13G INFO General (coverage_filling.hpp : 82) Processed 32000000 edges 1:40:33.709 5746M / 13G INFO General (coverage_filling.hpp : 82) Processed 33812882 edges 1:40:37.546 3824M / 13G INFO StageManager (stage.cpp : 185) STAGE == EC Threshold Finding (id: ec_threshold_finder) 1:40:39.839 4144M / 13G INFO General (kmer_coverage_model.cpp : 181) Kmer coverage valley at: 26 1:40:39.944 3824M / 13G INFO General (kmer_coverage_model.cpp : 201) K-mer histogram maximum: 124 1:40:39.984 3824M / 13G INFO General (kmer_coverage_model.cpp : 237) Estimated median coverage: 124. Coverage mad: 29.652 1:40:40.053 3824M / 13G INFO General (kmer_coverage_model.cpp : 259) Fitting coverage model 1:40:40.653 4144M / 13G INFO General (kmer_coverage_model.cpp : 295) ... iteration 2 1:40:41.165 4144M / 13G INFO General (kmer_coverage_model.cpp : 295) ... iteration 4 1:40:42.583 4144M / 13G INFO General (kmer_coverage_model.cpp : 295) ... iteration 8 1:40:45.815 4144M / 13G INFO General (kmer_coverage_model.cpp : 295) ... iteration 16 1:40:51.056 4144M / 13G INFO General (kmer_coverage_model.cpp : 295) ... iteration 32 1:40:52.413 4144M / 13G INFO General (kmer_coverage_model.cpp : 309) Fitted mean coverage: 112.95. Fitted coverage std. dev: 32.5473 1:40:52.418 4144M / 13G INFO General (kmer_coverage_model.cpp : 334) Probability of erroneous kmer at valley: 0.50129 1:40:52.418 4144M / 13G INFO General (kmer_coverage_model.cpp : 358) Preliminary threshold calculated as: 56 1:40:52.418 4144M / 13G INFO General (kmer_coverage_model.cpp : 362) Threshold adjusted to: 56 1:40:52.418 4144M / 13G INFO General (kmer_coverage_model.cpp : 375) Estimated genome size (ignoring repeats): 76239101 1:40:52.449 3824M / 13G INFO General (genomic_info_filler.cpp : 55) Mean coverage was calculated as 112.95 1:40:52.451 3824M / 13G INFO General (genomic_info_filler.cpp : 70) EC coverage threshold value was calculated as 56 1:40:52.451 3824M / 13G INFO General (genomic_info_filler.cpp : 71) Trusted kmer low bound: 0 1:40:52.451 3824M / 13G INFO StageManager (stage.cpp : 185) STAGE == Gap Closer (id: early_gapcloser) 1:40:52.591 3833M / 13G INFO General (edge_index.hpp : 113) Size of edge index entries: 12/8 1:40:58.052 4105M / 13G INFO General (gap_closer.cpp : 101) Total edges in tip neighborhood: 9518451 out of 33812882, length: 261110363 1:40:59.726 4233M / 13G INFO General (edge_index.hpp : 167) Using small index (max_id = 33981959) 1:41:02.770 4233M / 13G INFO K-mer Index Building (kmer_index_builder.hpp : 395) Building perfect hash indices 1:41:33.512 4384M / 13G INFO K-mer Index Building (kmer_index_builder.hpp : 431) Index built. Total 214506453 kmers, 154931064 bytes occupied (5.77814 bits per kmer). 1:41:34.560 6025M / 13G INFO General (edge_index_builders.hpp : 265) Collecting edge information from graph, this takes a while. 1:42:14.765 6025M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 200000 reads 1:42:14.885 6025M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 400000 reads 1:42:15.006 5897M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 600000 reads 1:42:15.279 5897M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 800000 reads 1:42:15.424 5897M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 1000000 reads 1:42:15.746 5897M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 1200000 reads 1:42:37.318 5897M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 2200000 reads 1:42:59.943 5898M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 4200000 reads 1:44:04.673 5898M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 8400000 reads 1:45:55.766 5905M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 16800000 reads 1:49:55.250 5908M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 33600000 reads 1:57:38.957 5913M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 67200000 reads 2:02:48.450 5991M / 13G INFO General (sequence_mapper_notifier.h: 95) Total 87973211 reads processed 2:02:50.841 3988M / 13G INFO General (gap_closer.cpp : 452) Initializing gap closer 2:02:50.868 3988M / 13G INFO GapCloser (gap_closer.cpp : 359) Closing short gaps 2:03:35.842 3988M / 13G INFO GapCloser (gap_closer.cpp : 391) Closing short gaps complete: filled 407 gaps after checking 56042 candidates 2:03:36.898 3988M / 13G INFO General (gap_closer.cpp : 456) Gap closer done 2:03:38.766 3824M / 13G INFO StageManager (stage.cpp : 185) STAGE == Raw Simplification (id: raw_simplification) 2:03:38.850 3824M / 13G INFO General (simplification.cpp : 127) PROCEDURE == Initial cleaning 2:03:39.161 3824M / 13G INFO General (graph_simplification.hpp : 662) Flanking coverage based disconnection disabled 2:03:39.181 3824M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Self conjugate edge remover 2:03:40.854 3824M / 13G INFO Simplification (parallel_processing.hpp : 170) Self conjugate edge remover triggered 1 times 2:03:40.855 3824M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Initial tip clipper 2:04:39.952 3465M / 13G INFO Simplification (parallel_processing.hpp : 170) Initial tip clipper triggered 6522347 times 2:04:40.067 3465M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Initial ec remover 2:04:49.952 3478M / 13G INFO Simplification (parallel_processing.hpp : 170) Initial ec remover triggered 666850 times 2:04:50.017 3478M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Initial isolated edge remover 2:04:53.178 3443M / 13G INFO Simplification (parallel_processing.hpp : 170) Initial isolated edge remover triggered 1290607 times 2:04:53.238 3278M / 13G INFO StageManager (stage.cpp : 185) STAGE == Simplification (id: simplification) 2:04:53.256 3278M / 13G INFO General (simplification.cpp : 368) Graph simplification started 2:04:53.276 3278M / 13G INFO General (graph_simplification.hpp : 634) Creating parallel br instance 2:04:53.276 3278M / 13G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 1 2:04:53.276 3278M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 2:04:54.606 3272M / 13G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 116582 times 2:04:54.626 3272M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 2:06:17.194 5871M / 13G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 227753 times 2:06:17.223 5871M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 2:06:18.760 5874M / 13G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 97184 times 2:06:18.777 5874M / 13G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 2 2:06:18.780 5874M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 2:06:19.066 5850M / 13G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 8922 times 2:06:19.092 5850M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 2:06:27.376 6082M / 13G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 19666 times 2:06:27.386 6082M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 2:06:27.615 6081M / 13G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 16051 times 2:06:27.622 6081M / 13G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 3 2:06:27.625 6081M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 2:06:27.669 6079M / 13G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 506 times 2:06:27.673 6079M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 2:06:28.995 6109M / 13G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 2845 times 2:06:29.001 6109M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 2:06:29.064 6110M / 13G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 4005 times 2:06:29.067 6110M / 13G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 4 2:06:29.071 6110M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 2:06:29.084 6109M / 13G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 75 times 2:06:29.088 6109M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 2:06:29.428 6117M / 13G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 705 times 2:06:29.432 6117M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 2:06:29.469 6117M / 13G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 2006 times 2:06:29.472 6117M / 13G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 5 2:06:29.475 6117M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 2:06:29.482 6117M / 13G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 36 times 2:06:29.485 6117M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 2:06:29.623 6120M / 13G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 297 times 2:06:29.628 6120M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 2:06:29.652 6120M / 13G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 1203 times 2:06:29.655 6120M / 13G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 6 2:06:29.659 6120M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 2:06:29.664 6120M / 13G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 12 times 2:06:29.667 6120M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 2:06:29.746 6122M / 13G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 163 times 2:06:29.750 6122M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 2:06:29.771 6122M / 13G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 880 times 2:06:29.774 6122M / 13G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 7 2:06:29.778 6122M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 2:06:29.782 6122M / 13G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 5 times 2:06:29.785 6122M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 2:06:29.834 6123M / 13G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 106 times 2:06:29.838 6123M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 2:06:29.859 6123M / 13G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 758 times 2:06:29.862 6123M / 13G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 8 2:06:29.866 6123M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 2:06:29.870 6123M / 13G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 3 times 2:06:29.873 6123M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 2:06:29.916 6124M / 13G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 91 times 2:06:29.919 6124M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 2:06:29.936 6124M / 13G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 604 times 2:06:29.939 6124M / 13G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 9 2:06:29.943 6124M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 2:06:29.946 6124M / 13G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 7 times 2:06:29.950 6124M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 2:06:29.976 6125M / 13G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 51 times 2:06:29.979 6125M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 2:06:29.995 6125M / 13G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 583 times 2:06:29.998 6125M / 13G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 10 2:06:30.001 6125M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 2:06:30.005 6125M / 13G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 3 times 2:06:30.008 6125M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 2:06:30.038 6125M / 13G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 53 times 2:06:30.041 6125M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 2:06:30.056 6125M / 13G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 525 times 2:06:30.059 6125M / 13G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 11 2:06:30.063 6125M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 2:06:30.125 6125M / 13G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 6 times 2:06:30.130 6125M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 2:06:30.340 6128M / 13G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 99 times 2:06:30.345 6128M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 2:06:30.408 6123M / 13G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 0 times 2:06:30.413 6123M / 13G INFO General (simplification.cpp : 373) PROCEDURE == Simplification cycle, iteration 12 2:06:30.416 6123M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 2:06:30.420 6123M / 13G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 0 times 2:06:30.423 6123M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 2:06:30.427 6123M / 13G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 0 times 2:06:30.430 6123M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Low coverage edge remover 2:06:30.433 6123M / 13G INFO Simplification (parallel_processing.hpp : 170) Low coverage edge remover triggered 0 times 2:06:30.547 6120M / 13G INFO StageManager (stage.cpp : 185) STAGE == Gap Closer (id: late_gapcloser) 2:06:30.566 6129M / 13G INFO General (edge_index.hpp : 113) Size of edge index entries: 12/8 2:06:30.689 6129M / 13G INFO General (gap_closer.cpp : 101) Total edges in tip neighborhood: 3280 out of 268542, length: 5125268 2:06:30.698 6129M / 13G INFO General (edge_index.hpp : 167) Using small index (max_id = 33981959) 2:06:30.781 6129M / 13G INFO K-mer Index Building (kmer_index_builder.hpp : 395) Building perfect hash indices 2:06:31.305 6133M / 13G INFO K-mer Index Building (kmer_index_builder.hpp : 431) Index built. Total 4734799 kmers, 3421554 bytes occupied (5.78112 bits per kmer). 2:06:31.341 6173M / 13G INFO General (edge_index_builders.hpp : 265) Collecting edge information from graph, this takes a while. 2:06:54.024 6173M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 200000 reads 2:06:54.055 6173M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 400000 reads 2:06:54.175 6173M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 600000 reads 2:06:54.337 6173M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 800000 reads 2:06:59.533 6173M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 1000000 reads 2:06:59.579 6173M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 1200000 reads 2:07:17.750 6173M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 2200000 reads 2:07:51.529 6173M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 4200000 reads 2:08:56.161 6173M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 8400000 reads 2:11:01.956 6173M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 16800000 reads 2:15:29.580 6173M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 33600000 reads 2:24:13.303 6173M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 67200000 reads 2:30:06.825 6173M / 13G INFO General (sequence_mapper_notifier.h: 95) Total 87973211 reads processed 2:30:06.849 6129M / 13G INFO General (gap_closer.cpp : 452) Initializing gap closer 2:30:06.855 6129M / 13G INFO GapCloser (gap_closer.cpp : 359) Closing short gaps 2:30:07.320 6129M / 13G INFO GapCloser (gap_closer.cpp : 391) Closing short gaps complete: filled 5 gaps after checking 708 candidates 2:30:07.426 6129M / 13G INFO General (gap_closer.cpp : 456) Gap closer done 2:30:08.832 6120M / 13G INFO StageManager (stage.cpp : 185) STAGE == Simplification Cleanup (id: simplification_cleanup) 2:30:08.846 6120M / 13G INFO General (simplification.cpp : 176) PROCEDURE == Post simplification 2:30:08.855 6120M / 13G INFO General (graph_simplification.hpp : 448) Disconnection of relatively low covered edges disabled 2:30:08.862 6120M / 13G INFO General (graph_simplification.hpp : 485) Complex tip clipping disabled 2:30:08.870 6120M / 13G INFO General (graph_simplification.hpp : 634) Creating parallel br instance 2:30:08.878 6120M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 2:30:09.027 6120M / 13G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 0 times 2:30:09.036 6120M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 2:30:09.306 6121M / 13G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 0 times 2:30:09.316 6121M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Tip clipper 2:30:09.435 6121M / 13G INFO Simplification (parallel_processing.hpp : 170) Tip clipper triggered 0 times 2:30:09.444 6121M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Bulge remover 2:30:09.709 6121M / 13G INFO Simplification (parallel_processing.hpp : 170) Bulge remover triggered 0 times 2:30:09.718 6121M / 13G INFO General (simplification.cpp : 327) Disrupting self-conjugate edges 2:30:09.893 6121M / 13G INFO Simplification (parallel_processing.hpp : 167) Running Removing isolated edges 2:30:09.981 6121M / 13G INFO Simplification (parallel_processing.hpp : 170) Removing isolated edges triggered 26 times 2:30:10.057 6120M / 13G INFO General (simplification.cpp : 495) After simplification: 2:30:10.064 6120M / 13G INFO General (simplification.cpp : 496) Average coverage = 146.023 2:30:10.174 6120M / 13G INFO General (simplification.cpp : 497) Total length = 82280628 2:30:10.313 6120M / 13G INFO General (simplification.cpp : 498) Median edge length: 6124 2:30:10.322 6120M / 13G INFO General (simplification.cpp : 499) Edges: 268480 2:30:10.329 6120M / 13G INFO General (simplification.cpp : 500) Vertices: 178282 2:30:10.337 6120M / 13G INFO StageManager (stage.cpp : 185) STAGE == Mismatch Correction (id: mismatch_correction) 2:30:10.357 6120M / 13G INFO General (graph_pack.cpp : 67) Index refill 2:30:10.373 6120M / 13G INFO General (edge_index.hpp : 156) Using small index (max_id = 33981959) 2:30:11.509 6120M / 13G INFO K-mer Index Building (kmer_index_builder.hpp : 395) Building perfect hash indices 2:30:19.846 6178M / 13G INFO K-mer Index Building (kmer_index_builder.hpp : 431) Index built. Total 82280628 kmers, 59430844 bytes occupied (5.77836 bits per kmer). 2:30:20.485 6806M / 13G INFO General (edge_index_builders.hpp : 252) Collecting edge information from graph, this takes a while. 2:30:25.360 6806M / 13G INFO General (graph_pack.cpp : 77) Normalizing k-mer map. Total 40799556 kmers to process 2:30:42.605 6806M / 13G INFO General (graph_pack.cpp : 79) Normalizing done 2:30:42.632 6806M / 13G INFO General (mismatch_correction.cpp : 400) Collect potential mismatches 2:30:57.787 6824M / 13G INFO General (mismatch_correction.cpp : 192) Total 173003 edges (out of 268480) with 1872586 potential mismatch positions (10.824 positions per edge) 2:30:57.800 6824M / 13G INFO General (mismatch_correction.cpp : 402) Potential mismatches collected 2:31:01.344 6838M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 200000 reads 2:31:01.433 6852M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 400000 reads 2:31:01.478 6865M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 600000 reads 2:31:01.505 6879M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 800000 reads 2:31:01.597 6893M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 1000000 reads 2:31:01.619 6906M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 1200000 reads 2:31:04.191 6915M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 2200000 reads 2:31:07.173 6928M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 4200000 reads 2:31:15.156 6949M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 8400000 reads 2:31:28.849 6993M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 16800000 reads 2:31:58.035 7023M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 33600000 reads 2:32:56.276 7031M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 67200000 reads 2:34:52.673 7094M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 134400000 reads 2:38:46.744 7138M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 268600000 reads 2:41:38.590 7544M / 13G INFO General (sequence_mapper_notifier.h: 95) Total 352335334 reads processed 2:41:40.399 6937M / 13G INFO General (mismatch_correction.cpp : 395) All edges processed 2:41:40.486 6807M / 13G INFO General (mismatch_correction.cpp : 450) Corrected 101 nucleotides 2:41:40.493 6807M / 13G INFO StageManager (stage.cpp : 185) STAGE == Contig Output (id: contig_output) 2:41:40.544 6807M / 13G INFO General (contig_output.hpp : 16) Outputting contigs to /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/before_rr.fasta 2:41:42.451 6807M / 13G INFO General (contig_output_stage.cpp : 151) Writing GFA graph to /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/assembly_graph_after_simplification.gfa 2:41:44.190 6807M / 13G INFO StageManager (stage.cpp : 185) STAGE == Hybrid Aligning (id: hybrid_aligning) 2:41:44.225 6807M / 13G INFO HybridAligning (hybrid_aligning.cpp : 341) Hybrid library detected: #1 2:41:44.245 6807M / 13G INFO General (hybrid_aligning.cpp : 294) Aligning long reads with bwa-mem based aligner 2:41:46.995 7069M / 13G INFO General (bwa_index.cpp : 176) Using BWA RopeBWT algorithm 2:43:41.893 7621M / 13G INFO General (hybrid_aligning.cpp : 274) Prepared batch 0 of 50000 reads. 2:49:40.437 7655M / 13G INFO General (hybrid_aligning.cpp : 236) Read batch of size: 50000 processed; 44660 of them longer than 500; among long reads aligned: 42638; paths of more than one edge received: 18472 2:49:40.674 7643M / 13G INFO General (hybrid_aligning.cpp : 279) Processed 50000 reads 2:49:46.287 7563M / 13G INFO General (hybrid_aligning.cpp : 274) Prepared batch 1 of 50000 reads. 2:54:32.010 7590M / 13G INFO General (hybrid_aligning.cpp : 236) Read batch of size: 50000 processed; 43423 of them longer than 500; among long reads aligned: 38827; paths of more than one edge received: 15149 2:54:32.199 7573M / 13G INFO General (hybrid_aligning.cpp : 279) Processed 100000 reads 2:54:37.833 7575M / 13G INFO General (hybrid_aligning.cpp : 274) Prepared batch 2 of 50000 reads. 2:59:42.126 7602M / 13G INFO General (hybrid_aligning.cpp : 236) Read batch of size: 50000 processed; 43527 of them longer than 500; among long reads aligned: 40949; paths of more than one edge received: 16210 2:59:42.347 7584M / 13G INFO General (hybrid_aligning.cpp : 279) Processed 150000 reads 2:59:48.792 7703M / 13G INFO General (hybrid_aligning.cpp : 274) Prepared batch 3 of 50000 reads. 3:05:30.557 7731M / 13G INFO General (hybrid_aligning.cpp : 236) Read batch of size: 50000 processed; 44976 of them longer than 500; among long reads aligned: 41402; paths of more than one edge received: 18836 3:05:30.795 7711M / 13G INFO General (hybrid_aligning.cpp : 279) Processed 200000 reads 3:05:36.950 7668M / 13G INFO General (hybrid_aligning.cpp : 274) Prepared batch 4 of 50000 reads. 3:11:11.326 7695M / 13G INFO General (hybrid_aligning.cpp : 236) Read batch of size: 50000 processed; 44412 of them longer than 500; among long reads aligned: 40631; paths of more than one edge received: 17871 3:11:11.566 7675M / 13G INFO General (hybrid_aligning.cpp : 279) Processed 250000 reads 3:11:17.921 7643M / 13G INFO General (hybrid_aligning.cpp : 274) Prepared batch 5 of 50000 reads. 3:16:03.365 7668M / 13G INFO General (hybrid_aligning.cpp : 236) Read batch of size: 50000 processed; 43985 of them longer than 500; among long reads aligned: 39569; paths of more than one edge received: 16590 3:16:03.584 7649M / 13G INFO General (hybrid_aligning.cpp : 279) Processed 300000 reads 3:16:10.827 7795M / 13G INFO General (hybrid_aligning.cpp : 274) Prepared batch 6 of 50000 reads. 3:22:27.607 7823M / 13G INFO General (hybrid_aligning.cpp : 236) Read batch of size: 50000 processed; 45241 of them longer than 500; among long reads aligned: 41831; paths of more than one edge received: 20770 3:22:27.823 7802M / 13G INFO General (hybrid_aligning.cpp : 279) Processed 350000 reads 3:22:34.418 7837M / 13G INFO General (hybrid_aligning.cpp : 274) Prepared batch 7 of 50000 reads. 3:28:30.729 7867M / 13G INFO General (hybrid_aligning.cpp : 236) Read batch of size: 50000 processed; 46916 of them longer than 500; among long reads aligned: 45736; paths of more than one edge received: 22618 3:28:30.954 7844M / 13G INFO General (hybrid_aligning.cpp : 279) Processed 400000 reads 3:28:35.993 7624M / 13G INFO General (hybrid_aligning.cpp : 274) Prepared batch 8 of 50000 reads. 3:34:02.336 7647M / 13G INFO General (hybrid_aligning.cpp : 236) Read batch of size: 50000 processed; 42507 of them longer than 500; among long reads aligned: 36766; paths of more than one edge received: 14732 3:34:02.508 7628M / 13G INFO General (hybrid_aligning.cpp : 279) Processed 450000 reads 3:34:08.117 7729M / 13G INFO General (hybrid_aligning.cpp : 274) Prepared batch 9 of 50000 reads. 3:39:33.038 7757M / 13G INFO General (hybrid_aligning.cpp : 236) Read batch of size: 50000 processed; 45377 of them longer than 500; among long reads aligned: 43577; paths of more than one edge received: 19353 3:39:33.335 7735M / 13G INFO General (hybrid_aligning.cpp : 279) Processed 500000 reads 3:39:39.327 7688M / 13G INFO General (hybrid_aligning.cpp : 274) Prepared batch 10 of 50000 reads. 3:45:26.130 7712M / 13G INFO General (hybrid_aligning.cpp : 236) Read batch of size: 50000 processed; 43824 of them longer than 500; among long reads aligned: 38063; paths of more than one edge received: 16234 3:45:26.362 7693M / 13G INFO General (hybrid_aligning.cpp : 279) Processed 550000 reads 3:45:32.616 7732M / 13G INFO General (hybrid_aligning.cpp : 274) Prepared batch 11 of 50000 reads. 3:51:19.452 7758M / 13G INFO General (hybrid_aligning.cpp : 236) Read batch of size: 50000 processed; 44737 of them longer than 500; among long reads aligned: 41386; paths of more than one edge received: 18642 3:51:19.622 7737M / 13G INFO General (hybrid_aligning.cpp : 279) Processed 600000 reads 3:51:24.405 7646M / 13G INFO General (hybrid_aligning.cpp : 274) Prepared batch 12 of 50000 reads. 3:56:00.852 7671M / 13G INFO General (hybrid_aligning.cpp : 236) Read batch of size: 50000 processed; 43320 of them longer than 500; among long reads aligned: 40888; paths of more than one edge received: 16375 3:56:01.092 7650M / 13G INFO General (hybrid_aligning.cpp : 279) Processed 650000 reads 3:56:07.505 7738M / 13G INFO General (hybrid_aligning.cpp : 274) Prepared batch 13 of 50000 reads. 4:01:47.671 7764M / 13G INFO General (hybrid_aligning.cpp : 236) Read batch of size: 50000 processed; 44344 of them longer than 500; among long reads aligned: 40924; paths of more than one edge received: 18195 4:01:47.853 7743M / 13G INFO General (hybrid_aligning.cpp : 279) Processed 700000 reads 4:01:51.611 7535M / 13G INFO General (hybrid_aligning.cpp : 274) Prepared batch 14 of 50000 reads. 4:05:01.802 7554M / 13G INFO General (hybrid_aligning.cpp : 236) Read batch of size: 50000 processed; 40594 of them longer than 500; among long reads aligned: 32182; paths of more than one edge received: 10844 4:05:01.936 7538M / 13G INFO General (hybrid_aligning.cpp : 279) Processed 750000 reads 4:05:07.835 7788M / 13G INFO General (hybrid_aligning.cpp : 274) Prepared batch 15 of 50000 reads. 4:10:31.820 7816M / 13G INFO General (hybrid_aligning.cpp : 236) Read batch of size: 50000 processed; 45946 of them longer than 500; among long reads aligned: 44359; paths of more than one edge received: 20512 4:10:32.003 7793M / 13G INFO General (hybrid_aligning.cpp : 279) Processed 800000 reads 4:10:37.495 7748M / 13G INFO General (hybrid_aligning.cpp : 274) Prepared batch 16 of 49424 reads. 4:16:07.289 7773M / 13G INFO General (hybrid_aligning.cpp : 236) Read batch of size: 49424 processed; 44525 of them longer than 500; among long reads aligned: 40967; paths of more than one edge received: 18364 4:16:07.546 7752M / 13G INFO General (hybrid_aligning.cpp : 279) Processed 849424 reads 4:16:07.714 7106M / 13G INFO General (hybrid_aligning.cpp : 308) For library of long reads 4:16:07.724 7106M / 13G INFO StatsCounter (pacbio_read_structures.hpp: 163) Median fraction of present seeds in maximal alignmnent among reads aligned to the graph: 0 4:16:07.732 7106M / 13G INFO General (hybrid_aligning.cpp : 310) Aligning of long reads finished 4:16:08.008 6870M / 13G INFO HybridAligning (hybrid_aligning.cpp : 388) Padding gaps 4:16:08.020 6870M / 13G INFO HybridAligning (hybrid_aligning.cpp : 392) Min gap weight set to 2 4:16:08.051 6869M / 13G INFO General (hybrid_aligning.cpp : 146) Closing gaps with long reads 4:16:13.917 6902M / 13G INFO MultiGapJoiner (hybrid_gap_closer.hpp : 531) Closed 297 gaps 4:16:16.875 6901M / 13G INFO General (hybrid_aligning.cpp : 167) Closing gaps with long reads finished 4:16:16.885 6875M / 13G INFO StageManager (stage.cpp : 185) STAGE == Paired Information Counting (id: late_pair_info_count) 4:16:18.132 6974M / 13G INFO General (graph_pack.cpp : 77) Normalizing k-mer map. Total 40808132 kmers to process 4:16:34.693 6974M / 13G INFO General (graph_pack.cpp : 79) Normalizing done 4:16:34.776 6974M / 13G INFO General (pair_info_count.cpp : 339) Min edge length for estimation: 6258 4:16:34.782 6974M / 13G INFO General (pair_info_count.cpp : 350) Estimating insert size for library #0 4:16:34.786 6974M / 13G INFO General (pair_info_count.cpp : 202) Estimating insert size (takes a while) 4:16:34.969 7118M / 13G INFO General (pair_info_count.cpp : 48) Selecting usual mapper 4:16:40.313 7118M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 200000 reads 4:16:40.665 7118M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 400000 reads 4:16:40.781 7118M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 600000 reads 4:16:40.950 7118M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 800000 reads 4:16:41.086 7118M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 1000000 reads 4:16:41.115 7118M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 1200000 reads 4:16:45.970 7118M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 2200000 reads 4:16:52.073 7118M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 4200000 reads 4:17:06.170 7118M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 8400000 reads 4:17:32.536 7118M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 16800000 reads 4:18:28.354 7118M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 33600000 reads 4:20:15.380 7118M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 67200000 reads 4:21:44.341 7118M / 13G INFO General (sequence_mapper_notifier.h: 95) Total 87973211 reads processed 4:21:44.803 7118M / 13G INFO General (pair_info_count.cpp : 220) Edge pairs: 31504422 4:21:44.809 7118M / 13G INFO General (pair_info_count.cpp : 222) 33168912 paired reads (37.7034% of all) aligned to long edges 4:21:44.816 6974M / 13G INFO General (pair_info_count.cpp : 369) Insert size = 576.43, deviation = 142.01, left quantile = 401, right quantile = 755, read length = 151 4:21:44.899 7066M / 13G INFO General (pair_info_count.cpp : 390) Filtering data for library #0 4:21:44.908 7066M / 13G INFO General (pair_info_count.cpp : 48) Selecting usual mapper 4:21:50.857 7066M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 200000 reads 4:21:50.925 7066M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 400000 reads 4:21:51.310 7066M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 600000 reads 4:21:51.396 7066M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 800000 reads 4:21:51.617 7066M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 1000000 reads 4:21:51.933 7066M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 1200000 reads 4:21:57.550 7066M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 2200000 reads 4:22:04.676 7066M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 4200000 reads 4:22:20.683 7066M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 8400000 reads 4:22:52.460 7066M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 16800000 reads 4:23:56.122 7066M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 33600000 reads 4:26:05.414 7066M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 67200000 reads 4:27:41.833 7066M / 13G INFO General (sequence_mapper_notifier.h: 95) Total 87973211 reads processed 4:27:41.844 7066M / 13G INFO General (pair_info_count.cpp : 402) Mapping library #0 4:27:41.851 7066M / 13G INFO General (pair_info_count.cpp : 404) Mapping paired reads (takes a while) 4:27:41.857 7066M / 13G INFO General (pair_info_count.cpp : 299) Left insert size quantile 401, right insert size quantile 755, filtering threshold 2, rounding threshold 0 4:27:41.875 7075M / 13G INFO General (pair_info_count.cpp : 48) Selecting usual mapper 4:27:53.189 7118M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 200000 reads 4:27:53.336 7165M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 400000 reads 4:27:53.448 7209M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 600000 reads 4:27:53.523 7254M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 800000 reads 4:27:53.586 7303M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 1000000 reads 4:27:53.600 7352M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 1200000 reads 4:28:04.383 7378M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 2200000 reads 4:28:15.949 7476M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 4200000 reads 4:28:48.894 7572M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 8400000 reads 4:29:45.485 7741M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 16800000 reads 4:31:44.853 8005M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 33600000 reads 4:35:49.348 8399M / 13G INFO General (sequence_mapper_notifier.h: 77) Processed 67200000 reads 4:38:26.910 12G / 13G INFO General (sequence_mapper_notifier.h: 95) Total 87973211 reads processed 4:38:27.208 11G / 13G INFO General (pair_info_count.cpp : 343) Library #1 was mapped earlier on hybrid aligning stage, skipping 4:38:27.219 11G / 13G INFO StageManager (stage.cpp : 185) STAGE == Distance Estimation (id: distance_estimation) 4:38:27.243 11G / 13G INFO General (distance_estimation.cpp : 179) Processing library #0 4:38:27.252 11G / 13G INFO General (distance_estimation.cpp : 150) Weight Filter Done 4:38:27.311 11G / 13G INFO DistanceEstimator (distance_estimation.hpp : 116) Using SIMPLE distance estimator 4:45:10.805 14G / 16G INFO General (distance_estimation.cpp : 37) Filtering info 4:45:11.402 14G / 16G INFO General (pair_info_filters.hpp : 242) Start filtering; library index size: 135908054 4:45:43.278 13G / 16G INFO General (pair_info_filters.hpp : 263) Done filtering; library index size: 29985840 4:45:43.305 13G / 16G INFO General (distance_estimation.cpp : 156) Refining clustered pair information 4:45:49.938 13G / 16G INFO General (distance_estimation.cpp : 158) The refining of clustered pair information has been finished 4:45:49.949 13G / 16G INFO General (distance_estimation.cpp : 160) Improving paired information 5:01:12.676 13G / 17G INFO PairInfoImprover (pair_info_improver.hpp : 104) Paired info stats: missing = 13516301; contradictional = 367906 5:07:36.924 13G / 18G INFO PairInfoImprover (pair_info_improver.hpp : 104) Paired info stats: missing = 3760642; contradictional = 6874 5:07:37.473 13G / 18G INFO General (distance_estimation.cpp : 104) Filling scaffolding index 5:07:37.839 13G / 18G INFO DistanceEstimator (distance_estimation.hpp : 116) Using SMOOTHING distance estimator 5:16:01.597 14G / 18G INFO General (distance_estimation.cpp : 37) Filtering info 5:16:01.665 14G / 18G INFO General (pair_info_filters.hpp : 242) Start filtering; library index size: 8777603 5:16:04.277 14G / 18G INFO General (pair_info_filters.hpp : 263) Done filtering; library index size: 8777603 5:16:04.295 14G / 18G INFO General (distance_estimation.cpp : 186) Clearing raw paired index 5:16:13.053 9226M / 18G INFO StageManager (stage.cpp : 185) STAGE == Repeat Resolving (id: repeat_resolving) 5:16:13.868 9226M / 18G INFO General (repeat_resolving.cpp : 87) Using Path-Extend repeat resolving 5:16:14.245 9226M / 18G INFO General (launcher.cpp : 600) ExSPAnder repeat resolving tool started 5:16:14.701 9226M / 18G INFO General (coverage_uniformity_analyz: 25) genomic coverage is 119.377 calculated of length 71972424 5:16:14.867 9226M / 18G WARN General (launcher.cpp : 178) Your data seems to have high uniform coverage depth. It is strongly recommended to use --isolate option. 5:16:14.911 9226M / 18G INFO General (launcher.cpp : 186) Autodetecting unique edge set parameters... 5:16:14.926 9226M / 18G INFO General (launcher.cpp : 188) Minimal unique edge length set to the smallest MP library IS: 2000 5:16:15.088 9226M / 18G INFO General (coverage_uniformity_analyz: 25) genomic coverage is 119.152 calculated of length 67270421 5:16:15.217 9226M / 18G INFO General (launcher.cpp : 193) median coverage for edges longer than 2000 is 119.152 uniformity 90% 5:16:15.532 9227M / 18G INFO ScaffoldingUniqueEdgeAna (scaff_supplementary.cpp : 68) With length cutoff: 2000, median long edge coverage: 119.154, and maximal unique coverage: 0.5 5:16:15.548 9227M / 18G INFO ScaffoldingUniqueEdgeAna (scaff_supplementary.cpp : 70) Unique edges quantity: 19822, unique edges length 122323544, total edges length 164568640 5:16:16.260 9759M / 18G INFO General (launcher.cpp : 418) Creating main extenders, unique edge length = 2000 5:16:16.283 10G / 18G INFO General (launcher.cpp : 342) filling path container 5:16:17.521 11G / 18G INFO General (extenders_logic.cpp : 334) Estimated coverage of library #0 is 146.023 5:16:17.556 11G / 18G INFO General (extenders_logic.cpp : 345) Creating extender; library index size: 64164800 5:16:17.597 11G / 18G INFO General (extenders_logic.cpp : 334) Estimated coverage of library #0 is 146.023 5:16:17.605 11G / 18G INFO General (extenders_logic.cpp : 345) Creating extender; library index size: 64164800 5:16:17.797 11G / 18G INFO General (extenders_logic.cpp : 47) resolvable_repeat_length_bound set to 132582 5:16:31.922 11G / 18G INFO General (extenders_logic.cpp : 543) Using 1 paired-end library 5:16:31.935 11G / 18G INFO General (extenders_logic.cpp : 544) Using 1 paired-end scaffolding library 5:16:31.942 11G / 18G INFO General (extenders_logic.cpp : 545) Using 1 single read library 5:16:32.077 11G / 18G INFO General (launcher.cpp : 385) Filling backbone edges for long reads scaffolding... 5:16:32.087 11G / 18G INFO General (launcher.cpp : 404) with coverage 5:16:32.194 11G / 18G INFO ScaffoldingUniqueEdgeAna (scaff_supplementary.cpp : 68) With length cutoff: 500, median long edge coverage: 119.566, and maximal unique coverage: 0.5 5:16:32.204 11G / 18G INFO ScaffoldingUniqueEdgeAna (scaff_supplementary.cpp : 70) Unique edges quantity: 31890, unique edges length 135018202, total edges length 164568640 5:16:32.212 11G / 18G INFO General (launcher.cpp : 407) 31890 unique edges 5:16:32.222 11G / 18G INFO General (extenders_logic.cpp : 420) Creating scaffolding extender for lib 1 5:16:32.265 11G / 18G INFO ExtensionChooser2015 (extension_chooser2015.hpp : 50) ExtensionChooser2015 created 5:16:32.454 11G / 18G INFO General (extenders_logic.cpp : 446) Using 1 long reads scaffolding library 5:16:32.475 11G / 18G INFO General (launcher.cpp : 447) Total number of extenders is 5 5:16:32.539 11G / 18G INFO General (path_extenders.cpp : 36) Processed 0 paths from 130634 (0%) 5:16:37.769 11G / 18G INFO General (path_extenders.cpp : 34) Processed 128 paths from 130634 (0%) 5:16:40.718 11G / 18G INFO General (path_extenders.cpp : 34) Processed 256 paths from 130634 (0%) 5:16:48.535 11G / 18G INFO General (path_extenders.cpp : 34) Processed 512 paths from 130634 (0%) 5:17:02.361 11G / 18G INFO General (path_extenders.cpp : 34) Processed 1024 paths from 130634 (0%) 5:17:22.856 11G / 18G INFO General (path_extenders.cpp : 34) Processed 2048 paths from 130634 (1%) 5:17:55.331 11G / 18G INFO General (path_extenders.cpp : 34) Processed 4096 paths from 130634 (3%) 5:18:34.921 12G / 18G INFO General (path_extenders.cpp : 34) Processed 8192 paths from 130634 (6%) 5:18:55.644 12G / 18G INFO General (path_extenders.cpp : 36) Processed 13064 paths from 130634 (10%) 5:19:00.688 12G / 18G INFO General (path_extenders.cpp : 34) Processed 16384 paths from 130634 (12%) 5:19:08.546 12G / 18G INFO General (path_extenders.cpp : 36) Processed 26128 paths from 130634 (20%) 5:19:14.493 12G / 18G INFO General (path_extenders.cpp : 34) Processed 32768 paths from 130634 (25%) 5:19:17.567 12G / 18G INFO General (path_extenders.cpp : 36) Processed 39192 paths from 130634 (30%) 5:19:22.280 12G / 18G INFO General (path_extenders.cpp : 36) Processed 52256 paths from 130634 (40%) 5:19:25.929 12G / 18G INFO General (path_extenders.cpp : 36) Processed 65320 paths from 130634 (50%) 5:19:25.952 12G / 18G INFO General (path_extenders.cpp : 34) Processed 65536 paths from 130634 (50%) 5:19:27.645 12G / 18G INFO General (path_extenders.cpp : 36) Processed 78384 paths from 130634 (60%) 5:19:28.400 12G / 18G INFO General (path_extenders.cpp : 36) Processed 91448 paths from 130634 (70%) 5:19:28.726 12G / 18G INFO General (path_extenders.cpp : 36) Processed 104512 paths from 130634 (80%) 5:19:29.674 12G / 18G INFO General (path_extenders.cpp : 36) Processed 117576 paths from 130634 (90%) 5:19:30.048 12G / 18G INFO General (launcher.cpp : 250) Finalizing paths 5:19:30.061 12G / 18G INFO General (launcher.cpp : 252) Deduplicating paths 5:21:02.174 12G / 18G INFO General (launcher.cpp : 256) Paths deduplicated 5:21:02.195 12G / 18G INFO PEResolver (pe_resolver.cpp : 60) Removing overlaps 5:21:02.203 12G / 18G INFO PEResolver (pe_resolver.cpp : 63) Sorting paths 5:21:02.228 12G / 18G INFO PEResolver (pe_resolver.cpp : 70) Marking overlaps 5:21:02.239 12G / 18G INFO OverlapRemover (overlap_remover.hpp : 116) Marking start/end overlaps 5:23:45.182 12G / 18G INFO OverlapRemover (overlap_remover.hpp : 119) Marking remaining overlaps 5:26:31.453 12G / 18G INFO PEResolver (pe_resolver.cpp : 73) Splitting paths 5:26:32.359 13G / 18G INFO PEResolver (pe_resolver.cpp : 78) Deduplicating paths 5:26:35.751 12G / 18G INFO PEResolver (pe_resolver.cpp : 80) Overlaps removed 5:26:35.986 12G / 18G INFO General (launcher.cpp : 273) Paths finalized 5:26:36.004 12G / 18G INFO General (launcher.cpp : 454) Closing gaps in paths 5:26:37.920 12G / 18G INFO General (launcher.cpp : 484) Gap closing completed 5:26:38.028 12G / 18G INFO General (launcher.cpp : 302) Traversing tandem repeats 5:26:39.454 12G / 18G INFO General (launcher.cpp : 312) Traversed 89 loops 5:26:39.474 12G / 18G INFO General (launcher.cpp : 250) Finalizing paths 5:26:39.485 12G / 18G INFO General (launcher.cpp : 252) Deduplicating paths 5:26:39.974 12G / 18G INFO General (launcher.cpp : 256) Paths deduplicated 5:26:39.993 12G / 18G INFO PEResolver (pe_resolver.cpp : 60) Removing overlaps 5:26:40.002 12G / 18G INFO PEResolver (pe_resolver.cpp : 63) Sorting paths 5:26:40.045 12G / 18G INFO PEResolver (pe_resolver.cpp : 70) Marking overlaps 5:26:40.060 12G / 18G INFO OverlapRemover (overlap_remover.hpp : 116) Marking start/end overlaps 5:26:41.058 12G / 18G INFO OverlapRemover (overlap_remover.hpp : 119) Marking remaining overlaps 5:26:41.916 12G / 18G INFO PEResolver (pe_resolver.cpp : 73) Splitting paths 5:26:42.002 13G / 18G INFO PEResolver (pe_resolver.cpp : 78) Deduplicating paths 5:26:42.219 12G / 18G INFO PEResolver (pe_resolver.cpp : 80) Overlaps removed 5:26:42.365 13G / 18G INFO General (launcher.cpp : 273) Paths finalized 5:26:42.382 13G / 18G INFO General (launcher.cpp : 664) ExSPAnder repeat resolving tool finished 5:26:44.672 9461M / 18G INFO StageManager (stage.cpp : 185) STAGE == Contig Output (id: contig_output) 5:26:44.706 9461M / 18G INFO General (contig_output.hpp : 16) Outputting contigs to /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/before_rr.fasta 5:26:46.783 9461M / 18G INFO General (contig_output_stage.cpp : 151) Writing GFA graph to /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/assembly_graph_with_scaffolds.gfa 5:26:47.982 9461M / 18G INFO General (contig_output_stage.cpp : 165) Outputting FastG graph to /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/assembly_graph.fastg 5:26:51.752 9461M / 18G INFO General (contig_output_stage.cpp : 196) Breaking scaffolds 5:26:52.747 10G / 18G INFO General (contig_output_stage.cpp : 98) Outputting contigs to /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/final_contigs.fasta 5:26:53.732 10G / 18G INFO General (contig_output_stage.cpp : 104) Outputting FastG paths to /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/final_contigs.paths 5:26:55.338 9588M / 18G INFO General (contig_output_stage.cpp : 98) Outputting contigs to /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/scaffolds.fasta 5:26:55.925 9588M / 18G INFO General (contig_output_stage.cpp : 104) Outputting FastG paths to /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/scaffolds.paths 5:26:56.272 9588M / 18G INFO General (contig_output_stage.cpp : 111) Populating GFA with scaffold paths 5:26:56.875 9461M / 18G INFO General (pipeline.cpp : 287) SPAdes finished 5:27:14.472 34M / 18G INFO General (main.cpp : 136) Assembling time: 5 hours 27 minutes 14 seconds ===== K77 finished. ===== Copy files started. == Running: /oak/stanford/groups/akundaje/marinovg/programs/Anaconda2/bin/python /oak/stanford/groups/akundaje/marinovg/programs/SPAdes-3.15.4-Linux/share/spades/spades_pipeline/scripts/copy_files.py /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/before_rr.fasta /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/before_rr.fasta /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/assembly_graph_after_simplification.gfa /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/assembly_graph_after_simplification.gfa /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/final_contigs.fasta /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/contigs.fasta /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/first_pe_contigs.fasta /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/first_pe_contigs.fasta /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/strain_graph.gfa /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/strain_graph.gfa /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/scaffolds.fasta /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/scaffolds.fasta /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/scaffolds.paths /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/scaffolds.paths /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/assembly_graph_with_scaffolds.gfa /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/assembly_graph_with_scaffolds.gfa /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/assembly_graph.fastg /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/assembly_graph.fastg /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/K77/final_contigs.paths /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/contigs.paths ===== Copy files finished. ===== Assembling finished. ===== Breaking scaffolds started. == Running: /oak/stanford/groups/akundaje/marinovg/programs/Anaconda2/bin/python /oak/stanford/groups/akundaje/marinovg/programs/SPAdes-3.15.4-Linux/share/spades/spades_pipeline/scripts/breaking_scaffolds_script.py --result_scaffolds_filename /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/scaffolds.fasta --misc_dir /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/misc --threshold_for_breaking_scaffolds 3 ===== Breaking scaffolds finished. ===== Terminate started. ===== Terminate finished. * Corrected reads are in /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/corrected/ * Assembled contigs are in /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/contigs.fasta * Assembled scaffolds are in /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/scaffolds.fasta * Paths in the assembly graph corresponding to the contigs are in /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/contigs.paths * Paths in the assembly graph corresponding to the scaffolds are in /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/scaffolds.paths * Assembly graph is in /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/assembly_graph.fastg * Assembly graph in GFA format is in /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/assembly_graph_with_scaffolds.gfa ======= SPAdes pipeline finished WITH WARNINGS! === Error correction and assembling warnings: * 5:16:14.867 9226M / 18G WARN General (launcher.cpp : 178) Your data seems to have high uniform coverage depth. It is strongly recommended to use --isolate option. ======= Warnings saved to /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/warnings.log SPAdes log can be found here: /oak/stanford/groups/akundaje/marinovg/papers/2024_ToL_Hi-C/2023-07-09/14-Prymnesium_parvum/SPAdes-3.15.4-Prymnesium_parvum_12B1/spades.log Thank you for using SPAdes!