cellbender:remove-background: Command: cellbender remove-background --input /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/inputs/1156-Control-dACC.raw_feature_bc_matrix.h5 --output /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1156-Control-dACC/1156-Control-dACC.cellbender.output.h5 --cuda --expected-cells 5077 --total-droplets-included 20000 --fpr 0.01 --epochs 150 cellbender:remove-background: CellBender 0.3.0 cellbender:remove-background: (Workflow hash 709f0d62d0) cellbender:remove-background: 2024-06-06 15:42:00 cellbender:remove-background: Running remove-background cellbender:remove-background: Loading data from /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/inputs/1156-Control-dACC.raw_feature_bc_matrix.h5 cellbender:remove-background: CellRanger v3 format cellbender:remove-background: Features in dataset: 36601 Gene Expression cellbender:remove-background: Trimming features for inference. cellbender:remove-background: 31228 features have nonzero counts. cellbender:remove-background: Prior on counts for cells is 2714 cellbender:remove-background: Prior on counts for empty droplets is 83 cellbender:remove-background: Excluding 8966 features that are estimated to have <= 0.1 background counts in cells. cellbender:remove-background: Including 22262 features in the analysis. cellbender:remove-background: Trimming barcodes for inference. cellbender:remove-background: Excluding barcodes with counts below 41 cellbender:remove-background: Using 5077 probable cell barcodes, plus an additional 14923 barcodes, and 50908 empty droplets. cellbender:remove-background: Largest surely-empty droplet has 83 UMI counts. cellbender:remove-background: Attempting to unpack tarball "ckpt.tar.gz" to /tmp/tmp194cq7in cellbender:remove-background: Successfully unpacked tarball to /tmp/tmp194cq7in /tmp/tmp194cq7in/709f0d62d0_random.pyro /tmp/tmp194cq7in/709f0d62d0_random.cuda /tmp/tmp194cq7in/709f0d62d0_model.torch /tmp/tmp194cq7in/709f0d62d0_optim.torch /tmp/tmp194cq7in/709f0d62d0_optim.pyro /tmp/tmp194cq7in/709f0d62d0_params.pyro /tmp/tmp194cq7in/709f0d62d0_train.loaderstate /tmp/tmp194cq7in/709f0d62d0_test.loaderstate /tmp/tmp194cq7in/709f0d62d0_args.npy cellbender:remove-background: Loaded partially-trained checkpoint from ckpt.tar.gz cellbender:remove-background: Checkpoint loaded successfully. cellbender:remove-background: Running inference... cellbender:remove-background: [epoch 051] average training loss: 1882.3825 cellbender:remove-background: [epoch 052] average training loss: 1884.3905 (14.1 seconds per epoch) cellbender:remove-background: Will checkpoint every 30 epochs cellbender:remove-background: [epoch 053] average training loss: 1903.8227 cellbender:remove-background: [epoch 054] average training loss: 1899.5478 cellbender:remove-background: [epoch 055] average training loss: 1885.0804 cellbender:remove-background: [epoch 055] average test loss: 1884.7390 cellbender:remove-background: [epoch 056] average training loss: 1884.8611 cellbender:remove-background: [epoch 057] average training loss: 1907.2440 cellbender:remove-background: [epoch 058] average training loss: 1896.4233 cellbender:remove-background: [epoch 059] average training loss: 1879.0668 cellbender:remove-background: [epoch 060] average training loss: 1893.8288 cellbender:remove-background: [epoch 060] average test loss: 1927.7678 cellbender:remove-background: Saving a checkpoint... cellbender:remove-background: Saved checkpoint as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1156-Control-dACC/ckpt.tar.gz cellbender:remove-background: [epoch 061] average training loss: 1893.1974 cellbender:remove-background: [epoch 062] average training loss: 1895.1596 cellbender:remove-background: [epoch 063] average training loss: 1892.3404 cellbender:remove-background: [epoch 064] average training loss: 1885.5676 cellbender:remove-background: [epoch 065] average training loss: 1892.4193 cellbender:remove-background: [epoch 065] average test loss: 1892.6619 cellbender:remove-background: [epoch 066] average training loss: 1885.1121 cellbender:remove-background: [epoch 067] average training loss: 1900.4924 cellbender:remove-background: [epoch 068] average training loss: 1904.5841 cellbender:remove-background: [epoch 069] average training loss: 1896.6059 cellbender:remove-background: [epoch 070] average training loss: 1906.7262 cellbender:remove-background: [epoch 070] average test loss: 1937.2875 cellbender:remove-background: [epoch 071] average training loss: 1900.7040 cellbender:remove-background: [epoch 072] average training loss: 1909.4369 cellbender:remove-background: [epoch 073] average training loss: 1908.2303 cellbender:remove-background: [epoch 074] average training loss: 1896.8993 cellbender:remove-background: [epoch 075] average training loss: 1909.7040 cellbender:remove-background: [epoch 075] average test loss: 1918.7549 cellbender:remove-background: [epoch 076] average training loss: 1904.7776 cellbender:remove-background: [epoch 077] average training loss: 1903.7228 cellbender:remove-background: [epoch 078] average training loss: 1911.9777 cellbender:remove-background: [epoch 079] average training loss: 1914.2305 cellbender:remove-background: [epoch 080] average training loss: 1898.5025 cellbender:remove-background: [epoch 080] average test loss: 1938.1745 cellbender:remove-background: [epoch 081] average training loss: 1900.6056 cellbender:remove-background: [epoch 082] average training loss: 1899.5480 cellbender:remove-background: [epoch 083] average training loss: 1899.3484 cellbender:remove-background: [epoch 084] average training loss: 1898.3000 cellbender:remove-background: [epoch 085] average training loss: 1899.6367 cellbender:remove-background: [epoch 085] average test loss: 1962.0418 cellbender:remove-background: [epoch 086] average training loss: 1902.6624 cellbender:remove-background: [epoch 087] average training loss: 1898.8776 cellbender:remove-background: [epoch 088] average training loss: 1908.5222 cellbender:remove-background: [epoch 089] average training loss: 1898.0747 cellbender:remove-background: [epoch 090] average training loss: 1905.9394 cellbender:remove-background: [epoch 090] average test loss: 1922.1105 cellbender:remove-background: Saving a checkpoint... cellbender:remove-background: Saved checkpoint as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1156-Control-dACC/ckpt.tar.gz cellbender:remove-background: [epoch 091] average training loss: 1902.9578 cellbender:remove-background: [epoch 092] average training loss: 1898.9108 cellbender:remove-background: [epoch 093] average training loss: 1903.4688 cellbender:remove-background: [epoch 094] average training loss: 1913.1994 cellbender:remove-background: [epoch 095] average training loss: 1906.9761 cellbender:remove-background: [epoch 095] average test loss: 1929.7253 cellbender:remove-background: [epoch 096] average training loss: 1908.8612 cellbender:remove-background: [epoch 097] average training loss: 1900.6211 cellbender:remove-background: [epoch 098] average training loss: 1909.0823 cellbender:remove-background: [epoch 099] average training loss: 1907.7362 cellbender:remove-background: [epoch 100] average training loss: 1907.6859 cellbender:remove-background: [epoch 100] average test loss: 1945.8278 cellbender:remove-background: [epoch 101] average training loss: 1907.8634 cellbender:remove-background: [epoch 102] average training loss: 1909.0992 cellbender:remove-background: [epoch 103] average training loss: 1907.4835 cellbender:remove-background: [epoch 104] average training loss: 1906.8513 cellbender:remove-background: [epoch 105] average training loss: 1914.1792 cellbender:remove-background: [epoch 105] average test loss: 1955.8374 cellbender:remove-background: [epoch 106] average training loss: 1906.3478 cellbender:remove-background: [epoch 107] average training loss: 1907.9539 cellbender:remove-background: [epoch 108] average training loss: 1919.2132 cellbender:remove-background: [epoch 109] average training loss: 1911.6623 cellbender:remove-background: [epoch 110] average training loss: 1914.3470 cellbender:remove-background: [epoch 110] average test loss: 1952.3639 cellbender:remove-background: [epoch 111] average training loss: 1907.2297 cellbender:remove-background: [epoch 112] average training loss: 1905.4281 cellbender:remove-background: [epoch 113] average training loss: 1907.6710 cellbender:remove-background: [epoch 114] average training loss: 1905.2195 cellbender:remove-background: [epoch 115] average training loss: 1903.4467 cellbender:remove-background: [epoch 115] average test loss: 1945.4815 cellbender:remove-background: [epoch 116] average training loss: 1901.0007 cellbender:remove-background: [epoch 117] average training loss: 1904.8103 cellbender:remove-background: [epoch 118] average training loss: 1902.5397 cellbender:remove-background: [epoch 119] average training loss: 1901.5015 cellbender:remove-background: [epoch 120] average training loss: 1904.6104 cellbender:remove-background: [epoch 120] average test loss: 1947.8350 cellbender:remove-background: Saving a checkpoint... cellbender:remove-background: Saved checkpoint as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1156-Control-dACC/ckpt.tar.gz cellbender:remove-background: [epoch 121] average training loss: 1904.8587 cellbender:remove-background: [epoch 122] average training loss: 1904.0494 cellbender:remove-background: [epoch 123] average training loss: 1898.9621 cellbender:remove-background: [epoch 124] average training loss: 1903.6549 cellbender:remove-background: [epoch 125] average training loss: 1896.6095 cellbender:remove-background: [epoch 125] average test loss: 1958.2469 cellbender:remove-background: [epoch 126] average training loss: 1903.2002 cellbender:remove-background: [epoch 127] average training loss: 1902.5485 cellbender:remove-background: [epoch 128] average training loss: 1905.8521 cellbender:remove-background: [epoch 129] average training loss: 1897.7300 cellbender:remove-background: [epoch 130] average training loss: 1904.0434 cellbender:remove-background: [epoch 130] average test loss: 1939.4079 cellbender:remove-background: [epoch 131] average training loss: 1892.6861 cellbender:remove-background: [epoch 132] average training loss: 1900.8516 cellbender:remove-background: [epoch 133] average training loss: 1901.4134 cellbender:remove-background: [epoch 134] average training loss: 1903.8543 cellbender:remove-background: [epoch 135] average training loss: 1900.7232 cellbender:remove-background: [epoch 135] average test loss: 1956.0818 cellbender:remove-background: [epoch 136] average training loss: 1896.6271 cellbender:remove-background: [epoch 137] average training loss: 1902.3572 cellbender:remove-background: [epoch 138] average training loss: 1901.1447 cellbender:remove-background: [epoch 139] average training loss: 1894.6750 cellbender:remove-background: [epoch 140] average training loss: 1899.4234 cellbender:remove-background: [epoch 140] average test loss: 1953.6230 cellbender:remove-background: [epoch 141] average training loss: 1901.3527 cellbender:remove-background: [epoch 142] average training loss: 1903.0780 cellbender:remove-background: [epoch 143] average training loss: 1900.5363 cellbender:remove-background: [epoch 144] average training loss: 1901.3474 cellbender:remove-background: [epoch 145] average training loss: 1902.2001 cellbender:remove-background: [epoch 145] average test loss: 1944.8245 cellbender:remove-background: [epoch 146] average training loss: 1897.7482 cellbender:remove-background: [epoch 147] average training loss: 1899.7961 cellbender:remove-background: [epoch 148] average training loss: 1901.4638 cellbender:remove-background: [epoch 149] average training loss: 1898.8705 cellbender:remove-background: [epoch 150] average training loss: 1898.4877 cellbender:remove-background: [epoch 150] average test loss: 1957.0018 cellbender:remove-background: Saving a checkpoint... cellbender:remove-background: Saved checkpoint as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1156-Control-dACC/ckpt.tar.gz cellbender:remove-background: 2024-06-06 16:20:59 cellbender:remove-background: Inference procedure complete. cellbender:remove-background: Attempting to unpack tarball "ckpt.tar.gz" to /tmp/tmpdkw5ojg_ cellbender:remove-background: Successfully unpacked tarball to /tmp/tmpdkw5ojg_ /tmp/tmpdkw5ojg_/709f0d62d0_random.pyro /tmp/tmpdkw5ojg_/709f0d62d0_random.cuda /tmp/tmpdkw5ojg_/709f0d62d0_model.torch /tmp/tmpdkw5ojg_/709f0d62d0_optim.torch /tmp/tmpdkw5ojg_/709f0d62d0_optim.pyro /tmp/tmpdkw5ojg_/709f0d62d0_params.pyro /tmp/tmpdkw5ojg_/709f0d62d0_train.loaderstate /tmp/tmpdkw5ojg_/709f0d62d0_test.loaderstate /tmp/tmpdkw5ojg_/709f0d62d0_args.npy cellbender:remove-background: Posterior not currently included in checkpoint. cellbender:remove-background: Computing posterior noise count probabilities in mini-batches. cellbender:remove-background: Working on chunk (1/62) cellbender:remove-background: [0.10 mins per chunk] cellbender:remove-background: Working on chunk (2/62) cellbender:remove-background: Working on chunk (3/62) cellbender:remove-background: Working on chunk (4/62) cellbender:remove-background: Working on chunk (5/62) cellbender:remove-background: Working on chunk (6/62) cellbender:remove-background: Working on chunk (7/62) cellbender:remove-background: Working on chunk (8/62) cellbender:remove-background: Working on chunk (9/62) cellbender:remove-background: Working on chunk (10/62) cellbender:remove-background: Working on chunk (11/62) cellbender:remove-background: Working on chunk (12/62) cellbender:remove-background: Working on chunk (13/62) cellbender:remove-background: Working on chunk (14/62) cellbender:remove-background: Working on chunk (15/62) cellbender:remove-background: Working on chunk (16/62) cellbender:remove-background: Working on chunk (17/62) cellbender:remove-background: Working on chunk (18/62) cellbender:remove-background: Working on chunk (19/62) cellbender:remove-background: Working on chunk (20/62) cellbender:remove-background: Working on chunk (21/62) cellbender:remove-background: Working on chunk (22/62) cellbender:remove-background: Working on chunk (23/62) cellbender:remove-background: Working on chunk (24/62) cellbender:remove-background: Working on chunk (25/62) cellbender:remove-background: Working on chunk (26/62) cellbender:remove-background: Working on chunk (27/62) cellbender:remove-background: Working on chunk (28/62) cellbender:remove-background: Working on chunk (29/62) cellbender:remove-background: Working on chunk (30/62) cellbender:remove-background: Working on chunk (31/62) cellbender:remove-background: Working on chunk (32/62) cellbender:remove-background: Working on chunk (33/62) cellbender:remove-background: Working on chunk (34/62) cellbender:remove-background: Working on chunk (35/62) cellbender:remove-background: Working on chunk (36/62) cellbender:remove-background: Working on chunk (37/62) cellbender:remove-background: Working on chunk (38/62) cellbender:remove-background: Working on chunk (39/62) cellbender:remove-background: Working on chunk (40/62) cellbender:remove-background: Working on chunk (41/62) cellbender:remove-background: Working on chunk (42/62) cellbender:remove-background: Working on chunk (43/62) cellbender:remove-background: Working on chunk (44/62) cellbender:remove-background: Working on chunk (45/62) cellbender:remove-background: Working on chunk (46/62) cellbender:remove-background: Working on chunk (47/62) cellbender:remove-background: Working on chunk (48/62) cellbender:remove-background: Working on chunk (49/62) cellbender:remove-background: Working on chunk (50/62) cellbender:remove-background: Working on chunk (51/62) cellbender:remove-background: Working on chunk (52/62) cellbender:remove-background: Working on chunk (53/62) cellbender:remove-background: Working on chunk (54/62) cellbender:remove-background: Working on chunk (55/62) cellbender:remove-background: Working on chunk (56/62) cellbender:remove-background: Working on chunk (57/62) cellbender:remove-background: Working on chunk (58/62) cellbender:remove-background: Working on chunk (59/62) cellbender:remove-background: Working on chunk (60/62) cellbender:remove-background: Working on chunk (61/62) cellbender:remove-background: Working on chunk (62/62) cellbender:remove-background: Writing full posterior to /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1156-Control-dACC/1156-Control-dACC.cellbender.output_posterior.h5 cellbender:remove-background: Succeeded in writing posterior to file /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1156-Control-dACC/1156-Control-dACC.cellbender.output_posterior.h5 cellbender:remove-background: Added posterior object to checkpoint file. cellbender:remove-background: 2024-06-06 16:32:50 cellbender:remove-background: Saved summary plots as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1156-Control-dACC/1156-Control-dACC.cellbender.output.pdf cellbender:remove-background: Saved cell barcodes in /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1156-Control-dACC/1156-Control-dACC.cellbender.output_cell_barcodes.csv cellbender:remove-background: Computing target noise counts per gene for MCKP estimator cellbender:remove-background: Using MCKP noise targets computed for FPR 0.01 cellbender:remove-background: Computing denoised counts using mckp estimator cellbender:remove-background: Working on chunk (1/7) cellbender:remove-background: [2.77 mins per chunk] cellbender:remove-background: Working on chunk (2/7) cellbender:remove-background: Working on chunk (3/7) cellbender:remove-background: Working on chunk (4/7) cellbender:remove-background: Working on chunk (5/7) cellbender:remove-background: Working on chunk (6/7) cellbender:remove-background: Working on chunk (7/7) cellbender:remove-background: 2024-06-06 16:53:01 Total MCKP estimation time = 1169.82 sec cellbender:remove-background: Succeeded in writing CellRanger format output to file /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1156-Control-dACC/1156-Control-dACC.cellbender.output.h5 cellbender:remove-background: Succeeded in writing CellRanger format output to file /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1156-Control-dACC/1156-Control-dACC.cellbender.output_filtered.h5 cellbender:remove-background: Saved output metrics as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1156-Control-dACC/1156-Control-dACC.cellbender.output_metrics.csv Traceback (most recent call last): File "/home/users/guozy/miniconda3/envs/cellbender/bin/jupyter-nbconvert", line 5, in from nbconvert.nbconvertapp import main File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/__init__.py", line 5, in from .exporters import * File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/__init__.py", line 1, in from .asciidoc import ASCIIDocExporter File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/asciidoc.py", line 9, in from .templateexporter import TemplateExporter File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/templateexporter.py", line 25, in from lxml.html.clean import clean_html File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/lxml/html/clean.py", line 21, in ) from None ImportError: lxml.html.clean module is now a separate project lxml_html_clean. Install lxml[html_clean] or lxml_html_clean directly. Traceback (most recent call last): File "/home/users/guozy/miniconda3/envs/cellbender/bin/jupyter-nbconvert", line 5, in from nbconvert.nbconvertapp import main File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/__init__.py", line 5, in from .exporters import * File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/__init__.py", line 1, in from .asciidoc import ASCIIDocExporter File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/asciidoc.py", line 9, in from .templateexporter import TemplateExporter File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/templateexporter.py", line 25, in from lxml.html.clean import clean_html File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/lxml/html/clean.py", line 21, in ) from None ImportError: lxml.html.clean module is now a separate project lxml_html_clean. Install lxml[html_clean] or lxml_html_clean directly. mv: cannot stat ‘tmp.report.nbconvert.html’: No such file or directory rm: cannot remove ‘tmp.report.nbconvert.ipynb’: No such file or directory cellbender:remove-background: Unable to create report. cellbender:remove-background: Traceback (most recent call last): File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/run.py", line 351, in compute_output_denoised_counts_reports_metrics output=html_report_file, File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/report.py", line 82, in run_notebook_make_html title=('CellBender: ' + os.path.basename(output).replace('_report.html', '')), File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/report.py", line 59, in _postprocess_html with open(file, mode='r') as f: FileNotFoundError: [Errno 2] No such file or directory: '/oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1156-Control-dACC/1156-Control-dACC.cellbender.output_report.html' cellbender:remove-background: Completed remove-background. cellbender:remove-background: 2024-06-06 16:53:30