cellbender:remove-background: Command: cellbender remove-background --input /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/inputs/1713-Bipolar-dACC.raw_feature_bc_matrix.h5 --output /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1713-Bipolar-dACC/1713-Bipolar-dACC.cellbender.output.h5 --cuda --expected-cells 7212 --total-droplets-included 20000 --fpr 0.01 --epochs 150 cellbender:remove-background: CellBender 0.3.0 cellbender:remove-background: (Workflow hash ef420f3ad3) cellbender:remove-background: 2024-06-06 15:37:51 cellbender:remove-background: Running remove-background cellbender:remove-background: Loading data from /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/inputs/1713-Bipolar-dACC.raw_feature_bc_matrix.h5 cellbender:remove-background: CellRanger v3 format cellbender:remove-background: Features in dataset: 36601 Gene Expression cellbender:remove-background: Trimming features for inference. cellbender:remove-background: 30688 features have nonzero counts. cellbender:remove-background: Prior on counts for cells is 614 cellbender:remove-background: Prior on counts for empty droplets is 54 cellbender:remove-background: Excluding 6029 features that are estimated to have <= 0.1 background counts in cells. cellbender:remove-background: Including 24659 features in the analysis. cellbender:remove-background: Trimming barcodes for inference. cellbender:remove-background: Excluding barcodes with counts below 27 cellbender:remove-background: Using 7212 probable cell barcodes, plus an additional 12788 barcodes, and 56617 empty droplets. cellbender:remove-background: Largest surely-empty droplet has 54 UMI counts. cellbender:remove-background: Attempting to unpack tarball "ckpt.tar.gz" to /tmp/tmpuhkgjlxy cellbender:remove-background: Successfully unpacked tarball to /tmp/tmpuhkgjlxy /tmp/tmpuhkgjlxy/ef420f3ad3_random.pyro /tmp/tmpuhkgjlxy/ef420f3ad3_random.cuda /tmp/tmpuhkgjlxy/ef420f3ad3_model.torch /tmp/tmpuhkgjlxy/ef420f3ad3_optim.torch /tmp/tmpuhkgjlxy/ef420f3ad3_optim.pyro /tmp/tmpuhkgjlxy/ef420f3ad3_params.pyro /tmp/tmpuhkgjlxy/ef420f3ad3_train.loaderstate /tmp/tmpuhkgjlxy/ef420f3ad3_test.loaderstate /tmp/tmpuhkgjlxy/ef420f3ad3_args.npy cellbender:remove-background: Loaded partially-trained checkpoint from ckpt.tar.gz cellbender:remove-background: Checkpoint loaded successfully. cellbender:remove-background: Running inference... cellbender:remove-background: [epoch 030] average training loss: 1269.6045 cellbender:remove-background: [epoch 030] average test loss: 1299.1706 cellbender:remove-background: [epoch 031] average training loss: 1269.3303 (7.5 seconds per epoch) cellbender:remove-background: Will checkpoint every 57 epochs cellbender:remove-background: [epoch 032] average training loss: 1268.9527 cellbender:remove-background: [epoch 033] average training loss: 1270.0718 cellbender:remove-background: [epoch 034] average training loss: 1266.8183 cellbender:remove-background: [epoch 035] average training loss: 1268.2741 cellbender:remove-background: [epoch 035] average test loss: 1309.8190 cellbender:remove-background: [epoch 036] average training loss: 1266.6598 cellbender:remove-background: [epoch 037] average training loss: 1265.7328 cellbender:remove-background: [epoch 038] average training loss: 1265.4669 cellbender:remove-background: [epoch 039] average training loss: 1264.5627 cellbender:remove-background: [epoch 040] average training loss: 1264.0862 cellbender:remove-background: [epoch 040] average test loss: 1295.6418 cellbender:remove-background: [epoch 041] average training loss: 1264.5675 cellbender:remove-background: [epoch 042] average training loss: 1264.6442 cellbender:remove-background: [epoch 043] average training loss: 1262.8455 cellbender:remove-background: [epoch 044] average training loss: 1262.6712 cellbender:remove-background: [epoch 045] average training loss: 1262.2990 cellbender:remove-background: [epoch 045] average test loss: 1300.6260 cellbender:remove-background: [epoch 046] average training loss: 1262.6119 cellbender:remove-background: [epoch 047] average training loss: 1260.7175 cellbender:remove-background: [epoch 048] average training loss: 1261.9572 cellbender:remove-background: [epoch 049] average training loss: 1261.0720 cellbender:remove-background: [epoch 050] average training loss: 1259.8489 cellbender:remove-background: [epoch 050] average test loss: 1298.9659 cellbender:remove-background: [epoch 051] average training loss: 1259.0024 cellbender:remove-background: [epoch 052] average training loss: 1257.9463 cellbender:remove-background: [epoch 053] average training loss: 1257.8083 cellbender:remove-background: [epoch 054] average training loss: 1257.7514 cellbender:remove-background: [epoch 055] average training loss: 1258.3291 cellbender:remove-background: [epoch 055] average test loss: 1297.2478 cellbender:remove-background: [epoch 056] average training loss: 1257.4000 cellbender:remove-background: [epoch 057] average training loss: 1256.9259 cellbender:remove-background: Saving a checkpoint... cellbender:remove-background: Saved checkpoint as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1713-Bipolar-dACC/ckpt.tar.gz cellbender:remove-background: [epoch 058] average training loss: 1256.9456 cellbender:remove-background: [epoch 059] average training loss: 1257.6292 cellbender:remove-background: [epoch 060] average training loss: 1255.5107 cellbender:remove-background: [epoch 060] average test loss: 1289.1286 cellbender:remove-background: [epoch 061] average training loss: 1255.5735 cellbender:remove-background: [epoch 062] average training loss: 1254.4811 cellbender:remove-background: [epoch 063] average training loss: 1253.7255 cellbender:remove-background: [epoch 064] average training loss: 1254.4728 cellbender:remove-background: [epoch 065] average training loss: 1254.5488 cellbender:remove-background: [epoch 065] average test loss: 1292.7883 cellbender:remove-background: [epoch 066] average training loss: 1254.3451 cellbender:remove-background: [epoch 067] average training loss: 1255.9714 cellbender:remove-background: [epoch 068] average training loss: 1254.6605 cellbender:remove-background: [epoch 069] average training loss: 1255.7853 cellbender:remove-background: [epoch 070] average training loss: 1257.3060 cellbender:remove-background: [epoch 070] average test loss: 1311.3064 cellbender:remove-background: [epoch 071] average training loss: 1255.5078 cellbender:remove-background: [epoch 072] average training loss: 1257.3156 cellbender:remove-background: [epoch 073] average training loss: 1256.2870 cellbender:remove-background: [epoch 074] average training loss: 1260.5976 cellbender:remove-background: [epoch 075] average training loss: 1261.1003 cellbender:remove-background: [epoch 075] average test loss: 1314.6228 cellbender:remove-background: [epoch 076] average training loss: 1262.9977 cellbender:remove-background: [epoch 077] average training loss: 1256.5196 cellbender:remove-background: [epoch 078] average training loss: 1261.1601 cellbender:remove-background: [epoch 079] average training loss: 1255.2653 cellbender:remove-background: [epoch 080] average training loss: 1255.9677 cellbender:remove-background: [epoch 080] average test loss: 1296.1256 cellbender:remove-background: [epoch 081] average training loss: 1254.5416 cellbender:remove-background: [epoch 082] average training loss: 1256.2263 cellbender:remove-background: [epoch 083] average training loss: 1255.9809 cellbender:remove-background: [epoch 084] average training loss: 1253.9114 cellbender:remove-background: [epoch 085] average training loss: 1255.0623 cellbender:remove-background: [epoch 085] average test loss: 1309.5721 cellbender:remove-background: [epoch 086] average training loss: 1253.4700 cellbender:remove-background: [epoch 087] average training loss: 1255.2275 cellbender:remove-background: [epoch 088] average training loss: 1255.4620 cellbender:remove-background: [epoch 089] average training loss: 1253.1880 cellbender:remove-background: [epoch 090] average training loss: 1256.0949 cellbender:remove-background: [epoch 090] average test loss: 1295.1523 cellbender:remove-background: [epoch 091] average training loss: 1251.0144 cellbender:remove-background: [epoch 092] average training loss: 1245.0098 cellbender:remove-background: [epoch 093] average training loss: 1244.0590 cellbender:remove-background: [epoch 094] average training loss: 1252.3945 cellbender:remove-background: [epoch 095] average training loss: 1244.4484 cellbender:remove-background: [epoch 095] average test loss: 1282.9595 cellbender:remove-background: [epoch 096] average training loss: 1246.7203 cellbender:remove-background: [epoch 097] average training loss: 1244.4224 cellbender:remove-background: [epoch 098] average training loss: 1250.4392 cellbender:remove-background: [epoch 099] average training loss: 1242.0271 cellbender:remove-background: [epoch 100] average training loss: 1242.3642 cellbender:remove-background: [epoch 100] average test loss: 1300.8133 cellbender:remove-background: [epoch 101] average training loss: 1241.9943 cellbender:remove-background: [epoch 102] average training loss: 1243.3267 cellbender:remove-background: [epoch 103] average training loss: 1243.1942 cellbender:remove-background: [epoch 104] average training loss: 1244.9466 cellbender:remove-background: [epoch 105] average training loss: 1243.8791 cellbender:remove-background: [epoch 105] average test loss: 1292.0963 cellbender:remove-background: [epoch 106] average training loss: 1243.3912 cellbender:remove-background: [epoch 107] average training loss: 1243.1408 cellbender:remove-background: [epoch 108] average training loss: 1243.8426 cellbender:remove-background: [epoch 109] average training loss: 1244.1435 cellbender:remove-background: [epoch 110] average training loss: 1242.2024 cellbender:remove-background: [epoch 110] average test loss: 1304.6732 cellbender:remove-background: [epoch 111] average training loss: 1242.7933 cellbender:remove-background: [epoch 112] average training loss: 1240.9035 cellbender:remove-background: [epoch 113] average training loss: 1240.3135 cellbender:remove-background: [epoch 114] average training loss: 1241.4403 cellbender:remove-background: Saving a checkpoint... cellbender:remove-background: Saved checkpoint as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1713-Bipolar-dACC/ckpt.tar.gz cellbender:remove-background: [epoch 115] average training loss: 1243.9050 cellbender:remove-background: [epoch 115] average test loss: 1292.7526 cellbender:remove-background: [epoch 116] average training loss: 1243.8235 cellbender:remove-background: [epoch 117] average training loss: 1238.5502 cellbender:remove-background: [epoch 118] average training loss: 1238.5028 cellbender:remove-background: [epoch 119] average training loss: 1239.1299 cellbender:remove-background: [epoch 120] average training loss: 1239.4382 cellbender:remove-background: [epoch 120] average test loss: 1291.6997 cellbender:remove-background: [epoch 121] average training loss: 1233.9984 cellbender:remove-background: [epoch 122] average training loss: 1235.9352 cellbender:remove-background: [epoch 123] average training loss: 1236.9536 cellbender:remove-background: [epoch 124] average training loss: 1234.1271 cellbender:remove-background: [epoch 125] average training loss: 1237.4961 cellbender:remove-background: [epoch 125] average test loss: 1297.3569 cellbender:remove-background: [epoch 126] average training loss: 1235.6023 cellbender:remove-background: [epoch 127] average training loss: 1235.9928 cellbender:remove-background: [epoch 128] average training loss: 1237.9398 cellbender:remove-background: [epoch 129] average training loss: 1239.5488 cellbender:remove-background: [epoch 130] average training loss: 1238.7377 cellbender:remove-background: [epoch 130] average test loss: 1310.3171 cellbender:remove-background: [epoch 131] average training loss: 1236.4616 cellbender:remove-background: [epoch 132] average training loss: 1237.7544 cellbender:remove-background: [epoch 133] average training loss: 1234.1596 cellbender:remove-background: [epoch 134] average training loss: 1235.8776 cellbender:remove-background: [epoch 135] average training loss: 1235.7531 cellbender:remove-background: [epoch 135] average test loss: 1306.9801 cellbender:remove-background: [epoch 136] average training loss: 1236.1652 cellbender:remove-background: [epoch 137] average training loss: 1238.7286 cellbender:remove-background: [epoch 138] average training loss: 1236.5666 cellbender:remove-background: [epoch 139] average training loss: 1235.7881 cellbender:remove-background: [epoch 140] average training loss: 1234.8922 cellbender:remove-background: [epoch 140] average test loss: 1305.9972 cellbender:remove-background: [epoch 141] average training loss: 1237.5388 cellbender:remove-background: [epoch 142] average training loss: 1238.0078 cellbender:remove-background: [epoch 143] average training loss: 1236.1793 cellbender:remove-background: [epoch 144] average training loss: 1237.3553 cellbender:remove-background: [epoch 145] average training loss: 1235.2325 cellbender:remove-background: [epoch 145] average test loss: 1299.8762 cellbender:remove-background: [epoch 146] average training loss: 1236.2047 cellbender:remove-background: [epoch 147] average training loss: 1240.0064 cellbender:remove-background: [epoch 148] average training loss: 1235.5464 cellbender:remove-background: [epoch 149] average training loss: 1237.7068 cellbender:remove-background: [epoch 150] average training loss: 1235.0146 cellbender:remove-background: [epoch 150] average test loss: 1302.9977 cellbender:remove-background: Saving a checkpoint... cellbender:remove-background: Saved checkpoint as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1713-Bipolar-dACC/ckpt.tar.gz cellbender:remove-background: 2024-06-06 15:59:54 cellbender:remove-background: Inference procedure complete. cellbender:remove-background: Attempting to unpack tarball "ckpt.tar.gz" to /tmp/tmphp104pns cellbender:remove-background: Successfully unpacked tarball to /tmp/tmphp104pns /tmp/tmphp104pns/ef420f3ad3_random.pyro /tmp/tmphp104pns/ef420f3ad3_random.cuda /tmp/tmphp104pns/ef420f3ad3_model.torch /tmp/tmphp104pns/ef420f3ad3_optim.torch /tmp/tmphp104pns/ef420f3ad3_optim.pyro /tmp/tmphp104pns/ef420f3ad3_params.pyro /tmp/tmphp104pns/ef420f3ad3_train.loaderstate /tmp/tmphp104pns/ef420f3ad3_test.loaderstate /tmp/tmphp104pns/ef420f3ad3_args.npy cellbender:remove-background: Posterior not currently included in checkpoint. cellbender:remove-background: Computing posterior noise count probabilities in mini-batches. cellbender:remove-background: Working on chunk (1/57) cellbender:remove-background: [0.04 mins per chunk] cellbender:remove-background: Working on chunk (2/57) cellbender:remove-background: Working on chunk (3/57) cellbender:remove-background: Working on chunk (4/57) cellbender:remove-background: Working on chunk (5/57) cellbender:remove-background: Working on chunk (6/57) cellbender:remove-background: Working on chunk (7/57) cellbender:remove-background: Working on chunk (8/57) cellbender:remove-background: Working on chunk (9/57) cellbender:remove-background: Working on chunk (10/57) cellbender:remove-background: Working on chunk (11/57) cellbender:remove-background: Working on chunk (12/57) cellbender:remove-background: Working on chunk (13/57) cellbender:remove-background: Working on chunk (14/57) cellbender:remove-background: Working on chunk (15/57) cellbender:remove-background: Working on chunk (16/57) cellbender:remove-background: Working on chunk (17/57) cellbender:remove-background: Working on chunk (18/57) cellbender:remove-background: Working on chunk (19/57) cellbender:remove-background: Working on chunk (20/57) cellbender:remove-background: Working on chunk (21/57) cellbender:remove-background: Working on chunk (22/57) cellbender:remove-background: Working on chunk (23/57) cellbender:remove-background: Working on chunk (24/57) cellbender:remove-background: Working on chunk (25/57) cellbender:remove-background: Working on chunk (26/57) cellbender:remove-background: Working on chunk (27/57) cellbender:remove-background: Working on chunk (28/57) cellbender:remove-background: Working on chunk (29/57) cellbender:remove-background: Working on chunk (30/57) cellbender:remove-background: Working on chunk (31/57) cellbender:remove-background: Working on chunk (32/57) cellbender:remove-background: Working on chunk (33/57) cellbender:remove-background: Working on chunk (34/57) cellbender:remove-background: Working on chunk (35/57) cellbender:remove-background: Working on chunk (36/57) cellbender:remove-background: Working on chunk (37/57) cellbender:remove-background: Working on chunk (38/57) cellbender:remove-background: Working on chunk (39/57) cellbender:remove-background: Working on chunk (40/57) cellbender:remove-background: Working on chunk (41/57) cellbender:remove-background: Working on chunk (42/57) cellbender:remove-background: Working on chunk (43/57) cellbender:remove-background: Working on chunk (44/57) cellbender:remove-background: Working on chunk (45/57) cellbender:remove-background: Working on chunk (46/57) cellbender:remove-background: Working on chunk (47/57) cellbender:remove-background: Working on chunk (48/57) cellbender:remove-background: Working on chunk (49/57) cellbender:remove-background: Working on chunk (50/57) cellbender:remove-background: Working on chunk (51/57) cellbender:remove-background: Working on chunk (52/57) cellbender:remove-background: Working on chunk (53/57) cellbender:remove-background: Working on chunk (54/57) cellbender:remove-background: Working on chunk (55/57) cellbender:remove-background: Working on chunk (56/57) cellbender:remove-background: Working on chunk (57/57) cellbender:remove-background: Writing full posterior to /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1713-Bipolar-dACC/1713-Bipolar-dACC.cellbender.output_posterior.h5 cellbender:remove-background: Succeeded in writing posterior to file /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1713-Bipolar-dACC/1713-Bipolar-dACC.cellbender.output_posterior.h5 cellbender:remove-background: Added posterior object to checkpoint file. cellbender:remove-background: 2024-06-06 16:04:55 cellbender:remove-background: Saved summary plots as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1713-Bipolar-dACC/1713-Bipolar-dACC.cellbender.output.pdf cellbender:remove-background: Saved cell barcodes in /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1713-Bipolar-dACC/1713-Bipolar-dACC.cellbender.output_cell_barcodes.csv cellbender:remove-background: Computing target noise counts per gene for MCKP estimator cellbender:remove-background: Using MCKP noise targets computed for FPR 0.01 cellbender:remove-background: Computing denoised counts using mckp estimator cellbender:remove-background: Working on chunk (1/7) cellbender:remove-background: [0.58 mins per chunk] cellbender:remove-background: Working on chunk (2/7) cellbender:remove-background: Working on chunk (3/7) cellbender:remove-background: Working on chunk (4/7) cellbender:remove-background: Working on chunk (5/7) cellbender:remove-background: Working on chunk (6/7) cellbender:remove-background: Working on chunk (7/7) cellbender:remove-background: 2024-06-06 16:09:08 Total MCKP estimation time = 243.55 sec cellbender:remove-background: Succeeded in writing CellRanger format output to file /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1713-Bipolar-dACC/1713-Bipolar-dACC.cellbender.output.h5 cellbender:remove-background: Succeeded in writing CellRanger format output to file /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1713-Bipolar-dACC/1713-Bipolar-dACC.cellbender.output_filtered.h5 cellbender:remove-background: Saved output metrics as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1713-Bipolar-dACC/1713-Bipolar-dACC.cellbender.output_metrics.csv Traceback (most recent call last): File "/home/users/guozy/miniconda3/envs/cellbender/bin/jupyter-nbconvert", line 5, in from nbconvert.nbconvertapp import main File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/__init__.py", line 5, in from .exporters import * File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/__init__.py", line 1, in from .asciidoc import ASCIIDocExporter File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/asciidoc.py", line 9, in from .templateexporter import TemplateExporter File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/templateexporter.py", line 25, in from lxml.html.clean import clean_html File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/lxml/html/clean.py", line 21, in ) from None ImportError: lxml.html.clean module is now a separate project lxml_html_clean. Install lxml[html_clean] or lxml_html_clean directly. Traceback (most recent call last): File "/home/users/guozy/miniconda3/envs/cellbender/bin/jupyter-nbconvert", line 5, in from nbconvert.nbconvertapp import main File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/__init__.py", line 5, in from .exporters import * File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/__init__.py", line 1, in from .asciidoc import ASCIIDocExporter File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/asciidoc.py", line 9, in from .templateexporter import TemplateExporter File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/templateexporter.py", line 25, in from lxml.html.clean import clean_html File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/lxml/html/clean.py", line 21, in ) from None ImportError: lxml.html.clean module is now a separate project lxml_html_clean. Install lxml[html_clean] or lxml_html_clean directly. mv: cannot stat ‘tmp.report.nbconvert.html’: No such file or directory rm: cannot remove ‘tmp.report.nbconvert.ipynb’: No such file or directory cellbender:remove-background: Unable to create report. cellbender:remove-background: Traceback (most recent call last): File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/run.py", line 351, in compute_output_denoised_counts_reports_metrics output=html_report_file, File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/report.py", line 82, in run_notebook_make_html title=('CellBender: ' + os.path.basename(output).replace('_report.html', '')), File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/report.py", line 59, in _postprocess_html with open(file, mode='r') as f: FileNotFoundError: [Errno 2] No such file or directory: '/oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1713-Bipolar-dACC/1713-Bipolar-dACC.cellbender.output_report.html' cellbender:remove-background: Completed remove-background. cellbender:remove-background: 2024-06-06 16:09:19