cellbender:remove-background: Command: cellbender remove-background --input /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/inputs/2291-Schizophrenia-dACC.raw_feature_bc_matrix.h5 --output /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2291-Schizophrenia-dACC/2291-Schizophrenia-dACC.cellbender.output.h5 --cuda --expected-cells 5304 --total-droplets-included 20000 --fpr 0.01 --epochs 150 cellbender:remove-background: CellBender 0.3.0 cellbender:remove-background: (Workflow hash 860e842553) cellbender:remove-background: 2024-06-06 15:42:37 cellbender:remove-background: Running remove-background cellbender:remove-background: Loading data from /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/inputs/2291-Schizophrenia-dACC.raw_feature_bc_matrix.h5 cellbender:remove-background: CellRanger v3 format cellbender:remove-background: Features in dataset: 36601 Gene Expression cellbender:remove-background: Trimming features for inference. cellbender:remove-background: 29375 features have nonzero counts. cellbender:remove-background: Prior on counts for cells is 1237 cellbender:remove-background: Prior on counts for empty droplets is 46 cellbender:remove-background: Excluding 10293 features that are estimated to have <= 0.1 background counts in cells. cellbender:remove-background: Including 19082 features in the analysis. cellbender:remove-background: Trimming barcodes for inference. cellbender:remove-background: Excluding barcodes with counts below 23 cellbender:remove-background: Using 5304 probable cell barcodes, plus an additional 14696 barcodes, and 23064 empty droplets. cellbender:remove-background: Largest surely-empty droplet has 46 UMI counts. cellbender:remove-background: Attempting to unpack tarball "ckpt.tar.gz" to /tmp/tmpm5ivsdqu cellbender:remove-background: Successfully unpacked tarball to /tmp/tmpm5ivsdqu /tmp/tmpm5ivsdqu/860e842553_random.pyro /tmp/tmpm5ivsdqu/860e842553_random.cuda /tmp/tmpm5ivsdqu/860e842553_model.torch /tmp/tmpm5ivsdqu/860e842553_optim.torch /tmp/tmpm5ivsdqu/860e842553_optim.pyro /tmp/tmpm5ivsdqu/860e842553_params.pyro /tmp/tmpm5ivsdqu/860e842553_train.loaderstate /tmp/tmpm5ivsdqu/860e842553_test.loaderstate /tmp/tmpm5ivsdqu/860e842553_args.npy cellbender:remove-background: Loaded partially-trained checkpoint from ckpt.tar.gz cellbender:remove-background: Checkpoint loaded successfully. cellbender:remove-background: Running inference... cellbender:remove-background: [epoch 057] average training loss: 1114.4454 cellbender:remove-background: [epoch 058] average training loss: 1112.1643 (4.9 seconds per epoch) cellbender:remove-background: Will checkpoint every 86 epochs cellbender:remove-background: [epoch 059] average training loss: 1109.8018 cellbender:remove-background: [epoch 060] average training loss: 1111.2692 cellbender:remove-background: [epoch 060] average test loss: 1107.9363 cellbender:remove-background: [epoch 061] average training loss: 1111.2063 cellbender:remove-background: [epoch 062] average training loss: 1105.9656 cellbender:remove-background: [epoch 063] average training loss: 1109.2881 cellbender:remove-background: [epoch 064] average training loss: 1114.0863 cellbender:remove-background: [epoch 065] average training loss: 1109.7907 cellbender:remove-background: [epoch 065] average test loss: 1126.2713 cellbender:remove-background: [epoch 066] average training loss: 1109.7108 cellbender:remove-background: [epoch 067] average training loss: 1105.1865 cellbender:remove-background: [epoch 068] average training loss: 1108.8992 cellbender:remove-background: [epoch 069] average training loss: 1109.8698 cellbender:remove-background: [epoch 070] average training loss: 1111.1959 cellbender:remove-background: [epoch 070] average test loss: 1122.1820 cellbender:remove-background: [epoch 071] average training loss: 1100.5155 cellbender:remove-background: [epoch 072] average training loss: 1103.0028 cellbender:remove-background: [epoch 073] average training loss: 1103.3733 cellbender:remove-background: [epoch 074] average training loss: 1104.0301 cellbender:remove-background: [epoch 075] average training loss: 1104.5659 cellbender:remove-background: [epoch 075] average test loss: 1124.4972 cellbender:remove-background: [epoch 076] average training loss: 1103.8970 cellbender:remove-background: [epoch 077] average training loss: 1098.3915 cellbender:remove-background: [epoch 078] average training loss: 1102.4486 cellbender:remove-background: [epoch 079] average training loss: 1101.5687 cellbender:remove-background: [epoch 080] average training loss: 1097.6485 cellbender:remove-background: [epoch 080] average test loss: 1122.3618 cellbender:remove-background: [epoch 081] average training loss: 1101.6829 cellbender:remove-background: [epoch 082] average training loss: 1102.4695 cellbender:remove-background: [epoch 083] average training loss: 1097.1607 cellbender:remove-background: [epoch 084] average training loss: 1102.9561 cellbender:remove-background: [epoch 085] average training loss: 1097.3409 cellbender:remove-background: [epoch 085] average test loss: 1124.8461 cellbender:remove-background: [epoch 086] average training loss: 1106.1087 cellbender:remove-background: Saving a checkpoint... cellbender:remove-background: Saved checkpoint as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2291-Schizophrenia-dACC/ckpt.tar.gz cellbender:remove-background: [epoch 087] average training loss: 1101.8793 cellbender:remove-background: [epoch 088] average training loss: 1104.3158 cellbender:remove-background: [epoch 089] average training loss: 1100.2072 cellbender:remove-background: [epoch 090] average training loss: 1102.0516 cellbender:remove-background: [epoch 090] average test loss: 1128.6240 cellbender:remove-background: [epoch 091] average training loss: 1099.1897 cellbender:remove-background: [epoch 092] average training loss: 1096.1626 cellbender:remove-background: [epoch 093] average training loss: 1103.0887 cellbender:remove-background: [epoch 094] average training loss: 1097.2444 cellbender:remove-background: [epoch 095] average training loss: 1096.5141 cellbender:remove-background: [epoch 095] average test loss: 1121.4993 cellbender:remove-background: [epoch 096] average training loss: 1096.3855 cellbender:remove-background: [epoch 097] average training loss: 1099.0154 cellbender:remove-background: [epoch 098] average training loss: 1096.0011 cellbender:remove-background: [epoch 099] average training loss: 1091.9989 cellbender:remove-background: [epoch 100] average training loss: 1091.3878 cellbender:remove-background: [epoch 100] average test loss: 1126.7895 cellbender:remove-background: [epoch 101] average training loss: 1092.0313 cellbender:remove-background: [epoch 102] average training loss: 1089.9163 cellbender:remove-background: [epoch 103] average training loss: 1096.2185 cellbender:remove-background: [epoch 104] average training loss: 1092.2522 cellbender:remove-background: [epoch 105] average training loss: 1091.7282 cellbender:remove-background: [epoch 105] average test loss: 1126.9693 cellbender:remove-background: [epoch 106] average training loss: 1090.4895 cellbender:remove-background: [epoch 107] average training loss: 1089.6058 cellbender:remove-background: [epoch 108] average training loss: 1092.1893 cellbender:remove-background: [epoch 109] average training loss: 1090.8570 cellbender:remove-background: [epoch 110] average training loss: 1094.3654 cellbender:remove-background: [epoch 110] average test loss: 1125.2405 cellbender:remove-background: [epoch 111] average training loss: 1091.3970 cellbender:remove-background: [epoch 112] average training loss: 1096.8999 cellbender:remove-background: [epoch 113] average training loss: 1091.1937 cellbender:remove-background: [epoch 114] average training loss: 1090.5371 cellbender:remove-background: [epoch 115] average training loss: 1093.3352 cellbender:remove-background: [epoch 115] average test loss: 1134.3236 cellbender:remove-background: [epoch 116] average training loss: 1095.9717 cellbender:remove-background: [epoch 117] average training loss: 1090.8005 cellbender:remove-background: [epoch 118] average training loss: 1092.5632 cellbender:remove-background: [epoch 119] average training loss: 1090.3622 cellbender:remove-background: [epoch 120] average training loss: 1091.6575 cellbender:remove-background: [epoch 120] average test loss: 1131.2991 cellbender:remove-background: [epoch 121] average training loss: 1092.0683 cellbender:remove-background: [epoch 122] average training loss: 1091.9754 cellbender:remove-background: [epoch 123] average training loss: 1092.7723 cellbender:remove-background: [epoch 124] average training loss: 1093.5890 cellbender:remove-background: [epoch 125] average training loss: 1092.3128 cellbender:remove-background: [epoch 125] average test loss: 1131.4250 cellbender:remove-background: [epoch 126] average training loss: 1090.6582 cellbender:remove-background: [epoch 127] average training loss: 1092.8134 cellbender:remove-background: [epoch 128] average training loss: 1091.7933 cellbender:remove-background: [epoch 129] average training loss: 1091.1942 cellbender:remove-background: [epoch 130] average training loss: 1090.5685 cellbender:remove-background: [epoch 130] average test loss: 1129.8231 cellbender:remove-background: [epoch 131] average training loss: 1090.7140 cellbender:remove-background: [epoch 132] average training loss: 1092.4446 cellbender:remove-background: [epoch 133] average training loss: 1092.2408 cellbender:remove-background: [epoch 134] average training loss: 1090.7183 cellbender:remove-background: [epoch 135] average training loss: 1093.0356 cellbender:remove-background: [epoch 135] average test loss: 1129.4974 cellbender:remove-background: [epoch 136] average training loss: 1090.1679 cellbender:remove-background: [epoch 137] average training loss: 1091.6772 cellbender:remove-background: [epoch 138] average training loss: 1092.9035 cellbender:remove-background: [epoch 139] average training loss: 1092.3720 cellbender:remove-background: [epoch 140] average training loss: 1091.9361 cellbender:remove-background: [epoch 140] average test loss: 1129.0792 cellbender:remove-background: [epoch 141] average training loss: 1091.3338 cellbender:remove-background: [epoch 142] average training loss: 1092.7540 cellbender:remove-background: [epoch 143] average training loss: 1092.5730 cellbender:remove-background: [epoch 144] average training loss: 1092.5372 cellbender:remove-background: [epoch 145] average training loss: 1091.6102 cellbender:remove-background: [epoch 145] average test loss: 1138.3812 cellbender:remove-background: [epoch 146] average training loss: 1092.3109 cellbender:remove-background: [epoch 147] average training loss: 1091.5636 cellbender:remove-background: [epoch 148] average training loss: 1093.1769 cellbender:remove-background: [epoch 149] average training loss: 1091.6029 cellbender:remove-background: [epoch 150] average training loss: 1091.8380 cellbender:remove-background: [epoch 150] average test loss: 1129.8927 cellbender:remove-background: Saving a checkpoint... cellbender:remove-background: Saved checkpoint as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2291-Schizophrenia-dACC/ckpt.tar.gz cellbender:remove-background: 2024-06-06 15:54:13 cellbender:remove-background: Inference procedure complete. cellbender:remove-background: Attempting to unpack tarball "ckpt.tar.gz" to /tmp/tmpj9cgzxd2 cellbender:remove-background: Successfully unpacked tarball to /tmp/tmpj9cgzxd2 /tmp/tmpj9cgzxd2/860e842553_random.pyro /tmp/tmpj9cgzxd2/860e842553_random.cuda /tmp/tmpj9cgzxd2/860e842553_model.torch /tmp/tmpj9cgzxd2/860e842553_optim.torch /tmp/tmpj9cgzxd2/860e842553_optim.pyro /tmp/tmpj9cgzxd2/860e842553_params.pyro /tmp/tmpj9cgzxd2/860e842553_train.loaderstate /tmp/tmpj9cgzxd2/860e842553_test.loaderstate /tmp/tmpj9cgzxd2/860e842553_args.npy cellbender:remove-background: Posterior not currently included in checkpoint. cellbender:remove-background: Computing posterior noise count probabilities in mini-batches. cellbender:remove-background: Working on chunk (1/60) cellbender:remove-background: [0.02 mins per chunk] cellbender:remove-background: Working on chunk (2/60) cellbender:remove-background: Working on chunk (3/60) cellbender:remove-background: Working on chunk (4/60) cellbender:remove-background: Working on chunk (5/60) cellbender:remove-background: Working on chunk (6/60) cellbender:remove-background: Working on chunk (7/60) cellbender:remove-background: Working on chunk (8/60) cellbender:remove-background: Working on chunk (9/60) cellbender:remove-background: Working on chunk (10/60) cellbender:remove-background: Working on chunk (11/60) cellbender:remove-background: Working on chunk (12/60) cellbender:remove-background: Working on chunk (13/60) cellbender:remove-background: Working on chunk (14/60) cellbender:remove-background: Working on chunk (15/60) cellbender:remove-background: Working on chunk (16/60) cellbender:remove-background: Working on chunk (17/60) cellbender:remove-background: Working on chunk (18/60) cellbender:remove-background: Working on chunk (19/60) cellbender:remove-background: Working on chunk (20/60) cellbender:remove-background: Working on chunk (21/60) cellbender:remove-background: Working on chunk (22/60) cellbender:remove-background: Working on chunk (23/60) cellbender:remove-background: Working on chunk (24/60) cellbender:remove-background: Working on chunk (25/60) cellbender:remove-background: Working on chunk (26/60) cellbender:remove-background: Working on chunk (27/60) cellbender:remove-background: Working on chunk (28/60) cellbender:remove-background: Working on chunk (29/60) cellbender:remove-background: Working on chunk (30/60) cellbender:remove-background: Working on chunk (31/60) cellbender:remove-background: Working on chunk (32/60) cellbender:remove-background: Working on chunk (33/60) cellbender:remove-background: Working on chunk (34/60) cellbender:remove-background: Working on chunk (35/60) cellbender:remove-background: Working on chunk (36/60) cellbender:remove-background: Working on chunk (37/60) cellbender:remove-background: Working on chunk (38/60) cellbender:remove-background: Working on chunk (39/60) cellbender:remove-background: Working on chunk (40/60) cellbender:remove-background: Working on chunk (41/60) cellbender:remove-background: Working on chunk (42/60) cellbender:remove-background: Working on chunk (43/60) cellbender:remove-background: Working on chunk (44/60) cellbender:remove-background: Working on chunk (45/60) cellbender:remove-background: Working on chunk (46/60) cellbender:remove-background: Working on chunk (47/60) cellbender:remove-background: Working on chunk (48/60) cellbender:remove-background: Working on chunk (49/60) cellbender:remove-background: Working on chunk (50/60) cellbender:remove-background: Working on chunk (51/60) cellbender:remove-background: Working on chunk (52/60) cellbender:remove-background: Working on chunk (53/60) cellbender:remove-background: Working on chunk (54/60) cellbender:remove-background: Working on chunk (55/60) cellbender:remove-background: Working on chunk (56/60) cellbender:remove-background: Working on chunk (57/60) cellbender:remove-background: Working on chunk (58/60) cellbender:remove-background: Working on chunk (59/60) cellbender:remove-background: Working on chunk (60/60) cellbender:remove-background: Writing full posterior to /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2291-Schizophrenia-dACC/2291-Schizophrenia-dACC.cellbender.output_posterior.h5 cellbender:remove-background: Succeeded in writing posterior to file /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2291-Schizophrenia-dACC/2291-Schizophrenia-dACC.cellbender.output_posterior.h5 cellbender:remove-background: Added posterior object to checkpoint file. cellbender:remove-background: 2024-06-06 15:57:52 cellbender:remove-background: Saved summary plots as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2291-Schizophrenia-dACC/2291-Schizophrenia-dACC.cellbender.output.pdf cellbender:remove-background: Saved cell barcodes in /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2291-Schizophrenia-dACC/2291-Schizophrenia-dACC.cellbender.output_cell_barcodes.csv cellbender:remove-background: Computing target noise counts per gene for MCKP estimator cellbender:remove-background: Using MCKP noise targets computed for FPR 0.01 cellbender:remove-background: Computing denoised counts using mckp estimator cellbender:remove-background: Working on chunk (1/7) cellbender:remove-background: [0.42 mins per chunk] cellbender:remove-background: Working on chunk (2/7) cellbender:remove-background: Working on chunk (3/7) cellbender:remove-background: Working on chunk (4/7) cellbender:remove-background: Working on chunk (5/7) cellbender:remove-background: Working on chunk (6/7) cellbender:remove-background: Working on chunk (7/7) cellbender:remove-background: 2024-06-06 16:00:59 Total MCKP estimation time = 177.93 sec cellbender:remove-background: Succeeded in writing CellRanger format output to file /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2291-Schizophrenia-dACC/2291-Schizophrenia-dACC.cellbender.output.h5 cellbender:remove-background: Succeeded in writing CellRanger format output to file /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2291-Schizophrenia-dACC/2291-Schizophrenia-dACC.cellbender.output_filtered.h5 cellbender:remove-background: Saved output metrics as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2291-Schizophrenia-dACC/2291-Schizophrenia-dACC.cellbender.output_metrics.csv Traceback (most recent call last): File "/home/users/guozy/miniconda3/envs/cellbender/bin/jupyter-nbconvert", line 5, in from nbconvert.nbconvertapp import main File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/__init__.py", line 5, in from .exporters import * File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/__init__.py", line 1, in from .asciidoc import ASCIIDocExporter File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/asciidoc.py", line 9, in from .templateexporter import TemplateExporter File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/templateexporter.py", line 25, in from lxml.html.clean import clean_html File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/lxml/html/clean.py", line 21, in ) from None ImportError: lxml.html.clean module is now a separate project lxml_html_clean. Install lxml[html_clean] or lxml_html_clean directly. Traceback (most recent call last): File "/home/users/guozy/miniconda3/envs/cellbender/bin/jupyter-nbconvert", line 5, in from nbconvert.nbconvertapp import main File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/__init__.py", line 5, in from .exporters import * File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/__init__.py", line 1, in from .asciidoc import ASCIIDocExporter File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/asciidoc.py", line 9, in from .templateexporter import TemplateExporter File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/templateexporter.py", line 25, in from lxml.html.clean import clean_html File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/lxml/html/clean.py", line 21, in ) from None ImportError: lxml.html.clean module is now a separate project lxml_html_clean. Install lxml[html_clean] or lxml_html_clean directly. mv: cannot stat ‘tmp.report.nbconvert.html’: No such file or directory rm: cannot remove ‘tmp.report.nbconvert.ipynb’: No such file or directory cellbender:remove-background: Unable to create report. cellbender:remove-background: Traceback (most recent call last): File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/run.py", line 351, in compute_output_denoised_counts_reports_metrics output=html_report_file, File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/report.py", line 82, in run_notebook_make_html title=('CellBender: ' + os.path.basename(output).replace('_report.html', '')), File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/report.py", line 59, in _postprocess_html with open(file, mode='r') as f: FileNotFoundError: [Errno 2] No such file or directory: '/oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2291-Schizophrenia-dACC/2291-Schizophrenia-dACC.cellbender.output_report.html' cellbender:remove-background: Completed remove-background. cellbender:remove-background: 2024-06-06 16:01:10