cellbender:remove-background: Command:
cellbender remove-background --input /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/inputs/2653-Schizophrenia-dACC.raw_feature_bc_matrix.h5 --output /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2653-Schizophrenia-dACC/2653-Schizophrenia-dACC.cellbender.output.h5 --cuda --expected-cells 8406 --total-droplets-included 20000 --fpr 0.01 --epochs 150
cellbender:remove-background: CellBender 0.3.0
cellbender:remove-background: (Workflow hash beb16b2e6e)
cellbender:remove-background: 2024-06-06 15:42:05
cellbender:remove-background: Running remove-background
cellbender:remove-background: Loading data from /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/inputs/2653-Schizophrenia-dACC.raw_feature_bc_matrix.h5
cellbender:remove-background: CellRanger v3 format
cellbender:remove-background: Features in dataset: 36601 Gene Expression
cellbender:remove-background: Trimming features for inference.
cellbender:remove-background: 30930 features have nonzero counts.
cellbender:remove-background: Prior on counts for cells is 2681
cellbender:remove-background: Prior on counts for empty droplets is 51
cellbender:remove-background: Excluding 8025 features that are estimated to have <= 0.1 background counts in cells.
cellbender:remove-background: Including 22905 features in the analysis.
cellbender:remove-background: Trimming barcodes for inference.
cellbender:remove-background: Excluding barcodes with counts below 25
cellbender:remove-background: Using 8406 probable cell barcodes, plus an additional 11594 barcodes, and 43332 empty droplets.
cellbender:remove-background: Largest surely-empty droplet has 51 UMI counts.
cellbender:remove-background: Attempting to unpack tarball "ckpt.tar.gz" to /tmp/tmp44bpt4fp
cellbender:remove-background: Successfully unpacked tarball to /tmp/tmp44bpt4fp
/tmp/tmp44bpt4fp/beb16b2e6e_random.pyro
/tmp/tmp44bpt4fp/beb16b2e6e_random.cuda
/tmp/tmp44bpt4fp/beb16b2e6e_model.torch
/tmp/tmp44bpt4fp/beb16b2e6e_optim.torch
/tmp/tmp44bpt4fp/beb16b2e6e_optim.pyro
/tmp/tmp44bpt4fp/beb16b2e6e_params.pyro
/tmp/tmp44bpt4fp/beb16b2e6e_train.loaderstate
/tmp/tmp44bpt4fp/beb16b2e6e_test.loaderstate
/tmp/tmp44bpt4fp/beb16b2e6e_args.npy
cellbender:remove-background: Loaded partially-trained checkpoint from ckpt.tar.gz
cellbender:remove-background: Checkpoint loaded successfully.
cellbender:remove-background: Running inference...
cellbender:remove-background: [epoch 058]  average training loss: 2300.2464
cellbender:remove-background: [epoch 059]  average training loss: 2301.0280  (7.4 seconds per epoch)
cellbender:remove-background: Will checkpoint every 57 epochs
cellbender:remove-background: [epoch 060]  average training loss: 2299.9976
cellbender:remove-background: [epoch 060] average test loss: 2291.1262
cellbender:remove-background: [epoch 061]  average training loss: 2296.3665
cellbender:remove-background: [epoch 062]  average training loss: 2299.1109
cellbender:remove-background: [epoch 063]  average training loss: 2300.0344
cellbender:remove-background: [epoch 064]  average training loss: 2299.9052
cellbender:remove-background: [epoch 065]  average training loss: 2298.5403
cellbender:remove-background: [epoch 065] average test loss: 2282.8993
cellbender:remove-background: [epoch 066]  average training loss: 2299.2709
cellbender:remove-background: [epoch 067]  average training loss: 2299.9512
cellbender:remove-background: [epoch 068]  average training loss: 2297.4127
cellbender:remove-background: [epoch 069]  average training loss: 2300.7630
cellbender:remove-background: [epoch 070]  average training loss: 2299.4573
cellbender:remove-background: [epoch 070] average test loss: 2283.5170
cellbender:remove-background: [epoch 071]  average training loss: 2300.0404
cellbender:remove-background: [epoch 072]  average training loss: 2301.4531
cellbender:remove-background: [epoch 073]  average training loss: 2302.5253
cellbender:remove-background: [epoch 074]  average training loss: 2299.9398
cellbender:remove-background: [epoch 075]  average training loss: 2300.3998
cellbender:remove-background: [epoch 075] average test loss: 2284.8585
cellbender:remove-background: [epoch 076]  average training loss: 2300.2210
cellbender:remove-background: [epoch 077]  average training loss: 2299.4933
cellbender:remove-background: [epoch 078]  average training loss: 2303.8042
cellbender:remove-background: [epoch 079]  average training loss: 2300.1209
cellbender:remove-background: [epoch 080]  average training loss: 2303.1483
cellbender:remove-background: [epoch 080] average test loss: 2298.5763
cellbender:remove-background: [epoch 081]  average training loss: 2299.9780
cellbender:remove-background: [epoch 082]  average training loss: 2303.0990
cellbender:remove-background: [epoch 083]  average training loss: 2303.5265
cellbender:remove-background: [epoch 084]  average training loss: 2300.9840
cellbender:remove-background: [epoch 085]  average training loss: 2304.1080
cellbender:remove-background: [epoch 085] average test loss: 2305.4313
cellbender:remove-background: [epoch 086]  average training loss: 2303.7979
cellbender:remove-background: [epoch 087]  average training loss: 2300.8340
cellbender:remove-background: [epoch 088]  average training loss: 2301.8554
cellbender:remove-background: [epoch 089]  average training loss: 2302.4206
cellbender:remove-background: [epoch 090]  average training loss: 2301.9085
cellbender:remove-background: [epoch 090] average test loss: 2307.3291
cellbender:remove-background: [epoch 091]  average training loss: 2303.0526
cellbender:remove-background: [epoch 092]  average training loss: 2300.9999
cellbender:remove-background: [epoch 093]  average training loss: 2303.1174
cellbender:remove-background: [epoch 094]  average training loss: 2303.6225
cellbender:remove-background: [epoch 095]  average training loss: 2302.6344
cellbender:remove-background: [epoch 095] average test loss: 2301.5657
cellbender:remove-background: [epoch 096]  average training loss: 2304.4290
cellbender:remove-background: [epoch 097]  average training loss: 2304.4885
cellbender:remove-background: [epoch 098]  average training loss: 2301.9608
cellbender:remove-background: [epoch 099]  average training loss: 2304.8898
cellbender:remove-background: [epoch 100]  average training loss: 2302.0036
cellbender:remove-background: [epoch 100] average test loss: 2307.4885
cellbender:remove-background: [epoch 101]  average training loss: 2305.3896
cellbender:remove-background: [epoch 102]  average training loss: 2304.1888
cellbender:remove-background: [epoch 103]  average training loss: 2302.6685
cellbender:remove-background: [epoch 104]  average training loss: 2303.4369
cellbender:remove-background: [epoch 105]  average training loss: 2305.3558
cellbender:remove-background: [epoch 105] average test loss: 2292.8733
cellbender:remove-background: [epoch 106]  average training loss: 2304.2994
cellbender:remove-background: [epoch 107]  average training loss: 2301.2396
cellbender:remove-background: [epoch 108]  average training loss: 2305.0594
cellbender:remove-background: [epoch 109]  average training loss: 2304.9511
cellbender:remove-background: [epoch 110]  average training loss: 2301.2232
cellbender:remove-background: [epoch 110] average test loss: 2313.0818
cellbender:remove-background: [epoch 111]  average training loss: 2303.1900
cellbender:remove-background: [epoch 112]  average training loss: 2304.9131
cellbender:remove-background: [epoch 113]  average training loss: 2301.6111
cellbender:remove-background: [epoch 114]  average training loss: 2306.6374
cellbender:remove-background: Saving a checkpoint...
cellbender:remove-background: Saved checkpoint as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2653-Schizophrenia-dACC/ckpt.tar.gz
cellbender:remove-background: [epoch 115]  average training loss: 2302.4731
cellbender:remove-background: [epoch 115] average test loss: 2305.6876
cellbender:remove-background: [epoch 116]  average training loss: 2301.6911
cellbender:remove-background: [epoch 117]  average training loss: 2302.9484
cellbender:remove-background: [epoch 118]  average training loss: 2302.1583
cellbender:remove-background: [epoch 119]  average training loss: 2303.1070
cellbender:remove-background: [epoch 120]  average training loss: 2303.8474
cellbender:remove-background: [epoch 120] average test loss: 2303.9519
cellbender:remove-background: [epoch 121]  average training loss: 2304.7056
cellbender:remove-background: [epoch 122]  average training loss: 2302.1269
cellbender:remove-background: [epoch 123]  average training loss: 2303.9205
cellbender:remove-background: [epoch 124]  average training loss: 2302.2780
cellbender:remove-background: [epoch 125]  average training loss: 2302.1455
cellbender:remove-background: [epoch 125] average test loss: 2308.5964
cellbender:remove-background: [epoch 126]  average training loss: 2304.3691
cellbender:remove-background: [epoch 127]  average training loss: 2300.4985
cellbender:remove-background: [epoch 128]  average training loss: 2303.9238
cellbender:remove-background: [epoch 129]  average training loss: 2303.5399
cellbender:remove-background: [epoch 130]  average training loss: 2301.4046
cellbender:remove-background: [epoch 130] average test loss: 2314.3129
cellbender:remove-background: [epoch 131]  average training loss: 2305.7132
cellbender:remove-background: [epoch 132]  average training loss: 2303.2011
cellbender:remove-background: [epoch 133]  average training loss: 2301.3046
cellbender:remove-background: [epoch 134]  average training loss: 2301.7055
cellbender:remove-background: [epoch 135]  average training loss: 2300.9859
cellbender:remove-background: [epoch 135] average test loss: 2308.3212
cellbender:remove-background: [epoch 136]  average training loss: 2302.6076
cellbender:remove-background: [epoch 137]  average training loss: 2302.3075
cellbender:remove-background: [epoch 138]  average training loss: 2301.7149
cellbender:remove-background: [epoch 139]  average training loss: 2303.2961
cellbender:remove-background: [epoch 140]  average training loss: 2300.4506
cellbender:remove-background: [epoch 140] average test loss: 2307.4410
cellbender:remove-background: [epoch 141]  average training loss: 2301.6114
cellbender:remove-background: [epoch 142]  average training loss: 2301.6623
cellbender:remove-background: [epoch 143]  average training loss: 2299.7240
cellbender:remove-background: [epoch 144]  average training loss: 2300.3984
cellbender:remove-background: [epoch 145]  average training loss: 2300.9802
cellbender:remove-background: [epoch 145] average test loss: 2315.1789
cellbender:remove-background: [epoch 146]  average training loss: 2300.6190
cellbender:remove-background: [epoch 147]  average training loss: 2301.1484
cellbender:remove-background: [epoch 148]  average training loss: 2300.3582
cellbender:remove-background: [epoch 149]  average training loss: 2302.1176
cellbender:remove-background: [epoch 150]  average training loss: 2302.0406
cellbender:remove-background: [epoch 150] average test loss: 2309.6488
cellbender:remove-background: Saving a checkpoint...
cellbender:remove-background: Saved checkpoint as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2653-Schizophrenia-dACC/ckpt.tar.gz
cellbender:remove-background: 2024-06-06 15:58:07
cellbender:remove-background: Inference procedure complete.
cellbender:remove-background: Attempting to unpack tarball "ckpt.tar.gz" to /tmp/tmpfy0w7xzp
cellbender:remove-background: Successfully unpacked tarball to /tmp/tmpfy0w7xzp
/tmp/tmpfy0w7xzp/beb16b2e6e_random.pyro
/tmp/tmpfy0w7xzp/beb16b2e6e_random.cuda
/tmp/tmpfy0w7xzp/beb16b2e6e_model.torch
/tmp/tmpfy0w7xzp/beb16b2e6e_optim.torch
/tmp/tmpfy0w7xzp/beb16b2e6e_optim.pyro
/tmp/tmpfy0w7xzp/beb16b2e6e_params.pyro
/tmp/tmpfy0w7xzp/beb16b2e6e_train.loaderstate
/tmp/tmpfy0w7xzp/beb16b2e6e_test.loaderstate
/tmp/tmpfy0w7xzp/beb16b2e6e_args.npy
cellbender:remove-background: Posterior not currently included in checkpoint.
cellbender:remove-background: Computing posterior noise count probabilities in mini-batches.
cellbender:remove-background: Working on chunk (1/82)
cellbender:remove-background:     [0.04 mins per chunk]
cellbender:remove-background: Working on chunk (2/82)
cellbender:remove-background: Working on chunk (3/82)
cellbender:remove-background: Working on chunk (4/82)
cellbender:remove-background: Working on chunk (5/82)
cellbender:remove-background: Working on chunk (6/82)
cellbender:remove-background: Working on chunk (7/82)
cellbender:remove-background: Working on chunk (8/82)
cellbender:remove-background: Working on chunk (9/82)
cellbender:remove-background: Working on chunk (10/82)
cellbender:remove-background: Working on chunk (11/82)
cellbender:remove-background: Working on chunk (12/82)
cellbender:remove-background: Working on chunk (13/82)
cellbender:remove-background: Working on chunk (14/82)
cellbender:remove-background: Working on chunk (15/82)
cellbender:remove-background: Working on chunk (16/82)
cellbender:remove-background: Working on chunk (17/82)
cellbender:remove-background: Working on chunk (18/82)
cellbender:remove-background: Working on chunk (19/82)
cellbender:remove-background: Working on chunk (20/82)
cellbender:remove-background: Working on chunk (21/82)
cellbender:remove-background: Working on chunk (22/82)
cellbender:remove-background: Working on chunk (23/82)
cellbender:remove-background: Working on chunk (24/82)
cellbender:remove-background: Working on chunk (25/82)
cellbender:remove-background: Working on chunk (26/82)
cellbender:remove-background: Working on chunk (27/82)
cellbender:remove-background: Working on chunk (28/82)
cellbender:remove-background: Working on chunk (29/82)
cellbender:remove-background: Working on chunk (30/82)
cellbender:remove-background: Working on chunk (31/82)
cellbender:remove-background: Working on chunk (32/82)
cellbender:remove-background: Working on chunk (33/82)
cellbender:remove-background: Working on chunk (34/82)
cellbender:remove-background: Working on chunk (35/82)
cellbender:remove-background: Working on chunk (36/82)
cellbender:remove-background: Working on chunk (37/82)
cellbender:remove-background: Working on chunk (38/82)
cellbender:remove-background: Working on chunk (39/82)
cellbender:remove-background: Working on chunk (40/82)
cellbender:remove-background: Working on chunk (41/82)
cellbender:remove-background: Working on chunk (42/82)
cellbender:remove-background: Working on chunk (43/82)
cellbender:remove-background: Working on chunk (44/82)
cellbender:remove-background: Working on chunk (45/82)
cellbender:remove-background: Working on chunk (46/82)
cellbender:remove-background: Working on chunk (47/82)
cellbender:remove-background: Working on chunk (48/82)
cellbender:remove-background: Working on chunk (49/82)
cellbender:remove-background: Working on chunk (50/82)
cellbender:remove-background: Working on chunk (51/82)
cellbender:remove-background: Working on chunk (52/82)
cellbender:remove-background: Working on chunk (53/82)
cellbender:remove-background: Working on chunk (54/82)
cellbender:remove-background: Working on chunk (55/82)
cellbender:remove-background: Working on chunk (56/82)
cellbender:remove-background: Working on chunk (57/82)
cellbender:remove-background: Working on chunk (58/82)
cellbender:remove-background: Working on chunk (59/82)
cellbender:remove-background: Working on chunk (60/82)
cellbender:remove-background: Working on chunk (61/82)
cellbender:remove-background: Working on chunk (62/82)
cellbender:remove-background: Working on chunk (63/82)
cellbender:remove-background: Working on chunk (64/82)
cellbender:remove-background: Working on chunk (65/82)
cellbender:remove-background: Working on chunk (66/82)
cellbender:remove-background: Working on chunk (67/82)
cellbender:remove-background: Working on chunk (68/82)
cellbender:remove-background: Working on chunk (69/82)
cellbender:remove-background: Working on chunk (70/82)
cellbender:remove-background: Working on chunk (71/82)
cellbender:remove-background: Working on chunk (72/82)
cellbender:remove-background: Working on chunk (73/82)
cellbender:remove-background: Working on chunk (74/82)
cellbender:remove-background: Working on chunk (75/82)
cellbender:remove-background: Working on chunk (76/82)
cellbender:remove-background: Working on chunk (77/82)
cellbender:remove-background: Working on chunk (78/82)
cellbender:remove-background: Working on chunk (79/82)
cellbender:remove-background: Working on chunk (80/82)
cellbender:remove-background: Working on chunk (81/82)
cellbender:remove-background: Working on chunk (82/82)
cellbender:remove-background: Writing full posterior to /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2653-Schizophrenia-dACC/2653-Schizophrenia-dACC.cellbender.output_posterior.h5
cellbender:remove-background: Succeeded in writing posterior to file /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2653-Schizophrenia-dACC/2653-Schizophrenia-dACC.cellbender.output_posterior.h5
cellbender:remove-background: Added posterior object to checkpoint file.
cellbender:remove-background: 2024-06-06 16:04:45

cellbender:remove-background: Saved summary plots as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2653-Schizophrenia-dACC/2653-Schizophrenia-dACC.cellbender.output.pdf
cellbender:remove-background: Saved cell barcodes in /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2653-Schizophrenia-dACC/2653-Schizophrenia-dACC.cellbender.output_cell_barcodes.csv
cellbender:remove-background: Computing target noise counts per gene for MCKP estimator
cellbender:remove-background: Using MCKP noise targets computed for FPR 0.01
cellbender:remove-background: Computing denoised counts using mckp estimator
cellbender:remove-background: Working on chunk (1/7)
cellbender:remove-background:     [1.58 mins per chunk]
cellbender:remove-background: Working on chunk (2/7)
cellbender:remove-background: Working on chunk (3/7)
cellbender:remove-background: Working on chunk (4/7)
cellbender:remove-background: Working on chunk (5/7)
cellbender:remove-background: Working on chunk (6/7)
cellbender:remove-background: Working on chunk (7/7)
cellbender:remove-background: 2024-06-06 16:16:12 Total MCKP estimation time = 662.42 sec
cellbender:remove-background: Succeeded in writing CellRanger format output to file /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2653-Schizophrenia-dACC/2653-Schizophrenia-dACC.cellbender.output.h5
cellbender:remove-background: Succeeded in writing CellRanger format output to file /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2653-Schizophrenia-dACC/2653-Schizophrenia-dACC.cellbender.output_filtered.h5
cellbender:remove-background: Saved output metrics as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2653-Schizophrenia-dACC/2653-Schizophrenia-dACC.cellbender.output_metrics.csv
Traceback (most recent call last):
  File "/home/users/guozy/miniconda3/envs/cellbender/bin/jupyter-nbconvert", line 5, in <module>
    from nbconvert.nbconvertapp import main
  File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/__init__.py", line 5, in <module>
    from .exporters import *
  File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/__init__.py", line 1, in <module>
    from .asciidoc import ASCIIDocExporter
  File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/asciidoc.py", line 9, in <module>
    from .templateexporter import TemplateExporter
  File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/templateexporter.py", line 25, in <module>
    from lxml.html.clean import clean_html
  File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/lxml/html/clean.py", line 21, in <module>
    ) from None
ImportError: lxml.html.clean module is now a separate project lxml_html_clean.
Install lxml[html_clean] or lxml_html_clean directly.
Traceback (most recent call last):
  File "/home/users/guozy/miniconda3/envs/cellbender/bin/jupyter-nbconvert", line 5, in <module>
    from nbconvert.nbconvertapp import main
  File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/__init__.py", line 5, in <module>
    from .exporters import *
  File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/__init__.py", line 1, in <module>
    from .asciidoc import ASCIIDocExporter
  File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/asciidoc.py", line 9, in <module>
    from .templateexporter import TemplateExporter
  File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/templateexporter.py", line 25, in <module>
    from lxml.html.clean import clean_html
  File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/lxml/html/clean.py", line 21, in <module>
    ) from None
ImportError: lxml.html.clean module is now a separate project lxml_html_clean.
Install lxml[html_clean] or lxml_html_clean directly.
mv: cannot stat ‘tmp.report.nbconvert.html’: No such file or directory
rm: cannot remove ‘tmp.report.nbconvert.ipynb’: No such file or directory
cellbender:remove-background: Unable to create report.
cellbender:remove-background: Traceback (most recent call last):
  File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/run.py", line 351, in compute_output_denoised_counts_reports_metrics
    output=html_report_file,
  File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/report.py", line 82, in run_notebook_make_html
    title=('CellBender: ' + os.path.basename(output).replace('_report.html', '')),
  File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/report.py", line 59, in _postprocess_html
    with open(file, mode='r') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/2653-Schizophrenia-dACC/2653-Schizophrenia-dACC.cellbender.output_report.html'

cellbender:remove-background: Completed remove-background.
cellbender:remove-background: 2024-06-06 16:16:29