cellbender:remove-background: Command: cellbender remove-background --input /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/inputs/1227-Control-DLPFC.raw_feature_bc_matrix.h5 --output /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1227-Control-DLPFC/1227-Control-DLPFC.cellbender.output.h5 --cuda --expected-cells 9135 --total-droplets-included 20000 --fpr 0.01 --epochs 150 cellbender:remove-background: CellBender 0.3.0 cellbender:remove-background: (Workflow hash 99f7018acf) cellbender:remove-background: 2024-06-06 15:51:36 cellbender:remove-background: Running remove-background cellbender:remove-background: Loading data from /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/inputs/1227-Control-DLPFC.raw_feature_bc_matrix.h5 cellbender:remove-background: CellRanger v3 format cellbender:remove-background: Features in dataset: 36601 Gene Expression cellbender:remove-background: Trimming features for inference. cellbender:remove-background: 32534 features have nonzero counts. cellbender:remove-background: Prior on counts for cells is 5505 cellbender:remove-background: Prior on counts for empty droplets is 111 cellbender:remove-background: Excluding 8445 features that are estimated to have <= 0.1 background counts in cells. cellbender:remove-background: Including 24089 features in the analysis. cellbender:remove-background: Trimming barcodes for inference. cellbender:remove-background: Excluding barcodes with counts below 55 cellbender:remove-background: Using 9135 probable cell barcodes, plus an additional 10865 barcodes, and 14758 empty droplets. cellbender:remove-background: Largest surely-empty droplet has 108 UMI counts. cellbender:remove-background: Attempting to unpack tarball "ckpt.tar.gz" to /tmp/tmpdujnseko cellbender:remove-background: Successfully unpacked tarball to /tmp/tmpdujnseko /tmp/tmpdujnseko/99f7018acf_random.pyro /tmp/tmpdujnseko/99f7018acf_random.cuda /tmp/tmpdujnseko/99f7018acf_model.torch /tmp/tmpdujnseko/99f7018acf_optim.torch /tmp/tmpdujnseko/99f7018acf_optim.pyro /tmp/tmpdujnseko/99f7018acf_params.pyro /tmp/tmpdujnseko/99f7018acf_train.loaderstate /tmp/tmpdujnseko/99f7018acf_test.loaderstate /tmp/tmpdujnseko/99f7018acf_args.npy cellbender:remove-background: Loaded partially-trained checkpoint from ckpt.tar.gz cellbender:remove-background: Checkpoint loaded successfully. cellbender:remove-background: Running inference... cellbender:remove-background: [epoch 088] average training loss: 3850.6375 cellbender:remove-background: [epoch 089] average training loss: 3854.2526 (14.7 seconds per epoch) cellbender:remove-background: Will checkpoint every 29 epochs cellbender:remove-background: [epoch 090] average training loss: 3876.0392 cellbender:remove-background: [epoch 090] average test loss: 3770.6884 cellbender:remove-background: [epoch 091] average training loss: 3870.5777 cellbender:remove-background: [epoch 092] average training loss: 3852.4758 cellbender:remove-background: [epoch 093] average training loss: 3856.5658 cellbender:remove-background: [epoch 094] average training loss: 3856.3615 cellbender:remove-background: [epoch 095] average training loss: 3851.6548 cellbender:remove-background: [epoch 095] average test loss: 3846.7681 cellbender:remove-background: [epoch 096] average training loss: 3854.0694 cellbender:remove-background: [epoch 097] average training loss: 3858.7604 cellbender:remove-background: [epoch 098] average training loss: 3834.3524 cellbender:remove-background: [epoch 099] average training loss: 3847.2904 cellbender:remove-background: [epoch 100] average training loss: 3844.5883 cellbender:remove-background: [epoch 100] average test loss: 3794.6849 cellbender:remove-background: [epoch 101] average training loss: 3829.7957 cellbender:remove-background: [epoch 102] average training loss: 3822.0630 cellbender:remove-background: [epoch 103] average training loss: 3823.0883 cellbender:remove-background: [epoch 104] average training loss: 3823.0927 cellbender:remove-background: [epoch 105] average training loss: 3811.9177 cellbender:remove-background: [epoch 105] average test loss: 3760.5868 cellbender:remove-background: [epoch 106] average training loss: 3812.4994 cellbender:remove-background: [epoch 107] average training loss: 3818.3598 cellbender:remove-background: [epoch 108] average training loss: 3805.4622 cellbender:remove-background: [epoch 109] average training loss: 3814.3872 cellbender:remove-background: [epoch 110] average training loss: 3816.9557 cellbender:remove-background: [epoch 110] average test loss: 3794.2535 cellbender:remove-background: [epoch 111] average training loss: 3813.5670 cellbender:remove-background: [epoch 112] average training loss: 3821.7093 cellbender:remove-background: [epoch 113] average training loss: 3827.8004 cellbender:remove-background: [epoch 114] average training loss: 3813.0087 cellbender:remove-background: [epoch 115] average training loss: 3809.0098 cellbender:remove-background: [epoch 115] average test loss: 3773.2447 cellbender:remove-background: [epoch 116] average training loss: 3806.2201 cellbender:remove-background: Saving a checkpoint... cellbender:remove-background: Saved checkpoint as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1227-Control-DLPFC/ckpt.tar.gz cellbender:remove-background: [epoch 117] average training loss: 3827.1473 cellbender:remove-background: [epoch 118] average training loss: 3806.0984 cellbender:remove-background: [epoch 119] average training loss: 3819.7937 cellbender:remove-background: [epoch 120] average training loss: 3790.1143 cellbender:remove-background: [epoch 120] average test loss: 3803.4286 cellbender:remove-background: [epoch 121] average training loss: 3802.9498 cellbender:remove-background: [epoch 122] average training loss: 3806.5331 cellbender:remove-background: [epoch 123] average training loss: 3800.7767 cellbender:remove-background: [epoch 124] average training loss: 3799.7704 cellbender:remove-background: [epoch 125] average training loss: 3794.5277 cellbender:remove-background: [epoch 125] average test loss: 3786.2585 cellbender:remove-background: [epoch 126] average training loss: 3794.9711 cellbender:remove-background: [epoch 127] average training loss: 3797.8113 cellbender:remove-background: [epoch 128] average training loss: 3801.0920 cellbender:remove-background: [epoch 129] average training loss: 3794.4571 cellbender:remove-background: [epoch 130] average training loss: 3793.2647 cellbender:remove-background: [epoch 130] average test loss: 3781.0933 cellbender:remove-background: [epoch 131] average training loss: 3798.2296 cellbender:remove-background: [epoch 132] average training loss: 3788.6215 cellbender:remove-background: [epoch 133] average training loss: 3794.0431 cellbender:remove-background: [epoch 134] average training loss: 3786.5509 cellbender:remove-background: [epoch 135] average training loss: 3795.1483 cellbender:remove-background: [epoch 135] average test loss: 3784.0424 cellbender:remove-background: [epoch 136] average training loss: 3790.8898 cellbender:remove-background: [epoch 137] average training loss: 3786.6136 cellbender:remove-background: [epoch 138] average training loss: 3801.2339 cellbender:remove-background: [epoch 139] average training loss: 3795.7524 cellbender:remove-background: [epoch 140] average training loss: 3791.3147 cellbender:remove-background: [epoch 140] average test loss: 3777.5466 cellbender:remove-background: [epoch 141] average training loss: 3787.7745 cellbender:remove-background: [epoch 142] average training loss: 3795.3329 cellbender:remove-background: [epoch 143] average training loss: 3791.9609 cellbender:remove-background: [epoch 144] average training loss: 3786.2472 cellbender:remove-background: [epoch 145] average training loss: 3794.9059 cellbender:remove-background: [epoch 145] average test loss: 3777.8286 cellbender:remove-background: Saving a checkpoint... cellbender:remove-background: Saved checkpoint as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1227-Control-DLPFC/ckpt.tar.gz cellbender:remove-background: [epoch 146] average training loss: 3793.8801 cellbender:remove-background: [epoch 147] average training loss: 3787.2106 cellbender:remove-background: [epoch 148] average training loss: 3792.8205 cellbender:remove-background: [epoch 149] average training loss: 3788.2016 cellbender:remove-background: [epoch 150] average training loss: 3790.5268 cellbender:remove-background: [epoch 150] average test loss: 3773.5482 cellbender:remove-background: Saving a checkpoint... cellbender:remove-background: Saved checkpoint as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1227-Control-DLPFC/ckpt.tar.gz cellbender:remove-background: 2024-06-06 16:15:17 cellbender:remove-background: Inference procedure complete. cellbender:remove-background: Attempting to unpack tarball "ckpt.tar.gz" to /tmp/tmpjulbmspo cellbender:remove-background: Successfully unpacked tarball to /tmp/tmpjulbmspo /tmp/tmpjulbmspo/99f7018acf_random.pyro /tmp/tmpjulbmspo/99f7018acf_random.cuda /tmp/tmpjulbmspo/99f7018acf_model.torch /tmp/tmpjulbmspo/99f7018acf_optim.torch /tmp/tmpjulbmspo/99f7018acf_optim.pyro /tmp/tmpjulbmspo/99f7018acf_params.pyro /tmp/tmpjulbmspo/99f7018acf_train.loaderstate /tmp/tmpjulbmspo/99f7018acf_test.loaderstate /tmp/tmpjulbmspo/99f7018acf_args.npy cellbender:remove-background: Posterior not currently included in checkpoint. cellbender:remove-background: Computing posterior noise count probabilities in mini-batches. cellbender:remove-background: Working on chunk (1/114) cellbender:remove-background: [0.06 mins per chunk] cellbender:remove-background: Working on chunk (2/114) cellbender:remove-background: Working on chunk (3/114) cellbender:remove-background: Working on chunk (4/114) cellbender:remove-background: Working on chunk (5/114) cellbender:remove-background: Working on chunk (6/114) cellbender:remove-background: Working on chunk (7/114) cellbender:remove-background: Working on chunk (8/114) cellbender:remove-background: Working on chunk (9/114) cellbender:remove-background: Working on chunk (10/114) cellbender:remove-background: Working on chunk (11/114) cellbender:remove-background: Working on chunk (12/114) cellbender:remove-background: Working on chunk (13/114) cellbender:remove-background: Working on chunk (14/114) cellbender:remove-background: Working on chunk (15/114) cellbender:remove-background: Working on chunk (16/114) cellbender:remove-background: Working on chunk (17/114) cellbender:remove-background: Working on chunk (18/114) cellbender:remove-background: Working on chunk (19/114) cellbender:remove-background: Working on chunk (20/114) cellbender:remove-background: Working on chunk (21/114) cellbender:remove-background: Working on chunk (22/114) cellbender:remove-background: Working on chunk (23/114) cellbender:remove-background: Working on chunk (24/114) cellbender:remove-background: Working on chunk (25/114) cellbender:remove-background: Working on chunk (26/114) cellbender:remove-background: Working on chunk (27/114) cellbender:remove-background: Working on chunk (28/114) cellbender:remove-background: Working on chunk (29/114) cellbender:remove-background: Working on chunk (30/114) cellbender:remove-background: Working on chunk (31/114) cellbender:remove-background: Working on chunk (32/114) cellbender:remove-background: Working on chunk (33/114) cellbender:remove-background: Working on chunk (34/114) cellbender:remove-background: Working on chunk (35/114) cellbender:remove-background: Working on chunk (36/114) cellbender:remove-background: Working on chunk (37/114) cellbender:remove-background: Working on chunk (38/114) cellbender:remove-background: Working on chunk (39/114) cellbender:remove-background: Working on chunk (40/114) cellbender:remove-background: Working on chunk (41/114) cellbender:remove-background: Working on chunk (42/114) cellbender:remove-background: Working on chunk (43/114) cellbender:remove-background: Working on chunk (44/114) cellbender:remove-background: Working on chunk (45/114) cellbender:remove-background: Working on chunk (46/114) cellbender:remove-background: Working on chunk (47/114) cellbender:remove-background: Working on chunk (48/114) cellbender:remove-background: Working on chunk (49/114) cellbender:remove-background: Working on chunk (50/114) cellbender:remove-background: Working on chunk (51/114) cellbender:remove-background: Working on chunk (52/114) cellbender:remove-background: Working on chunk (53/114) cellbender:remove-background: Working on chunk (54/114) cellbender:remove-background: Working on chunk (55/114) cellbender:remove-background: Working on chunk (56/114) cellbender:remove-background: Working on chunk (57/114) cellbender:remove-background: Working on chunk (58/114) cellbender:remove-background: Working on chunk (59/114) cellbender:remove-background: Working on chunk (60/114) cellbender:remove-background: Working on chunk (61/114) cellbender:remove-background: Working on chunk (62/114) cellbender:remove-background: Working on chunk (63/114) cellbender:remove-background: Working on chunk (64/114) cellbender:remove-background: Working on chunk (65/114) cellbender:remove-background: Working on chunk (66/114) cellbender:remove-background: Working on chunk (67/114) cellbender:remove-background: Working on chunk (68/114) cellbender:remove-background: Working on chunk (69/114) cellbender:remove-background: Working on chunk (70/114) cellbender:remove-background: Working on chunk (71/114) cellbender:remove-background: Working on chunk (72/114) cellbender:remove-background: Working on chunk (73/114) cellbender:remove-background: Working on chunk (74/114) cellbender:remove-background: Working on chunk (75/114) cellbender:remove-background: Working on chunk (76/114) cellbender:remove-background: Working on chunk (77/114) cellbender:remove-background: Working on chunk (78/114) cellbender:remove-background: Working on chunk (79/114) cellbender:remove-background: Working on chunk (80/114) cellbender:remove-background: Working on chunk (81/114) cellbender:remove-background: Working on chunk (82/114) cellbender:remove-background: Working on chunk (83/114) cellbender:remove-background: Working on chunk (84/114) cellbender:remove-background: Working on chunk (85/114) cellbender:remove-background: Working on chunk (86/114) cellbender:remove-background: Working on chunk (87/114) cellbender:remove-background: Working on chunk (88/114) cellbender:remove-background: Working on chunk (89/114) cellbender:remove-background: Working on chunk (90/114) cellbender:remove-background: Working on chunk (91/114) cellbender:remove-background: Working on chunk (92/114) cellbender:remove-background: Working on chunk (93/114) cellbender:remove-background: Working on chunk (94/114) cellbender:remove-background: Working on chunk (95/114) cellbender:remove-background: Working on chunk (96/114) cellbender:remove-background: Working on chunk (97/114) cellbender:remove-background: Working on chunk (98/114) cellbender:remove-background: Working on chunk (99/114) cellbender:remove-background: Working on chunk (100/114) cellbender:remove-background: Working on chunk (101/114) cellbender:remove-background: Working on chunk (102/114) cellbender:remove-background: Working on chunk (103/114) cellbender:remove-background: Working on chunk (104/114) cellbender:remove-background: Working on chunk (105/114) cellbender:remove-background: Working on chunk (106/114) cellbender:remove-background: Working on chunk (107/114) cellbender:remove-background: Working on chunk (108/114) cellbender:remove-background: Working on chunk (109/114) cellbender:remove-background: Working on chunk (110/114) cellbender:remove-background: Working on chunk (111/114) cellbender:remove-background: Working on chunk (112/114) cellbender:remove-background: Working on chunk (113/114) cellbender:remove-background: Working on chunk (114/114) cellbender:remove-background: Writing full posterior to /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1227-Control-DLPFC/1227-Control-DLPFC.cellbender.output_posterior.h5 cellbender:remove-background: Succeeded in writing posterior to file /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1227-Control-DLPFC/1227-Control-DLPFC.cellbender.output_posterior.h5 cellbender:remove-background: Added posterior object to checkpoint file. cellbender:remove-background: 2024-06-06 16:26:50 cellbender:remove-background: Saved summary plots as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1227-Control-DLPFC/1227-Control-DLPFC.cellbender.output.pdf cellbender:remove-background: Saved cell barcodes in /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1227-Control-DLPFC/1227-Control-DLPFC.cellbender.output_cell_barcodes.csv cellbender:remove-background: Computing target noise counts per gene for MCKP estimator cellbender:remove-background: Using MCKP noise targets computed for FPR 0.01 cellbender:remove-background: Computing denoised counts using mckp estimator cellbender:remove-background: Working on chunk (1/7) cellbender:remove-background: [2.83 mins per chunk] cellbender:remove-background: Working on chunk (2/7) cellbender:remove-background: Working on chunk (3/7) cellbender:remove-background: Working on chunk (4/7) cellbender:remove-background: Working on chunk (5/7) cellbender:remove-background: Working on chunk (6/7) cellbender:remove-background: Working on chunk (7/7) cellbender:remove-background: 2024-06-06 16:47:19 Total MCKP estimation time = 1166.57 sec cellbender:remove-background: Succeeded in writing CellRanger format output to file /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1227-Control-DLPFC/1227-Control-DLPFC.cellbender.output.h5 cellbender:remove-background: Succeeded in writing CellRanger format output to file /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1227-Control-DLPFC/1227-Control-DLPFC.cellbender.output_filtered.h5 cellbender:remove-background: Saved output metrics as /oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1227-Control-DLPFC/1227-Control-DLPFC.cellbender.output_metrics.csv Traceback (most recent call last): File "/home/users/guozy/miniconda3/envs/cellbender/bin/jupyter-nbconvert", line 5, in from nbconvert.nbconvertapp import main File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/__init__.py", line 5, in from .exporters import * File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/__init__.py", line 1, in from .asciidoc import ASCIIDocExporter File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/asciidoc.py", line 9, in from .templateexporter import TemplateExporter File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/templateexporter.py", line 25, in from lxml.html.clean import clean_html File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/lxml/html/clean.py", line 21, in ) from None ImportError: lxml.html.clean module is now a separate project lxml_html_clean. Install lxml[html_clean] or lxml_html_clean directly. Traceback (most recent call last): File "/home/users/guozy/miniconda3/envs/cellbender/bin/jupyter-nbconvert", line 5, in from nbconvert.nbconvertapp import main File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/__init__.py", line 5, in from .exporters import * File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/__init__.py", line 1, in from .asciidoc import ASCIIDocExporter File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/asciidoc.py", line 9, in from .templateexporter import TemplateExporter File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/nbconvert/exporters/templateexporter.py", line 25, in from lxml.html.clean import clean_html File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/lxml/html/clean.py", line 21, in ) from None ImportError: lxml.html.clean module is now a separate project lxml_html_clean. Install lxml[html_clean] or lxml_html_clean directly. mv: cannot stat ‘tmp.report.nbconvert.html’: No such file or directory rm: cannot remove ‘tmp.report.nbconvert.ipynb’: No such file or directory cellbender:remove-background: Unable to create report. cellbender:remove-background: Traceback (most recent call last): File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/run.py", line 351, in compute_output_denoised_counts_reports_metrics output=html_report_file, File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/report.py", line 82, in run_notebook_make_html title=('CellBender: ' + os.path.basename(output).replace('_report.html', '')), File "/home/users/guozy/miniconda3/envs/cellbender/lib/python3.7/site-packages/cellbender/remove_background/report.py", line 59, in _postprocess_html with open(file, mode='r') as f: FileNotFoundError: [Errno 2] No such file or directory: '/oak/stanford/groups/akundaje/projects/psychencode/outs/20231009_multiome_analysis/01-cellbender/outputs/1227-Control-DLPFC/1227-Control-DLPFC.cellbender.output_report.html' cellbender:remove-background: Completed remove-background. cellbender:remove-background: 2024-06-06 16:47:57