Lmod Warning:
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: curl/8.4.0
(required by: htslib/1.16)
-------------------------------------------------------------------------------




The following have been reloaded with a version change:
  1) curl/8.4.0 => curl/8.17.0     2) openssl/3.0.7 => openssl/3.6.0

2026-05-11 02:50:32.233951: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 02:54:49.736033: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 02:54:49.738624: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-11 02:54:49.893791: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:03:00.0 name: Tesla P100-PCIE-16GB computeCapability: 6.0
coreClock: 1.3285GHz coreCount: 56 deviceMemorySize: 15.89GiB deviceMemoryBandwidth: 681.88GiB/s
2026-05-11 02:54:49.893951: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 02:54:50.062782: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 02:54:50.063001: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 02:54:50.072610: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 02:54:50.455286: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 02:54:51.279148: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 02:54:51.468315: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 02:54:51.847559: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 02:54:51.848991: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 02:54:51.849721: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-11 02:54:51.850045: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 02:54:51.850906: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:03:00.0 name: Tesla P100-PCIE-16GB computeCapability: 6.0
coreClock: 1.3285GHz coreCount: 56 deviceMemorySize: 15.89GiB deviceMemoryBandwidth: 681.88GiB/s
2026-05-11 02:54:51.851028: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 02:54:51.851100: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 02:54:51.851157: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 02:54:51.851214: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 02:54:51.851267: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 02:54:51.851326: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 02:54:51.851376: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 02:54:51.851463: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 02:54:51.852767: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 02:54:51.852868: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 02:54:54.646112: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-11 02:54:54.646203: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-11 02:54:54.646234: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-11 02:54:54.647260: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14950 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:03:00.0, compute capability: 6.0)
2026-05-11 02:55:01.247400: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-05-11 02:55:01.248154: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2400130000 Hz
2026-05-11 02:55:13.497843: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 02:55:16.162394: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 02:55:16.215156: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 02:55:29.816772: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-11 02:55:29.960809: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'jitters', 'index', 'status', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
2026-05-11 03:22:35.869872: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2026-05-11 03:22:53.021010: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:24:07.139671: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 03:24:07.147420: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-11 03:24:07.172916: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:03:00.0 name: Tesla P100-PCIE-16GB computeCapability: 6.0
coreClock: 1.3285GHz coreCount: 56 deviceMemorySize: 15.89GiB deviceMemoryBandwidth: 681.88GiB/s
2026-05-11 03:24:07.173042: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:24:07.184243: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:24:07.184357: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:24:07.190737: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 03:24:07.194981: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 03:24:07.203903: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 03:24:07.209752: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 03:24:07.214532: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:24:07.215168: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 03:24:07.215543: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-11 03:24:07.215680: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 03:24:07.216151: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:03:00.0 name: Tesla P100-PCIE-16GB computeCapability: 6.0
coreClock: 1.3285GHz coreCount: 56 deviceMemorySize: 15.89GiB deviceMemoryBandwidth: 681.88GiB/s
2026-05-11 03:24:07.216203: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:24:07.216242: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:24:07.216273: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:24:07.216305: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 03:24:07.216336: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 03:24:07.216366: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 03:24:07.216396: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 03:24:07.216426: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:24:07.216861: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 03:24:07.216912: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:24:07.803262: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-11 03:24:07.803364: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-11 03:24:07.803388: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-11 03:24:07.804418: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14950 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:03:00.0, compute capability: 6.0)
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)

batch:   0%|          | 0/104 [00:00<?, ?it/s]2026-05-11 03:24:10.202310: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-05-11 03:24:10.203038: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2400130000 Hz
2026-05-11 03:24:10.712435: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:24:11.009823: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:24:11.011632: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:24:13.107727: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-11 03:24:13.282247: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'true_profiles', 'true_logcounts', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])

batch:   1%|          | 1/104 [00:22<38:59, 22.71s/it]
batch:   2%|▏         | 2/104 [00:23<16:23,  9.64s/it]
batch:   3%|▎         | 3/104 [00:23<09:12,  5.47s/it]
batch:   4%|▍         | 4/104 [00:24<05:51,  3.52s/it]
batch:   5%|▍         | 5/104 [00:24<04:00,  2.43s/it]
batch:   6%|▌         | 6/104 [00:25<02:50,  1.74s/it]
batch:   7%|▋         | 7/104 [00:25<02:07,  1.31s/it]
batch:   8%|▊         | 8/104 [00:26<01:42,  1.06s/it]
batch:   9%|▊         | 9/104 [00:26<01:25,  1.11it/s]
batch:  10%|▉         | 10/104 [00:27<01:12,  1.30it/s]
batch:  11%|█         | 11/104 [00:27<01:04,  1.44it/s]
batch:  12%|█▏        | 12/104 [00:28<00:57,  1.59it/s]
batch:  12%|█▎        | 13/104 [00:28<00:51,  1.77it/s]
batch:  13%|█▎        | 14/104 [00:28<00:46,  1.92it/s]
batch:  14%|█▍        | 15/104 [00:29<00:43,  2.03it/s]
batch:  15%|█▌        | 16/104 [00:29<00:44,  2.00it/s]
batch:  16%|█▋        | 17/104 [00:30<00:43,  1.99it/s]
batch:  17%|█▋        | 18/104 [00:30<00:43,  1.97it/s]
batch:  18%|█▊        | 19/104 [00:31<00:41,  2.06it/s]
batch:  19%|█▉        | 20/104 [00:31<00:39,  2.13it/s]
batch:  20%|██        | 21/104 [00:32<00:37,  2.20it/s]
batch:  21%|██        | 22/104 [00:32<00:36,  2.24it/s]
batch:  22%|██▏       | 23/104 [00:33<00:37,  2.15it/s]
batch:  23%|██▎       | 24/104 [00:33<00:37,  2.13it/s]
batch:  24%|██▍       | 25/104 [00:34<00:35,  2.20it/s]
batch:  25%|██▌       | 26/104 [00:34<00:34,  2.25it/s]
batch:  26%|██▌       | 27/104 [00:34<00:33,  2.27it/s]
batch:  27%|██▋       | 28/104 [00:35<00:35,  2.16it/s]
batch:  28%|██▊       | 29/104 [00:35<00:36,  2.08it/s]
batch:  29%|██▉       | 30/104 [00:36<00:34,  2.12it/s]
batch:  30%|██▉       | 31/104 [00:36<00:35,  2.06it/s]
batch:  31%|███       | 32/104 [00:37<00:35,  2.03it/s]
batch:  32%|███▏      | 33/104 [00:37<00:35,  2.02it/s]
batch:  33%|███▎      | 34/104 [00:38<00:35,  1.99it/s]
batch:  34%|███▎      | 35/104 [00:38<00:35,  1.96it/s]
batch:  35%|███▍      | 36/104 [00:39<00:34,  1.97it/s]
batch:  36%|███▌      | 37/104 [00:39<00:34,  1.97it/s]
batch:  37%|███▋      | 38/104 [00:40<00:33,  1.98it/s]
batch:  38%|███▊      | 39/104 [00:41<00:33,  1.95it/s]
batch:  38%|███▊      | 40/104 [00:41<00:32,  1.97it/s]
batch:  39%|███▉      | 41/104 [00:42<00:31,  1.97it/s]
batch:  40%|████      | 42/104 [00:42<00:31,  1.97it/s]
batch:  41%|████▏     | 43/104 [00:43<00:30,  1.98it/s]
batch:  42%|████▏     | 44/104 [00:43<00:30,  1.98it/s]
batch:  43%|████▎     | 45/104 [00:44<00:29,  1.98it/s]
batch:  44%|████▍     | 46/104 [00:44<00:29,  1.98it/s]
batch:  45%|████▌     | 47/104 [00:45<00:29,  1.93it/s]
batch:  46%|████▌     | 48/104 [00:45<00:28,  1.95it/s]
batch:  47%|████▋     | 49/104 [00:46<00:28,  1.92it/s]
batch:  48%|████▊     | 50/104 [00:46<00:27,  1.96it/s]
batch:  49%|████▉     | 51/104 [00:47<00:27,  1.94it/s]
batch:  50%|█████     | 52/104 [00:47<00:26,  1.98it/s]
batch:  51%|█████     | 53/104 [00:48<00:26,  1.96it/s]
batch:  52%|█████▏    | 54/104 [00:48<00:25,  1.99it/s]
batch:  53%|█████▎    | 55/104 [00:49<00:24,  1.98it/s]
batch:  54%|█████▍    | 56/104 [00:49<00:24,  1.99it/s]
batch:  55%|█████▍    | 57/104 [00:50<00:23,  1.98it/s]
batch:  56%|█████▌    | 58/104 [00:50<00:22,  2.03it/s]
batch:  57%|█████▋    | 59/104 [00:51<00:21,  2.12it/s]
batch:  58%|█████▊    | 60/104 [00:51<00:19,  2.20it/s]
batch:  59%|█████▊    | 61/104 [00:51<00:19,  2.25it/s]
batch:  60%|█████▉    | 62/104 [00:52<00:19,  2.15it/s]
batch:  61%|██████    | 63/104 [00:52<00:19,  2.15it/s]
batch:  62%|██████▏   | 64/104 [00:53<00:18,  2.22it/s]
batch:  62%|██████▎   | 65/104 [00:53<00:17,  2.27it/s]
batch:  63%|██████▎   | 66/104 [00:54<00:19,  1.95it/s]
batch:  64%|██████▍   | 67/104 [00:54<00:18,  2.05it/s]
batch:  65%|██████▌   | 68/104 [00:55<00:17,  2.02it/s]
batch:  66%|██████▋   | 69/104 [00:55<00:16,  2.06it/s]
batch:  67%|██████▋   | 70/104 [00:56<00:15,  2.16it/s]
batch:  68%|██████▊   | 71/104 [00:56<00:14,  2.22it/s]
batch:  69%|██████▉   | 72/104 [00:57<00:14,  2.26it/s]
batch:  70%|███████   | 73/104 [00:57<00:13,  2.29it/s]
batch:  71%|███████   | 74/104 [00:57<00:12,  2.33it/s]
batch:  72%|███████▏  | 75/104 [00:58<00:13,  2.21it/s]
batch:  73%|███████▎  | 76/104 [00:58<00:13,  2.13it/s]
batch:  74%|███████▍  | 77/104 [00:59<00:13,  2.06it/s]
batch:  75%|███████▌  | 78/104 [00:59<00:12,  2.12it/s]
batch:  76%|███████▌  | 79/104 [01:00<00:12,  2.06it/s]
batch:  77%|███████▋  | 80/104 [01:00<00:11,  2.08it/s]
batch:  78%|███████▊  | 81/104 [01:01<00:11,  2.04it/s]
batch:  79%|███████▉  | 82/104 [01:01<00:10,  2.06it/s]
batch:  80%|███████▉  | 83/104 [01:02<00:09,  2.15it/s]
batch:  81%|████████  | 84/104 [01:02<00:09,  2.21it/s]
batch:  82%|████████▏ | 85/104 [01:03<00:08,  2.26it/s]
batch:  83%|████████▎ | 86/104 [01:03<00:08,  2.12it/s]
batch:  84%|████████▎ | 87/104 [01:04<00:08,  2.09it/s]
batch:  85%|████████▍ | 88/104 [01:04<00:07,  2.17it/s]
batch:  86%|████████▌ | 89/104 [01:04<00:06,  2.17it/s]
batch:  87%|████████▋ | 90/104 [01:05<00:06,  2.09it/s]
batch:  88%|████████▊ | 91/104 [01:05<00:06,  2.06it/s]
batch:  88%|████████▊ | 92/104 [01:06<00:05,  2.04it/s]
batch:  89%|████████▉ | 93/104 [01:07<00:05,  2.02it/s]
batch:  90%|█████████ | 94/104 [01:07<00:04,  2.01it/s]
batch:  91%|█████████▏| 95/104 [01:08<00:04,  1.98it/s]
batch:  92%|█████████▏| 96/104 [01:08<00:04,  1.94it/s]
batch:  93%|█████████▎| 97/104 [01:09<00:03,  1.91it/s]
batch:  94%|█████████▍| 98/104 [01:09<00:03,  1.91it/s]
batch:  95%|█████████▌| 99/104 [01:10<00:02,  1.92it/s]
batch:  96%|█████████▌| 100/104 [01:10<00:02,  1.97it/s]
batch:  97%|█████████▋| 101/104 [01:11<00:01,  1.97it/s]
batch:  98%|█████████▊| 102/104 [01:11<00:01,  1.95it/s]
batch:  99%|█████████▉| 103/104 [01:11<00:00,  2.18it/s]
batch: 100%|██████████| 104/104 [01:12<00:00,  2.84it/s]
batch: 100%|██████████| 104/104 [01:12<00:00,  1.44it/s]

  0%|          | 0/6562 [00:00<?, ?it/s]
  1%|          | 57/6562 [00:00<00:11, 567.69it/s]
  2%|▏         | 128/6562 [00:00<00:09, 644.51it/s]
  3%|▎         | 199/6562 [00:00<00:09, 668.19it/s]
  4%|▍         | 273/6562 [00:00<00:09, 690.29it/s]
  5%|▌         | 344/6562 [00:00<00:08, 696.76it/s]
  6%|▋         | 417/6562 [00:00<00:08, 702.39it/s]
  7%|▋         | 488/6562 [00:00<00:08, 700.77it/s]
  9%|▊         | 562/6562 [00:00<00:08, 712.28it/s]
 10%|▉         | 653/6562 [00:00<00:07, 772.84it/s]
 11%|█▏        | 741/6562 [00:01<00:07, 802.04it/s]
 13%|█▎        | 831/6562 [00:01<00:06, 828.67it/s]
 14%|█▍        | 920/6562 [00:01<00:06, 843.02it/s]
 15%|█▌        | 1009/6562 [00:01<00:06, 854.17it/s]
 17%|█▋        | 1098/6562 [00:01<00:06, 859.43it/s]
 18%|█▊        | 1185/6562 [00:01<00:06, 862.49it/s]
 19%|█▉        | 1272/6562 [00:01<00:06, 861.75it/s]
 21%|██        | 1359/6562 [00:02<00:13, 392.87it/s]
 22%|██▏       | 1425/6562 [00:02<00:12, 425.31it/s]
 23%|██▎       | 1507/6562 [00:02<00:10, 497.08it/s]
 24%|██▍       | 1595/6562 [00:02<00:08, 574.97it/s]
 26%|██▌       | 1679/6562 [00:02<00:07, 633.03it/s]
 27%|██▋       | 1765/6562 [00:02<00:06, 688.13it/s]
 28%|██▊       | 1850/6562 [00:02<00:06, 729.78it/s]
 30%|██▉       | 1936/6562 [00:02<00:06, 763.42it/s]
 31%|███       | 2024/6562 [00:02<00:05, 793.74it/s]
 32%|███▏      | 2110/6562 [00:03<00:05, 808.78it/s]
 33%|███▎      | 2197/6562 [00:03<00:05, 823.72it/s]
 35%|███▍      | 2283/6562 [00:03<00:05, 828.39it/s]
 36%|███▌      | 2370/6562 [00:03<00:05, 836.35it/s]
 37%|███▋      | 2455/6562 [00:03<00:04, 837.67it/s]
 39%|███▊      | 2540/6562 [00:03<00:04, 839.11it/s]
 40%|████      | 2625/6562 [00:03<00:04, 828.12it/s]
 41%|████▏     | 2709/6562 [00:03<00:04, 831.20it/s]
 43%|████▎     | 2795/6562 [00:03<00:04, 834.46it/s]
 44%|████▍     | 2882/6562 [00:03<00:04, 843.42it/s]
 45%|████▌     | 2968/6562 [00:04<00:04, 848.22it/s]
 47%|████▋     | 3053/6562 [00:04<00:04, 841.78it/s]
 48%|████▊     | 3138/6562 [00:04<00:04, 834.58it/s]
 49%|████▉     | 3222/6562 [00:04<00:04, 833.25it/s]
 50%|█████     | 3306/6562 [00:04<00:03, 833.01it/s]
 52%|█████▏    | 3390/6562 [00:04<00:03, 831.75it/s]
 53%|█████▎    | 3475/6562 [00:04<00:03, 831.57it/s]
 54%|█████▍    | 3560/6562 [00:04<00:03, 836.14it/s]
 56%|█████▌    | 3644/6562 [00:04<00:03, 835.74it/s]
 57%|█████▋    | 3728/6562 [00:04<00:03, 829.32it/s]
 58%|█████▊    | 3811/6562 [00:05<00:03, 826.10it/s]
 59%|█████▉    | 3894/6562 [00:05<00:03, 823.75it/s]
 61%|██████    | 3977/6562 [00:05<00:03, 824.30it/s]
 62%|██████▏   | 4060/6562 [00:05<00:03, 819.93it/s]
 63%|██████▎   | 4143/6562 [00:05<00:02, 818.52it/s]
 64%|██████▍   | 4225/6562 [00:05<00:02, 817.18it/s]
 66%|██████▌   | 4307/6562 [00:05<00:02, 810.46it/s]
 67%|██████▋   | 4389/6562 [00:05<00:02, 813.13it/s]
 68%|██████▊   | 4471/6562 [00:05<00:02, 814.39it/s]
 69%|██████▉   | 4553/6562 [00:05<00:02, 812.98it/s]
 71%|███████   | 4636/6562 [00:06<00:02, 813.27it/s]
 72%|███████▏  | 4718/6562 [00:06<00:02, 814.39it/s]
 73%|███████▎  | 4800/6562 [00:06<00:02, 810.35it/s]
 74%|███████▍  | 4882/6562 [00:06<00:02, 810.50it/s]
 76%|███████▌  | 4964/6562 [00:06<00:01, 808.99it/s]
 77%|███████▋  | 5045/6562 [00:06<00:01, 806.26it/s]
 78%|███████▊  | 5126/6562 [00:06<00:01, 796.27it/s]
 79%|███████▉  | 5207/6562 [00:06<00:01, 799.94it/s]
 81%|████████  | 5288/6562 [00:06<00:01, 792.29it/s]
 82%|████████▏ | 5368/6562 [00:07<00:01, 787.94it/s]
 83%|████████▎ | 5449/6562 [00:07<00:01, 792.24it/s]
 84%|████████▍ | 5531/6562 [00:07<00:01, 800.31it/s]
 86%|████████▌ | 5612/6562 [00:07<00:01, 800.73it/s]
 87%|████████▋ | 5693/6562 [00:07<00:01, 794.37it/s]
 88%|████████▊ | 5773/6562 [00:07<00:01, 778.50it/s]
 89%|████████▉ | 5851/6562 [00:07<00:00, 778.36it/s]
 90%|█████████ | 5929/6562 [00:07<00:00, 776.84it/s]
 92%|█████████▏| 6007/6562 [00:07<00:00, 774.76it/s]
 93%|█████████▎| 6085/6562 [00:07<00:00, 774.36it/s]
 94%|█████████▍| 6164/6562 [00:08<00:00, 772.79it/s]
 95%|█████████▌| 6242/6562 [00:08<00:00, 772.15it/s]
 96%|█████████▋| 6320/6562 [00:08<00:00, 772.01it/s]
 98%|█████████▊| 6399/6562 [00:08<00:00, 771.67it/s]
 99%|█████████▊| 6477/6562 [00:08<00:00, 767.39it/s]
100%|█████████▉| 6554/6562 [00:08<00:00, 761.32it/s]
100%|██████████| 6562/6562 [00:08<00:00, 766.94it/s]

  0%|          | 0/6562 [00:00<?, ?it/s]
  1%|          | 48/6562 [00:00<00:13, 476.34it/s]
  2%|▏         | 114/6562 [00:00<00:11, 580.92it/s]
  3%|▎         | 185/6562 [00:00<00:10, 635.57it/s]
  4%|▍         | 259/6562 [00:00<00:09, 670.66it/s]
  5%|▌         | 332/6562 [00:00<00:09, 688.63it/s]
  6%|▌         | 405/6562 [00:00<00:08, 702.30it/s]
  7%|▋         | 476/6562 [00:00<00:08, 701.11it/s]
  8%|▊         | 547/6562 [00:00<00:08, 700.73it/s]
  9%|▉         | 623/6562 [00:00<00:08, 718.36it/s]
 11%|█         | 711/6562 [00:01<00:07, 767.61it/s]
 12%|█▏        | 801/6562 [00:01<00:07, 803.08it/s]
 14%|█▎        | 890/6562 [00:01<00:06, 829.04it/s]
 15%|█▍        | 979/6562 [00:01<00:06, 843.46it/s]
 16%|█▋        | 1068/6562 [00:01<00:06, 857.13it/s]
 18%|█▊        | 1156/6562 [00:01<00:06, 862.02it/s]
 19%|█▉        | 1245/6562 [00:01<00:06, 869.45it/s]
 20%|██        | 1334/6562 [00:01<00:06, 871.13it/s]
 22%|██▏       | 1423/6562 [00:01<00:05, 872.23it/s]
 23%|██▎       | 1512/6562 [00:01<00:05, 875.26it/s]
 24%|██▍       | 1601/6562 [00:02<00:05, 878.28it/s]
 26%|██▌       | 1689/6562 [00:02<00:05, 878.68it/s]
 27%|██▋       | 1777/6562 [00:02<00:05, 874.52it/s]
 28%|██▊       | 1865/6562 [00:02<00:05, 862.80it/s]
 30%|██▉       | 1952/6562 [00:02<00:05, 860.91it/s]
 31%|███       | 2039/6562 [00:02<00:05, 860.00it/s]
 32%|███▏      | 2126/6562 [00:02<00:05, 857.62it/s]
 34%|███▎      | 2212/6562 [00:02<00:05, 853.60it/s]
 35%|███▌      | 2298/6562 [00:02<00:05, 849.78it/s]
 36%|███▋      | 2384/6562 [00:02<00:04, 850.18it/s]
 38%|███▊      | 2470/6562 [00:03<00:04, 850.67it/s]
 39%|███▉      | 2556/6562 [00:03<00:04, 846.11it/s]
 40%|████      | 2641/6562 [00:03<00:04, 843.72it/s]
 42%|████▏     | 2726/6562 [00:03<00:04, 839.44it/s]
 43%|████▎     | 2814/6562 [00:03<00:04, 848.23it/s]
 44%|████▍     | 2902/6562 [00:03<00:04, 857.44it/s]
 46%|████▌     | 2988/6562 [00:03<00:04, 850.50it/s]
 47%|████▋     | 3074/6562 [00:03<00:04, 843.95it/s]
 48%|████▊     | 3159/6562 [00:03<00:04, 839.08it/s]
 49%|████▉     | 3243/6562 [00:03<00:03, 830.90it/s]
 51%|█████     | 3330/6562 [00:04<00:03, 838.51it/s]
 52%|█████▏    | 3414/6562 [00:04<00:03, 836.80it/s]
 53%|█████▎    | 3499/6562 [00:04<00:03, 839.59it/s]
 55%|█████▍    | 3588/6562 [00:04<00:03, 853.89it/s]
 56%|█████▌    | 3674/6562 [00:04<00:03, 854.21it/s]
 57%|█████▋    | 3760/6562 [00:04<00:03, 843.75it/s]
 59%|█████▊    | 3845/6562 [00:04<00:03, 844.99it/s]
 60%|█████▉    | 3930/6562 [00:04<00:03, 843.37it/s]
 61%|██████    | 4015/6562 [00:04<00:03, 831.08it/s]
 62%|██████▏   | 4099/6562 [00:04<00:02, 831.66it/s]
 64%|██████▎   | 4183/6562 [00:05<00:02, 826.08it/s]
 65%|██████▌   | 4266/6562 [00:05<00:02, 822.16it/s]
 66%|██████▋   | 4350/6562 [00:05<00:02, 824.48it/s]
 68%|██████▊   | 4433/6562 [00:05<00:02, 821.84it/s]
 69%|██████▉   | 4517/6562 [00:05<00:02, 821.05it/s]
 70%|███████   | 4602/6562 [00:05<00:02, 824.42it/s]
 71%|███████▏  | 4686/6562 [00:05<00:02, 824.12it/s]
 73%|███████▎  | 4770/6562 [00:05<00:02, 824.87it/s]
 74%|███████▍  | 4853/6562 [00:05<00:02, 821.88it/s]
 75%|███████▌  | 4937/6562 [00:05<00:01, 826.28it/s]
 77%|███████▋  | 5020/6562 [00:06<00:01, 819.25it/s]
 78%|███████▊  | 5102/6562 [00:06<00:01, 814.16it/s]
 79%|███████▉  | 5184/6562 [00:06<00:01, 809.13it/s]
 80%|████████  | 5265/6562 [00:06<00:01, 804.56it/s]
 81%|████████▏ | 5346/6562 [00:06<00:01, 797.39it/s]
 83%|████████▎ | 5427/6562 [00:06<00:01, 796.02it/s]
 84%|████████▍ | 5508/6562 [00:06<00:01, 797.20it/s]
 85%|████████▌ | 5588/6562 [00:06<00:01, 700.66it/s]
 86%|████████▋ | 5661/6562 [00:06<00:01, 670.08it/s]
 87%|████████▋ | 5730/6562 [00:07<00:01, 653.17it/s]
 88%|████████▊ | 5797/6562 [00:07<00:01, 650.68it/s]
 89%|████████▉ | 5863/6562 [00:07<00:01, 634.51it/s]
 90%|█████████ | 5928/6562 [00:07<00:00, 634.11it/s]
 91%|█████████▏| 5994/6562 [00:07<00:00, 640.86it/s]
 92%|█████████▏| 6069/6562 [00:07<00:00, 669.64it/s]
 94%|█████████▎| 6147/6562 [00:07<00:00, 701.46it/s]
 95%|█████████▍| 6223/6562 [00:07<00:00, 718.10it/s]
 96%|█████████▌| 6300/6562 [00:07<00:00, 732.98it/s]
 97%|█████████▋| 6380/6562 [00:08<00:00, 747.51it/s]
 98%|█████████▊| 6457/6562 [00:08<00:00, 748.83it/s]
100%|█████████▉| 6533/6562 [00:08<00:00, 749.57it/s]
100%|██████████| 6562/6562 [00:08<00:00, 794.53it/s]
2026-05-11 03:26:20.346079: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:27:03.641910: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 03:27:03.650200: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-11 03:27:03.681320: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:03:00.0 name: Tesla P100-PCIE-16GB computeCapability: 6.0
coreClock: 1.3285GHz coreCount: 56 deviceMemorySize: 15.89GiB deviceMemoryBandwidth: 681.88GiB/s
2026-05-11 03:27:03.681470: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:27:03.692682: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:27:03.692817: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:27:03.699307: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 03:27:03.704425: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 03:27:03.712123: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 03:27:03.716780: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 03:27:03.721124: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:27:03.721806: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 03:27:03.722337: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-11 03:27:03.722477: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 03:27:03.722906: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:03:00.0 name: Tesla P100-PCIE-16GB computeCapability: 6.0
coreClock: 1.3285GHz coreCount: 56 deviceMemorySize: 15.89GiB deviceMemoryBandwidth: 681.88GiB/s
2026-05-11 03:27:03.722979: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:27:03.723026: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:27:03.723066: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:27:03.723105: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 03:27:03.723144: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 03:27:03.723183: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 03:27:03.723221: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 03:27:03.723260: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:27:03.724216: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 03:27:03.724369: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:27:04.413553: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-11 03:27:04.413657: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-11 03:27:04.413682: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-11 03:27:04.414799: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14950 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:03:00.0, compute capability: 6.0)
2026-05-11 03:27:04.503252: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2026-05-11 03:27:04.532692: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2400130000 Hz
2026-05-11 03:27:12.904750: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:27:13.227996: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:27:13.234304: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:27:14.302791: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-11 03:27:14.411896: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)