Lmod Warning:
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: curl/8.4.0
(required by: htslib/1.16)
-------------------------------------------------------------------------------




The following have been reloaded with a version change:
  1) curl/8.4.0 => curl/8.17.0     2) openssl/3.0.7 => openssl/3.6.0

2026-05-11 03:51:15.880627: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:01:20.624357: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 04:01:20.631037: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-11 04:01:20.768656: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:03:00.0 name: Tesla P100-PCIE-16GB computeCapability: 6.0
coreClock: 1.3285GHz coreCount: 56 deviceMemorySize: 15.89GiB deviceMemoryBandwidth: 681.88GiB/s
2026-05-11 04:01:20.768814: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:01:20.893542: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:01:20.893786: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:01:20.956624: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 04:01:21.019042: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 04:01:21.081015: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 04:01:21.139117: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 04:01:21.195732: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:01:21.196898: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 04:01:21.197723: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-11 04:01:21.198070: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 04:01:21.198803: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:03:00.0 name: Tesla P100-PCIE-16GB computeCapability: 6.0
coreClock: 1.3285GHz coreCount: 56 deviceMemorySize: 15.89GiB deviceMemoryBandwidth: 681.88GiB/s
2026-05-11 04:01:21.198888: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:01:21.198950: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:01:21.199018: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:01:21.199068: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 04:01:21.199117: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 04:01:21.199176: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 04:01:21.199223: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 04:01:21.199304: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:01:21.200082: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 04:01:21.200170: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:01:21.886919: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-11 04:01:21.887035: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-11 04:01:21.887062: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-11 04:01:21.888116: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14950 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:03:00.0, compute capability: 6.0)
2026-05-11 04:01:25.887112: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-05-11 04:01:25.887867: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2400130000 Hz
2026-05-11 04:01:40.923677: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:01:41.201050: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:01:41.209920: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:01:42.615548: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-11 04:01:42.720533: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'jitters', 'index', 'status', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
2026-05-11 04:24:10.143328: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2026-05-11 04:24:42.059289: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:25:04.148123: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 04:25:04.149186: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-11 04:25:04.182325: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:03:00.0 name: Tesla P100-PCIE-16GB computeCapability: 6.0
coreClock: 1.3285GHz coreCount: 56 deviceMemorySize: 15.89GiB deviceMemoryBandwidth: 681.88GiB/s
2026-05-11 04:25:04.182482: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:25:04.195353: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:25:04.195485: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:25:04.200847: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 04:25:04.205012: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 04:25:04.213795: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 04:25:04.218975: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 04:25:04.222622: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:25:04.223424: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 04:25:04.223890: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-11 04:25:04.224064: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 04:25:04.224789: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:03:00.0 name: Tesla P100-PCIE-16GB computeCapability: 6.0
coreClock: 1.3285GHz coreCount: 56 deviceMemorySize: 15.89GiB deviceMemoryBandwidth: 681.88GiB/s
2026-05-11 04:25:04.224886: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:25:04.224954: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:25:04.225035: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:25:04.225094: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 04:25:04.225152: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 04:25:04.225210: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 04:25:04.225266: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 04:25:04.225323: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:25:04.226135: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 04:25:04.226234: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:25:04.887978: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-11 04:25:04.888094: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-11 04:25:04.888120: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-11 04:25:04.889147: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14950 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:03:00.0, compute capability: 6.0)
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)

batch:   0%|          | 0/48 [00:00<?, ?it/s]2026-05-11 04:25:07.350370: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-05-11 04:25:07.351040: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2400130000 Hz
2026-05-11 04:25:08.042554: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:25:08.615155: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:25:08.629219: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:25:10.641072: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-11 04:25:10.811177: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'true_profiles', 'true_logcounts', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])

batch:   2%|▏         | 1/48 [00:17<13:20, 17.04s/it]
batch:   4%|▍         | 2/48 [00:17<05:37,  7.33s/it]
batch:   6%|▋         | 3/48 [00:18<03:08,  4.18s/it]
batch:   8%|▊         | 4/48 [00:18<01:58,  2.70s/it]
batch:  10%|█         | 5/48 [00:18<01:20,  1.88s/it]
batch:  12%|█▎        | 6/48 [00:19<00:59,  1.42s/it]
batch:  15%|█▍        | 7/48 [00:19<00:46,  1.13s/it]
batch:  17%|█▋        | 8/48 [00:20<00:37,  1.07it/s]
batch:  19%|█▉        | 9/48 [00:20<00:31,  1.24it/s]
batch:  21%|██        | 10/48 [00:21<00:27,  1.37it/s]
batch:  23%|██▎       | 11/48 [00:22<00:24,  1.49it/s]
batch:  25%|██▌       | 12/48 [00:22<00:22,  1.61it/s]
batch:  27%|██▋       | 13/48 [00:22<00:19,  1.78it/s]
batch:  29%|██▉       | 14/48 [00:23<00:17,  1.91it/s]
batch:  31%|███▏      | 15/48 [00:23<00:16,  2.01it/s]
batch:  33%|███▎      | 16/48 [00:24<00:15,  2.09it/s]
batch:  35%|███▌      | 17/48 [00:24<00:14,  2.15it/s]
batch:  38%|███▊      | 18/48 [00:25<00:13,  2.20it/s]
batch:  40%|███▉      | 19/48 [00:25<00:12,  2.23it/s]
batch:  42%|████▏     | 20/48 [00:26<00:12,  2.26it/s]
batch:  44%|████▍     | 21/48 [00:26<00:11,  2.27it/s]
batch:  46%|████▌     | 22/48 [00:26<00:11,  2.28it/s]
batch:  48%|████▊     | 23/48 [00:27<00:10,  2.29it/s]
batch:  50%|█████     | 24/48 [00:27<00:11,  2.15it/s]
batch:  52%|█████▏    | 25/48 [00:28<00:11,  2.07it/s]
batch:  54%|█████▍    | 26/48 [00:28<00:10,  2.02it/s]
batch:  56%|█████▋    | 27/48 [00:29<00:10,  1.99it/s]
batch:  58%|█████▊    | 28/48 [00:29<00:10,  1.97it/s]
batch:  60%|██████    | 29/48 [00:30<00:09,  1.95it/s]
batch:  62%|██████▎   | 30/48 [00:31<00:09,  1.90it/s]
batch:  65%|██████▍   | 31/48 [00:31<00:08,  1.93it/s]
batch:  67%|██████▋   | 32/48 [00:32<00:08,  1.91it/s]
batch:  69%|██████▉   | 33/48 [00:32<00:07,  1.97it/s]
batch:  71%|███████   | 34/48 [00:32<00:06,  2.06it/s]
batch:  73%|███████▎  | 35/48 [00:33<00:06,  2.14it/s]
batch:  75%|███████▌  | 36/48 [00:33<00:05,  2.18it/s]
batch:  77%|███████▋  | 37/48 [00:34<00:04,  2.22it/s]
batch:  79%|███████▉  | 38/48 [00:34<00:04,  2.11it/s]
batch:  81%|████████▏ | 39/48 [00:35<00:04,  2.16it/s]
batch:  83%|████████▎ | 40/48 [00:35<00:03,  2.17it/s]
batch:  85%|████████▌ | 41/48 [00:36<00:03,  2.21it/s]
batch:  88%|████████▊ | 42/48 [00:36<00:02,  2.23it/s]
batch:  90%|████████▉ | 43/48 [00:37<00:02,  1.90it/s]
batch:  92%|█████████▏| 44/48 [00:37<00:02,  1.96it/s]
batch:  94%|█████████▍| 45/48 [00:38<00:01,  2.05it/s]
batch:  96%|█████████▌| 46/48 [00:38<00:01,  2.00it/s]
batch:  98%|█████████▊| 47/48 [00:38<00:00,  2.59it/s]
batch: 100%|██████████| 48/48 [00:39<00:00,  2.34it/s]
batch: 100%|██████████| 48/48 [00:39<00:00,  1.22it/s]

  0%|          | 0/3007 [00:00<?, ?it/s]
  3%|▎         | 91/3007 [00:00<00:03, 893.69it/s]
  6%|▌         | 181/3007 [00:00<00:03, 893.86it/s]
  9%|▉         | 273/3007 [00:00<00:03, 897.76it/s]
 12%|█▏        | 366/3007 [00:00<00:02, 902.72it/s]
 15%|█▌        | 457/3007 [00:00<00:02, 898.61it/s]
 18%|█▊        | 547/3007 [00:00<00:02, 888.49it/s]
 21%|██        | 636/3007 [00:00<00:02, 880.97it/s]
 24%|██▍       | 730/3007 [00:00<00:02, 894.44it/s]
 27%|██▋       | 820/3007 [00:00<00:02, 892.57it/s]
 30%|███       | 912/3007 [00:01<00:02, 899.42it/s]
 33%|███▎      | 1002/3007 [00:01<00:02, 888.55it/s]
 36%|███▋      | 1091/3007 [00:01<00:02, 874.06it/s]
 39%|███▉      | 1179/3007 [00:01<00:02, 875.33it/s]
 42%|████▏     | 1267/3007 [00:01<00:02, 858.15it/s]
 45%|████▍     | 1353/3007 [00:01<00:01, 851.36it/s]
 48%|████▊     | 1439/3007 [00:01<00:01, 846.22it/s]
 51%|█████     | 1526/3007 [00:01<00:01, 847.76it/s]
 54%|█████▎    | 1611/3007 [00:01<00:01, 842.15it/s]
 56%|█████▋    | 1696/3007 [00:01<00:01, 840.85it/s]
 59%|█████▉    | 1781/3007 [00:02<00:01, 838.15it/s]
 62%|██████▏   | 1866/3007 [00:02<00:01, 835.33it/s]
 65%|██████▍   | 1951/3007 [00:02<00:01, 835.48it/s]
 68%|██████▊   | 2036/3007 [00:02<00:01, 838.18it/s]
 71%|███████   | 2120/3007 [00:02<00:01, 823.53it/s]
 73%|███████▎  | 2203/3007 [00:02<00:00, 813.05it/s]
 76%|███████▌  | 2290/3007 [00:02<00:00, 829.06it/s]
 79%|███████▉  | 2375/3007 [00:02<00:00, 829.67it/s]
 82%|████████▏ | 2461/3007 [00:02<00:00, 838.18it/s]
 85%|████████▍ | 2553/3007 [00:02<00:00, 856.54it/s]
 88%|████████▊ | 2639/3007 [00:03<00:00, 846.95it/s]
 91%|█████████ | 2724/3007 [00:03<00:00, 839.02it/s]
 93%|█████████▎| 2808/3007 [00:03<00:00, 832.32it/s]
 96%|█████████▌| 2892/3007 [00:03<00:00, 827.32it/s]
 99%|█████████▉| 2975/3007 [00:03<00:00, 826.16it/s]
100%|██████████| 3007/3007 [00:03<00:00, 852.61it/s]

  0%|          | 0/3007 [00:00<?, ?it/s]
  3%|▎         | 93/3007 [00:00<00:03, 923.99it/s]
  6%|▌         | 186/3007 [00:00<00:03, 921.14it/s]
  9%|▉         | 282/3007 [00:00<00:02, 938.01it/s]
 13%|█▎        | 377/3007 [00:00<00:02, 935.13it/s]
 16%|█▌        | 471/3007 [00:00<00:02, 924.49it/s]
 19%|█▉        | 564/3007 [00:00<00:02, 916.54it/s]
 22%|██▏       | 656/3007 [00:00<00:02, 917.64it/s]
 25%|██▍       | 751/3007 [00:00<00:02, 922.87it/s]
 28%|██▊       | 844/3007 [00:00<00:02, 917.28it/s]
 31%|███       | 939/3007 [00:01<00:02, 924.88it/s]
 34%|███▍      | 1032/3007 [00:01<00:02, 904.64it/s]
 37%|███▋      | 1123/3007 [00:01<00:02, 889.66it/s]
 40%|████      | 1216/3007 [00:01<00:01, 896.34it/s]
 43%|████▎     | 1306/3007 [00:01<00:01, 882.50it/s]
 46%|████▋     | 1395/3007 [00:01<00:01, 879.54it/s]
 49%|████▉     | 1484/3007 [00:01<00:01, 881.91it/s]
 52%|█████▏    | 1573/3007 [00:01<00:01, 871.24it/s]
 55%|█████▌    | 1661/3007 [00:01<00:01, 868.33it/s]
 58%|█████▊    | 1748/3007 [00:01<00:01, 862.70it/s]
 61%|██████    | 1835/3007 [00:02<00:01, 862.23it/s]
 64%|██████▍   | 1922/3007 [00:02<00:01, 860.02it/s]
 67%|██████▋   | 2009/3007 [00:02<00:01, 857.55it/s]
 70%|██████▉   | 2095/3007 [00:02<00:01, 850.82it/s]
 73%|███████▎  | 2181/3007 [00:02<00:00, 845.70it/s]
 75%|███████▌  | 2267/3007 [00:02<00:00, 849.68it/s]
 78%|███████▊  | 2352/3007 [00:02<00:00, 848.95it/s]
 81%|████████  | 2439/3007 [00:02<00:00, 848.99it/s]
 84%|████████▍ | 2531/3007 [00:02<00:00, 864.66it/s]
 87%|████████▋ | 2618/3007 [00:02<00:00, 836.50it/s]
 90%|████████▉ | 2702/3007 [00:03<00:00, 820.77it/s]
 93%|█████████▎| 2785/3007 [00:03<00:00, 809.78it/s]
 95%|█████████▌| 2868/3007 [00:03<00:00, 812.80it/s]
 98%|█████████▊| 2950/3007 [00:03<00:00, 805.16it/s]
100%|██████████| 3007/3007 [00:03<00:00, 867.56it/s]
2026-05-11 04:26:09.164699: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:26:24.746033: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 04:26:24.750854: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-11 04:26:24.783327: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:03:00.0 name: Tesla P100-PCIE-16GB computeCapability: 6.0
coreClock: 1.3285GHz coreCount: 56 deviceMemorySize: 15.89GiB deviceMemoryBandwidth: 681.88GiB/s
2026-05-11 04:26:24.783510: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:26:24.791574: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:26:24.791708: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:26:24.795699: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 04:26:24.797389: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 04:26:24.803817: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 04:26:24.806309: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 04:26:24.807952: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:26:24.808857: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 04:26:24.809626: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-11 04:26:24.809869: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 04:26:24.810334: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:03:00.0 name: Tesla P100-PCIE-16GB computeCapability: 6.0
coreClock: 1.3285GHz coreCount: 56 deviceMemorySize: 15.89GiB deviceMemoryBandwidth: 681.88GiB/s
2026-05-11 04:26:24.810421: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:26:24.810487: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:26:24.810549: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:26:24.810609: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 04:26:24.810696: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 04:26:24.810759: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 04:26:24.810819: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 04:26:24.810878: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:26:24.811579: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 04:26:24.811676: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:26:25.466163: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-11 04:26:25.466272: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-11 04:26:25.466298: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-11 04:26:25.467361: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 14950 MB memory) -> physical GPU (device: 0, name: Tesla P100-PCIE-16GB, pci bus id: 0000:03:00.0, compute capability: 6.0)
2026-05-11 04:26:25.550967: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2026-05-11 04:26:25.575283: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2400130000 Hz
2026-05-11 04:26:30.088131: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:26:30.406694: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:26:30.412219: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:26:31.400868: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-11 04:26:31.492255: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)