Lmod Warning:
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: curl/8.4.0
(required by: htslib/1.16)
-------------------------------------------------------------------------------




The following have been reloaded with a version change:
  1) curl/8.4.0 => curl/8.17.0     2) openssl/3.0.7 => openssl/3.6.0

2026-01-05 00:02:45.668960: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:05:04.370661: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-01-05 00:05:04.399211: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-01-05 00:05:04.490743: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-01-05 00:05:04.490792: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:05:05.473724: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:05:05.473801: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:05:06.036996: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-01-05 00:05:06.716230: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-01-05 00:05:07.473090: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-01-05 00:05:07.914399: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-01-05 00:05:08.025097: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:05:08.025490: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-01-05 00:05:08.025866: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-01-05 00:05:08.027162: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-01-05 00:05:08.027421: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-01-05 00:05:08.027440: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:05:08.027455: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:05:08.027464: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:05:08.027473: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-01-05 00:05:08.027482: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-01-05 00:05:08.027491: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-01-05 00:05:08.027500: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-01-05 00:05:08.027521: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:05:08.027805: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-01-05 00:05:08.027824: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:05:11.160846: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-01-05 00:05:11.160957: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-01-05 00:05:11.160968: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-01-05 00:05:11.161641: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 42313 MB memory) -> physical GPU (device: 0, name: NVIDIA L40S, pci bus id: 0000:61:00.0, compute capability: 8.9)
2026-01-05 00:05:23.446460: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-01-05 00:05:23.447111: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500000000 Hz
2026-01-05 00:05:24.633186: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:05:26.597159: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:05:26.603661: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:05:41.512159: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-01-05 00:05:41.594254: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2026-01-05 00:06:04.257344: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'jitters', 'index', 'status', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
2026-01-05 00:18:39.350337: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2026-01-05 00:18:43.442764: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:19:00.142563: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-01-05 00:19:00.143363: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-01-05 00:19:00.172578: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-01-05 00:19:00.172627: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:19:00.181377: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:19:00.181456: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:19:00.185994: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-01-05 00:19:00.189492: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-01-05 00:19:00.194897: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-01-05 00:19:00.198849: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-01-05 00:19:00.202569: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:19:00.202941: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-01-05 00:19:00.203158: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-01-05 00:19:00.204755: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-01-05 00:19:00.205056: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-01-05 00:19:00.205085: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:19:00.205101: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:19:00.205112: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:19:00.205122: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-01-05 00:19:00.205132: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-01-05 00:19:00.205142: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-01-05 00:19:00.205151: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-01-05 00:19:00.205161: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:19:00.205448: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-01-05 00:19:00.205470: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:19:00.609578: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-01-05 00:19:00.609677: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-01-05 00:19:00.609687: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-01-05 00:19:00.610373: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 42313 MB memory) -> physical GPU (device: 0, name: NVIDIA L40S, pci bus id: 0000:61:00.0, compute capability: 8.9)
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)

batch:   0%|          | 0/78 [00:00<?, ?it/s]2026-01-05 00:19:02.401385: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-01-05 00:19:02.401961: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500000000 Hz
2026-01-05 00:19:02.601935: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:19:03.106041: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:19:03.108857: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:19:04.618457: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-01-05 00:19:04.706841: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'true_profiles', 'true_logcounts', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])

batch:   1%|▏         | 1/78 [00:25<32:30, 25.34s/it]
batch:   3%|▎         | 2/78 [00:25<13:24, 10.58s/it]
batch:   4%|▍         | 3/78 [00:25<07:19,  5.86s/it]
batch:   5%|▌         | 4/78 [00:26<04:29,  3.64s/it]
batch:   6%|▋         | 5/78 [00:26<02:55,  2.41s/it]
batch:   8%|▊         | 6/78 [00:26<02:00,  1.67s/it]
batch:   9%|▉         | 7/78 [00:26<01:25,  1.20s/it]
batch:  10%|█         | 8/78 [00:26<01:02,  1.12it/s]
batch:  12%|█▏        | 9/78 [00:27<00:47,  1.45it/s]
batch:  13%|█▎        | 10/78 [00:27<00:37,  1.82it/s]
batch:  14%|█▍        | 11/78 [00:27<00:30,  2.21it/s]
batch:  15%|█▌        | 12/78 [00:27<00:25,  2.58it/s]
batch:  17%|█▋        | 13/78 [00:28<00:22,  2.95it/s]
batch:  18%|█▊        | 14/78 [00:28<00:19,  3.25it/s]
batch:  19%|█▉        | 15/78 [00:28<00:18,  3.49it/s]
batch:  21%|██        | 16/78 [00:28<00:16,  3.71it/s]
batch:  22%|██▏       | 17/78 [00:29<00:15,  3.85it/s]
batch:  23%|██▎       | 18/78 [00:29<00:15,  3.96it/s]
batch:  24%|██▍       | 19/78 [00:29<00:14,  4.06it/s]
batch:  26%|██▌       | 20/78 [00:29<00:14,  4.09it/s]
batch:  27%|██▋       | 21/78 [00:30<00:13,  4.17it/s]
batch:  28%|██▊       | 22/78 [00:30<00:13,  4.18it/s]
batch:  29%|██▉       | 23/78 [00:30<00:13,  4.23it/s]
batch:  31%|███       | 24/78 [00:30<00:12,  4.23it/s]
batch:  32%|███▏      | 25/78 [00:30<00:12,  4.24it/s]
batch:  33%|███▎      | 26/78 [00:31<00:12,  4.24it/s]
batch:  35%|███▍      | 27/78 [00:31<00:12,  4.25it/s]
batch:  36%|███▌      | 28/78 [00:31<00:11,  4.23it/s]
batch:  37%|███▋      | 29/78 [00:31<00:11,  4.22it/s]
batch:  38%|███▊      | 30/78 [00:32<00:11,  4.23it/s]
batch:  40%|███▉      | 31/78 [00:32<00:11,  4.26it/s]
batch:  41%|████      | 32/78 [00:32<00:10,  4.25it/s]
batch:  42%|████▏     | 33/78 [00:32<00:10,  4.23it/s]
batch:  44%|████▎     | 34/78 [00:33<00:10,  4.22it/s]
batch:  45%|████▍     | 35/78 [00:33<00:10,  4.24it/s]
batch:  46%|████▌     | 36/78 [00:33<00:09,  4.24it/s]
batch:  47%|████▋     | 37/78 [00:33<00:09,  4.24it/s]
batch:  49%|████▊     | 38/78 [00:34<00:09,  4.26it/s]
batch:  50%|█████     | 39/78 [00:34<00:09,  4.25it/s]
batch:  51%|█████▏    | 40/78 [00:34<00:08,  4.26it/s]
batch:  53%|█████▎    | 41/78 [00:34<00:08,  4.24it/s]
batch:  54%|█████▍    | 42/78 [00:35<00:08,  4.18it/s]
batch:  55%|█████▌    | 43/78 [00:35<00:08,  4.13it/s]
batch:  56%|█████▋    | 44/78 [00:35<00:08,  4.11it/s]
batch:  58%|█████▊    | 45/78 [00:35<00:08,  4.12it/s]
batch:  59%|█████▉    | 46/78 [00:35<00:07,  4.12it/s]
batch:  60%|██████    | 47/78 [00:36<00:07,  4.10it/s]
batch:  62%|██████▏   | 48/78 [00:36<00:07,  4.10it/s]
batch:  63%|██████▎   | 49/78 [00:36<00:07,  4.12it/s]
batch:  64%|██████▍   | 50/78 [00:36<00:06,  4.12it/s]
batch:  65%|██████▌   | 51/78 [00:37<00:06,  4.16it/s]
batch:  67%|██████▋   | 52/78 [00:37<00:06,  4.17it/s]
batch:  68%|██████▊   | 53/78 [00:37<00:05,  4.18it/s]
batch:  69%|██████▉   | 54/78 [00:37<00:05,  4.17it/s]
batch:  71%|███████   | 55/78 [00:38<00:05,  4.20it/s]
batch:  72%|███████▏  | 56/78 [00:38<00:05,  4.19it/s]
batch:  73%|███████▎  | 57/78 [00:38<00:04,  4.23it/s]
batch:  74%|███████▍  | 58/78 [00:38<00:04,  4.24it/s]
batch:  76%|███████▌  | 59/78 [00:39<00:04,  4.24it/s]
batch:  77%|███████▋  | 60/78 [00:39<00:04,  4.24it/s]
batch:  78%|███████▊  | 61/78 [00:39<00:04,  4.23it/s]
batch:  79%|███████▉  | 62/78 [00:39<00:03,  4.25it/s]
batch:  81%|████████  | 63/78 [00:40<00:03,  4.24it/s]
batch:  82%|████████▏ | 64/78 [00:40<00:03,  4.24it/s]
batch:  83%|████████▎ | 65/78 [00:40<00:03,  4.25it/s]
batch:  85%|████████▍ | 66/78 [00:40<00:03,  3.62it/s]
batch:  86%|████████▌ | 67/78 [00:41<00:02,  3.79it/s]
batch:  87%|████████▋ | 68/78 [00:41<00:02,  3.88it/s]
batch:  88%|████████▊ | 69/78 [00:41<00:02,  3.95it/s]
batch:  90%|████████▉ | 70/78 [00:41<00:01,  4.01it/s]
batch:  91%|█████████ | 71/78 [00:42<00:01,  4.06it/s]
batch:  92%|█████████▏| 72/78 [00:42<00:01,  4.11it/s]
batch:  94%|█████████▎| 73/78 [00:42<00:01,  4.14it/s]
batch:  95%|█████████▍| 74/78 [00:42<00:00,  4.16it/s]
batch:  96%|█████████▌| 75/78 [00:43<00:00,  4.20it/s]
batch:  97%|█████████▋| 76/78 [00:43<00:00,  4.19it/s]
batch:  99%|█████████▊| 77/78 [00:43<00:00,  4.22it/s]
batch: 100%|██████████| 78/78 [00:43<00:00,  1.79it/s]

  0%|          | 0/4945 [00:00<?, ?it/s]
  2%|▏         | 120/4945 [00:00<00:04, 1193.63it/s]
  5%|▍         | 240/4945 [00:00<00:03, 1192.14it/s]
  7%|▋         | 360/4945 [00:00<00:03, 1187.98it/s]
 10%|▉         | 479/4945 [00:00<00:03, 1184.43it/s]
 12%|█▏        | 598/4945 [00:00<00:03, 1175.82it/s]
 14%|█▍        | 716/4945 [00:00<00:03, 1164.57it/s]
 17%|█▋        | 834/4945 [00:00<00:03, 1166.70it/s]
 19%|█▉        | 952/4945 [00:00<00:03, 1165.15it/s]
 22%|██▏       | 1069/4945 [00:00<00:03, 1161.66it/s]
 24%|██▍       | 1186/4945 [00:01<00:03, 1162.09it/s]
 26%|██▋       | 1303/4945 [00:01<00:03, 1149.99it/s]
 29%|██▊       | 1419/4945 [00:01<00:03, 1146.45it/s]
 31%|███       | 1534/4945 [00:01<00:02, 1137.11it/s]
 33%|███▎      | 1648/4945 [00:01<00:02, 1132.72it/s]
 36%|███▌      | 1762/4945 [00:01<00:02, 1124.33it/s]
 38%|███▊      | 1875/4945 [00:01<00:02, 1114.87it/s]
 40%|████      | 1987/4945 [00:01<00:02, 1113.80it/s]
 42%|████▏     | 2099/4945 [00:01<00:02, 1093.23it/s]
 45%|████▍     | 2212/4945 [00:01<00:02, 1097.91it/s]
 47%|████▋     | 2325/4945 [00:02<00:02, 1106.80it/s]
 49%|████▉     | 2436/4945 [00:02<00:02, 1098.75it/s]
 52%|█████▏    | 2548/4945 [00:02<00:02, 1104.99it/s]
 54%|█████▍    | 2659/4945 [00:02<00:02, 1093.50it/s]
 56%|█████▌    | 2770/4945 [00:02<00:01, 1093.37it/s]
 58%|█████▊    | 2881/4945 [00:02<00:01, 1097.24it/s]
 61%|██████    | 2993/4945 [00:02<00:01, 1103.54it/s]
 63%|██████▎   | 3104/4945 [00:02<00:01, 1098.07it/s]
 65%|██████▍   | 3214/4945 [00:02<00:01, 1096.00it/s]
 67%|██████▋   | 3324/4945 [00:02<00:01, 1087.20it/s]
 69%|██████▉   | 3433/4945 [00:03<00:01, 1083.20it/s]
 72%|███████▏  | 3542/4945 [00:03<00:01, 1079.78it/s]
 74%|███████▍  | 3650/4945 [00:03<00:01, 1075.84it/s]
 76%|███████▌  | 3758/4945 [00:03<00:01, 1074.13it/s]
 78%|███████▊  | 3866/4945 [00:03<00:01, 1056.35it/s]
 80%|████████  | 3972/4945 [00:03<00:00, 1054.95it/s]
 82%|████████▏ | 4078/4945 [00:03<00:00, 1051.51it/s]
 85%|████████▍ | 4184/4945 [00:03<00:00, 1048.04it/s]
 87%|████████▋ | 4289/4945 [00:03<00:00, 1048.42it/s]
 89%|████████▉ | 4394/4945 [00:03<00:00, 1038.70it/s]
 91%|█████████ | 4498/4945 [00:04<00:00, 1038.61it/s]
 93%|█████████▎| 4607/4945 [00:04<00:00, 1048.62it/s]
 95%|█████████▌| 4712/4945 [00:04<00:00, 1047.59it/s]
 97%|█████████▋| 4817/4945 [00:04<00:00, 1046.31it/s]
100%|█████████▉| 4922/4945 [00:04<00:00, 1037.21it/s]
100%|██████████| 4945/4945 [00:04<00:00, 1098.04it/s]

  0%|          | 0/4945 [00:00<?, ?it/s]
  2%|▏         | 122/4945 [00:00<00:04, 1197.98it/s]
  5%|▍         | 244/4945 [00:00<00:03, 1199.60it/s]
  7%|▋         | 364/4945 [00:00<00:03, 1184.30it/s]
 10%|▉         | 484/4945 [00:00<00:03, 1189.93it/s]
 12%|█▏        | 604/4945 [00:00<00:03, 1184.47it/s]
 15%|█▍        | 723/4945 [00:00<00:03, 1178.15it/s]
 17%|█▋        | 841/4945 [00:00<00:03, 1175.60it/s]
 19%|█▉        | 959/4945 [00:00<00:03, 1174.64it/s]
 22%|██▏       | 1077/4945 [00:00<00:03, 1165.61it/s]
 24%|██▍       | 1195/4945 [00:01<00:03, 1163.63it/s]
 27%|██▋       | 1312/4945 [00:01<00:03, 1155.58it/s]
 29%|██▉       | 1428/4945 [00:01<00:03, 1135.93it/s]
 31%|███       | 1543/4945 [00:01<00:03, 1133.96it/s]
 34%|███▎      | 1660/4945 [00:01<00:02, 1139.31it/s]
 36%|███▌      | 1774/4945 [00:01<00:02, 1129.06it/s]
 38%|███▊      | 1887/4945 [00:01<00:02, 1122.79it/s]
 40%|████      | 2000/4945 [00:01<00:02, 1105.78it/s]
 43%|████▎     | 2113/4945 [00:01<00:02, 1108.85it/s]
 45%|████▌     | 2226/4945 [00:01<00:02, 1109.61it/s]
 47%|████▋     | 2339/4945 [00:02<00:02, 1110.28it/s]
 50%|████▉     | 2451/4945 [00:02<00:02, 1109.01it/s]
 52%|█████▏    | 2562/4945 [00:02<00:02, 1104.70it/s]
 54%|█████▍    | 2673/4945 [00:02<00:02, 1103.86it/s]
 56%|█████▋    | 2784/4945 [00:02<00:01, 1094.22it/s]
 59%|█████▊    | 2895/4945 [00:02<00:01, 1094.95it/s]
 61%|██████    | 3005/4945 [00:02<00:01, 1093.74it/s]
 63%|██████▎   | 3115/4945 [00:02<00:01, 1073.24it/s]
 65%|██████▌   | 3225/4945 [00:02<00:01, 1075.94it/s]
 67%|██████▋   | 3334/4945 [00:02<00:01, 1079.49it/s]
 70%|██████▉   | 3442/4945 [00:03<00:01, 1068.43it/s]
 72%|███████▏  | 3550/4945 [00:03<00:01, 1065.91it/s]
 74%|███████▍  | 3657/4945 [00:03<00:01, 1054.16it/s]
 76%|███████▌  | 3765/4945 [00:03<00:01, 1056.42it/s]
 78%|███████▊  | 3872/4945 [00:03<00:01, 1056.30it/s]
 80%|████████  | 3978/4945 [00:03<00:00, 1056.19it/s]
 83%|████████▎ | 4084/4945 [00:03<00:00, 1048.17it/s]
 85%|████████▍ | 4192/4945 [00:03<00:00, 1053.06it/s]
 87%|████████▋ | 4298/4945 [00:03<00:00, 1051.82it/s]
 89%|████████▉ | 4404/4945 [00:03<00:00, 1043.83it/s]
 91%|█████████ | 4509/4945 [00:04<00:00, 1029.97it/s]
 93%|█████████▎| 4618/4945 [00:04<00:00, 1041.72it/s]
 96%|█████████▌| 4723/4945 [00:04<00:00, 1037.34it/s]
 98%|█████████▊| 4827/4945 [00:04<00:00, 1037.32it/s]
100%|█████████▉| 4931/4945 [00:04<00:00, 1025.90it/s]
100%|██████████| 4945/4945 [00:04<00:00, 1095.95it/s]
2026-01-05 00:20:10.301559: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:20:19.700682: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-01-05 00:20:19.701799: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-01-05 00:20:19.729507: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-01-05 00:20:19.729564: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:20:19.734740: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:20:19.734834: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:20:19.737551: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-01-05 00:20:19.739371: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-01-05 00:20:19.743050: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-01-05 00:20:19.745060: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-01-05 00:20:19.746610: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:20:19.747007: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-01-05 00:20:19.747483: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-01-05 00:20:19.749062: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-01-05 00:20:19.749340: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-01-05 00:20:19.749372: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:20:19.749388: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:20:19.749401: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:20:19.749414: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-01-05 00:20:19.749427: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-01-05 00:20:19.749440: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-01-05 00:20:19.749453: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-01-05 00:20:19.749466: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:20:19.749779: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-01-05 00:20:19.749807: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:20:20.161677: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-01-05 00:20:20.161781: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-01-05 00:20:20.161791: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-01-05 00:20:20.162502: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 42313 MB memory) -> physical GPU (device: 0, name: NVIDIA L40S, pci bus id: 0000:61:00.0, compute capability: 8.9)
2026-01-05 00:20:20.202153: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2026-01-05 00:20:20.215994: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500000000 Hz
2026-01-05 00:20:24.225913: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:20:24.733399: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:20:24.737735: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:20:26.235131: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-01-05 00:20:26.316923: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2026-01-05 00:20:43.951863: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)