Lmod Warning:
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: curl/8.4.0
(required by: htslib/1.16)
-------------------------------------------------------------------------------




The following have been reloaded with a version change:
  1) curl/8.4.0 => curl/8.17.0     2) openssl/3.0.7 => openssl/3.6.0

2026-01-05 00:02:45.668256: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:05:04.592586: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-01-05 00:05:04.627342: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-01-05 00:05:04.724847: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-01-05 00:05:04.724895: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:05:05.681680: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:05:05.681752: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:05:05.989878: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-01-05 00:05:06.714537: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-01-05 00:05:07.524413: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-01-05 00:05:07.904326: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-01-05 00:05:08.017096: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:05:08.017483: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-01-05 00:05:08.017853: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-01-05 00:05:08.019555: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-01-05 00:05:08.019811: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-01-05 00:05:08.019832: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:05:08.019846: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:05:08.019855: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:05:08.019864: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-01-05 00:05:08.019873: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-01-05 00:05:08.019881: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-01-05 00:05:08.019890: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-01-05 00:05:08.019910: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:05:08.020202: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-01-05 00:05:08.020221: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:05:10.841623: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-01-05 00:05:10.841730: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-01-05 00:05:10.841743: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-01-05 00:05:10.842454: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 42313 MB memory) -> physical GPU (device: 0, name: NVIDIA L40S, pci bus id: 0000:61:00.0, compute capability: 8.9)
2026-01-05 00:05:23.826670: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-01-05 00:05:23.827249: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500000000 Hz
2026-01-05 00:05:24.970410: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:05:26.778249: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:05:26.783482: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:05:41.342946: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-01-05 00:05:41.477638: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2026-01-05 00:06:16.682643: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'jitters', 'index', 'status', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
2026-01-05 00:18:57.801048: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2026-01-05 00:19:01.424672: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:19:06.986465: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-01-05 00:19:06.987328: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-01-05 00:19:07.014221: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-01-05 00:19:07.014272: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:19:07.018506: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:19:07.018586: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:19:07.020614: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-01-05 00:19:07.022023: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-01-05 00:19:07.025321: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-01-05 00:19:07.026838: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-01-05 00:19:07.027904: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:19:07.028280: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-01-05 00:19:07.028533: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-01-05 00:19:07.030108: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-01-05 00:19:07.030358: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-01-05 00:19:07.030380: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:19:07.030394: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:19:07.030404: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:19:07.030414: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-01-05 00:19:07.030424: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-01-05 00:19:07.030434: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-01-05 00:19:07.030444: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-01-05 00:19:07.030454: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:19:07.030734: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-01-05 00:19:07.030754: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:19:07.453482: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-01-05 00:19:07.453580: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-01-05 00:19:07.453591: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-01-05 00:19:07.454284: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 42313 MB memory) -> physical GPU (device: 0, name: NVIDIA L40S, pci bus id: 0000:61:00.0, compute capability: 8.9)
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)

batch:   0%|          | 0/78 [00:00<?, ?it/s]2026-01-05 00:19:09.048614: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-01-05 00:19:09.049220: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500000000 Hz
2026-01-05 00:19:09.250375: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:19:09.780114: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:19:09.782098: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:19:11.413995: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-01-05 00:19:11.538302: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'true_profiles', 'true_logcounts', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])

batch:   1%|▏         | 1/78 [00:33<43:02, 33.53s/it]
batch:   3%|▎         | 2/78 [00:33<17:40, 13.95s/it]
batch:   4%|▍         | 3/78 [00:34<09:36,  7.69s/it]
batch:   5%|▌         | 4/78 [00:34<05:51,  4.75s/it]
batch:   6%|▋         | 5/78 [00:34<03:47,  3.12s/it]
batch:   8%|▊         | 6/78 [00:34<02:33,  2.14s/it]
batch:   9%|▉         | 7/78 [00:34<01:47,  1.52s/it]
batch:  10%|█         | 8/78 [00:35<01:17,  1.11s/it]
batch:  12%|█▏        | 9/78 [00:35<00:57,  1.20it/s]
batch:  13%|█▎        | 10/78 [00:35<00:44,  1.54it/s]
batch:  14%|█▍        | 11/78 [00:35<00:34,  1.92it/s]
batch:  15%|█▌        | 12/78 [00:36<00:28,  2.31it/s]
batch:  17%|█▋        | 13/78 [00:36<00:24,  2.68it/s]
batch:  18%|█▊        | 14/78 [00:36<00:21,  3.03it/s]
batch:  19%|█▉        | 15/78 [00:36<00:18,  3.32it/s]
batch:  21%|██        | 16/78 [00:37<00:17,  3.56it/s]
batch:  22%|██▏       | 17/78 [00:37<00:16,  3.74it/s]
batch:  23%|██▎       | 18/78 [00:37<00:15,  3.89it/s]
batch:  24%|██▍       | 19/78 [00:37<00:14,  3.99it/s]
batch:  26%|██▌       | 20/78 [00:37<00:14,  4.04it/s]
batch:  27%|██▋       | 21/78 [00:38<00:13,  4.11it/s]
batch:  28%|██▊       | 22/78 [00:38<00:13,  4.13it/s]
batch:  29%|██▉       | 23/78 [00:38<00:13,  4.16it/s]
batch:  31%|███       | 24/78 [00:38<00:12,  4.19it/s]
batch:  32%|███▏      | 25/78 [00:39<00:12,  4.17it/s]
batch:  33%|███▎      | 26/78 [00:39<00:12,  4.19it/s]
batch:  35%|███▍      | 27/78 [00:39<00:12,  4.20it/s]
batch:  36%|███▌      | 28/78 [00:39<00:11,  4.20it/s]
batch:  37%|███▋      | 29/78 [00:40<00:11,  4.21it/s]
batch:  38%|███▊      | 30/78 [00:40<00:11,  4.22it/s]
batch:  40%|███▉      | 31/78 [00:40<00:11,  4.20it/s]
batch:  41%|████      | 32/78 [00:40<00:10,  4.19it/s]
batch:  42%|████▏     | 33/78 [00:41<00:10,  4.21it/s]
batch:  44%|████▎     | 34/78 [00:41<00:10,  4.21it/s]
batch:  45%|████▍     | 35/78 [00:41<00:10,  4.21it/s]
batch:  46%|████▌     | 36/78 [00:41<00:09,  4.22it/s]
batch:  47%|████▋     | 37/78 [00:42<00:09,  4.20it/s]
batch:  49%|████▊     | 38/78 [00:42<00:09,  4.20it/s]
batch:  50%|█████     | 39/78 [00:42<00:09,  4.21it/s]
batch:  51%|█████▏    | 40/78 [00:42<00:08,  4.23it/s]
batch:  53%|█████▎    | 41/78 [00:42<00:08,  4.23it/s]
batch:  54%|█████▍    | 42/78 [00:43<00:08,  4.23it/s]
batch:  55%|█████▌    | 43/78 [00:43<00:08,  4.21it/s]
batch:  56%|█████▋    | 44/78 [00:43<00:08,  4.23it/s]
batch:  58%|█████▊    | 45/78 [00:43<00:07,  4.24it/s]
batch:  59%|█████▉    | 46/78 [00:44<00:07,  4.23it/s]
batch:  60%|██████    | 47/78 [00:44<00:07,  4.22it/s]
batch:  62%|██████▏   | 48/78 [00:44<00:07,  4.24it/s]
batch:  63%|██████▎   | 49/78 [00:44<00:06,  4.23it/s]
batch:  64%|██████▍   | 50/78 [00:45<00:06,  4.24it/s]
batch:  65%|██████▌   | 51/78 [00:45<00:06,  4.24it/s]
batch:  67%|██████▋   | 52/78 [00:45<00:06,  4.23it/s]
batch:  68%|██████▊   | 53/78 [00:45<00:05,  4.20it/s]
batch:  69%|██████▉   | 54/78 [00:46<00:05,  4.17it/s]
batch:  71%|███████   | 55/78 [00:46<00:05,  4.17it/s]
batch:  72%|███████▏  | 56/78 [00:46<00:05,  4.19it/s]
batch:  73%|███████▎  | 57/78 [00:46<00:05,  4.20it/s]
batch:  74%|███████▍  | 58/78 [00:47<00:04,  4.20it/s]
batch:  76%|███████▌  | 59/78 [00:47<00:04,  4.21it/s]
batch:  77%|███████▋  | 60/78 [00:47<00:04,  4.23it/s]
batch:  78%|███████▊  | 61/78 [00:47<00:04,  4.23it/s]
batch:  79%|███████▉  | 62/78 [00:47<00:03,  4.21it/s]
batch:  81%|████████  | 63/78 [00:48<00:03,  4.21it/s]
batch:  82%|████████▏ | 64/78 [00:48<00:03,  4.25it/s]
batch:  83%|████████▎ | 65/78 [00:48<00:03,  4.25it/s]
batch:  85%|████████▍ | 66/78 [00:49<00:03,  3.61it/s]
batch:  86%|████████▌ | 67/78 [00:49<00:02,  3.77it/s]
batch:  87%|████████▋ | 68/78 [00:49<00:02,  3.89it/s]
batch:  88%|████████▊ | 69/78 [00:49<00:02,  4.00it/s]
batch:  90%|████████▉ | 70/78 [00:49<00:01,  4.08it/s]
batch:  91%|█████████ | 71/78 [00:50<00:01,  4.09it/s]
batch:  92%|█████████▏| 72/78 [00:50<00:01,  4.11it/s]
batch:  94%|█████████▎| 73/78 [00:50<00:01,  4.14it/s]
batch:  95%|█████████▍| 74/78 [00:50<00:00,  4.15it/s]
batch:  96%|█████████▌| 75/78 [00:51<00:00,  4.16it/s]
batch:  97%|█████████▋| 76/78 [00:51<00:00,  4.15it/s]
batch:  99%|█████████▊| 77/78 [00:51<00:00,  4.17it/s]
batch: 100%|██████████| 78/78 [00:51<00:00,  1.51it/s]

  0%|          | 0/4945 [00:00<?, ?it/s]
  2%|▏         | 120/4945 [00:00<00:04, 1195.23it/s]
  5%|▍         | 240/4945 [00:00<00:03, 1186.96it/s]
  7%|▋         | 359/4945 [00:00<00:03, 1179.77it/s]
 10%|▉         | 477/4945 [00:00<00:03, 1170.82it/s]
 12%|█▏        | 595/4945 [00:00<00:03, 1173.27it/s]
 14%|█▍        | 713/4945 [00:00<00:03, 1173.82it/s]
 17%|█▋        | 831/4945 [00:00<00:03, 1164.55it/s]
 19%|█▉        | 950/4945 [00:00<00:03, 1169.23it/s]
 22%|██▏       | 1067/4945 [00:00<00:03, 1162.73it/s]
 24%|██▍       | 1184/4945 [00:01<00:03, 1160.58it/s]
 26%|██▋       | 1301/4945 [00:01<00:03, 1157.12it/s]
 29%|██▊       | 1417/4945 [00:01<00:03, 1146.46it/s]
 31%|███       | 1532/4945 [00:01<00:02, 1142.57it/s]
 33%|███▎      | 1647/4945 [00:01<00:02, 1141.05it/s]
 36%|███▌      | 1762/4945 [00:01<00:02, 1137.99it/s]
 38%|███▊      | 1876/4945 [00:01<00:02, 1115.78it/s]
 40%|████      | 1991/4945 [00:01<00:02, 1120.36it/s]
 43%|████▎     | 2104/4945 [00:01<00:02, 1119.02it/s]
 45%|████▍     | 2217/4945 [00:01<00:02, 1116.54it/s]
 47%|████▋     | 2329/4945 [00:02<00:02, 1101.58it/s]
 49%|████▉     | 2440/4945 [00:02<00:02, 1102.62it/s]
 52%|█████▏    | 2551/4945 [00:02<00:02, 1101.10it/s]
 54%|█████▍    | 2663/4945 [00:02<00:02, 1103.77it/s]
 56%|█████▌    | 2776/4945 [00:02<00:01, 1105.28it/s]
 58%|█████▊    | 2887/4945 [00:02<00:01, 1103.17it/s]
 61%|██████    | 2998/4945 [00:02<00:01, 1097.99it/s]
 63%|██████▎   | 3108/4945 [00:02<00:01, 1096.65it/s]
 65%|██████▌   | 3218/4945 [00:02<00:01, 1095.26it/s]
 67%|██████▋   | 3328/4945 [00:02<00:01, 1091.36it/s]
 70%|██████▉   | 3438/4945 [00:03<00:01, 1048.40it/s]
 72%|███████▏  | 3547/4945 [00:03<00:01, 1055.33it/s]
 74%|███████▍  | 3656/4945 [00:03<00:01, 1065.28it/s]
 76%|███████▌  | 3763/4945 [00:03<00:01, 1064.03it/s]
 78%|███████▊  | 3871/4945 [00:03<00:01, 1067.79it/s]
 80%|████████  | 3978/4945 [00:03<00:00, 1034.42it/s]
 83%|████████▎ | 4083/4945 [00:03<00:00, 1034.45it/s]
 85%|████████▍ | 4192/4945 [00:03<00:00, 1046.71it/s]
 87%|████████▋ | 4299/4945 [00:03<00:00, 1048.80it/s]
 89%|████████▉ | 4404/4945 [00:03<00:00, 1031.85it/s]
 91%|█████████ | 4509/4945 [00:04<00:00, 1036.64it/s]
 93%|█████████▎| 4616/4945 [00:04<00:00, 1045.05it/s]
 95%|█████████▌| 4722/4945 [00:04<00:00, 1044.56it/s]
 98%|█████████▊| 4827/4945 [00:04<00:00, 1040.11it/s]
100%|█████████▉| 4932/4945 [00:04<00:00, 1041.00it/s]
100%|██████████| 4945/4945 [00:04<00:00, 1097.23it/s]

  0%|          | 0/4945 [00:00<?, ?it/s]
  2%|▏         | 121/4945 [00:00<00:03, 1207.69it/s]
  5%|▍         | 242/4945 [00:00<00:04, 1155.05it/s]
  7%|▋         | 361/4945 [00:00<00:03, 1164.03it/s]
 10%|▉         | 483/4945 [00:00<00:03, 1177.42it/s]
 12%|█▏        | 602/4945 [00:00<00:03, 1181.71it/s]
 15%|█▍        | 721/4945 [00:00<00:03, 1177.45it/s]
 17%|█▋        | 839/4945 [00:00<00:03, 1175.23it/s]
 19%|█▉        | 958/4945 [00:00<00:03, 1177.19it/s]
 22%|██▏       | 1076/4945 [00:00<00:03, 1168.25it/s]
 24%|██▍       | 1194/4945 [00:01<00:03, 1171.53it/s]
 27%|██▋       | 1312/4945 [00:01<00:03, 1165.77it/s]
 29%|██▉       | 1429/4945 [00:01<00:03, 1153.26it/s]
 31%|███       | 1545/4945 [00:01<00:02, 1148.11it/s]
 34%|███▎      | 1662/4945 [00:01<00:02, 1150.00it/s]
 36%|███▌      | 1778/4945 [00:01<00:02, 1139.50it/s]
 38%|███▊      | 1892/4945 [00:01<00:02, 1136.24it/s]
 41%|████      | 2006/4945 [00:01<00:02, 1135.94it/s]
 43%|████▎     | 2120/4945 [00:01<00:02, 1096.61it/s]
 45%|████▌     | 2233/4945 [00:01<00:02, 1101.67it/s]
 47%|████▋     | 2347/4945 [00:02<00:02, 1111.20it/s]
 50%|████▉     | 2460/4945 [00:02<00:02, 1112.29it/s]
 52%|█████▏    | 2574/4945 [00:02<00:02, 1114.90it/s]
 54%|█████▍    | 2687/4945 [00:02<00:02, 1119.09it/s]
 57%|█████▋    | 2799/4945 [00:02<00:01, 1115.82it/s]
 59%|█████▉    | 2911/4945 [00:02<00:01, 1079.13it/s]
 61%|██████    | 3023/4945 [00:02<00:01, 1090.09it/s]
 63%|██████▎   | 3133/4945 [00:02<00:01, 1090.80it/s]
 66%|██████▌   | 3245/4945 [00:02<00:01, 1094.07it/s]
 68%|██████▊   | 3355/4945 [00:02<00:01, 1091.03it/s]
 70%|███████   | 3465/4945 [00:03<00:01, 1066.33it/s]
 72%|███████▏  | 3574/4945 [00:03<00:01, 1067.29it/s]
 74%|███████▍  | 3683/4945 [00:03<00:01, 1071.75it/s]
 77%|███████▋  | 3791/4945 [00:03<00:01, 1073.36it/s]
 79%|███████▉  | 3899/4945 [00:03<00:00, 1074.41it/s]
 81%|████████  | 4007/4945 [00:03<00:00, 1063.60it/s]
 83%|████████▎ | 4114/4945 [00:03<00:00, 1062.31it/s]
 85%|████████▌ | 4222/4945 [00:03<00:00, 1065.15it/s]
 88%|████████▊ | 4330/4945 [00:03<00:00, 1064.20it/s]
 90%|████████▉ | 4437/4945 [00:03<00:00, 1061.67it/s]
 92%|█████████▏| 4544/4945 [00:04<00:00, 1060.30it/s]
 94%|█████████▍| 4651/4945 [00:04<00:00, 1060.34it/s]
 96%|█████████▌| 4758/4945 [00:04<00:00, 1055.30it/s]
 98%|█████████▊| 4864/4945 [00:04<00:00, 1052.78it/s]
100%|██████████| 4945/4945 [00:04<00:00, 1105.63it/s]
2026-01-05 00:20:25.153218: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:20:48.505560: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-01-05 00:20:48.506523: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-01-05 00:20:48.543799: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-01-05 00:20:48.543884: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:20:48.553561: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:20:48.553653: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:20:48.559060: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-01-05 00:20:48.562502: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-01-05 00:20:48.568406: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-01-05 00:20:48.572030: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-01-05 00:20:48.575569: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:20:48.575969: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-01-05 00:20:48.576411: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-01-05 00:20:48.578089: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-01-05 00:20:48.578368: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-01-05 00:20:48.578401: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:20:48.578418: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:20:48.578431: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:20:48.578444: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-01-05 00:20:48.578458: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-01-05 00:20:48.578471: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-01-05 00:20:48.578484: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-01-05 00:20:48.578497: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:20:48.578804: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-01-05 00:20:48.578832: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:20:49.004432: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-01-05 00:20:49.004533: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-01-05 00:20:49.004544: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-01-05 00:20:49.005569: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 42313 MB memory) -> physical GPU (device: 0, name: NVIDIA L40S, pci bus id: 0000:61:00.0, compute capability: 8.9)
2026-01-05 00:20:49.065719: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2026-01-05 00:20:49.082279: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500000000 Hz
2026-01-05 00:20:53.159100: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:20:53.685478: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:20:53.690702: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:20:55.327597: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-01-05 00:20:55.462446: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2026-01-05 00:21:19.244805: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)