Lmod Warning:
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: curl/8.4.0
(required by: htslib/1.16)
-------------------------------------------------------------------------------




The following have been reloaded with a version change:
  1) curl/8.4.0 => curl/8.17.0     2) openssl/3.0.7 => openssl/3.6.0

2026-01-05 00:02:45.669378: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:05:04.470055: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-01-05 00:05:04.514464: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-01-05 00:05:04.599002: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-01-05 00:05:04.599052: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:05:05.687782: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:05:05.687848: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:05:05.989993: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-01-05 00:05:06.715593: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-01-05 00:05:07.467605: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-01-05 00:05:07.928660: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-01-05 00:05:08.028169: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:05:08.028568: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-01-05 00:05:08.029007: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-01-05 00:05:08.030623: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-01-05 00:05:08.030884: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-01-05 00:05:08.030906: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:05:08.030919: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:05:08.030929: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:05:08.030938: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-01-05 00:05:08.030947: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-01-05 00:05:08.030956: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-01-05 00:05:08.030964: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-01-05 00:05:08.030985: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:05:08.031272: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-01-05 00:05:08.031292: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:05:10.805862: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-01-05 00:05:10.805961: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-01-05 00:05:10.805973: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-01-05 00:05:10.806673: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 42313 MB memory) -> physical GPU (device: 0, name: NVIDIA L40S, pci bus id: 0000:61:00.0, compute capability: 8.9)
2026-01-05 00:05:24.028335: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-01-05 00:05:24.028913: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500000000 Hz
2026-01-05 00:05:25.164382: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:05:27.032794: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:05:27.037659: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:05:41.570060: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-01-05 00:05:41.651093: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2026-01-05 00:06:03.901205: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'jitters', 'index', 'status', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
2026-01-05 00:17:55.701222: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2026-01-05 00:17:59.836189: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:18:06.900154: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-01-05 00:18:06.900944: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-01-05 00:18:06.928352: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-01-05 00:18:06.928400: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:18:06.933246: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:18:06.933326: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:18:06.935586: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-01-05 00:18:06.937139: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-01-05 00:18:06.940630: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-01-05 00:18:06.942355: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-01-05 00:18:06.943649: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:18:06.944033: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-01-05 00:18:06.944262: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-01-05 00:18:06.945899: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-01-05 00:18:06.946140: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-01-05 00:18:06.946161: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:18:06.946174: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:18:06.946184: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:18:06.946195: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-01-05 00:18:06.946205: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-01-05 00:18:06.946214: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-01-05 00:18:06.946224: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-01-05 00:18:06.946234: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:18:06.946518: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-01-05 00:18:06.946540: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:18:07.358708: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-01-05 00:18:07.358797: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-01-05 00:18:07.358807: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-01-05 00:18:07.359481: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 42313 MB memory) -> physical GPU (device: 0, name: NVIDIA L40S, pci bus id: 0000:61:00.0, compute capability: 8.9)
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)

batch:   0%|          | 0/78 [00:00<?, ?it/s]2026-01-05 00:18:08.916769: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-01-05 00:18:08.917371: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500000000 Hz
2026-01-05 00:18:09.127173: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:18:09.650493: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:18:09.652813: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:18:11.233911: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-01-05 00:18:11.317079: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'true_profiles', 'true_logcounts', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])

batch:   1%|▏         | 1/78 [00:23<30:09, 23.50s/it]
batch:   3%|▎         | 2/78 [00:23<12:26,  9.83s/it]
batch:   4%|▍         | 3/78 [00:23<06:48,  5.45s/it]
batch:   5%|▌         | 4/78 [00:24<04:11,  3.39s/it]
batch:   6%|▋         | 5/78 [00:24<02:44,  2.26s/it]
batch:   8%|▊         | 6/78 [00:24<01:53,  1.57s/it]
batch:   9%|▉         | 7/78 [00:24<01:20,  1.13s/it]
batch:  10%|█         | 8/78 [00:25<00:59,  1.18it/s]
batch:  12%|█▏        | 9/78 [00:25<00:45,  1.52it/s]
batch:  13%|█▎        | 10/78 [00:25<00:35,  1.89it/s]
batch:  14%|█▍        | 11/78 [00:25<00:29,  2.28it/s]
batch:  15%|█▌        | 12/78 [00:26<00:24,  2.65it/s]
batch:  17%|█▋        | 13/78 [00:26<00:21,  2.97it/s]
batch:  18%|█▊        | 14/78 [00:26<00:19,  3.24it/s]
batch:  19%|█▉        | 15/78 [00:26<00:18,  3.46it/s]
batch:  21%|██        | 16/78 [00:27<00:16,  3.67it/s]
batch:  22%|██▏       | 17/78 [00:27<00:16,  3.80it/s]
batch:  23%|██▎       | 18/78 [00:27<00:15,  3.92it/s]
batch:  24%|██▍       | 19/78 [00:27<00:14,  3.98it/s]
batch:  26%|██▌       | 20/78 [00:28<00:14,  4.03it/s]
batch:  27%|██▋       | 21/78 [00:28<00:14,  4.07it/s]
batch:  28%|██▊       | 22/78 [00:28<00:13,  4.11it/s]
batch:  29%|██▉       | 23/78 [00:28<00:13,  4.11it/s]
batch:  31%|███       | 24/78 [00:29<00:13,  4.13it/s]
batch:  32%|███▏      | 25/78 [00:29<00:12,  4.16it/s]
batch:  33%|███▎      | 26/78 [00:29<00:12,  4.16it/s]
batch:  35%|███▍      | 27/78 [00:29<00:12,  4.16it/s]
batch:  36%|███▌      | 28/78 [00:29<00:11,  4.18it/s]
batch:  37%|███▋      | 29/78 [00:30<00:11,  4.16it/s]
batch:  38%|███▊      | 30/78 [00:30<00:11,  4.13it/s]
batch:  40%|███▉      | 31/78 [00:30<00:11,  4.15it/s]
batch:  41%|████      | 32/78 [00:30<00:11,  4.13it/s]
batch:  42%|████▏     | 33/78 [00:31<00:10,  4.15it/s]
batch:  44%|████▎     | 34/78 [00:31<00:10,  4.14it/s]
batch:  45%|████▍     | 35/78 [00:31<00:10,  4.14it/s]
batch:  46%|████▌     | 36/78 [00:31<00:10,  4.19it/s]
batch:  47%|████▋     | 37/78 [00:32<00:09,  4.20it/s]
batch:  49%|████▊     | 38/78 [00:32<00:09,  4.19it/s]
batch:  50%|█████     | 39/78 [00:32<00:09,  4.15it/s]
batch:  51%|█████▏    | 40/78 [00:32<00:09,  4.18it/s]
batch:  53%|█████▎    | 41/78 [00:33<00:08,  4.19it/s]
batch:  54%|█████▍    | 42/78 [00:33<00:08,  4.19it/s]
batch:  55%|█████▌    | 43/78 [00:33<00:08,  4.18it/s]
batch:  56%|█████▋    | 44/78 [00:33<00:08,  4.18it/s]
batch:  58%|█████▊    | 45/78 [00:34<00:07,  4.18it/s]
batch:  59%|█████▉    | 46/78 [00:34<00:07,  4.20it/s]
batch:  60%|██████    | 47/78 [00:34<00:07,  4.20it/s]
batch:  62%|██████▏   | 48/78 [00:34<00:07,  4.21it/s]
batch:  63%|██████▎   | 49/78 [00:35<00:06,  4.22it/s]
batch:  64%|██████▍   | 50/78 [00:35<00:06,  4.21it/s]
batch:  65%|██████▌   | 51/78 [00:35<00:06,  4.23it/s]
batch:  67%|██████▋   | 52/78 [00:35<00:06,  4.24it/s]
batch:  68%|██████▊   | 53/78 [00:35<00:05,  4.23it/s]
batch:  69%|██████▉   | 54/78 [00:36<00:05,  4.22it/s]
batch:  71%|███████   | 55/78 [00:36<00:05,  4.21it/s]
batch:  72%|███████▏  | 56/78 [00:36<00:05,  4.22it/s]
batch:  73%|███████▎  | 57/78 [00:36<00:05,  4.20it/s]
batch:  74%|███████▍  | 58/78 [00:37<00:04,  4.22it/s]
batch:  76%|███████▌  | 59/78 [00:37<00:04,  4.18it/s]
batch:  77%|███████▋  | 60/78 [00:37<00:04,  4.16it/s]
batch:  78%|███████▊  | 61/78 [00:37<00:04,  4.17it/s]
batch:  79%|███████▉  | 62/78 [00:38<00:03,  4.19it/s]
batch:  81%|████████  | 63/78 [00:38<00:03,  4.20it/s]
batch:  82%|████████▏ | 64/78 [00:38<00:03,  4.21it/s]
batch:  83%|████████▎ | 65/78 [00:38<00:03,  4.22it/s]
batch:  85%|████████▍ | 66/78 [00:39<00:03,  3.61it/s]
batch:  86%|████████▌ | 67/78 [00:39<00:02,  3.77it/s]
batch:  87%|████████▋ | 68/78 [00:39<00:02,  3.89it/s]
batch:  88%|████████▊ | 69/78 [00:39<00:02,  4.00it/s]
batch:  90%|████████▉ | 70/78 [00:40<00:01,  4.08it/s]
batch:  91%|█████████ | 71/78 [00:40<00:01,  4.11it/s]
batch:  92%|█████████▏| 72/78 [00:40<00:01,  4.13it/s]
batch:  94%|█████████▎| 73/78 [00:40<00:01,  4.17it/s]
batch:  95%|█████████▍| 74/78 [00:41<00:00,  4.18it/s]
batch:  96%|█████████▌| 75/78 [00:41<00:00,  4.18it/s]
batch:  97%|█████████▋| 76/78 [00:41<00:00,  5.04it/s]
batch:  99%|█████████▊| 77/78 [00:41<00:00,  4.83it/s]
batch: 100%|██████████| 78/78 [00:41<00:00,  4.66it/s]
batch: 100%|██████████| 78/78 [00:41<00:00,  1.86it/s]

  0%|          | 0/4945 [00:00<?, ?it/s]
  2%|▏         | 120/4945 [00:00<00:04, 1197.11it/s]
  5%|▍         | 241/4945 [00:00<00:03, 1198.22it/s]
  7%|▋         | 361/4945 [00:00<00:03, 1191.01it/s]
 10%|▉         | 483/4945 [00:00<00:03, 1196.15it/s]
 12%|█▏        | 603/4945 [00:00<00:03, 1195.84it/s]
 15%|█▍        | 723/4945 [00:00<00:03, 1188.94it/s]
 17%|█▋        | 842/4945 [00:00<00:03, 1186.06it/s]
 19%|█▉        | 961/4945 [00:00<00:03, 1187.19it/s]
 22%|██▏       | 1080/4945 [00:00<00:03, 1182.47it/s]
 24%|██▍       | 1199/4945 [00:01<00:03, 1179.87it/s]
 27%|██▋       | 1317/4945 [00:01<00:03, 1174.08it/s]
 29%|██▉       | 1435/4945 [00:01<00:03, 1166.07it/s]
 31%|███▏      | 1552/4945 [00:01<00:02, 1162.00it/s]
 34%|███▍      | 1669/4945 [00:01<00:02, 1154.98it/s]
 36%|███▌      | 1785/4945 [00:01<00:02, 1151.25it/s]
 38%|███▊      | 1901/4945 [00:01<00:02, 1138.82it/s]
 41%|████      | 2015/4945 [00:01<00:02, 1134.78it/s]
 43%|████▎     | 2129/4945 [00:01<00:02, 1118.96it/s]
 45%|████▌     | 2241/4945 [00:01<00:02, 1113.45it/s]
 48%|████▊     | 2355/4945 [00:02<00:02, 1120.80it/s]
 50%|████▉     | 2469/4945 [00:02<00:02, 1122.65it/s]
 52%|█████▏    | 2582/4945 [00:02<00:02, 1117.69it/s]
 54%|█████▍    | 2694/4945 [00:02<00:02, 1116.50it/s]
 57%|█████▋    | 2806/4945 [00:02<00:01, 1106.04it/s]
 59%|█████▉    | 2917/4945 [00:02<00:01, 1105.55it/s]
 61%|██████    | 3028/4945 [00:02<00:01, 1105.53it/s]
 63%|██████▎   | 3139/4945 [00:02<00:01, 1096.47it/s]
 66%|██████▌   | 3250/4945 [00:02<00:01, 1095.77it/s]
 68%|██████▊   | 3360/4945 [00:02<00:01, 1086.02it/s]
 70%|███████   | 3469/4945 [00:03<00:01, 1085.37it/s]
 72%|███████▏  | 3578/4945 [00:03<00:01, 1077.28it/s]
 75%|███████▍  | 3686/4945 [00:03<00:01, 1066.66it/s]
 77%|███████▋  | 3793/4945 [00:03<00:01, 1066.93it/s]
 79%|███████▉  | 3900/4945 [00:03<00:00, 1053.46it/s]
 81%|████████  | 4006/4945 [00:03<00:00, 1052.72it/s]
 83%|████████▎ | 4113/4945 [00:03<00:00, 1057.81it/s]
 85%|████████▌ | 4219/4945 [00:03<00:00, 1047.44it/s]
 88%|████████▊ | 4327/4945 [00:03<00:00, 1052.33it/s]
 90%|████████▉ | 4433/4945 [00:03<00:00, 1042.28it/s]
 92%|█████████▏| 4540/4945 [00:04<00:00, 1046.27it/s]
 94%|█████████▍| 4647/4945 [00:04<00:00, 1051.35it/s]
 96%|█████████▌| 4753/4945 [00:04<00:00, 1049.88it/s]
 98%|█████████▊| 4859/4945 [00:04<00:00, 1044.90it/s]
100%|██████████| 4945/4945 [00:04<00:00, 1108.50it/s]

  0%|          | 0/4945 [00:00<?, ?it/s]
  2%|▏         | 118/4945 [00:00<00:04, 1179.26it/s]
  5%|▍         | 238/4945 [00:00<00:03, 1185.30it/s]
  7%|▋         | 357/4945 [00:00<00:03, 1166.66it/s]
 10%|▉         | 477/4945 [00:00<00:03, 1174.01it/s]
 12%|█▏        | 595/4945 [00:00<00:03, 1169.68it/s]
 14%|█▍        | 712/4945 [00:00<00:03, 1165.75it/s]
 17%|█▋        | 829/4945 [00:00<00:03, 1155.04it/s]
 19%|█▉        | 947/4945 [00:00<00:03, 1159.52it/s]
 21%|██▏       | 1063/4945 [00:00<00:03, 1154.98it/s]
 24%|██▍       | 1179/4945 [00:01<00:03, 1155.60it/s]
 26%|██▌       | 1295/4945 [00:01<00:03, 1149.78it/s]
 29%|██▊       | 1410/4945 [00:01<00:03, 1142.29it/s]
 31%|███       | 1525/4945 [00:01<00:03, 1138.88it/s]
 33%|███▎      | 1640/4945 [00:01<00:02, 1142.12it/s]
 35%|███▌      | 1755/4945 [00:01<00:02, 1123.34it/s]
 38%|███▊      | 1868/4945 [00:01<00:02, 1108.47it/s]
 40%|████      | 1980/4945 [00:01<00:02, 1111.27it/s]
 42%|████▏     | 2092/4945 [00:01<00:02, 1109.00it/s]
 45%|████▍     | 2204/4945 [00:01<00:02, 1109.05it/s]
 47%|████▋     | 2315/4945 [00:02<00:02, 1105.61it/s]
 49%|████▉     | 2426/4945 [00:02<00:02, 1106.70it/s]
 51%|█████▏    | 2537/4945 [00:02<00:02, 1104.88it/s]
 54%|█████▎    | 2648/4945 [00:02<00:02, 1101.81it/s]
 56%|█████▌    | 2759/4945 [00:02<00:01, 1095.55it/s]
 58%|█████▊    | 2871/4945 [00:02<00:01, 1099.17it/s]
 60%|██████    | 2981/4945 [00:02<00:01, 1099.10it/s]
 63%|██████▎   | 3091/4945 [00:02<00:01, 1080.88it/s]
 65%|██████▍   | 3200/4945 [00:02<00:01, 1081.65it/s]
 67%|██████▋   | 3310/4945 [00:02<00:01, 1081.09it/s]
 69%|██████▉   | 3419/4945 [00:03<00:01, 1073.73it/s]
 71%|███████▏  | 3527/4945 [00:03<00:01, 1061.70it/s]
 74%|███████▎  | 3635/4945 [00:03<00:01, 1062.27it/s]
 76%|███████▌  | 3742/4945 [00:03<00:01, 1062.48it/s]
 78%|███████▊  | 3849/4945 [00:03<00:01, 1060.15it/s]
 80%|████████  | 3956/4945 [00:03<00:00, 1058.46it/s]
 82%|████████▏ | 4062/4945 [00:03<00:00, 1055.57it/s]
 84%|████████▍ | 4170/4945 [00:03<00:00, 1057.18it/s]
 86%|████████▋ | 4276/4945 [00:03<00:00, 1054.43it/s]
 89%|████████▊ | 4382/4945 [00:03<00:00, 1046.75it/s]
 91%|█████████ | 4487/4945 [00:04<00:00, 1045.34it/s]
 93%|█████████▎| 4593/4945 [00:04<00:00, 1044.66it/s]
 95%|█████████▌| 4700/4945 [00:04<00:00, 1049.07it/s]
 97%|█████████▋| 4805/4945 [00:04<00:00, 1037.20it/s]
 99%|█████████▉| 4911/4945 [00:04<00:00, 1038.00it/s]
100%|██████████| 4945/4945 [00:04<00:00, 1095.82it/s]
2026-01-05 00:19:17.597977: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:19:28.740768: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-01-05 00:19:28.741944: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-01-05 00:19:28.770850: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-01-05 00:19:28.770937: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:19:28.776921: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:19:28.777031: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:19:28.779822: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-01-05 00:19:28.781954: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-01-05 00:19:28.785947: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-01-05 00:19:28.788061: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-01-05 00:19:28.789982: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:19:28.790413: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-01-05 00:19:28.791106: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-01-05 00:19:28.792811: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-01-05 00:19:28.793122: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-01-05 00:19:28.793159: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:19:28.793175: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:19:28.793188: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:19:28.793201: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-01-05 00:19:28.793214: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-01-05 00:19:28.793227: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-01-05 00:19:28.793240: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-01-05 00:19:28.793252: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:19:28.793575: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-01-05 00:19:28.793603: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-01-05 00:19:29.222953: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-01-05 00:19:29.223047: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-01-05 00:19:29.223058: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-01-05 00:19:29.223849: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 42313 MB memory) -> physical GPU (device: 0, name: NVIDIA L40S, pci bus id: 0000:61:00.0, compute capability: 8.9)
2026-01-05 00:19:29.267182: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2026-01-05 00:19:29.282052: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500000000 Hz
2026-01-05 00:19:33.369561: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-01-05 00:19:33.903515: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-01-05 00:19:33.907887: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-01-05 00:19:35.397095: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-01-05 00:19:35.483372: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2026-01-05 00:19:52.140892: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)