Lmod Warning:
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: curl/8.4.0
(required by: htslib/1.16)
-------------------------------------------------------------------------------




The following have been reloaded with a version change:
  1) curl/8.4.0 => curl/8.17.0     2) openssl/3.0.7 => openssl/3.6.0

2026-05-11 02:49:42.959799: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 02:50:06.423241: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 02:50:06.424625: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-11 02:50:06.724249: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:c0:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2026-05-11 02:50:06.724356: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 02:50:06.751497: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 02:50:06.751677: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 02:50:06.755183: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 02:50:06.757512: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 02:50:06.761571: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 02:50:06.763849: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 02:50:06.766752: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 02:50:06.772676: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 02:50:06.773042: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-11 02:50:06.773262: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 02:50:06.775737: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:c0:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2026-05-11 02:50:06.775775: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 02:50:06.775801: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 02:50:06.775818: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 02:50:06.775834: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 02:50:06.775849: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 02:50:06.775864: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 02:50:06.775880: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 02:50:06.775916: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 02:50:06.780830: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 02:50:06.780883: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 02:50:07.288548: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-11 02:50:07.288646: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-11 02:50:07.288664: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-11 02:50:07.292685: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 75642 MB memory) -> physical GPU (device: 0, name: NVIDIA A100-SXM4-80GB, pci bus id: 0000:c0:00.0, compute capability: 8.0)
2026-05-11 02:50:09.214774: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-05-11 02:50:09.215385: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2450225000 Hz
2026-05-11 02:50:11.800100: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 02:50:12.223458: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 02:50:12.228536: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 02:50:24.204193: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-11 02:50:24.286678: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2026-05-11 02:52:03.128605: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'jitters', 'index', 'status', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
2026-05-11 02:57:32.945248: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2026-05-11 02:59:05.677299: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:04:12.773457: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 03:04:12.780202: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-11 03:04:13.096455: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:c0:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2026-05-11 03:04:13.096546: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:04:13.186208: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:04:13.186336: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:04:13.232307: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 03:04:13.279252: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 03:04:13.333809: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 03:04:13.381120: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 03:04:13.425929: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:04:13.431024: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 03:04:13.431493: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-11 03:04:13.431829: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 03:04:13.434456: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:c0:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2026-05-11 03:04:13.434569: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:04:13.434627: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:04:13.434668: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:04:13.434721: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 03:04:13.434762: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 03:04:13.434798: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 03:04:13.434834: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 03:04:13.434865: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:04:13.439559: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 03:04:13.439594: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:04:13.962434: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-11 03:04:13.962583: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-11 03:04:13.962601: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-11 03:04:13.966530: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 75642 MB memory) -> physical GPU (device: 0, name: NVIDIA A100-SXM4-80GB, pci bus id: 0000:c0:00.0, compute capability: 8.0)
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)

batch:   0%|          | 0/30 [00:00<?, ?it/s]2026-05-11 03:04:16.851156: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-05-11 03:04:16.851774: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2450225000 Hz
2026-05-11 03:04:17.953412: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:04:18.416178: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:04:18.417912: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:04:33.514493: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-11 03:04:34.114639: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'true_profiles', 'true_logcounts', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])

batch:   3%|▎         | 1/30 [02:57<1:25:46, 177.46s/it]
batch:   7%|▋         | 2/30 [02:57<34:10, 73.23s/it]   
batch:  10%|█         | 3/30 [02:58<17:58, 39.94s/it]
batch:  13%|█▎        | 4/30 [02:58<10:31, 24.28s/it]
batch:  17%|█▋        | 5/30 [02:58<06:30, 15.62s/it]
batch:  20%|██        | 6/30 [02:58<04:09, 10.40s/it]
batch:  23%|██▎       | 7/30 [02:59<02:42,  7.09s/it]
batch:  27%|██▋       | 8/30 [02:59<01:48,  4.91s/it]
batch:  30%|███       | 9/30 [02:59<01:12,  3.46s/it]
batch:  33%|███▎      | 10/30 [02:59<00:49,  2.47s/it]
batch:  37%|███▋      | 11/30 [03:00<00:34,  1.80s/it]
batch:  40%|████      | 12/30 [03:00<00:23,  1.33s/it]
batch:  43%|████▎     | 13/30 [03:00<00:17,  1.01s/it]
batch:  47%|████▋     | 14/30 [03:00<00:12,  1.28it/s]
batch:  50%|█████     | 15/30 [03:01<00:09,  1.60it/s]
batch:  53%|█████▎    | 16/30 [03:01<00:07,  1.95it/s]
batch:  57%|█████▋    | 17/30 [03:01<00:05,  2.28it/s]
batch:  60%|██████    | 18/30 [03:01<00:04,  2.59it/s]
batch:  63%|██████▎   | 19/30 [03:02<00:03,  2.86it/s]
batch:  67%|██████▋   | 20/30 [03:02<00:03,  3.10it/s]
batch:  70%|███████   | 21/30 [03:02<00:02,  3.29it/s]
batch:  73%|███████▎  | 22/30 [03:03<00:02,  3.43it/s]
batch:  77%|███████▋  | 23/30 [03:03<00:01,  3.54it/s]
batch:  80%|████████  | 24/30 [03:03<00:01,  3.61it/s]
batch:  83%|████████▎ | 25/30 [03:03<00:01,  3.67it/s]
batch:  87%|████████▋ | 26/30 [03:04<00:01,  3.72it/s]
batch:  90%|█████████ | 27/30 [03:04<00:00,  3.72it/s]
batch:  93%|█████████▎| 28/30 [03:04<00:00,  3.75it/s]
batch:  97%|█████████▋| 29/30 [03:04<00:00,  3.76it/s]
batch: 100%|██████████| 30/30 [03:04<00:00,  6.17s/it]

  0%|          | 0/1859 [00:00<?, ?it/s]
  6%|▌         | 115/1859 [00:00<00:01, 1139.14it/s]
 12%|█▏        | 229/1859 [00:00<00:01, 1124.35it/s]
 18%|█▊        | 343/1859 [00:00<00:01, 1126.15it/s]
 25%|██▍       | 456/1859 [00:00<00:01, 1117.18it/s]
 31%|███       | 568/1859 [00:00<00:01, 1100.37it/s]
 37%|███▋      | 679/1859 [00:00<00:01, 1091.86it/s]
 42%|████▏     | 789/1859 [00:00<00:00, 1091.46it/s]
 48%|████▊     | 899/1859 [00:00<00:00, 1084.53it/s]
 54%|█████▍    | 1009/1859 [00:00<00:00, 1085.63it/s]
 60%|██████    | 1118/1859 [00:01<00:00, 1080.70it/s]
 66%|██████▌   | 1227/1859 [00:01<00:00, 1077.21it/s]
 72%|███████▏  | 1335/1859 [00:01<00:00, 1071.89it/s]
 78%|███████▊  | 1443/1859 [00:01<00:00, 1057.47it/s]
 83%|████████▎ | 1549/1859 [00:01<00:00, 1048.86it/s]
 89%|████████▉ | 1654/1859 [00:01<00:00, 1047.87it/s]
 95%|█████████▍| 1759/1859 [00:01<00:00, 1042.90it/s]
100%|██████████| 1859/1859 [00:01<00:00, 1070.85it/s]

  0%|          | 0/1859 [00:00<?, ?it/s]
  6%|▌         | 109/1859 [00:00<00:01, 1083.20it/s]
 12%|█▏        | 222/1859 [00:00<00:01, 1110.57it/s]
 18%|█▊        | 334/1859 [00:00<00:01, 1111.68it/s]
 24%|██▍       | 446/1859 [00:00<00:01, 1107.09it/s]
 30%|██▉       | 557/1859 [00:00<00:01, 1100.05it/s]
 36%|███▌      | 668/1859 [00:00<00:01, 1092.84it/s]
 42%|████▏     | 778/1859 [00:00<00:00, 1093.83it/s]
 48%|████▊     | 888/1859 [00:00<00:00, 1084.69it/s]
 54%|█████▎    | 997/1859 [00:00<00:00, 1083.49it/s]
 59%|█████▉    | 1106/1859 [00:01<00:00, 1083.94it/s]
 65%|██████▌   | 1215/1859 [00:01<00:00, 1077.75it/s]
 71%|███████   | 1323/1859 [00:01<00:00, 1074.64it/s]
 77%|███████▋  | 1431/1859 [00:01<00:00, 1058.14it/s]
 83%|████████▎ | 1537/1859 [00:01<00:00, 1049.19it/s]
 88%|████████▊ | 1642/1859 [00:01<00:00, 1048.59it/s]
 94%|█████████▍| 1747/1859 [00:01<00:00, 1044.33it/s]
100%|█████████▉| 1852/1859 [00:01<00:00, 1035.64it/s]
100%|██████████| 1859/1859 [00:01<00:00, 1069.13it/s]
2026-05-11 03:12:46.176382: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:17:39.741530: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 03:17:39.747263: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-11 03:17:40.092075: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:c0:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2026-05-11 03:17:40.092232: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:17:40.188464: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:17:40.188601: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:17:40.208652: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 03:17:40.226403: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 03:17:40.247705: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 03:17:40.265777: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 03:17:40.288636: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:17:40.293918: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 03:17:40.294939: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-11 03:17:40.295308: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 03:17:40.297865: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:c0:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2026-05-11 03:17:40.297930: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:17:40.297968: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:17:40.298001: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:17:40.298031: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 03:17:40.298061: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 03:17:40.298091: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 03:17:40.298121: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 03:17:40.298150: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:17:40.304363: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 03:17:40.304436: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:17:40.761227: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-11 03:17:40.761377: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-11 03:17:40.761390: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-11 03:17:40.765286: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 75642 MB memory) -> physical GPU (device: 0, name: NVIDIA A100-SXM4-80GB, pci bus id: 0000:c0:00.0, compute capability: 8.0)
2026-05-11 03:17:40.808504: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2026-05-11 03:17:40.819765: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2450225000 Hz
2026-05-11 03:17:43.229058: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:17:43.662779: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:17:43.669130: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:17:45.255028: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-11 03:17:45.356080: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2026-05-11 03:20:05.554288: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)