Lmod Warning:
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: curl/8.4.0
(required by: htslib/1.16)
-------------------------------------------------------------------------------




The following have been reloaded with a version change:
  1) curl/8.4.0 => curl/8.17.0     2) openssl/3.0.7 => openssl/3.6.0

2026-05-11 02:49:35.185807: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 02:49:58.008633: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 02:49:58.010250: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-11 02:49:58.268297: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2026-05-11 02:49:58.268402: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 02:49:58.453122: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 02:49:58.453247: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 02:49:58.518179: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 02:49:58.772836: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 02:49:58.962186: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 02:49:59.084920: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 02:49:59.294866: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 02:49:59.303487: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 02:49:59.304075: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-11 02:49:59.304359: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 02:49:59.306837: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2026-05-11 02:49:59.306894: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 02:49:59.306920: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 02:49:59.306939: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 02:49:59.306955: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 02:49:59.306970: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 02:49:59.306985: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 02:49:59.307002: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 02:49:59.307029: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 02:49:59.311779: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 02:49:59.311824: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 02:50:02.167200: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-11 02:50:02.167380: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-11 02:50:02.167399: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-11 02:50:02.172569: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 75642 MB memory) -> physical GPU (device: 0, name: NVIDIA A100-SXM4-80GB, pci bus id: 0000:07:00.0, compute capability: 8.0)
2026-05-11 02:50:06.419597: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-05-11 02:50:06.420138: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2450225000 Hz
2026-05-11 02:50:08.889285: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 02:50:09.360493: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 02:50:09.366117: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 02:50:24.209644: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-11 02:50:24.300436: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2026-05-11 02:50:42.402880: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'jitters', 'index', 'status', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
2026-05-11 02:56:02.966161: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2026-05-11 02:57:34.107922: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:02:35.851871: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 03:02:35.854884: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-11 03:02:36.077341: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2026-05-11 03:02:36.077472: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:02:36.129477: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:02:36.129601: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:02:36.163023: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 03:02:36.189030: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 03:02:36.224211: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 03:02:36.257061: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 03:02:36.301193: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:02:36.307450: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 03:02:36.307834: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-11 03:02:36.308027: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 03:02:36.310323: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2026-05-11 03:02:36.310359: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:02:36.310380: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:02:36.310397: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:02:36.310413: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 03:02:36.310429: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 03:02:36.310444: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 03:02:36.310459: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 03:02:36.310475: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:02:36.314862: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 03:02:36.314901: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:02:36.776589: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-11 03:02:36.776742: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-11 03:02:36.776757: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-11 03:02:36.781475: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 75642 MB memory) -> physical GPU (device: 0, name: NVIDIA A100-SXM4-80GB, pci bus id: 0000:07:00.0, compute capability: 8.0)
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)

batch:   0%|          | 0/30 [00:00<?, ?it/s]2026-05-11 03:02:38.910638: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-05-11 03:02:38.911253: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2450225000 Hz
2026-05-11 03:02:39.263509: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:02:39.753563: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:02:39.755266: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:02:41.449421: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-11 03:02:41.546721: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'true_profiles', 'true_logcounts', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])

batch:   3%|▎         | 1/30 [02:05<1:00:48, 125.81s/it]
batch:   7%|▋         | 2/30 [02:06<24:15, 51.97s/it]   
batch:  10%|█         | 3/30 [02:06<12:45, 28.36s/it]
batch:  13%|█▎        | 4/30 [02:06<07:29, 17.27s/it]
batch:  17%|█▋        | 5/30 [02:06<04:38, 11.15s/it]
batch:  20%|██        | 6/30 [02:07<02:58,  7.45s/it]
batch:  23%|██▎       | 7/30 [02:07<01:57,  5.10s/it]
batch:  27%|██▋       | 8/30 [02:07<01:18,  3.57s/it]
batch:  30%|███       | 9/30 [02:08<00:53,  2.54s/it]
batch:  33%|███▎      | 10/30 [02:08<00:36,  1.84s/it]
batch:  37%|███▋      | 11/30 [02:08<00:25,  1.36s/it]
batch:  40%|████      | 12/30 [02:08<00:18,  1.03s/it]
batch:  43%|████▎     | 13/30 [02:09<00:13,  1.25it/s]
batch:  47%|████▋     | 14/30 [02:09<00:10,  1.56it/s]
batch:  50%|█████     | 15/30 [02:09<00:07,  1.89it/s]
batch:  53%|█████▎    | 16/30 [02:09<00:06,  2.21it/s]
batch:  57%|█████▋    | 17/30 [02:10<00:05,  2.52it/s]
batch:  60%|██████    | 18/30 [02:10<00:04,  2.78it/s]
batch:  63%|██████▎   | 19/30 [02:10<00:03,  3.01it/s]
batch:  67%|██████▋   | 20/30 [02:11<00:03,  3.18it/s]
batch:  70%|███████   | 21/30 [02:11<00:02,  3.34it/s]
batch:  73%|███████▎  | 22/30 [02:11<00:02,  3.45it/s]
batch:  77%|███████▋  | 23/30 [02:11<00:01,  3.54it/s]
batch:  80%|████████  | 24/30 [02:12<00:01,  3.62it/s]
batch:  83%|████████▎ | 25/30 [02:12<00:01,  3.63it/s]
batch:  87%|████████▋ | 26/30 [02:12<00:01,  3.67it/s]
batch:  90%|█████████ | 27/30 [02:12<00:00,  3.65it/s]
batch:  93%|█████████▎| 28/30 [02:13<00:00,  3.67it/s]
batch:  97%|█████████▋| 29/30 [02:13<00:00,  3.69it/s]
batch: 100%|██████████| 30/30 [02:13<00:00,  4.45s/it]

  0%|          | 0/1859 [00:00<?, ?it/s]
  6%|▌         | 113/1859 [00:00<00:01, 1128.96it/s]
 12%|█▏        | 226/1859 [00:00<00:01, 1115.63it/s]
 18%|█▊        | 340/1859 [00:00<00:01, 1119.57it/s]
 24%|██▍       | 452/1859 [00:00<00:01, 1105.03it/s]
 30%|███       | 563/1859 [00:00<00:01, 1099.22it/s]
 36%|███▌      | 673/1859 [00:00<00:01, 1089.66it/s]
 42%|████▏     | 782/1859 [00:00<00:00, 1079.20it/s]
 48%|████▊     | 890/1859 [00:00<00:00, 1074.34it/s]
 54%|█████▎    | 998/1859 [00:00<00:00, 1060.94it/s]
 59%|█████▉    | 1105/1859 [00:01<00:00, 1061.41it/s]
 65%|██████▌   | 1212/1859 [00:01<00:00, 1030.99it/s]
 71%|███████   | 1316/1859 [00:01<00:00, 1032.87it/s]
 76%|███████▋  | 1420/1859 [00:01<00:00, 1030.50it/s]
 82%|████████▏ | 1524/1859 [00:01<00:00, 1032.15it/s]
 88%|████████▊ | 1628/1859 [00:01<00:00, 1031.81it/s]
 93%|█████████▎| 1732/1859 [00:01<00:00, 1031.80it/s]
 99%|█████████▉| 1836/1859 [00:01<00:00, 1023.69it/s]
100%|██████████| 1859/1859 [00:01<00:00, 1054.95it/s]

  0%|          | 0/1859 [00:00<?, ?it/s]
  6%|▌         | 109/1859 [00:00<00:01, 1080.62it/s]
 12%|█▏        | 221/1859 [00:00<00:01, 1103.35it/s]
 18%|█▊        | 332/1859 [00:00<00:01, 1104.01it/s]
 24%|██▍       | 443/1859 [00:00<00:01, 1102.95it/s]
 30%|██▉       | 554/1859 [00:00<00:01, 1092.34it/s]
 36%|███▌      | 664/1859 [00:00<00:01, 1073.66it/s]
 42%|████▏     | 772/1859 [00:00<00:01, 1074.23it/s]
 47%|████▋     | 880/1859 [00:00<00:00, 1065.71it/s]
 53%|█████▎    | 987/1859 [00:00<00:00, 1063.24it/s]
 59%|█████▉    | 1094/1859 [00:01<00:00, 1055.37it/s]
 65%|██████▍   | 1200/1859 [00:01<00:00, 1047.79it/s]
 70%|███████   | 1305/1859 [00:01<00:00, 1047.86it/s]
 76%|███████▌  | 1410/1859 [00:01<00:00, 1031.62it/s]
 81%|████████▏ | 1514/1859 [00:01<00:00, 1018.17it/s]
 87%|████████▋ | 1616/1859 [00:01<00:00, 1015.66it/s]
 92%|█████████▏| 1718/1859 [00:01<00:00, 1014.24it/s]
 98%|█████████▊| 1820/1859 [00:01<00:00, 1005.35it/s]
100%|██████████| 1859/1859 [00:01<00:00, 1044.76it/s]
2026-05-11 03:10:35.117170: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:15:43.531704: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 03:15:43.533294: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-11 03:15:43.888658: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2026-05-11 03:15:43.888881: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:15:44.120989: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:15:44.121153: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:15:44.237571: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 03:15:44.357762: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 03:15:44.487688: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 03:15:44.619481: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 03:15:44.759504: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:15:44.764880: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 03:15:44.765548: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-11 03:15:44.765837: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 03:15:44.769981: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2026-05-11 03:15:44.770146: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:15:44.770185: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:15:44.770218: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:15:44.770244: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 03:15:44.770272: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 03:15:44.770300: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 03:15:44.770323: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 03:15:44.770346: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:15:44.775164: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 03:15:44.775279: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:15:45.219292: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-11 03:15:45.219449: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-11 03:15:45.219464: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-11 03:15:45.224661: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 75642 MB memory) -> physical GPU (device: 0, name: NVIDIA A100-SXM4-80GB, pci bus id: 0000:07:00.0, compute capability: 8.0)
2026-05-11 03:15:45.269808: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2026-05-11 03:15:45.281643: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2450225000 Hz
2026-05-11 03:16:06.540229: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:16:06.986119: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:16:06.992223: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:16:08.538939: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-11 03:16:08.631059: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2026-05-11 03:17:40.957913: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)