Lmod Warning:
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: curl/8.4.0
(required by: htslib/1.16)
-------------------------------------------------------------------------------




The following have been reloaded with a version change:
  1) curl/8.4.0 => curl/8.17.0     2) openssl/3.0.7 => openssl/3.6.0

2025-12-29 21:28:06.694784: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:28:18.307377: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-12-29 21:28:18.308774: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2025-12-29 21:28:18.335168: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2025-12-29 21:28:18.335218: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:28:18.340073: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:28:18.340139: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:28:18.342583: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-12-29 21:28:18.343969: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-12-29 21:28:18.347467: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-12-29 21:28:18.349202: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-12-29 21:28:18.350500: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:28:18.350876: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-12-29 21:28:18.351235: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-12-29 21:28:18.352789: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-12-29 21:28:18.353047: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2025-12-29 21:28:18.353069: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:28:18.353083: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:28:18.353093: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:28:18.353102: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-12-29 21:28:18.353111: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-12-29 21:28:18.353120: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-12-29 21:28:18.353129: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-12-29 21:28:18.353148: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:28:18.353431: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-12-29 21:28:18.353450: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:28:18.794682: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-12-29 21:28:18.794786: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2025-12-29 21:28:18.794798: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2025-12-29 21:28:18.795519: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 42313 MB memory) -> physical GPU (device: 0, name: NVIDIA L40S, pci bus id: 0000:61:00.0, compute capability: 8.9)
2025-12-29 21:28:33.097563: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2025-12-29 21:28:33.098167: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500000000 Hz
2025-12-29 21:28:34.256180: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:28:34.775610: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:28:34.782123: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:28:36.321224: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2025-12-29 21:28:36.444628: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2025-12-29 21:29:02.404128: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'jitters', 'index', 'status', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
2025-12-29 21:31:07.446217: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2025-12-29 21:31:11.483335: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:31:18.175819: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-12-29 21:31:18.176689: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2025-12-29 21:31:18.203446: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2025-12-29 21:31:18.203495: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:31:18.208425: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:31:18.208512: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:31:18.210845: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-12-29 21:31:18.212305: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-12-29 21:31:18.215918: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-12-29 21:31:18.217700: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-12-29 21:31:18.219107: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:31:18.219485: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-12-29 21:31:18.219730: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-12-29 21:31:18.221305: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-12-29 21:31:18.221561: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2025-12-29 21:31:18.221583: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:31:18.221597: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:31:18.221607: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:31:18.221617: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-12-29 21:31:18.221627: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-12-29 21:31:18.221637: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-12-29 21:31:18.221646: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-12-29 21:31:18.221656: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:31:18.221941: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-12-29 21:31:18.221963: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:31:18.637827: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-12-29 21:31:18.637927: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2025-12-29 21:31:18.637937: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2025-12-29 21:31:18.638632: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 42313 MB memory) -> physical GPU (device: 0, name: NVIDIA L40S, pci bus id: 0000:61:00.0, compute capability: 8.9)
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)

batch:   0%|          | 0/24 [00:00<?, ?it/s]2025-12-29 21:31:20.210871: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2025-12-29 21:31:20.211347: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500000000 Hz
2025-12-29 21:31:20.404005: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:31:20.897362: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:31:20.899487: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:31:22.506287: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2025-12-29 21:31:22.629092: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'true_profiles', 'true_logcounts', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])

batch:   4%|▍         | 1/24 [00:29<11:20, 29.59s/it]
batch:   8%|▊         | 2/24 [00:29<04:31, 12.34s/it]
batch:  12%|█▎        | 3/24 [00:30<02:23,  6.82s/it]
batch:  17%|█▋        | 4/24 [00:30<01:24,  4.22s/it]
batch:  21%|██        | 5/24 [00:30<00:53,  2.79s/it]
batch:  25%|██▌       | 6/24 [00:30<00:34,  1.92s/it]
batch:  29%|██▉       | 7/24 [00:31<00:23,  1.37s/it]
batch:  33%|███▎      | 8/24 [00:31<00:16,  1.02s/it]
batch:  38%|███▊      | 9/24 [00:31<00:11,  1.29it/s]
batch:  42%|████▏     | 10/24 [00:31<00:08,  1.64it/s]
batch:  46%|████▌     | 11/24 [00:32<00:06,  2.01it/s]
batch:  50%|█████     | 12/24 [00:32<00:05,  2.38it/s]
batch:  54%|█████▍    | 13/24 [00:32<00:04,  2.73it/s]
batch:  58%|█████▊    | 14/24 [00:32<00:03,  3.05it/s]
batch:  62%|██████▎   | 15/24 [00:33<00:02,  3.29it/s]
batch:  67%|██████▋   | 16/24 [00:33<00:02,  3.50it/s]
batch:  71%|███████   | 17/24 [00:33<00:01,  3.65it/s]
batch:  75%|███████▌  | 18/24 [00:33<00:01,  3.80it/s]
batch:  79%|███████▉  | 19/24 [00:34<00:01,  3.88it/s]
batch:  83%|████████▎ | 20/24 [00:34<00:01,  3.94it/s]
batch:  88%|████████▊ | 21/24 [00:34<00:00,  4.00it/s]
batch:  92%|█████████▏| 22/24 [00:34<00:00,  4.05it/s]
batch:  96%|█████████▌| 23/24 [00:34<00:00,  4.11it/s]
batch: 100%|██████████| 24/24 [00:35<00:00,  1.46s/it]

  0%|          | 0/1487 [00:00<?, ?it/s]
  8%|▊         | 121/1487 [00:00<00:01, 1191.42it/s]
 16%|█▌        | 241/1487 [00:00<00:01, 1191.06it/s]
 24%|██▍       | 361/1487 [00:00<00:00, 1179.46it/s]
 32%|███▏      | 479/1487 [00:00<00:00, 1166.89it/s]
 40%|████      | 596/1487 [00:00<00:00, 1156.93it/s]
 48%|████▊     | 712/1487 [00:00<00:00, 1142.67it/s]
 56%|█████▌    | 827/1487 [00:00<00:00, 1132.93it/s]
 63%|██████▎   | 941/1487 [00:00<00:00, 1124.27it/s]
 71%|███████   | 1054/1487 [00:00<00:00, 1107.87it/s]
 78%|███████▊  | 1165/1487 [00:01<00:00, 1098.95it/s]
 86%|████████▌ | 1275/1487 [00:01<00:00, 1089.51it/s]
 93%|█████████▎| 1384/1487 [00:01<00:00, 1069.81it/s]
100%|██████████| 1487/1487 [00:01<00:00, 1112.42it/s]

  0%|          | 0/1487 [00:00<?, ?it/s]
  8%|▊         | 121/1487 [00:00<00:01, 1191.15it/s]
 16%|█▌        | 241/1487 [00:00<00:01, 1192.10it/s]
 24%|██▍       | 361/1487 [00:00<00:00, 1179.96it/s]
 32%|███▏      | 480/1487 [00:00<00:00, 1169.61it/s]
 40%|████      | 597/1487 [00:00<00:00, 1153.01it/s]
 48%|████▊     | 713/1487 [00:00<00:00, 1145.91it/s]
 56%|█████▌    | 828/1487 [00:00<00:00, 1135.19it/s]
 63%|██████▎   | 942/1487 [00:00<00:00, 1119.67it/s]
 71%|███████   | 1054/1487 [00:00<00:00, 1108.88it/s]
 78%|███████▊  | 1165/1487 [00:01<00:00, 1099.87it/s]
 86%|████████▌ | 1275/1487 [00:01<00:00, 1089.52it/s]
 93%|█████████▎| 1384/1487 [00:01<00:00, 1075.52it/s]
100%|██████████| 1487/1487 [00:01<00:00, 1114.21it/s]
2025-12-29 21:32:06.885465: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:32:13.441553: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-12-29 21:32:13.449826: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2025-12-29 21:32:13.476654: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2025-12-29 21:32:13.476713: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:32:13.480958: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:32:13.481048: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:32:13.483144: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-12-29 21:32:13.484317: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-12-29 21:32:13.487469: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-12-29 21:32:13.488936: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-12-29 21:32:13.489953: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:32:13.490355: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-12-29 21:32:13.490695: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-12-29 21:32:13.492197: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-12-29 21:32:13.492466: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:61:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2025-12-29 21:32:13.492495: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:32:13.492510: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:32:13.492523: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:32:13.492540: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-12-29 21:32:13.492553: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-12-29 21:32:13.492580: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-12-29 21:32:13.492593: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-12-29 21:32:13.492606: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:32:13.492909: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-12-29 21:32:13.492935: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:32:13.911418: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-12-29 21:32:13.911520: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2025-12-29 21:32:13.911534: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2025-12-29 21:32:13.912268: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 42313 MB memory) -> physical GPU (device: 0, name: NVIDIA L40S, pci bus id: 0000:61:00.0, compute capability: 8.9)
2025-12-29 21:32:13.952136: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2025-12-29 21:32:13.965673: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500000000 Hz
2025-12-29 21:32:15.783565: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:32:16.297474: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:32:16.301819: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:32:17.900473: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2025-12-29 21:32:18.023896: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2025-12-29 21:32:40.914691: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)