Lmod Warning:
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: curl/8.4.0
(required by: htslib/1.16)
-------------------------------------------------------------------------------




The following have been reloaded with a version change:
  1) curl/8.4.0 => curl/8.17.0     2) openssl/3.0.7 => openssl/3.6.0

2026-05-28 11:41:17.567413: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:42:21.569908: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-28 11:42:21.571129: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-28 11:42:21.669868: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:4a:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-05-28 11:42:21.669950: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:42:22.232989: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-28 11:42:22.233068: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-28 11:42:22.654709: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-28 11:42:23.231238: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-28 11:42:23.801969: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-28 11:42:24.086883: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-28 11:42:24.247774: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-28 11:42:24.248502: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-28 11:42:24.248884: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-28 11:42:24.249038: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-28 11:42:24.249385: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:4a:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-05-28 11:42:24.249403: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:42:24.249417: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-28 11:42:24.249427: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-28 11:42:24.249437: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-28 11:42:24.249446: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-28 11:42:24.249456: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-28 11:42:24.249465: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-28 11:42:24.249495: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-28 11:42:24.250019: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-28 11:42:24.250041: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:42:26.684796: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-28 11:42:26.684903: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-28 11:42:26.684913: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-28 11:42:26.685555: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 42313 MB memory) -> physical GPU (device: 0, name: NVIDIA L40S, pci bus id: 0000:4a:00.0, compute capability: 8.9)
2026-05-28 11:42:28.426259: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-05-28 11:42:28.426802: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500000000 Hz
2026-05-28 11:42:30.469549: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-28 11:42:32.080347: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-28 11:42:32.085270: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-28 11:42:45.348806: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-28 11:42:45.419486: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2026-05-28 11:42:58.784608: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'jitters', 'index', 'status', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
2026-05-28 11:47:33.342607: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2026-05-28 11:47:38.020093: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:47:54.407951: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-28 11:47:54.408983: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-28 11:47:54.437316: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:4a:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-05-28 11:47:54.437410: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:47:54.448883: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-28 11:47:54.448993: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-28 11:47:54.453478: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-28 11:47:54.458298: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-28 11:47:54.464946: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-28 11:47:54.468889: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-28 11:47:54.472315: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-28 11:47:54.472759: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-28 11:47:54.473028: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-28 11:47:54.473185: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-28 11:47:54.473396: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:4a:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-05-28 11:47:54.473416: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:47:54.473430: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-28 11:47:54.473441: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-28 11:47:54.473451: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-28 11:47:54.473461: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-28 11:47:54.473471: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-28 11:47:54.473480: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-28 11:47:54.473491: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-28 11:47:54.473773: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-28 11:47:54.473794: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:47:54.908700: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-28 11:47:54.908812: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-28 11:47:54.908822: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-28 11:47:54.909490: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 42313 MB memory) -> physical GPU (device: 0, name: NVIDIA L40S, pci bus id: 0000:4a:00.0, compute capability: 8.9)
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)

batch:   0%|          | 0/32 [00:00<?, ?it/s]2026-05-28 11:47:56.558504: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-05-28 11:47:56.559030: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500000000 Hz
2026-05-28 11:47:56.781843: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-28 11:47:57.373671: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-28 11:47:57.375662: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-28 11:47:59.001325: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-28 11:47:59.091889: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'true_profiles', 'true_logcounts', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])

batch:   3%|▎         | 1/32 [00:21<10:52, 21.06s/it]
batch:   6%|▋         | 2/32 [00:21<04:24,  8.82s/it]
batch:   9%|▉         | 3/32 [00:21<02:22,  4.90s/it]
batch:  12%|█▎        | 4/32 [00:21<01:25,  3.06s/it]
batch:  16%|█▌        | 5/32 [00:22<00:55,  2.05s/it]
batch:  19%|█▉        | 6/32 [00:22<00:37,  1.43s/it]
batch:  22%|██▏       | 7/32 [00:22<00:26,  1.04s/it]
batch:  25%|██▌       | 8/32 [00:22<00:18,  1.27it/s]
batch:  28%|██▊       | 9/32 [00:22<00:14,  1.62it/s]
batch:  31%|███▏      | 10/32 [00:23<00:11,  2.00it/s]
batch:  34%|███▍      | 11/32 [00:23<00:08,  2.38it/s]
batch:  38%|███▊      | 12/32 [00:23<00:07,  2.73it/s]
batch:  41%|████      | 13/32 [00:23<00:06,  3.05it/s]
batch:  44%|████▍     | 14/32 [00:24<00:05,  3.32it/s]
batch:  47%|████▋     | 15/32 [00:24<00:04,  3.52it/s]
batch:  50%|█████     | 16/32 [00:24<00:04,  3.69it/s]
batch:  53%|█████▎    | 17/32 [00:24<00:03,  3.81it/s]
batch:  56%|█████▋    | 18/32 [00:25<00:03,  3.90it/s]
batch:  59%|█████▉    | 19/32 [00:25<00:03,  3.98it/s]
batch:  62%|██████▎   | 20/32 [00:25<00:02,  4.01it/s]
batch:  66%|██████▌   | 21/32 [00:25<00:02,  4.06it/s]
batch:  69%|██████▉   | 22/32 [00:26<00:02,  4.07it/s]
batch:  72%|███████▏  | 23/32 [00:26<00:02,  4.12it/s]
batch:  75%|███████▌  | 24/32 [00:26<00:01,  4.15it/s]
batch:  78%|███████▊  | 25/32 [00:26<00:01,  4.16it/s]
batch:  81%|████████▏ | 26/32 [00:27<00:01,  4.18it/s]
batch:  84%|████████▍ | 27/32 [00:27<00:01,  4.18it/s]
batch:  88%|████████▊ | 28/32 [00:27<00:00,  4.19it/s]
batch:  91%|█████████ | 29/32 [00:27<00:00,  4.19it/s]
batch:  94%|█████████▍| 30/32 [00:28<00:00,  4.19it/s]
batch: 100%|██████████| 32/32 [00:28<00:00,  1.14it/s]

  0%|          | 0/1924 [00:00<?, ?it/s]
  6%|▋         | 121/1924 [00:00<00:01, 1189.76it/s]
 12%|█▏        | 240/1924 [00:00<00:01, 1189.56it/s]
 19%|█▊        | 359/1924 [00:00<00:01, 1174.97it/s]
 25%|██▍       | 477/1924 [00:00<00:01, 1169.17it/s]
 31%|███       | 594/1924 [00:00<00:01, 1161.52it/s]
 37%|███▋      | 711/1924 [00:00<00:01, 1151.48it/s]
 43%|████▎     | 827/1924 [00:00<00:00, 1147.68it/s]
 49%|████▉     | 942/1924 [00:00<00:00, 1136.80it/s]
 55%|█████▍    | 1056/1924 [00:00<00:00, 1127.49it/s]
 61%|██████    | 1169/1924 [00:01<00:00, 1117.60it/s]
 67%|██████▋   | 1281/1924 [00:01<00:00, 1112.25it/s]
 72%|███████▏  | 1393/1924 [00:01<00:00, 1103.78it/s]
 78%|███████▊  | 1504/1924 [00:01<00:00, 1098.40it/s]
 84%|████████▍ | 1614/1924 [00:01<00:00, 1093.83it/s]
 90%|████████▉ | 1724/1924 [00:01<00:00, 1086.36it/s]
 95%|█████████▌| 1833/1924 [00:01<00:00, 1082.43it/s]
100%|██████████| 1924/1924 [00:01<00:00, 1115.82it/s]

  0%|          | 0/1924 [00:00<?, ?it/s]
  6%|▌         | 120/1924 [00:00<00:01, 1198.93it/s]
 12%|█▏        | 240/1924 [00:00<00:01, 1186.84it/s]
 19%|█▊        | 359/1924 [00:00<00:01, 1175.32it/s]
 25%|██▍       | 477/1924 [00:00<00:01, 1170.48it/s]
 31%|███       | 595/1924 [00:00<00:01, 1163.24it/s]
 37%|███▋      | 712/1924 [00:00<00:01, 1163.48it/s]
 43%|████▎     | 829/1924 [00:00<00:00, 1156.53it/s]
 49%|████▉     | 945/1924 [00:00<00:00, 1148.05it/s]
 55%|█████▌    | 1060/1924 [00:00<00:00, 1135.51it/s]
 61%|██████    | 1174/1924 [00:01<00:00, 1134.46it/s]
 67%|██████▋   | 1288/1924 [00:01<00:00, 1120.13it/s]
 73%|███████▎  | 1401/1924 [00:01<00:00, 1114.22it/s]
 79%|███████▊  | 1513/1924 [00:01<00:00, 1112.53it/s]
 84%|████████▍ | 1625/1924 [00:01<00:00, 1105.90it/s]
 90%|█████████ | 1736/1924 [00:01<00:00, 1099.79it/s]
 96%|█████████▌| 1846/1924 [00:01<00:00, 1092.42it/s]
100%|██████████| 1924/1924 [00:01<00:00, 1126.46it/s]
2026-05-28 11:48:38.400885: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:48:53.012749: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-28 11:48:53.014241: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-28 11:48:53.041257: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:4a:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-05-28 11:48:53.041334: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:48:53.046508: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-28 11:48:53.046599: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-28 11:48:53.049736: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-28 11:48:53.051477: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-28 11:48:53.054972: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-28 11:48:53.056696: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-28 11:48:53.058306: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-28 11:48:53.058771: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-28 11:48:53.059214: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-28 11:48:53.059385: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-28 11:48:53.059643: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:4a:00.0 name: NVIDIA L40S computeCapability: 8.9
coreClock: 2.52GHz coreCount: 142 deviceMemorySize: 44.52GiB deviceMemoryBandwidth: 804.75GiB/s
2026-05-28 11:48:53.059672: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:48:53.059687: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-28 11:48:53.059701: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-28 11:48:53.059714: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-28 11:48:53.059726: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-28 11:48:53.059739: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-28 11:48:53.059752: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-28 11:48:53.059765: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-28 11:48:53.060053: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-28 11:48:53.060079: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:48:53.501296: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-28 11:48:53.501404: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-28 11:48:53.501413: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-28 11:48:53.502118: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 42313 MB memory) -> physical GPU (device: 0, name: NVIDIA L40S, pci bus id: 0000:4a:00.0, compute capability: 8.9)
2026-05-28 11:48:53.543064: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2026-05-28 11:48:53.558794: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500000000 Hz
2026-05-28 11:48:55.731145: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-28 11:48:56.246920: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-28 11:48:56.250717: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-28 11:48:57.741324: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-28 11:48:57.831024: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2026-05-28 11:49:12.752521: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)