Lmod Warning:
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: curl/8.4.0
(required by: htslib/1.16)
-------------------------------------------------------------------------------




The following have been reloaded with a version change:
  1) curl/8.4.0 => curl/8.17.0     2) openssl/3.0.7 => openssl/3.6.0

2026-05-11 03:36:38.110641: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:51:52.729389: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 03:51:52.732058: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-11 03:51:53.034470: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:8a:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2026-05-11 03:51:53.034590: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:51:53.154852: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:51:53.154998: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:51:53.196260: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 03:51:53.238173: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 03:51:53.282840: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 03:51:53.326378: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 03:51:53.370085: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:51:53.375312: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 03:51:53.375783: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-11 03:51:53.376119: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 03:51:53.380091: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:8a:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2026-05-11 03:51:53.380144: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:51:53.380177: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:51:53.380196: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:51:53.380213: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 03:51:53.380230: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 03:51:53.380246: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 03:51:53.380262: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 03:51:53.380298: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:51:53.385093: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 03:51:53.385177: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:51:53.910074: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-11 03:51:53.910169: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-11 03:51:53.910185: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-11 03:51:53.914014: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 75642 MB memory) -> physical GPU (device: 0, name: NVIDIA A100-SXM4-80GB, pci bus id: 0000:8a:00.0, compute capability: 8.0)
2026-05-11 03:51:57.296438: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-05-11 03:51:57.297060: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2450225000 Hz
2026-05-11 03:52:00.277512: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:52:00.689541: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:52:00.694046: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:53:10.705214: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-11 03:53:10.825685: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2026-05-11 03:54:43.567854: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'jitters', 'index', 'status', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
2026-05-11 04:04:41.281304: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2026-05-11 04:07:13.099681: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:14:45.566176: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 04:14:45.593232: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-11 04:14:45.816959: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:8a:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2026-05-11 04:14:45.817102: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:14:45.878874: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:14:45.879012: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:14:45.908510: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 04:14:45.937590: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 04:14:45.970389: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 04:14:46.000043: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 04:14:46.024081: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:14:46.030453: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 04:14:46.031040: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-11 04:14:46.031369: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 04:14:46.034035: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:8a:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2026-05-11 04:14:46.034106: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:14:46.034149: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:14:46.034181: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:14:46.034212: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 04:14:46.034241: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 04:14:46.034271: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 04:14:46.034302: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 04:14:46.034327: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:14:46.039147: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 04:14:46.039206: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:14:46.516425: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-11 04:14:46.516572: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-11 04:14:46.516586: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-11 04:14:46.522054: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 75642 MB memory) -> physical GPU (device: 0, name: NVIDIA A100-SXM4-80GB, pci bus id: 0000:8a:00.0, compute capability: 8.0)
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)

batch:   0%|          | 0/42 [00:00<?, ?it/s]2026-05-11 04:14:48.707672: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-05-11 04:14:48.708342: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2450225000 Hz
2026-05-11 04:14:49.157292: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:14:49.665073: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:14:49.667050: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:14:51.483910: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-11 04:14:51.621584: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'true_profiles', 'true_logcounts', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])

batch:   2%|▏         | 1/42 [01:56<1:19:38, 116.55s/it]
batch:   5%|▍         | 2/42 [01:56<32:06, 48.15s/it]   
batch:   7%|▋         | 3/42 [01:57<17:05, 26.29s/it]
batch:  10%|▉         | 4/42 [01:57<10:08, 16.02s/it]
batch:  12%|█▏        | 5/42 [01:57<06:22, 10.34s/it]
batch:  14%|█▍        | 6/42 [01:57<04:08,  6.91s/it]
batch:  17%|█▋        | 7/42 [01:58<02:45,  4.74s/it]
batch:  19%|█▉        | 8/42 [01:58<01:52,  3.32s/it]
batch:  21%|██▏       | 9/42 [01:58<01:18,  2.37s/it]
batch:  24%|██▍       | 10/42 [01:58<00:55,  1.72s/it]
batch:  26%|██▌       | 11/42 [01:59<00:39,  1.28s/it]
batch:  29%|██▊       | 12/42 [01:59<00:29,  1.03it/s]
batch:  31%|███       | 13/42 [01:59<00:22,  1.32it/s]
batch:  33%|███▎      | 14/42 [02:00<00:17,  1.64it/s]
batch:  36%|███▌      | 15/42 [02:00<00:13,  1.97it/s]
batch:  38%|███▊      | 16/42 [02:00<00:11,  2.29it/s]
batch:  40%|████      | 17/42 [02:00<00:09,  2.59it/s]
batch:  43%|████▎     | 18/42 [02:01<00:08,  2.85it/s]
batch:  45%|████▌     | 19/42 [02:01<00:07,  3.08it/s]
batch:  48%|████▊     | 20/42 [02:01<00:06,  3.23it/s]
batch:  50%|█████     | 21/42 [02:01<00:06,  3.36it/s]
batch:  52%|█████▏    | 22/42 [02:02<00:05,  3.44it/s]
batch:  55%|█████▍    | 23/42 [02:02<00:05,  3.51it/s]
batch:  57%|█████▋    | 24/42 [02:02<00:05,  3.58it/s]
batch:  60%|█████▉    | 25/42 [02:03<00:04,  3.61it/s]
batch:  62%|██████▏   | 26/42 [02:03<00:04,  3.64it/s]
batch:  64%|██████▍   | 27/42 [02:03<00:04,  3.66it/s]
batch:  67%|██████▋   | 28/42 [02:03<00:03,  3.68it/s]
batch:  69%|██████▉   | 29/42 [02:04<00:03,  3.68it/s]
batch:  71%|███████▏  | 30/42 [02:04<00:03,  3.71it/s]
batch:  74%|███████▍  | 31/42 [02:04<00:02,  3.70it/s]
batch:  76%|███████▌  | 32/42 [02:04<00:02,  3.69it/s]
batch:  79%|███████▊  | 33/42 [02:05<00:02,  3.70it/s]
batch:  81%|████████  | 34/42 [02:05<00:02,  3.70it/s]
batch:  83%|████████▎ | 35/42 [02:05<00:01,  3.70it/s]
batch:  86%|████████▌ | 36/42 [02:06<00:01,  3.71it/s]
batch:  88%|████████▊ | 37/42 [02:06<00:01,  3.70it/s]
batch:  90%|█████████ | 38/42 [02:06<00:01,  3.70it/s]
batch:  93%|█████████▎| 39/42 [02:06<00:00,  3.59it/s]
batch:  95%|█████████▌| 40/42 [02:07<00:00,  3.60it/s]
batch:  98%|█████████▊| 41/42 [02:07<00:00,  3.63it/s]
batch: 100%|██████████| 42/42 [02:07<00:00,  3.90it/s]
batch: 100%|██████████| 42/42 [02:07<00:00,  3.04s/it]

  0%|          | 0/2671 [00:00<?, ?it/s]
  4%|▍         | 113/2671 [00:00<00:02, 1126.49it/s]
  9%|▊         | 228/2671 [00:00<00:02, 1130.16it/s]
 13%|█▎        | 342/2671 [00:00<00:02, 1131.55it/s]
 17%|█▋        | 457/2671 [00:00<00:01, 1137.65it/s]
 21%|██▏       | 571/2671 [00:00<00:01, 1126.85it/s]
 26%|██▌       | 684/2671 [00:00<00:01, 1116.09it/s]
 30%|██▉       | 796/2671 [00:00<00:01, 1115.78it/s]
 34%|███▍      | 908/2671 [00:00<00:01, 1111.51it/s]
 38%|███▊      | 1020/2671 [00:00<00:01, 1110.83it/s]
 42%|████▏     | 1132/2671 [00:01<00:01, 1105.35it/s]
 47%|████▋     | 1243/2671 [00:01<00:01, 1099.86it/s]
 51%|█████     | 1353/2671 [00:01<00:01, 1085.63it/s]
 55%|█████▍    | 1464/2671 [00:01<00:01, 1092.23it/s]
 59%|█████▉    | 1574/2671 [00:01<00:01, 1093.56it/s]
 63%|██████▎   | 1684/2671 [00:01<00:00, 1089.89it/s]
 67%|██████▋   | 1794/2671 [00:01<00:00, 1085.14it/s]
 71%|███████   | 1903/2671 [00:01<00:00, 1082.45it/s]
 75%|███████▌  | 2012/2671 [00:01<00:00, 1073.20it/s]
 79%|███████▉  | 2120/2671 [00:01<00:00, 1058.65it/s]
 83%|████████▎ | 2226/2671 [00:02<00:00, 1057.73it/s]
 87%|████████▋ | 2333/2671 [00:02<00:00, 1055.80it/s]
 91%|█████████▏| 2439/2671 [00:02<00:00, 1050.85it/s]
 95%|█████████▌| 2545/2671 [00:02<00:00, 1037.06it/s]
 99%|█████████▉| 2649/2671 [00:02<00:00, 1036.56it/s]
100%|██████████| 2671/2671 [00:02<00:00, 1082.54it/s]

  0%|          | 0/2671 [00:00<?, ?it/s]
  4%|▍         | 110/2671 [00:00<00:02, 1080.05it/s]
  8%|▊         | 225/2671 [00:00<00:02, 1109.72it/s]
 13%|█▎        | 339/2671 [00:00<00:02, 1117.32it/s]
 17%|█▋        | 453/2671 [00:00<00:01, 1125.16it/s]
 21%|██        | 566/2671 [00:00<00:01, 1122.90it/s]
 25%|██▌       | 679/2671 [00:00<00:01, 1118.55it/s]
 30%|██▉       | 791/2671 [00:00<00:01, 1115.91it/s]
 34%|███▍      | 903/2671 [00:00<00:01, 1111.62it/s]
 38%|███▊      | 1015/2671 [00:00<00:01, 1109.04it/s]
 42%|████▏     | 1126/2671 [00:01<00:01, 1108.14it/s]
 46%|████▋     | 1237/2671 [00:01<00:01, 1096.87it/s]
 50%|█████     | 1347/2671 [00:01<00:01, 1090.45it/s]
 55%|█████▍    | 1458/2671 [00:01<00:01, 1095.73it/s]
 59%|█████▊    | 1568/2671 [00:01<00:01, 1096.34it/s]
 63%|██████▎   | 1678/2671 [00:01<00:00, 1090.01it/s]
 67%|██████▋   | 1788/2671 [00:01<00:00, 1085.90it/s]
 71%|███████   | 1897/2671 [00:01<00:00, 1074.61it/s]
 75%|███████▌  | 2005/2671 [00:01<00:00, 1065.45it/s]
 79%|███████▉  | 2112/2671 [00:01<00:00, 1057.37it/s]
 83%|████████▎ | 2218/2671 [00:02<00:00, 1057.06it/s]
 87%|████████▋ | 2324/2671 [00:02<00:00, 1053.67it/s]
 91%|█████████ | 2430/2671 [00:02<00:00, 1051.19it/s]
 95%|█████████▍| 2536/2671 [00:02<00:00, 1039.23it/s]
 99%|█████████▉| 2640/2671 [00:02<00:00, 1038.20it/s]
100%|██████████| 2671/2671 [00:02<00:00, 1080.67it/s]
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
Matplotlib is building the font cache; this may take a moment.
2026-05-11 04:21:52.300248: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:24:53.693438: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 04:24:53.703661: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-11 04:24:53.923388: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:8a:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2026-05-11 04:24:53.923541: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:24:53.969097: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:24:53.969228: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:24:53.991436: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 04:24:54.013158: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 04:24:54.034968: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 04:24:54.054088: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 04:24:54.073784: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:24:54.079404: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 04:24:54.080153: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-11 04:24:54.080457: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 04:24:54.082835: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:8a:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2026-05-11 04:24:54.082922: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:24:54.082974: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:24:54.083020: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:24:54.083062: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 04:24:54.083102: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 04:24:54.083142: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 04:24:54.083183: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 04:24:54.083224: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:24:54.087467: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 04:24:54.087511: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:24:54.542815: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-11 04:24:54.542953: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-11 04:24:54.542966: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-11 04:24:54.547998: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 75642 MB memory) -> physical GPU (device: 0, name: NVIDIA A100-SXM4-80GB, pci bus id: 0000:8a:00.0, compute capability: 8.0)
2026-05-11 04:24:54.588828: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2026-05-11 04:24:54.600084: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2450225000 Hz
2026-05-11 04:24:57.792938: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:24:58.201096: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:24:58.207479: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:24:59.965410: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-11 04:25:00.094849: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2026-05-11 04:25:47.362394: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)