Lmod Warning:
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: curl/8.4.0
(required by: htslib/1.16)
-------------------------------------------------------------------------------




The following have been reloaded with a version change:
  1) curl/8.4.0 => curl/8.17.0     2) openssl/3.0.7 => openssl/3.6.0

2025-12-29 21:51:06.572606: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:51:24.528387: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-12-29 21:51:24.530012: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2025-12-29 21:51:24.739857: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2025-12-29 21:51:24.739938: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:51:24.746935: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:51:24.747028: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:51:24.750641: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-12-29 21:51:24.753569: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-12-29 21:51:24.763495: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-12-29 21:51:24.766811: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-12-29 21:51:24.768743: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:51:24.773721: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-12-29 21:51:24.774339: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-12-29 21:51:24.777253: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-12-29 21:51:24.782569: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2025-12-29 21:51:24.782616: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:51:24.782644: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:51:24.782667: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:51:24.782688: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-12-29 21:51:24.782707: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-12-29 21:51:24.782727: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-12-29 21:51:24.782747: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-12-29 21:51:24.782788: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:51:24.789725: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-12-29 21:51:24.789764: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:51:25.309672: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-12-29 21:51:25.309778: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2025-12-29 21:51:25.309795: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2025-12-29 21:51:25.313676: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 75642 MB memory) -> physical GPU (device: 0, name: NVIDIA A100-SXM4-80GB, pci bus id: 0000:07:00.0, compute capability: 8.0)
2025-12-29 21:51:26.947817: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2025-12-29 21:51:26.948366: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2450135000 Hz
2025-12-29 21:51:28.808994: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:51:29.498209: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:51:29.506724: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:51:31.232802: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2025-12-29 21:51:31.432205: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2025-12-29 21:52:18.054925: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'jitters', 'index', 'status', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
2025-12-29 21:55:12.055666: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2025-12-29 21:55:17.049392: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:55:24.257838: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-12-29 21:55:24.259268: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2025-12-29 21:55:24.487007: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2025-12-29 21:55:24.487124: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:55:24.492282: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:55:24.492360: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:55:24.494761: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-12-29 21:55:24.496467: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-12-29 21:55:24.500607: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-12-29 21:55:24.502715: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-12-29 21:55:24.504485: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:55:24.509779: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-12-29 21:55:24.510351: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-12-29 21:55:24.513316: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-12-29 21:55:24.515434: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2025-12-29 21:55:24.515470: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:55:24.515491: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:55:24.515508: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:55:24.515523: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-12-29 21:55:24.515538: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-12-29 21:55:24.515552: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-12-29 21:55:24.515567: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-12-29 21:55:24.515581: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:55:24.519499: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-12-29 21:55:24.519540: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:55:25.015164: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-12-29 21:55:25.015287: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2025-12-29 21:55:25.015303: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2025-12-29 21:55:25.020197: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 75642 MB memory) -> physical GPU (device: 0, name: NVIDIA A100-SXM4-80GB, pci bus id: 0000:07:00.0, compute capability: 8.0)
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)

batch:   0%|          | 0/54 [00:00<?, ?it/s]2025-12-29 21:55:27.156402: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2025-12-29 21:55:27.156961: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2450135000 Hz
2025-12-29 21:55:27.411168: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:55:27.966381: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:55:27.968557: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:55:29.775682: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2025-12-29 21:55:30.010077: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'true_profiles', 'true_logcounts', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])

batch:   2%|▏         | 1/54 [00:47<42:08, 47.71s/it]
batch:   4%|▎         | 2/54 [00:48<17:11, 19.84s/it]
batch:   6%|▌         | 3/54 [00:48<09:17, 10.92s/it]
batch:   7%|▋         | 4/54 [00:48<05:36,  6.73s/it]
batch:   9%|▉         | 5/54 [00:48<03:36,  4.42s/it]
batch:  11%|█         | 6/54 [00:49<02:24,  3.02s/it]
batch:  13%|█▎        | 7/54 [00:49<01:40,  2.13s/it]
batch:  15%|█▍        | 8/54 [00:49<01:11,  1.55s/it]
batch:  17%|█▋        | 9/54 [00:50<00:52,  1.16s/it]
batch:  19%|█▊        | 10/54 [00:50<00:39,  1.11it/s]
batch:  20%|██        | 11/54 [00:50<00:30,  1.39it/s]
batch:  22%|██▏       | 12/54 [00:51<00:25,  1.68it/s]
batch:  24%|██▍       | 13/54 [00:51<00:20,  1.97it/s]
batch:  26%|██▌       | 14/54 [00:51<00:17,  2.23it/s]
batch:  28%|██▊       | 15/54 [00:52<00:15,  2.45it/s]
batch:  30%|██▉       | 16/54 [00:52<00:14,  2.65it/s]
batch:  31%|███▏      | 17/54 [00:52<00:13,  2.80it/s]
batch:  33%|███▎      | 18/54 [00:52<00:12,  2.92it/s]
batch:  35%|███▌      | 19/54 [00:53<00:11,  3.01it/s]
batch:  37%|███▋      | 20/54 [00:53<00:11,  3.05it/s]
batch:  39%|███▉      | 21/54 [00:53<00:10,  3.09it/s]
batch:  41%|████      | 22/54 [00:54<00:10,  3.11it/s]
batch:  43%|████▎     | 23/54 [00:54<00:09,  3.15it/s]
batch:  44%|████▍     | 24/54 [00:54<00:09,  3.16it/s]
batch:  46%|████▋     | 25/54 [00:55<00:09,  3.17it/s]
batch:  48%|████▊     | 26/54 [00:55<00:08,  3.20it/s]
batch:  50%|█████     | 27/54 [00:55<00:08,  3.21it/s]
batch:  52%|█████▏    | 28/54 [00:56<00:08,  3.23it/s]
batch:  54%|█████▎    | 29/54 [00:56<00:07,  3.23it/s]
batch:  56%|█████▌    | 30/54 [00:56<00:07,  3.23it/s]
batch:  57%|█████▋    | 31/54 [00:57<00:07,  3.23it/s]
batch:  59%|█████▉    | 32/54 [00:57<00:06,  3.22it/s]
batch:  61%|██████    | 33/54 [00:57<00:06,  3.21it/s]
batch:  63%|██████▎   | 34/54 [00:57<00:06,  3.20it/s]
batch:  65%|██████▍   | 35/54 [00:58<00:05,  3.22it/s]
batch:  67%|██████▋   | 36/54 [00:58<00:05,  3.23it/s]
batch:  69%|██████▊   | 37/54 [00:58<00:05,  3.21it/s]
batch:  70%|███████   | 38/54 [00:59<00:05,  3.20it/s]
batch:  72%|███████▏  | 39/54 [00:59<00:04,  3.21it/s]
batch:  74%|███████▍  | 40/54 [00:59<00:04,  3.21it/s]
batch:  76%|███████▌  | 41/54 [01:00<00:04,  3.20it/s]
batch:  78%|███████▊  | 42/54 [01:00<00:03,  3.18it/s]
batch:  80%|███████▉  | 43/54 [01:00<00:03,  3.20it/s]
batch:  81%|████████▏ | 44/54 [01:01<00:03,  3.22it/s]
batch:  83%|████████▎ | 45/54 [01:01<00:02,  3.22it/s]
batch:  85%|████████▌ | 46/54 [01:01<00:02,  3.21it/s]
batch:  87%|████████▋ | 47/54 [01:02<00:02,  3.19it/s]
batch:  89%|████████▉ | 48/54 [01:02<00:01,  3.19it/s]
batch:  91%|█████████ | 49/54 [01:02<00:01,  3.21it/s]
batch:  93%|█████████▎| 50/54 [01:02<00:01,  3.21it/s]
batch:  94%|█████████▍| 51/54 [01:03<00:00,  3.21it/s]
batch:  96%|█████████▋| 52/54 [01:03<00:00,  3.22it/s]
batch:  98%|█████████▊| 53/54 [01:03<00:00,  3.83it/s]
batch: 100%|██████████| 54/54 [01:03<00:00,  1.18s/it]

  0%|          | 0/3350 [00:00<?, ?it/s]
  3%|▎         | 95/3350 [00:00<00:03, 948.73it/s]
  6%|▌         | 192/3350 [00:00<00:03, 953.56it/s]
  9%|▊         | 288/3350 [00:00<00:03, 943.26it/s]
 12%|█▏        | 388/3350 [00:00<00:03, 964.28it/s]
 14%|█▍        | 485/3350 [00:00<00:02, 964.80it/s]
 17%|█▋        | 584/3350 [00:00<00:02, 972.51it/s]
 20%|██        | 682/3350 [00:00<00:02, 966.56it/s]
 23%|██▎       | 779/3350 [00:00<00:02, 965.90it/s]
 26%|██▌       | 876/3350 [00:00<00:02, 965.16it/s]
 29%|██▉       | 973/3350 [00:01<00:02, 955.92it/s]
 32%|███▏      | 1069/3350 [00:01<00:02, 950.80it/s]
 35%|███▍      | 1165/3350 [00:01<00:02, 948.87it/s]
 38%|███▊      | 1260/3350 [00:01<00:02, 941.37it/s]
 40%|████      | 1356/3350 [00:01<00:02, 944.66it/s]
 43%|████▎     | 1451/3350 [00:01<00:02, 941.50it/s]
 46%|████▌     | 1546/3350 [00:01<00:01, 937.21it/s]
 49%|████▉     | 1640/3350 [00:01<00:01, 928.56it/s]
 52%|█████▏    | 1733/3350 [00:01<00:01, 928.19it/s]
 55%|█████▍    | 1826/3350 [00:01<00:01, 927.12it/s]
 57%|█████▋    | 1919/3350 [00:02<00:01, 926.64it/s]
 60%|██████    | 2012/3350 [00:02<00:01, 917.99it/s]
 63%|██████▎   | 2104/3350 [00:02<00:01, 913.87it/s]
 66%|██████▌   | 2196/3350 [00:02<00:01, 909.37it/s]
 68%|██████▊   | 2288/3350 [00:02<00:01, 911.63it/s]
 71%|███████   | 2380/3350 [00:02<00:01, 909.50it/s]
 74%|███████▍  | 2471/3350 [00:02<00:00, 901.30it/s]
 76%|███████▋  | 2562/3350 [00:02<00:00, 899.07it/s]
 79%|███████▉  | 2652/3350 [00:02<00:00, 899.03it/s]
 82%|████████▏ | 2742/3350 [00:02<00:00, 897.00it/s]
 85%|████████▍ | 2832/3350 [00:03<00:00, 889.65it/s]
 87%|████████▋ | 2921/3350 [00:03<00:00, 881.77it/s]
 90%|████████▉ | 3010/3350 [00:03<00:00, 881.95it/s]
 93%|█████████▎| 3099/3350 [00:03<00:00, 881.07it/s]
 95%|█████████▌| 3188/3350 [00:03<00:00, 875.07it/s]
 98%|█████████▊| 3277/3350 [00:03<00:00, 874.17it/s]
100%|██████████| 3350/3350 [00:03<00:00, 919.56it/s]

  0%|          | 0/3350 [00:00<?, ?it/s]
  3%|▎         | 95/3350 [00:00<00:03, 949.76it/s]
  6%|▌         | 192/3350 [00:00<00:03, 952.05it/s]
  9%|▊         | 288/3350 [00:00<00:03, 952.52it/s]
 12%|█▏        | 389/3350 [00:00<00:03, 967.02it/s]
 15%|█▍        | 488/3350 [00:00<00:02, 969.84it/s]
 17%|█▋        | 586/3350 [00:00<00:02, 973.08it/s]
 20%|██        | 684/3350 [00:00<00:02, 971.73it/s]
 23%|██▎       | 782/3350 [00:00<00:02, 964.85it/s]
 26%|██▌       | 879/3350 [00:00<00:02, 965.04it/s]
 29%|██▉       | 976/3350 [00:01<00:02, 957.39it/s]
 32%|███▏      | 1072/3350 [00:01<00:02, 953.09it/s]
 35%|███▍      | 1168/3350 [00:01<00:02, 946.43it/s]
 38%|███▊      | 1263/3350 [00:01<00:02, 941.87it/s]
 41%|████      | 1359/3350 [00:01<00:02, 944.25it/s]
 43%|████▎     | 1454/3350 [00:01<00:02, 942.76it/s]
 46%|████▌     | 1549/3350 [00:01<00:01, 938.89it/s]
 49%|████▉     | 1643/3350 [00:01<00:01, 936.76it/s]
 52%|█████▏    | 1737/3350 [00:01<00:01, 931.19it/s]
 55%|█████▍    | 1831/3350 [00:01<00:01, 932.10it/s]
 57%|█████▋    | 1925/3350 [00:02<00:01, 932.10it/s]
 60%|██████    | 2019/3350 [00:02<00:01, 923.08it/s]
 63%|██████▎   | 2112/3350 [00:02<00:01, 914.99it/s]
 66%|██████▌   | 2204/3350 [00:02<00:01, 909.81it/s]
 69%|██████▊   | 2296/3350 [00:02<00:01, 912.02it/s]
 71%|███████▏  | 2388/3350 [00:02<00:01, 908.37it/s]
 74%|███████▍  | 2479/3350 [00:02<00:00, 906.14it/s]
 77%|███████▋  | 2570/3350 [00:02<00:00, 898.42it/s]
 79%|███████▉  | 2660/3350 [00:02<00:00, 898.80it/s]
 82%|████████▏ | 2750/3350 [00:02<00:00, 896.50it/s]
 85%|████████▍ | 2840/3350 [00:03<00:00, 896.10it/s]
 87%|████████▋ | 2930/3350 [00:03<00:00, 889.32it/s]
 90%|█████████ | 3019/3350 [00:03<00:00, 888.64it/s]
 93%|█████████▎| 3108/3350 [00:03<00:00, 885.97it/s]
 95%|█████████▌| 3197/3350 [00:03<00:00, 879.14it/s]
 98%|█████████▊| 3285/3350 [00:03<00:00, 875.90it/s]
100%|██████████| 3350/3350 [00:03<00:00, 922.26it/s]
2025-12-29 21:56:51.363243: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:56:58.911830: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-12-29 21:56:58.913568: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2025-12-29 21:56:59.151699: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2025-12-29 21:56:59.151811: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:56:59.157742: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:56:59.157825: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:56:59.160616: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-12-29 21:56:59.162343: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-12-29 21:56:59.166529: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-12-29 21:56:59.168758: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-12-29 21:56:59.170386: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:56:59.175645: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-12-29 21:56:59.176231: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-12-29 21:56:59.179190: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-12-29 21:56:59.181313: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2025-12-29 21:56:59.181381: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:56:59.181415: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:56:59.181442: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:56:59.181467: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-12-29 21:56:59.181492: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-12-29 21:56:59.181516: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-12-29 21:56:59.181540: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-12-29 21:56:59.181564: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:56:59.185586: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-12-29 21:56:59.185648: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:56:59.694928: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-12-29 21:56:59.695026: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2025-12-29 21:56:59.695042: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2025-12-29 21:56:59.699746: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 75642 MB memory) -> physical GPU (device: 0, name: NVIDIA A100-SXM4-80GB, pci bus id: 0000:07:00.0, compute capability: 8.0)
2025-12-29 21:56:59.747792: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2025-12-29 21:56:59.760250: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2450135000 Hz
2025-12-29 21:57:03.059135: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:57:03.566833: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:57:03.574106: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:57:05.262535: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2025-12-29 21:57:05.473625: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2025-12-29 21:57:44.884490: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)