Lmod Warning:
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: curl/8.4.0
(required by: htslib/1.16)
-------------------------------------------------------------------------------




The following have been reloaded with a version change:
  1) curl/8.4.0 => curl/8.17.0     2) openssl/3.0.7 => openssl/3.6.0

2025-12-29 20:58:21.070150: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 20:58:32.688084: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-12-29 20:58:32.689976: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2025-12-29 20:58:33.004877: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2025-12-29 20:58:33.005015: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 20:58:33.012061: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 20:58:33.012131: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 20:58:33.014242: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-12-29 20:58:33.015455: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-12-29 20:58:33.018737: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-12-29 20:58:33.020375: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-12-29 20:58:33.021548: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 20:58:33.027787: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-12-29 20:58:33.028332: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-12-29 20:58:33.031500: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-12-29 20:58:33.034302: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2025-12-29 20:58:33.034363: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 20:58:33.034403: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 20:58:33.034422: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 20:58:33.034441: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-12-29 20:58:33.034459: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-12-29 20:58:33.034477: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-12-29 20:58:33.034494: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-12-29 20:58:33.034528: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 20:58:33.039285: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-12-29 20:58:33.039325: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 20:58:33.563649: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-12-29 20:58:33.563738: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2025-12-29 20:58:33.563756: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2025-12-29 20:58:33.567709: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 75642 MB memory) -> physical GPU (device: 0, name: NVIDIA A100-SXM4-80GB, pci bus id: 0000:07:00.0, compute capability: 8.0)
2025-12-29 20:58:36.165351: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2025-12-29 20:58:36.166461: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2450135000 Hz
2025-12-29 20:58:37.697753: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 20:58:38.192296: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 20:58:38.198746: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 20:58:39.770334: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2025-12-29 20:58:39.982550: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2025-12-29 20:59:23.838158: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'jitters', 'index', 'status', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
2025-12-29 21:05:33.729340: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2025-12-29 21:05:38.274380: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:05:46.154354: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-12-29 21:05:46.155383: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2025-12-29 21:05:46.429500: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2025-12-29 21:05:46.429612: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:05:46.433807: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:05:46.433884: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:05:46.435774: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-12-29 21:05:46.437125: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-12-29 21:05:46.440934: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-12-29 21:05:46.442724: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-12-29 21:05:46.443995: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:05:46.448745: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-12-29 21:05:46.449199: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-12-29 21:05:46.452096: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-12-29 21:05:46.455350: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2025-12-29 21:05:46.455407: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:05:46.455439: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:05:46.455465: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:05:46.455489: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-12-29 21:05:46.455513: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-12-29 21:05:46.455536: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-12-29 21:05:46.455559: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-12-29 21:05:46.455583: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:05:46.459563: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-12-29 21:05:46.459602: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:05:46.946389: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-12-29 21:05:46.946648: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2025-12-29 21:05:46.946665: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2025-12-29 21:05:46.950778: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 75642 MB memory) -> physical GPU (device: 0, name: NVIDIA A100-SXM4-80GB, pci bus id: 0000:07:00.0, compute capability: 8.0)
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)

batch:   0%|          | 0/56 [00:00<?, ?it/s]2025-12-29 21:05:49.111570: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2025-12-29 21:05:49.112323: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2450135000 Hz
2025-12-29 21:05:49.373416: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:05:49.845543: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:05:49.847766: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:05:51.362043: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2025-12-29 21:05:51.533160: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'true_profiles', 'true_logcounts', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])

batch:   2%|▏         | 1/56 [00:53<49:05, 53.56s/it]
batch:   4%|▎         | 2/56 [00:53<20:01, 22.25s/it]
batch:   5%|▌         | 3/56 [00:54<10:48, 12.23s/it]
batch:   7%|▋         | 4/56 [00:54<06:31,  7.53s/it]
batch:   9%|▉         | 5/56 [00:54<04:11,  4.93s/it]
batch:  11%|█         | 6/56 [00:55<02:47,  3.36s/it]
batch:  12%|█▎        | 7/56 [00:55<01:55,  2.36s/it]
batch:  14%|█▍        | 8/56 [00:55<01:22,  1.71s/it]
batch:  16%|█▌        | 9/56 [00:56<01:00,  1.28s/it]
batch:  18%|█▊        | 10/56 [00:56<00:45,  1.02it/s]
batch:  20%|█▉        | 11/56 [00:56<00:34,  1.29it/s]
batch:  21%|██▏       | 12/56 [00:57<00:28,  1.56it/s]
batch:  23%|██▎       | 13/56 [00:57<00:23,  1.85it/s]
batch:  25%|██▌       | 14/56 [00:57<00:19,  2.11it/s]
batch:  27%|██▋       | 15/56 [00:58<00:17,  2.34it/s]
batch:  29%|██▊       | 16/56 [00:58<00:15,  2.55it/s]
batch:  30%|███       | 17/56 [00:58<00:14,  2.73it/s]
batch:  32%|███▏      | 18/56 [00:58<00:13,  2.88it/s]
batch:  34%|███▍      | 19/56 [00:59<00:12,  3.00it/s]
batch:  36%|███▌      | 20/56 [00:59<00:11,  3.06it/s]
batch:  38%|███▊      | 21/56 [00:59<00:11,  3.12it/s]
batch:  39%|███▉      | 22/56 [01:00<00:10,  3.16it/s]
batch:  41%|████      | 23/56 [01:00<00:10,  3.20it/s]
batch:  43%|████▎     | 24/56 [01:00<00:09,  3.23it/s]
batch:  45%|████▍     | 25/56 [01:01<00:09,  3.24it/s]
batch:  46%|████▋     | 26/56 [01:01<00:09,  3.25it/s]
batch:  48%|████▊     | 27/56 [01:01<00:08,  3.27it/s]
batch:  50%|█████     | 28/56 [01:01<00:08,  3.28it/s]
batch:  52%|█████▏    | 29/56 [01:02<00:08,  3.27it/s]
batch:  54%|█████▎    | 30/56 [01:02<00:08,  3.25it/s]
batch:  55%|█████▌    | 31/56 [01:02<00:07,  3.24it/s]
batch:  57%|█████▋    | 32/56 [01:03<00:07,  3.19it/s]
batch:  59%|█████▉    | 33/56 [01:03<00:07,  3.19it/s]
batch:  61%|██████    | 34/56 [01:03<00:06,  3.17it/s]
batch:  62%|██████▎   | 35/56 [01:04<00:06,  3.19it/s]
batch:  64%|██████▍   | 36/56 [01:04<00:06,  3.22it/s]
batch:  66%|██████▌   | 37/56 [01:04<00:05,  3.24it/s]
batch:  68%|██████▊   | 38/56 [01:05<00:05,  3.25it/s]
batch:  70%|██████▉   | 39/56 [01:05<00:05,  3.26it/s]
batch:  71%|███████▏  | 40/56 [01:05<00:04,  3.22it/s]
batch:  73%|███████▎  | 41/56 [01:06<00:04,  3.26it/s]
batch:  75%|███████▌  | 42/56 [01:06<00:04,  3.27it/s]
batch:  77%|███████▋  | 43/56 [01:06<00:03,  3.27it/s]
batch:  79%|███████▊  | 44/56 [01:06<00:03,  3.26it/s]
batch:  80%|████████  | 45/56 [01:07<00:03,  3.28it/s]
batch:  82%|████████▏ | 46/56 [01:07<00:03,  3.25it/s]
batch:  84%|████████▍ | 47/56 [01:07<00:02,  3.26it/s]
batch:  86%|████████▌ | 48/56 [01:08<00:02,  3.24it/s]
batch:  88%|████████▊ | 49/56 [01:08<00:02,  3.21it/s]
batch:  89%|████████▉ | 50/56 [01:08<00:01,  3.13it/s]
batch:  91%|█████████ | 51/56 [01:09<00:01,  3.19it/s]
batch:  93%|█████████▎| 52/56 [01:09<00:01,  3.19it/s]
batch:  95%|█████████▍| 53/56 [01:09<00:00,  3.26it/s]
batch:  96%|█████████▋| 54/56 [01:10<00:00,  3.30it/s]
batch:  98%|█████████▊| 55/56 [01:10<00:00,  3.25it/s]
batch: 100%|██████████| 56/56 [01:10<00:00,  3.43it/s]
batch: 100%|██████████| 56/56 [01:10<00:00,  1.26s/it]

  0%|          | 0/3568 [00:00<?, ?it/s]
  3%|▎         | 94/3568 [00:00<00:03, 937.41it/s]
  5%|▌         | 193/3568 [00:00<00:03, 956.27it/s]
  8%|▊         | 292/3568 [00:00<00:03, 970.60it/s]
 11%|█         | 390/3568 [00:00<00:03, 918.24it/s]
 14%|█▎        | 489/3568 [00:00<00:03, 941.99it/s]
 16%|█▋        | 587/3568 [00:00<00:03, 952.16it/s]
 19%|█▉        | 685/3568 [00:00<00:03, 960.86it/s]
 22%|██▏       | 782/3568 [00:00<00:02, 960.94it/s]
 25%|██▍       | 879/3568 [00:00<00:02, 955.60it/s]
 27%|██▋       | 976/3568 [00:01<00:02, 959.38it/s]
 30%|███       | 1073/3568 [00:01<00:02, 953.01it/s]
 33%|███▎      | 1169/3568 [00:01<00:02, 946.85it/s]
 35%|███▌      | 1264/3568 [00:01<00:02, 944.98it/s]
 38%|███▊      | 1359/3568 [00:01<00:02, 944.40it/s]
 41%|████      | 1454/3568 [00:01<00:02, 936.56it/s]
 43%|████▎     | 1548/3568 [00:01<00:02, 928.88it/s]
 46%|████▌     | 1643/3568 [00:01<00:02, 929.92it/s]
 49%|████▊     | 1737/3568 [00:01<00:02, 905.19it/s]
 51%|█████▏    | 1829/3568 [00:01<00:01, 909.11it/s]
 54%|█████▍    | 1922/3568 [00:02<00:01, 914.56it/s]
 57%|█████▋    | 2016/3568 [00:02<00:01, 921.25it/s]
 59%|█████▉    | 2109/3568 [00:02<00:01, 888.83it/s]
 62%|██████▏   | 2200/3568 [00:02<00:01, 894.72it/s]
 64%|██████▍   | 2291/3568 [00:02<00:01, 896.53it/s]
 67%|██████▋   | 2383/3568 [00:02<00:01, 903.10it/s]
 69%|██████▉   | 2475/3568 [00:02<00:01, 904.10it/s]
 72%|███████▏  | 2566/3568 [00:02<00:01, 866.44it/s]
 74%|███████▍  | 2658/3568 [00:02<00:01, 877.34it/s]
 77%|███████▋  | 2749/3568 [00:02<00:00, 885.23it/s]
 80%|███████▉  | 2839/3568 [00:03<00:00, 889.02it/s]
 82%|████████▏ | 2929/3568 [00:03<00:00, 890.01it/s]
 85%|████████▍ | 3020/3568 [00:03<00:00, 890.99it/s]
 87%|████████▋ | 3110/3568 [00:03<00:00, 880.31it/s]
 90%|████████▉ | 3199/3568 [00:03<00:00, 878.76it/s]
 92%|█████████▏| 3289/3568 [00:03<00:00, 880.52it/s]
 95%|█████████▍| 3378/3568 [00:03<00:00, 881.90it/s]
 97%|█████████▋| 3467/3568 [00:03<00:00, 880.14it/s]
100%|█████████▉| 3556/3568 [00:03<00:00, 880.68it/s]
100%|██████████| 3568/3568 [00:03<00:00, 911.55it/s]

  0%|          | 0/3568 [00:00<?, ?it/s]
  2%|▏         | 86/3568 [00:00<00:04, 845.51it/s]
  5%|▌         | 185/3568 [00:00<00:03, 928.91it/s]
  8%|▊         | 283/3568 [00:00<00:03, 951.79it/s]
 11%|█         | 382/3568 [00:00<00:03, 963.76it/s]
 13%|█▎        | 481/3568 [00:00<00:03, 971.17it/s]
 16%|█▌        | 579/3568 [00:00<00:03, 971.54it/s]
 19%|█▉        | 677/3568 [00:00<00:03, 929.22it/s]
 22%|██▏       | 776/3568 [00:00<00:02, 942.91it/s]
 24%|██▍       | 874/3568 [00:00<00:02, 952.79it/s]
 27%|██▋       | 972/3568 [00:01<00:02, 954.81it/s]
 30%|██▉       | 1068/3568 [00:01<00:02, 956.06it/s]
 33%|███▎      | 1164/3568 [00:01<00:02, 944.84it/s]
 35%|███▌      | 1260/3568 [00:01<00:02, 945.45it/s]
 38%|███▊      | 1355/3568 [00:01<00:02, 945.02it/s]
 41%|████      | 1450/3568 [00:01<00:02, 941.73it/s]
 43%|████▎     | 1545/3568 [00:01<00:02, 935.72it/s]
 46%|████▌     | 1640/3568 [00:01<00:02, 935.12it/s]
 49%|████▊     | 1734/3568 [00:01<00:01, 935.71it/s]
 51%|█████     | 1828/3568 [00:01<00:01, 931.75it/s]
 54%|█████▍    | 1922/3568 [00:02<00:01, 917.91it/s]
 56%|█████▋    | 2014/3568 [00:02<00:01, 898.03it/s]
 59%|█████▉    | 2107/3568 [00:02<00:01, 905.98it/s]
 62%|██████▏   | 2199/3568 [00:02<00:01, 908.95it/s]
 64%|██████▍   | 2291/3568 [00:02<00:01, 908.33it/s]
 67%|██████▋   | 2382/3568 [00:02<00:01, 878.59it/s]
 69%|██████▉   | 2475/3568 [00:02<00:01, 888.70it/s]
 72%|███████▏  | 2566/3568 [00:02<00:01, 894.83it/s]
 74%|███████▍  | 2658/3568 [00:02<00:01, 898.00it/s]
 77%|███████▋  | 2749/3568 [00:02<00:00, 900.07it/s]
 80%|███████▉  | 2840/3568 [00:03<00:00, 901.61it/s]
 82%|████████▏ | 2931/3568 [00:03<00:00, 900.80it/s]
 85%|████████▍ | 3022/3568 [00:03<00:00, 867.83it/s]
 87%|████████▋ | 3111/3568 [00:03<00:00, 872.10it/s]
 90%|████████▉ | 3201/3568 [00:03<00:00, 879.88it/s]
 92%|█████████▏| 3290/3568 [00:03<00:00, 879.30it/s]
 95%|█████████▍| 3379/3568 [00:03<00:00, 880.92it/s]
 97%|█████████▋| 3469/3568 [00:03<00:00, 880.78it/s]
100%|█████████▉| 3558/3568 [00:03<00:00, 879.95it/s]
100%|██████████| 3568/3568 [00:03<00:00, 913.71it/s]
2025-12-29 21:07:18.939484: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:07:25.249100: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-12-29 21:07:25.250921: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2025-12-29 21:07:25.478158: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2025-12-29 21:07:25.478275: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:07:25.482600: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:07:25.482664: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:07:25.484672: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-12-29 21:07:25.486281: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-12-29 21:07:25.490918: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-12-29 21:07:25.492874: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-12-29 21:07:25.494179: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:07:25.501372: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-12-29 21:07:25.502266: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-12-29 21:07:25.504675: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-12-29 21:07:25.507167: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:07:00.0 name: NVIDIA A100-SXM4-80GB computeCapability: 8.0
coreClock: 1.41GHz coreCount: 108 deviceMemorySize: 79.14GiB deviceMemoryBandwidth: 1.85TiB/s
2025-12-29 21:07:25.507211: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:07:25.507237: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:07:25.507259: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:07:25.507279: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-12-29 21:07:25.507299: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-12-29 21:07:25.507318: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-12-29 21:07:25.507338: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-12-29 21:07:25.507357: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:07:25.513198: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-12-29 21:07:25.513246: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-12-29 21:07:25.986447: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-12-29 21:07:25.986558: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2025-12-29 21:07:25.986574: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2025-12-29 21:07:25.990586: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 75642 MB memory) -> physical GPU (device: 0, name: NVIDIA A100-SXM4-80GB, pci bus id: 0000:07:00.0, compute capability: 8.0)
2025-12-29 21:07:26.040840: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2025-12-29 21:07:26.053932: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2450135000 Hz
2025-12-29 21:07:30.006335: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-12-29 21:07:30.524074: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-12-29 21:07:30.531108: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-12-29 21:07:32.145852: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2025-12-29 21:07:32.364821: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
2025-12-29 21:08:10.024649: I tensorflow/stream_executor/cuda/cuda_blas.cc:1838] TensorFloat-32 will be used for the matrix multiplication. This will only be logged once.
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)