Lmod Warning:
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: curl/8.4.0
(required by: htslib/1.16)
-------------------------------------------------------------------------------




The following have been reloaded with a version change:
  1) curl/8.4.0 => curl/8.17.0     2) openssl/3.0.7 => openssl/3.6.0

2025-11-17 16:46:03.095192: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:47:48.069091: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-11-17 16:47:48.083785: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2025-11-17 16:47:48.145040: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:89:00.0 name: Tesla V100S-PCIE-32GB computeCapability: 7.0
coreClock: 1.597GHz coreCount: 80 deviceMemorySize: 31.73GiB deviceMemoryBandwidth: 1.03TiB/s
2025-11-17 16:47:48.145144: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:47:48.702711: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:47:48.702947: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:47:49.112873: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-11-17 16:47:49.621056: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-11-17 16:47:50.152493: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-11-17 16:47:50.446000: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-11-17 16:47:50.576496: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:47:50.577704: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-11-17 16:47:50.578376: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-11-17 16:47:50.581569: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-11-17 16:47:50.582140: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:89:00.0 name: Tesla V100S-PCIE-32GB computeCapability: 7.0
coreClock: 1.597GHz coreCount: 80 deviceMemorySize: 31.73GiB deviceMemoryBandwidth: 1.03TiB/s
2025-11-17 16:47:50.582201: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:47:50.582262: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:47:50.582298: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:47:50.582331: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-11-17 16:47:50.582363: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-11-17 16:47:50.582395: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-11-17 16:47:50.582426: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-11-17 16:47:50.582483: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:47:50.583364: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-11-17 16:47:50.583434: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:47:53.010707: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-11-17 16:47:53.010805: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2025-11-17 16:47:53.010821: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2025-11-17 16:47:53.011934: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 30119 MB memory) -> physical GPU (device: 0, name: Tesla V100S-PCIE-32GB, pci bus id: 0000:89:00.0, compute capability: 7.0)
2025-11-17 16:48:06.101861: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2025-11-17 16:48:06.102528: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2499920000 Hz
2025-11-17 16:48:07.518192: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:48:09.096960: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:48:09.104021: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:48:22.737659: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2025-11-17 16:48:22.815674: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'jitters', 'index', 'status', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
2025-11-17 16:53:04.168362: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2025-11-17 16:53:09.338320: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:53:19.527745: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-11-17 16:53:19.528820: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2025-11-17 16:53:19.564310: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:89:00.0 name: Tesla V100S-PCIE-32GB computeCapability: 7.0
coreClock: 1.597GHz coreCount: 80 deviceMemorySize: 31.73GiB deviceMemoryBandwidth: 1.03TiB/s
2025-11-17 16:53:19.564415: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:53:19.569893: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:53:19.569967: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:53:19.572570: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-11-17 16:53:19.574149: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-11-17 16:53:19.578233: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-11-17 16:53:19.580383: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-11-17 16:53:19.581900: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:53:19.582746: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-11-17 16:53:19.583134: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-11-17 16:53:19.586198: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-11-17 16:53:19.586753: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:89:00.0 name: Tesla V100S-PCIE-32GB computeCapability: 7.0
coreClock: 1.597GHz coreCount: 80 deviceMemorySize: 31.73GiB deviceMemoryBandwidth: 1.03TiB/s
2025-11-17 16:53:19.586798: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:53:19.586825: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:53:19.586849: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:53:19.586872: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-11-17 16:53:19.586895: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-11-17 16:53:19.586918: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-11-17 16:53:19.586941: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-11-17 16:53:19.586964: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:53:19.587720: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-11-17 16:53:19.587776: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:53:20.072961: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-11-17 16:53:20.073114: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2025-11-17 16:53:20.073128: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2025-11-17 16:53:20.074419: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 30119 MB memory) -> physical GPU (device: 0, name: Tesla V100S-PCIE-32GB, pci bus id: 0000:89:00.0, compute capability: 7.0)
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)

batch:   0%|          | 0/58 [00:00<?, ?it/s]2025-11-17 16:53:21.793205: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2025-11-17 16:53:21.793727: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2499920000 Hz
2025-11-17 16:53:22.029264: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:53:22.300953: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:53:22.303103: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:53:23.481674: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2025-11-17 16:53:23.566990: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'true_profiles', 'true_logcounts', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])

batch:   2%|▏         | 1/58 [00:21<20:48, 21.91s/it]
batch:   3%|▎         | 2/58 [00:22<08:35,  9.21s/it]
batch:   5%|▌         | 3/58 [00:22<04:43,  5.15s/it]
batch:   7%|▋         | 4/58 [00:22<02:54,  3.24s/it]
batch:   9%|▊         | 5/58 [00:23<01:55,  2.18s/it]
batch:  10%|█         | 6/58 [00:23<01:20,  1.54s/it]
batch:  12%|█▏        | 7/58 [00:23<00:58,  1.14s/it]
batch:  14%|█▍        | 8/58 [00:24<00:43,  1.14it/s]
batch:  16%|█▌        | 9/58 [00:24<00:34,  1.43it/s]
batch:  17%|█▋        | 10/58 [00:24<00:27,  1.73it/s]
batch:  19%|█▉        | 11/58 [00:25<00:23,  2.02it/s]
batch:  21%|██        | 12/58 [00:25<00:20,  2.29it/s]
batch:  22%|██▏       | 13/58 [00:25<00:17,  2.52it/s]
batch:  24%|██▍       | 14/58 [00:25<00:16,  2.72it/s]
batch:  26%|██▌       | 15/58 [00:26<00:15,  2.86it/s]
batch:  28%|██▊       | 16/58 [00:26<00:14,  2.98it/s]
batch:  29%|██▉       | 17/58 [00:26<00:13,  3.07it/s]
batch:  31%|███       | 18/58 [00:27<00:12,  3.14it/s]
batch:  33%|███▎      | 19/58 [00:27<00:12,  3.17it/s]
batch:  34%|███▍      | 20/58 [00:27<00:11,  3.21it/s]
batch:  36%|███▌      | 21/58 [00:28<00:11,  3.22it/s]
batch:  38%|███▊      | 22/58 [00:28<00:11,  3.24it/s]
batch:  40%|███▉      | 23/58 [00:28<00:10,  3.25it/s]
batch:  41%|████▏     | 24/58 [00:28<00:10,  3.26it/s]
batch:  43%|████▎     | 25/58 [00:29<00:10,  3.26it/s]
batch:  45%|████▍     | 26/58 [00:29<00:09,  3.26it/s]
batch:  47%|████▋     | 27/58 [00:29<00:09,  3.25it/s]
batch:  48%|████▊     | 28/58 [00:30<00:09,  3.23it/s]
batch:  50%|█████     | 29/58 [00:30<00:08,  3.24it/s]
batch:  52%|█████▏    | 30/58 [00:30<00:08,  3.24it/s]
batch:  53%|█████▎    | 31/58 [00:31<00:08,  3.23it/s]
batch:  55%|█████▌    | 32/58 [00:31<00:08,  3.24it/s]
batch:  57%|█████▋    | 33/58 [00:31<00:07,  3.24it/s]
batch:  59%|█████▊    | 34/58 [00:32<00:07,  3.26it/s]
batch:  60%|██████    | 35/58 [00:32<00:07,  3.25it/s]
batch:  62%|██████▏   | 36/58 [00:32<00:06,  3.26it/s]
batch:  64%|██████▍   | 37/58 [00:32<00:06,  3.27it/s]
batch:  66%|██████▌   | 38/58 [00:33<00:06,  3.26it/s]
batch:  67%|██████▋   | 39/58 [00:33<00:05,  3.26it/s]
batch:  69%|██████▉   | 40/58 [00:33<00:05,  3.26it/s]
batch:  71%|███████   | 41/58 [00:34<00:05,  3.27it/s]
batch:  72%|███████▏  | 42/58 [00:34<00:04,  3.25it/s]
batch:  74%|███████▍  | 43/58 [00:34<00:04,  3.25it/s]
batch:  76%|███████▌  | 44/58 [00:35<00:04,  3.25it/s]
batch:  78%|███████▊  | 45/58 [00:35<00:03,  3.26it/s]
batch:  79%|███████▉  | 46/58 [00:35<00:03,  3.26it/s]
batch:  81%|████████  | 47/58 [00:36<00:03,  3.27it/s]
batch:  83%|████████▎ | 48/58 [00:36<00:03,  3.27it/s]
batch:  84%|████████▍ | 49/58 [00:36<00:02,  3.28it/s]
batch:  86%|████████▌ | 50/58 [00:36<00:02,  3.28it/s]
batch:  88%|████████▊ | 51/58 [00:37<00:02,  3.27it/s]
batch:  90%|████████▉ | 52/58 [00:37<00:01,  3.28it/s]
batch:  91%|█████████▏| 53/58 [00:37<00:01,  3.30it/s]
batch:  93%|█████████▎| 54/58 [00:38<00:01,  3.30it/s]
batch:  95%|█████████▍| 55/58 [00:38<00:00,  3.30it/s]
batch:  97%|█████████▋| 56/58 [00:38<00:00,  3.30it/s]
batch:  98%|█████████▊| 57/58 [00:38<00:00,  4.10it/s]
batch: 100%|██████████| 58/58 [00:39<00:00,  3.87it/s]
batch: 100%|██████████| 58/58 [00:39<00:00,  1.48it/s]

  0%|          | 0/3656 [00:00<?, ?it/s]
  3%|▎         | 97/3656 [00:00<00:03, 954.47it/s]
  5%|▌         | 197/3656 [00:00<00:03, 968.73it/s]
  8%|▊         | 297/3656 [00:00<00:03, 981.38it/s]
 11%|█         | 396/3656 [00:00<00:03, 982.82it/s]
 14%|█▎        | 496/3656 [00:00<00:03, 983.66it/s]
 16%|█▋        | 596/3656 [00:00<00:03, 983.20it/s]
 19%|█▉        | 695/3656 [00:00<00:03, 981.48it/s]
 22%|██▏       | 794/3656 [00:00<00:02, 978.49it/s]
 24%|██▍       | 892/3656 [00:00<00:02, 977.00it/s]
 27%|██▋       | 990/3656 [00:01<00:02, 974.18it/s]
 30%|██▉       | 1088/3656 [00:01<00:02, 969.74it/s]
 32%|███▏      | 1185/3656 [00:01<00:02, 968.28it/s]
 35%|███▌      | 1282/3656 [00:01<00:02, 964.42it/s]
 38%|███▊      | 1379/3656 [00:01<00:02, 961.75it/s]
 40%|████      | 1476/3656 [00:01<00:02, 954.83it/s]
 43%|████▎     | 1572/3656 [00:01<00:02, 953.39it/s]
 46%|████▌     | 1668/3656 [00:01<00:02, 948.73it/s]
 48%|████▊     | 1763/3656 [00:01<00:01, 947.07it/s]
 51%|█████     | 1858/3656 [00:01<00:01, 941.82it/s]
 53%|█████▎    | 1953/3656 [00:02<00:01, 940.93it/s]
 56%|█████▌    | 2048/3656 [00:02<00:01, 942.76it/s]
 59%|█████▊    | 2143/3656 [00:02<00:01, 938.43it/s]
 61%|██████    | 2238/3656 [00:02<00:01, 940.96it/s]
 64%|██████▍   | 2333/3656 [00:02<00:01, 933.85it/s]
 66%|██████▋   | 2427/3656 [00:02<00:01, 930.69it/s]
 69%|██████▉   | 2521/3656 [00:02<00:01, 925.11it/s]
 71%|███████▏  | 2614/3656 [00:02<00:01, 923.23it/s]
 74%|███████▍  | 2707/3656 [00:02<00:01, 918.50it/s]
 77%|███████▋  | 2799/3656 [00:02<00:00, 914.38it/s]
 79%|███████▉  | 2891/3656 [00:03<00:00, 906.26it/s]
 82%|████████▏ | 2982/3656 [00:03<00:00, 899.34it/s]
 84%|████████▍ | 3072/3656 [00:03<00:00, 897.38it/s]
 86%|████████▋ | 3162/3656 [00:03<00:00, 889.03it/s]
 89%|████████▉ | 3251/3656 [00:03<00:00, 879.52it/s]
 91%|█████████▏| 3339/3656 [00:03<00:00, 875.12it/s]
 94%|█████████▎| 3427/3656 [00:03<00:00, 874.54it/s]
 96%|█████████▌| 3515/3656 [00:03<00:00, 871.24it/s]
 99%|█████████▊| 3603/3656 [00:03<00:00, 868.28it/s]
100%|██████████| 3656/3656 [00:03<00:00, 930.07it/s]

  0%|          | 0/3656 [00:00<?, ?it/s]
  3%|▎         | 97/3656 [00:00<00:03, 960.92it/s]
  5%|▌         | 199/3656 [00:00<00:03, 983.19it/s]
  8%|▊         | 299/3656 [00:00<00:03, 990.21it/s]
 11%|█         | 399/3656 [00:00<00:03, 991.30it/s]
 14%|█▎        | 499/3656 [00:00<00:03, 988.86it/s]
 16%|█▋        | 599/3656 [00:00<00:03, 986.28it/s]
 19%|█▉        | 698/3656 [00:00<00:03, 969.13it/s]
 22%|██▏       | 796/3656 [00:00<00:02, 968.42it/s]
 24%|██▍       | 894/3656 [00:00<00:02, 970.93it/s]
 27%|██▋       | 992/3656 [00:01<00:02, 970.26it/s]
 30%|██▉       | 1090/3656 [00:01<00:02, 972.29it/s]
 32%|███▏      | 1188/3656 [00:01<00:02, 966.50it/s]
 35%|███▌      | 1285/3656 [00:01<00:02, 963.29it/s]
 38%|███▊      | 1382/3656 [00:01<00:02, 955.53it/s]
 40%|████      | 1478/3656 [00:01<00:02, 953.38it/s]
 43%|████▎     | 1574/3656 [00:01<00:02, 950.14it/s]
 46%|████▌     | 1670/3656 [00:01<00:02, 945.35it/s]
 48%|████▊     | 1765/3656 [00:01<00:02, 942.68it/s]
 51%|█████     | 1860/3656 [00:01<00:01, 937.25it/s]
 53%|█████▎    | 1954/3656 [00:02<00:01, 935.01it/s]
 56%|█████▌    | 2048/3656 [00:02<00:01, 935.03it/s]
 59%|█████▊    | 2142/3656 [00:02<00:01, 934.49it/s]
 61%|██████    | 2236/3656 [00:02<00:01, 934.64it/s]
 64%|██████▎   | 2330/3656 [00:02<00:01, 925.84it/s]
 66%|██████▋   | 2423/3656 [00:02<00:01, 925.75it/s]
 69%|██████▉   | 2516/3656 [00:02<00:01, 918.12it/s]
 71%|███████▏  | 2609/3656 [00:02<00:01, 916.43it/s]
 74%|███████▍  | 2701/3656 [00:02<00:01, 915.77it/s]
 76%|███████▋  | 2793/3656 [00:02<00:00, 910.77it/s]
 79%|███████▉  | 2885/3656 [00:03<00:00, 906.65it/s]
 81%|████████▏ | 2976/3656 [00:03<00:00, 899.82it/s]
 84%|████████▍ | 3066/3656 [00:03<00:00, 893.03it/s]
 86%|████████▋ | 3156/3656 [00:03<00:00, 885.46it/s]
 89%|████████▉ | 3245/3656 [00:03<00:00, 876.67it/s]
 91%|█████████ | 3333/3656 [00:03<00:00, 872.88it/s]
 94%|█████████▎| 3421/3656 [00:03<00:00, 873.03it/s]
 96%|█████████▌| 3509/3656 [00:03<00:00, 870.14it/s]
 98%|█████████▊| 3597/3656 [00:03<00:00, 867.65it/s]
100%|██████████| 3656/3656 [00:03<00:00, 927.53it/s]
2025-11-17 16:54:38.914293: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:55:08.399147: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-11-17 16:55:08.400081: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2025-11-17 16:55:08.454954: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:89:00.0 name: Tesla V100S-PCIE-32GB computeCapability: 7.0
coreClock: 1.597GHz coreCount: 80 deviceMemorySize: 31.73GiB deviceMemoryBandwidth: 1.03TiB/s
2025-11-17 16:55:08.455081: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:55:08.472998: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:55:08.473119: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:55:08.478291: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-11-17 16:55:08.483215: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-11-17 16:55:08.491439: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-11-17 16:55:08.496249: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-11-17 16:55:08.500193: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:55:08.501079: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-11-17 16:55:08.501614: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-11-17 16:55:08.504191: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-11-17 16:55:08.504686: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:89:00.0 name: Tesla V100S-PCIE-32GB computeCapability: 7.0
coreClock: 1.597GHz coreCount: 80 deviceMemorySize: 31.73GiB deviceMemoryBandwidth: 1.03TiB/s
2025-11-17 16:55:08.504744: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:55:08.504779: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:55:08.504811: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:55:08.504842: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-11-17 16:55:08.504872: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-11-17 16:55:08.504902: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-11-17 16:55:08.504932: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-11-17 16:55:08.504962: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:55:08.505700: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-11-17 16:55:08.505774: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:55:08.986475: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-11-17 16:55:08.986574: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2025-11-17 16:55:08.986587: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2025-11-17 16:55:08.987740: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 30119 MB memory) -> physical GPU (device: 0, name: Tesla V100S-PCIE-32GB, pci bus id: 0000:89:00.0, compute capability: 7.0)
2025-11-17 16:55:09.036635: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2025-11-17 16:55:09.053669: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2499920000 Hz
2025-11-17 16:55:12.987296: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:55:13.251065: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:55:13.256248: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:55:14.392006: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2025-11-17 16:55:14.479502: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)