Lmod Warning:
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: curl/8.4.0
(required by: htslib/1.16)
-------------------------------------------------------------------------------




The following have been reloaded with a version change:
  1) curl/8.4.0 => curl/8.17.0     2) openssl/3.0.7 => openssl/3.6.0

2025-11-17 16:46:02.942615: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:47:44.308959: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-11-17 16:47:44.341958: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2025-11-17 16:47:44.394100: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:89:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.75GiB deviceMemoryBandwidth: 573.69GiB/s
2025-11-17 16:47:44.394171: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:47:45.130800: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:47:45.130999: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:47:45.540267: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-11-17 16:47:45.915838: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-11-17 16:47:46.508715: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-11-17 16:47:46.723526: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-11-17 16:47:46.886161: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:47:46.887443: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-11-17 16:47:46.888165: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-11-17 16:47:46.891057: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-11-17 16:47:46.891701: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:89:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.75GiB deviceMemoryBandwidth: 573.69GiB/s
2025-11-17 16:47:46.891762: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:47:46.891802: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:47:46.891835: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:47:46.891867: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-11-17 16:47:46.891912: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-11-17 16:47:46.891947: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-11-17 16:47:46.891977: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-11-17 16:47:46.892036: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:47:46.893073: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-11-17 16:47:46.893147: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:47:49.597505: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-11-17 16:47:49.597619: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2025-11-17 16:47:49.597635: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2025-11-17 16:47:49.598905: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10064 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:89:00.0, compute capability: 7.5)
2025-11-17 16:48:01.547012: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2025-11-17 16:48:01.547719: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500115000 Hz
2025-11-17 16:48:03.023273: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:48:04.728794: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:48:04.735958: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:48:18.000775: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2025-11-17 16:48:18.087967: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'jitters', 'index', 'status', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
2025-11-17 16:54:01.649150: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2025-11-17 16:54:07.549537: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:54:22.058029: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-11-17 16:54:22.059164: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2025-11-17 16:54:22.092167: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:89:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.75GiB deviceMemoryBandwidth: 573.69GiB/s
2025-11-17 16:54:22.092307: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:54:22.100396: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:54:22.100452: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:54:22.104744: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-11-17 16:54:22.108133: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-11-17 16:54:22.116270: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-11-17 16:54:22.120926: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-11-17 16:54:22.123952: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:54:22.125017: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-11-17 16:54:22.125677: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-11-17 16:54:22.128238: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-11-17 16:54:22.128780: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:89:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.75GiB deviceMemoryBandwidth: 573.69GiB/s
2025-11-17 16:54:22.128831: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:54:22.128860: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:54:22.128885: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:54:22.128923: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-11-17 16:54:22.128948: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-11-17 16:54:22.128971: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-11-17 16:54:22.128994: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-11-17 16:54:22.129017: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:54:22.129838: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-11-17 16:54:22.129900: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:54:22.624774: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-11-17 16:54:22.624873: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2025-11-17 16:54:22.624887: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2025-11-17 16:54:22.626176: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10064 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:89:00.0, compute capability: 7.5)
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)

batch:   0%|          | 0/52 [00:00<?, ?it/s]2025-11-17 16:54:24.222345: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2025-11-17 16:54:24.222864: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500115000 Hz
2025-11-17 16:54:24.488598: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:54:24.859229: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:54:24.861492: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:54:26.246556: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2025-11-17 16:54:26.332034: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'true_profiles', 'true_logcounts', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])

batch:   2%|▏         | 1/52 [00:28<24:29, 28.82s/it]
batch:   4%|▍         | 2/52 [00:29<10:02, 12.05s/it]
batch:   6%|▌         | 3/52 [00:29<05:27,  6.69s/it]
batch:   8%|▊         | 4/52 [00:29<03:20,  4.17s/it]
batch:  10%|▉         | 5/52 [00:30<02:10,  2.78s/it]
batch:  12%|█▏        | 6/52 [00:30<01:29,  1.94s/it]
batch:  13%|█▎        | 7/52 [00:30<01:03,  1.41s/it]
batch:  15%|█▌        | 8/52 [00:31<00:46,  1.06s/it]
batch:  17%|█▋        | 9/52 [00:31<00:35,  1.21it/s]
batch:  19%|█▉        | 10/52 [00:31<00:28,  1.50it/s]
batch:  21%|██        | 11/52 [00:31<00:22,  1.79it/s]
batch:  23%|██▎       | 12/52 [00:32<00:19,  2.07it/s]
batch:  25%|██▌       | 13/52 [00:32<00:16,  2.33it/s]
batch:  27%|██▋       | 14/52 [00:32<00:15,  2.53it/s]
batch:  29%|██▉       | 15/52 [00:33<00:13,  2.71it/s]
batch:  31%|███       | 16/52 [00:33<00:12,  2.85it/s]
batch:  33%|███▎      | 17/52 [00:33<00:11,  2.95it/s]
batch:  35%|███▍      | 18/52 [00:34<00:11,  3.03it/s]
batch:  37%|███▋      | 19/52 [00:34<00:10,  3.09it/s]
batch:  38%|███▊      | 20/52 [00:34<00:10,  3.11it/s]
batch:  40%|████      | 21/52 [00:35<00:09,  3.15it/s]
batch:  42%|████▏     | 22/52 [00:35<00:09,  3.17it/s]
batch:  44%|████▍     | 23/52 [00:35<00:09,  3.19it/s]
batch:  46%|████▌     | 24/52 [00:35<00:08,  3.18it/s]
batch:  48%|████▊     | 25/52 [00:36<00:08,  3.20it/s]
batch:  50%|█████     | 26/52 [00:36<00:08,  3.20it/s]
batch:  52%|█████▏    | 27/52 [00:36<00:07,  3.21it/s]
batch:  54%|█████▍    | 28/52 [00:37<00:07,  3.22it/s]
batch:  56%|█████▌    | 29/52 [00:37<00:07,  3.23it/s]
batch:  58%|█████▊    | 30/52 [00:37<00:06,  3.23it/s]
batch:  60%|█████▉    | 31/52 [00:38<00:06,  3.24it/s]
batch:  62%|██████▏   | 32/52 [00:38<00:06,  3.24it/s]
batch:  63%|██████▎   | 33/52 [00:38<00:05,  3.24it/s]
batch:  65%|██████▌   | 34/52 [00:39<00:05,  3.24it/s]
batch:  67%|██████▋   | 35/52 [00:39<00:05,  3.23it/s]
batch:  69%|██████▉   | 36/52 [00:39<00:04,  3.23it/s]
batch:  71%|███████   | 37/52 [00:40<00:04,  3.23it/s]
batch:  73%|███████▎  | 38/52 [00:40<00:04,  3.23it/s]
batch:  75%|███████▌  | 39/52 [00:40<00:04,  3.22it/s]
batch:  77%|███████▋  | 40/52 [00:40<00:03,  3.20it/s]
batch:  79%|███████▉  | 41/52 [00:41<00:03,  3.19it/s]
batch:  81%|████████  | 42/52 [00:41<00:03,  3.19it/s]
batch:  83%|████████▎ | 43/52 [00:41<00:02,  3.21it/s]
batch:  85%|████████▍ | 44/52 [00:42<00:02,  3.19it/s]
batch:  87%|████████▋ | 45/52 [00:42<00:02,  3.21it/s]
batch:  88%|████████▊ | 46/52 [00:42<00:01,  3.22it/s]
batch:  90%|█████████ | 47/52 [00:43<00:01,  3.24it/s]
batch:  92%|█████████▏| 48/52 [00:43<00:01,  3.25it/s]
batch:  94%|█████████▍| 49/52 [00:43<00:00,  3.25it/s]
batch:  96%|█████████▌| 50/52 [00:43<00:00,  3.73it/s]
batch:  98%|█████████▊| 51/52 [00:44<00:00,  3.58it/s]
batch: 100%|██████████| 52/52 [00:44<00:00,  1.17it/s]

  0%|          | 0/3229 [00:00<?, ?it/s]
  3%|▎         | 98/3229 [00:00<00:03, 979.90it/s]
  6%|▌         | 196/3229 [00:00<00:03, 970.92it/s]
  9%|▉         | 296/3229 [00:00<00:02, 982.63it/s]
 12%|█▏        | 397/3229 [00:00<00:02, 989.28it/s]
 15%|█▌        | 496/3229 [00:00<00:02, 988.10it/s]
 18%|█▊        | 595/3229 [00:00<00:02, 987.23it/s]
 21%|██▏       | 694/3229 [00:00<00:02, 984.36it/s]
 25%|██▍       | 793/3229 [00:00<00:02, 982.75it/s]
 28%|██▊       | 892/3229 [00:00<00:02, 984.22it/s]
 31%|███       | 991/3229 [00:01<00:02, 976.39it/s]
 34%|███▎      | 1089/3229 [00:01<00:02, 971.25it/s]
 37%|███▋      | 1187/3229 [00:01<00:02, 970.75it/s]
 40%|███▉      | 1285/3229 [00:01<00:02, 963.84it/s]
 43%|████▎     | 1382/3229 [00:01<00:01, 955.40it/s]
 46%|████▌     | 1478/3229 [00:01<00:01, 955.43it/s]
 49%|████▊     | 1574/3229 [00:01<00:01, 945.12it/s]
 52%|█████▏    | 1670/3229 [00:01<00:01, 944.99it/s]
 55%|█████▍    | 1765/3229 [00:01<00:01, 939.80it/s]
 58%|█████▊    | 1861/3229 [00:01<00:01, 941.58it/s]
 61%|██████    | 1956/3229 [00:02<00:01, 940.00it/s]
 64%|██████▎   | 2051/3229 [00:02<00:01, 937.68it/s]
 66%|██████▋   | 2145/3229 [00:02<00:01, 929.27it/s]
 69%|██████▉   | 2239/3229 [00:02<00:01, 931.72it/s]
 72%|███████▏  | 2333/3229 [00:02<00:00, 924.46it/s]
 75%|███████▌  | 2426/3229 [00:02<00:00, 923.55it/s]
 78%|███████▊  | 2519/3229 [00:02<00:00, 918.97it/s]
 81%|████████  | 2611/3229 [00:02<00:00, 907.63it/s]
 84%|████████▎ | 2702/3229 [00:02<00:00, 904.80it/s]
 86%|████████▋ | 2793/3229 [00:02<00:00, 893.32it/s]
 89%|████████▉ | 2883/3229 [00:03<00:00, 893.72it/s]
 92%|█████████▏| 2973/3229 [00:03<00:00, 885.58it/s]
 95%|█████████▍| 3062/3229 [00:03<00:00, 878.45it/s]
 98%|█████████▊| 3150/3229 [00:03<00:00, 876.09it/s]
100%|██████████| 3229/3229 [00:03<00:00, 934.62it/s]

  0%|          | 0/3229 [00:00<?, ?it/s]
  3%|▎         | 98/3229 [00:00<00:03, 979.80it/s]
  6%|▌         | 197/3229 [00:00<00:03, 977.84it/s]
  9%|▉         | 299/3229 [00:00<00:02, 993.23it/s]
 12%|█▏        | 399/3229 [00:00<00:02, 994.14it/s]
 15%|█▌        | 499/3229 [00:00<00:02, 995.38it/s]
 19%|█▊        | 599/3229 [00:00<00:02, 989.36it/s]
 22%|██▏       | 700/3229 [00:00<00:02, 989.90it/s]
 25%|██▍       | 799/3229 [00:00<00:02, 986.89it/s]
 28%|██▊       | 898/3229 [00:00<00:02, 981.67it/s]
 31%|███       | 997/3229 [00:01<00:02, 980.43it/s]
 34%|███▍      | 1096/3229 [00:01<00:02, 976.24it/s]
 37%|███▋      | 1194/3229 [00:01<00:02, 969.71it/s]
 40%|███▉      | 1291/3229 [00:01<00:02, 967.84it/s]
 43%|████▎     | 1388/3229 [00:01<00:01, 961.21it/s]
 46%|████▌     | 1485/3229 [00:01<00:01, 957.82it/s]
 49%|████▉     | 1581/3229 [00:01<00:01, 954.57it/s]
 52%|█████▏    | 1677/3229 [00:01<00:01, 950.29it/s]
 55%|█████▍    | 1773/3229 [00:01<00:01, 948.57it/s]
 58%|█████▊    | 1868/3229 [00:01<00:01, 945.88it/s]
 61%|██████    | 1963/3229 [00:02<00:01, 942.16it/s]
 64%|██████▎   | 2058/3229 [00:02<00:01, 937.11it/s]
 67%|██████▋   | 2152/3229 [00:02<00:01, 935.08it/s]
 70%|██████▉   | 2246/3229 [00:02<00:01, 931.43it/s]
 72%|███████▏  | 2340/3229 [00:02<00:00, 929.64it/s]
 75%|███████▌  | 2433/3229 [00:02<00:00, 920.80it/s]
 78%|███████▊  | 2526/3229 [00:02<00:00, 916.87it/s]
 81%|████████  | 2618/3229 [00:02<00:00, 912.12it/s]
 84%|████████▍ | 2710/3229 [00:02<00:00, 904.79it/s]
 87%|████████▋ | 2801/3229 [00:02<00:00, 898.98it/s]
 90%|████████▉ | 2891/3229 [00:03<00:00, 893.71it/s]
 92%|█████████▏| 2981/3229 [00:03<00:00, 888.59it/s]
 95%|█████████▌| 3070/3229 [00:03<00:00, 882.83it/s]
 98%|█████████▊| 3159/3229 [00:03<00:00, 882.21it/s]
100%|██████████| 3229/3229 [00:03<00:00, 938.29it/s]
2025-11-17 16:55:50.052733: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:56:10.420984: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-11-17 16:56:10.421978: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2025-11-17 16:56:10.452877: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:89:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.75GiB deviceMemoryBandwidth: 573.69GiB/s
2025-11-17 16:56:10.453079: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:56:10.457404: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:56:10.457463: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:56:10.460608: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-11-17 16:56:10.462534: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-11-17 16:56:10.467083: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-11-17 16:56:10.469276: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-11-17 16:56:10.471467: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:56:10.472478: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-11-17 16:56:10.472991: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-11-17 16:56:10.475627: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-11-17 16:56:10.476190: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:89:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.75GiB deviceMemoryBandwidth: 573.69GiB/s
2025-11-17 16:56:10.476249: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:56:10.476285: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:56:10.476317: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:56:10.476372: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-11-17 16:56:10.476405: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-11-17 16:56:10.476436: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-11-17 16:56:10.476467: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-11-17 16:56:10.476497: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:56:10.477353: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-11-17 16:56:10.477417: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:56:10.955567: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-11-17 16:56:10.955665: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2025-11-17 16:56:10.955678: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2025-11-17 16:56:10.957007: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10064 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:89:00.0, compute capability: 7.5)
2025-11-17 16:56:11.006659: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2025-11-17 16:56:11.023429: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500115000 Hz
2025-11-17 16:56:14.345277: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:56:14.709416: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:56:14.714653: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:56:16.032092: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2025-11-17 16:56:16.116711: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)