Lmod Warning:
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: curl/8.4.0
(required by: htslib/1.16)
-------------------------------------------------------------------------------




The following have been reloaded with a version change:
  1) curl/8.4.0 => curl/8.17.0     2) openssl/3.0.7 => openssl/3.6.0

2025-11-17 16:46:02.253841: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:47:39.944186: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-11-17 16:47:39.960815: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2025-11-17 16:47:40.012600: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:8a:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.75GiB deviceMemoryBandwidth: 573.69GiB/s
2025-11-17 16:47:40.012710: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:47:40.658346: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:47:40.658517: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:47:41.016839: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-11-17 16:47:41.578962: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-11-17 16:47:42.125429: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-11-17 16:47:42.368226: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-11-17 16:47:42.525094: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:47:42.526412: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-11-17 16:47:42.527136: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-11-17 16:47:42.529946: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-11-17 16:47:42.530616: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:8a:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.75GiB deviceMemoryBandwidth: 573.69GiB/s
2025-11-17 16:47:42.530673: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:47:42.530710: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:47:42.530741: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:47:42.530770: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-11-17 16:47:42.530799: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-11-17 16:47:42.530828: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-11-17 16:47:42.530856: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-11-17 16:47:42.530907: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:47:42.531919: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-11-17 16:47:42.531993: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:47:45.062669: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-11-17 16:47:45.062765: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2025-11-17 16:47:45.062780: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2025-11-17 16:47:45.064086: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10064 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:8a:00.0, compute capability: 7.5)
2025-11-17 16:47:56.450986: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2025-11-17 16:47:56.451734: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500205000 Hz
2025-11-17 16:47:57.858133: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:47:59.483120: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:47:59.489684: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:48:13.148803: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2025-11-17 16:48:13.211025: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'jitters', 'index', 'status', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
2025-11-17 16:53:16.908248: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2025-11-17 16:53:21.632670: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:53:30.546542: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-11-17 16:53:30.547682: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2025-11-17 16:53:30.577428: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:8a:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.75GiB deviceMemoryBandwidth: 573.69GiB/s
2025-11-17 16:53:30.577524: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:53:30.583320: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:53:30.583374: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:53:30.585715: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-11-17 16:53:30.587715: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-11-17 16:53:30.592194: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-11-17 16:53:30.594231: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-11-17 16:53:30.595962: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:53:30.596966: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-11-17 16:53:30.597372: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-11-17 16:53:30.600176: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-11-17 16:53:30.600722: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:8a:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.75GiB deviceMemoryBandwidth: 573.69GiB/s
2025-11-17 16:53:30.600766: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:53:30.600794: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:53:30.600818: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:53:30.600841: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-11-17 16:53:30.600864: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-11-17 16:53:30.600887: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-11-17 16:53:30.600909: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-11-17 16:53:30.600941: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:53:30.601785: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-11-17 16:53:30.601847: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:53:31.078793: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-11-17 16:53:31.078951: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2025-11-17 16:53:31.078966: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2025-11-17 16:53:31.080796: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10064 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:8a:00.0, compute capability: 7.5)
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)

batch:   0%|          | 0/52 [00:00<?, ?it/s]2025-11-17 16:53:32.623457: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2025-11-17 16:53:32.623983: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500205000 Hz
2025-11-17 16:53:32.877745: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:53:33.232257: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:53:33.234727: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:53:34.548443: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2025-11-17 16:53:34.619277: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'true_profiles', 'true_logcounts', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])

batch:   2%|▏         | 1/52 [00:18<16:01, 18.86s/it]
batch:   4%|▍         | 2/52 [00:19<06:37,  7.95s/it]
batch:   6%|▌         | 3/52 [00:19<03:38,  4.46s/it]
batch:   8%|▊         | 4/52 [00:19<02:15,  2.82s/it]
batch:  10%|▉         | 5/52 [00:20<01:30,  1.92s/it]
batch:  12%|█▏        | 6/52 [00:20<01:03,  1.37s/it]
batch:  13%|█▎        | 7/52 [00:20<00:46,  1.03s/it]
batch:  15%|█▌        | 8/52 [00:21<00:35,  1.25it/s]
batch:  17%|█▋        | 9/52 [00:21<00:27,  1.55it/s]
batch:  19%|█▉        | 10/52 [00:21<00:22,  1.85it/s]
batch:  21%|██        | 11/52 [00:21<00:19,  2.13it/s]
batch:  23%|██▎       | 12/52 [00:22<00:16,  2.38it/s]
batch:  25%|██▌       | 13/52 [00:22<00:15,  2.58it/s]
batch:  27%|██▋       | 14/52 [00:22<00:13,  2.75it/s]
batch:  29%|██▉       | 15/52 [00:23<00:12,  2.88it/s]
batch:  31%|███       | 16/52 [00:23<00:12,  2.96it/s]
batch:  33%|███▎      | 17/52 [00:23<00:11,  3.04it/s]
batch:  35%|███▍      | 18/52 [00:24<00:10,  3.09it/s]
batch:  37%|███▋      | 19/52 [00:24<00:10,  3.14it/s]
batch:  38%|███▊      | 20/52 [00:24<00:10,  3.17it/s]
batch:  40%|████      | 21/52 [00:25<00:09,  3.18it/s]
batch:  42%|████▏     | 22/52 [00:25<00:09,  3.20it/s]
batch:  44%|████▍     | 23/52 [00:25<00:09,  3.21it/s]
batch:  46%|████▌     | 24/52 [00:25<00:08,  3.21it/s]
batch:  48%|████▊     | 25/52 [00:26<00:08,  3.22it/s]
batch:  50%|█████     | 26/52 [00:26<00:08,  3.22it/s]
batch:  52%|█████▏    | 27/52 [00:26<00:07,  3.23it/s]
batch:  54%|█████▍    | 28/52 [00:27<00:07,  3.24it/s]
batch:  56%|█████▌    | 29/52 [00:27<00:07,  3.24it/s]
batch:  58%|█████▊    | 30/52 [00:27<00:06,  3.24it/s]
batch:  60%|█████▉    | 31/52 [00:28<00:06,  3.25it/s]
batch:  62%|██████▏   | 32/52 [00:28<00:06,  3.23it/s]
batch:  63%|██████▎   | 33/52 [00:28<00:05,  3.23it/s]
batch:  65%|██████▌   | 34/52 [00:29<00:05,  3.23it/s]
batch:  67%|██████▋   | 35/52 [00:29<00:05,  3.22it/s]
batch:  69%|██████▉   | 36/52 [00:29<00:04,  3.23it/s]
batch:  71%|███████   | 37/52 [00:30<00:04,  3.22it/s]
batch:  73%|███████▎  | 38/52 [00:30<00:04,  3.22it/s]
batch:  75%|███████▌  | 39/52 [00:30<00:04,  3.23it/s]
batch:  77%|███████▋  | 40/52 [00:30<00:03,  3.23it/s]
batch:  79%|███████▉  | 41/52 [00:31<00:03,  3.23it/s]
batch:  81%|████████  | 42/52 [00:31<00:03,  3.23it/s]
batch:  83%|████████▎ | 43/52 [00:31<00:02,  3.24it/s]
batch:  85%|████████▍ | 44/52 [00:32<00:02,  3.23it/s]
batch:  87%|████████▋ | 45/52 [00:32<00:02,  3.25it/s]
batch:  88%|████████▊ | 46/52 [00:32<00:01,  3.25it/s]
batch:  90%|█████████ | 47/52 [00:33<00:01,  3.25it/s]
batch:  92%|█████████▏| 48/52 [00:33<00:01,  3.24it/s]
batch:  94%|█████████▍| 49/52 [00:33<00:00,  3.23it/s]
batch:  96%|█████████▌| 50/52 [00:33<00:00,  3.71it/s]
batch:  98%|█████████▊| 51/52 [00:34<00:00,  3.57it/s]
batch: 100%|██████████| 52/52 [00:34<00:00,  1.52it/s]

  0%|          | 0/3229 [00:00<?, ?it/s]
  3%|▎         | 97/3229 [00:00<00:03, 967.30it/s]
  6%|▌         | 197/3229 [00:00<00:03, 981.83it/s]
  9%|▉         | 299/3229 [00:00<00:02, 990.93it/s]
 12%|█▏        | 399/3229 [00:00<00:02, 988.97it/s]
 15%|█▌        | 498/3229 [00:00<00:02, 986.92it/s]
 18%|█▊        | 597/3229 [00:00<00:02, 986.71it/s]
 22%|██▏       | 697/3229 [00:00<00:02, 985.11it/s]
 25%|██▍       | 796/3229 [00:00<00:02, 982.26it/s]
 28%|██▊       | 895/3229 [00:00<00:02, 983.29it/s]
 31%|███       | 994/3229 [00:01<00:02, 973.56it/s]
 34%|███▍      | 1092/3229 [00:01<00:02, 968.64it/s]
 37%|███▋      | 1189/3229 [00:01<00:02, 966.89it/s]
 40%|███▉      | 1286/3229 [00:01<00:02, 959.60it/s]
 43%|████▎     | 1382/3229 [00:01<00:01, 951.57it/s]
 46%|████▌     | 1478/3229 [00:01<00:01, 953.98it/s]
 49%|████▊     | 1574/3229 [00:01<00:01, 946.69it/s]
 52%|█████▏    | 1670/3229 [00:01<00:01, 946.21it/s]
 55%|█████▍    | 1765/3229 [00:01<00:01, 942.80it/s]
 58%|█████▊    | 1861/3229 [00:01<00:01, 942.01it/s]
 61%|██████    | 1956/3229 [00:02<00:01, 939.11it/s]
 63%|██████▎   | 2050/3229 [00:02<00:01, 933.95it/s]
 66%|██████▋   | 2144/3229 [00:02<00:01, 930.13it/s]
 69%|██████▉   | 2238/3229 [00:02<00:01, 927.47it/s]
 72%|███████▏  | 2331/3229 [00:02<00:00, 924.36it/s]
 75%|███████▌  | 2424/3229 [00:02<00:00, 923.73it/s]
 78%|███████▊  | 2517/3229 [00:02<00:00, 919.24it/s]
 81%|████████  | 2609/3229 [00:02<00:00, 907.63it/s]
 84%|████████▎ | 2700/3229 [00:02<00:00, 898.53it/s]
 86%|████████▋ | 2790/3229 [00:02<00:00, 896.93it/s]
 89%|████████▉ | 2880/3229 [00:03<00:00, 890.85it/s]
 92%|█████████▏| 2970/3229 [00:03<00:00, 887.10it/s]
 95%|█████████▍| 3059/3229 [00:03<00:00, 880.62it/s]
 97%|█████████▋| 3148/3229 [00:03<00:00, 880.54it/s]
100%|██████████| 3229/3229 [00:03<00:00, 934.53it/s]

  0%|          | 0/3229 [00:00<?, ?it/s]
  3%|▎         | 98/3229 [00:00<00:03, 978.90it/s]
  6%|▌         | 197/3229 [00:00<00:03, 984.41it/s]
  9%|▉         | 299/3229 [00:00<00:02, 994.40it/s]
 12%|█▏        | 399/3229 [00:00<00:02, 994.19it/s]
 15%|█▌        | 499/3229 [00:00<00:02, 995.07it/s]
 19%|█▊        | 599/3229 [00:00<00:02, 989.45it/s]
 22%|██▏       | 700/3229 [00:00<00:02, 990.35it/s]
 25%|██▍       | 800/3229 [00:00<00:02, 989.27it/s]
 28%|██▊       | 899/3229 [00:00<00:02, 983.50it/s]
 31%|███       | 998/3229 [00:01<00:02, 976.07it/s]
 34%|███▍      | 1096/3229 [00:01<00:02, 976.72it/s]
 37%|███▋      | 1194/3229 [00:01<00:02, 970.10it/s]
 40%|████      | 1292/3229 [00:01<00:02, 964.01it/s]
 43%|████▎     | 1389/3229 [00:01<00:01, 963.34it/s]
 46%|████▌     | 1486/3229 [00:01<00:01, 959.16it/s]
 49%|████▉     | 1582/3229 [00:01<00:01, 955.92it/s]
 52%|█████▏    | 1678/3229 [00:01<00:01, 947.55it/s]
 55%|█████▍    | 1774/3229 [00:01<00:01, 946.38it/s]
 58%|█████▊    | 1870/3229 [00:01<00:01, 945.83it/s]
 61%|██████    | 1965/3229 [00:02<00:01, 942.50it/s]
 64%|██████▍   | 2060/3229 [00:02<00:01, 933.72it/s]
 67%|██████▋   | 2154/3229 [00:02<00:01, 931.55it/s]
 70%|██████▉   | 2248/3229 [00:02<00:01, 928.65it/s]
 72%|███████▏  | 2341/3229 [00:02<00:00, 926.15it/s]
 75%|███████▌  | 2434/3229 [00:02<00:00, 925.77it/s]
 78%|███████▊  | 2527/3229 [00:02<00:00, 920.14it/s]
 81%|████████  | 2620/3229 [00:02<00:00, 911.47it/s]
 84%|████████▍ | 2712/3229 [00:02<00:00, 904.25it/s]
 87%|████████▋ | 2803/3229 [00:02<00:00, 898.59it/s]
 90%|████████▉ | 2893/3229 [00:03<00:00, 893.40it/s]
 92%|█████████▏| 2983/3229 [00:03<00:00, 888.47it/s]
 95%|█████████▌| 3072/3229 [00:03<00:00, 887.60it/s]
 98%|█████████▊| 3161/3229 [00:03<00:00, 880.98it/s]
100%|██████████| 3229/3229 [00:03<00:00, 938.35it/s]
2025-11-17 16:54:43.581372: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:55:18.579106: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-11-17 16:55:18.580085: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2025-11-17 16:55:18.623136: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:8a:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.75GiB deviceMemoryBandwidth: 573.69GiB/s
2025-11-17 16:55:18.623301: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:55:18.640391: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:55:18.640521: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:55:18.647985: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-11-17 16:55:18.654721: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-11-17 16:55:18.664919: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-11-17 16:55:18.671721: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-11-17 16:55:18.678432: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:55:18.679701: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-11-17 16:55:18.680329: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2025-11-17 16:55:18.683322: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2025-11-17 16:55:18.684095: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:8a:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5
coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.75GiB deviceMemoryBandwidth: 573.69GiB/s
2025-11-17 16:55:18.684182: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:55:18.684236: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:55:18.684282: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:55:18.684353: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2025-11-17 16:55:18.684401: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2025-11-17 16:55:18.684446: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2025-11-17 16:55:18.684490: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2025-11-17 16:55:18.684533: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:55:18.685631: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2025-11-17 16:55:18.685723: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2025-11-17 16:55:19.147685: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2025-11-17 16:55:19.147782: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2025-11-17 16:55:19.147795: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2025-11-17 16:55:19.149164: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10064 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:8a:00.0, compute capability: 7.5)
2025-11-17 16:55:19.202486: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2025-11-17 16:55:19.226995: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2500205000 Hz
2025-11-17 16:55:22.619196: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2025-11-17 16:55:23.775041: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2025-11-17 16:55:23.780156: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2025-11-17 16:55:25.274916: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2025-11-17 16:55:25.346282: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)