Lmod Warning:
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: curl/8.4.0
(required by: htslib/1.16)
-------------------------------------------------------------------------------




The following have been reloaded with a version change:
  1) curl/8.4.0 => curl/8.17.0     2) openssl/3.0.7 => openssl/3.6.0

2026-05-28 11:40:38.844812: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:41:57.938353: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-28 11:41:57.949624: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-28 11:41:57.970034: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:02:00.0 name: NVIDIA TITAN Xp computeCapability: 6.1
coreClock: 1.911GHz coreCount: 30 deviceMemorySize: 11.90GiB deviceMemoryBandwidth: 510.07GiB/s
2026-05-28 11:41:57.970151: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:41:58.511526: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-28 11:41:58.511765: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-28 11:41:58.908639: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-28 11:41:59.447722: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-28 11:42:00.020763: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-28 11:42:00.234369: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-28 11:42:00.392263: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-28 11:42:00.393207: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-28 11:42:00.393858: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-28 11:42:00.394052: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-28 11:42:00.394457: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:02:00.0 name: NVIDIA TITAN Xp computeCapability: 6.1
coreClock: 1.911GHz coreCount: 30 deviceMemorySize: 11.90GiB deviceMemoryBandwidth: 510.07GiB/s
2026-05-28 11:42:00.394515: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:42:00.394562: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-28 11:42:00.394599: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-28 11:42:00.394675: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-28 11:42:00.394713: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-28 11:42:00.394748: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-28 11:42:00.394783: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-28 11:42:00.394844: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-28 11:42:00.395452: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-28 11:42:00.395518: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:42:03.112696: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-28 11:42:03.112822: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-28 11:42:03.112846: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-28 11:42:03.113737: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11209 MB memory) -> physical GPU (device: 0, name: NVIDIA TITAN Xp, pci bus id: 0000:02:00.0, compute capability: 6.1)
2026-05-28 11:42:05.486893: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-05-28 11:42:05.487798: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2399915000 Hz
2026-05-28 11:42:09.569805: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-28 11:42:11.262987: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-28 11:42:11.271051: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-28 11:42:24.675284: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-28 11:42:24.761126: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'jitters', 'index', 'status', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
2026-05-28 11:53:41.218453: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2026-05-28 11:53:46.072584: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:53:53.357316: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-28 11:53:53.358523: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-28 11:53:53.391730: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:02:00.0 name: NVIDIA TITAN Xp computeCapability: 6.1
coreClock: 1.911GHz coreCount: 30 deviceMemorySize: 11.90GiB deviceMemoryBandwidth: 510.07GiB/s
2026-05-28 11:53:53.391846: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:53:53.397072: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-28 11:53:53.397142: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-28 11:53:53.399525: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-28 11:53:53.400741: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-28 11:53:53.404584: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-28 11:53:53.406283: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-28 11:53:53.407469: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-28 11:53:53.408041: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-28 11:53:53.408390: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-28 11:53:53.408538: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-28 11:53:53.409511: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:02:00.0 name: NVIDIA TITAN Xp computeCapability: 6.1
coreClock: 1.911GHz coreCount: 30 deviceMemorySize: 11.90GiB deviceMemoryBandwidth: 510.07GiB/s
2026-05-28 11:53:53.409711: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:53:53.409785: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-28 11:53:53.409841: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-28 11:53:53.409894: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-28 11:53:53.409954: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-28 11:53:53.410007: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-28 11:53:53.410059: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-28 11:53:53.410112: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-28 11:53:53.410947: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-28 11:53:53.411048: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:53:53.997110: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-28 11:53:53.997223: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-28 11:53:53.997247: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-28 11:53:53.999041: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11209 MB memory) -> physical GPU (device: 0, name: NVIDIA TITAN Xp, pci bus id: 0000:02:00.0, compute capability: 6.1)
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)

batch:   0%|          | 0/54 [00:00<?, ?it/s]2026-05-28 11:53:56.098365: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-05-28 11:53:56.099167: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2399915000 Hz
2026-05-28 11:53:56.443418: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-28 11:53:56.755895: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-28 11:53:56.758132: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-28 11:53:57.861843: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-28 11:53:57.985763: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'true_profiles', 'true_logcounts', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])

batch:   2%|▏         | 1/54 [00:15<13:54, 15.75s/it]
batch:   4%|▎         | 2/54 [00:16<05:50,  6.75s/it]
batch:   6%|▌         | 3/54 [00:16<03:17,  3.88s/it]
batch:   7%|▋         | 4/54 [00:17<02:05,  2.51s/it]
batch:   9%|▉         | 5/54 [00:17<01:26,  1.76s/it]
batch:  11%|█         | 6/54 [00:17<01:02,  1.31s/it]
batch:  13%|█▎        | 7/54 [00:18<00:47,  1.01s/it]
batch:  15%|█▍        | 8/54 [00:18<00:37,  1.23it/s]
batch:  17%|█▋        | 9/54 [00:19<00:31,  1.42it/s]
batch:  19%|█▊        | 10/54 [00:19<00:26,  1.65it/s]
batch:  20%|██        | 11/54 [00:19<00:23,  1.83it/s]
batch:  22%|██▏       | 12/54 [00:20<00:21,  1.98it/s]
batch:  24%|██▍       | 13/54 [00:20<00:19,  2.07it/s]
batch:  26%|██▌       | 14/54 [00:21<00:18,  2.20it/s]
batch:  28%|██▊       | 15/54 [00:21<00:17,  2.17it/s]
batch:  30%|██▉       | 16/54 [00:22<00:16,  2.27it/s]
batch:  31%|███▏      | 17/54 [00:22<00:16,  2.31it/s]
batch:  33%|███▎      | 18/54 [00:23<00:16,  2.21it/s]
batch:  35%|███▌      | 19/54 [00:23<00:15,  2.29it/s]
batch:  37%|███▋      | 20/54 [00:23<00:14,  2.34it/s]
batch:  39%|███▉      | 21/54 [00:24<00:14,  2.34it/s]
batch:  41%|████      | 22/54 [00:24<00:13,  2.35it/s]
batch:  43%|████▎     | 23/54 [00:25<00:13,  2.28it/s]
batch:  44%|████▍     | 24/54 [00:25<00:13,  2.30it/s]
batch:  46%|████▋     | 25/54 [00:25<00:12,  2.37it/s]
batch:  48%|████▊     | 26/54 [00:26<00:11,  2.38it/s]
batch:  50%|█████     | 27/54 [00:26<00:11,  2.36it/s]
batch:  52%|█████▏    | 28/54 [00:27<00:10,  2.44it/s]
batch:  54%|█████▎    | 29/54 [00:27<00:10,  2.48it/s]
batch:  56%|█████▌    | 30/54 [00:27<00:09,  2.44it/s]
batch:  57%|█████▋    | 31/54 [00:28<00:09,  2.43it/s]
batch:  59%|█████▉    | 32/54 [00:28<00:09,  2.37it/s]
batch:  61%|██████    | 33/54 [00:29<00:08,  2.37it/s]
batch:  63%|██████▎   | 34/54 [00:29<00:08,  2.29it/s]
batch:  65%|██████▍   | 35/54 [00:30<00:08,  2.27it/s]
batch:  67%|██████▋   | 36/54 [00:30<00:08,  2.23it/s]
batch:  69%|██████▊   | 37/54 [00:31<00:07,  2.33it/s]
batch:  70%|███████   | 38/54 [00:31<00:06,  2.35it/s]
batch:  72%|███████▏  | 39/54 [00:31<00:06,  2.42it/s]
batch:  74%|███████▍  | 40/54 [00:32<00:06,  2.33it/s]
batch:  76%|███████▌  | 41/54 [00:32<00:05,  2.35it/s]
batch:  78%|███████▊  | 42/54 [00:33<00:04,  2.42it/s]
batch:  80%|███████▉  | 43/54 [00:33<00:04,  2.41it/s]
batch:  81%|████████▏ | 44/54 [00:33<00:04,  2.35it/s]
batch:  83%|████████▎ | 45/54 [00:34<00:03,  2.42it/s]
batch:  85%|████████▌ | 46/54 [00:34<00:03,  2.38it/s]
batch:  87%|████████▋ | 47/54 [00:35<00:03,  2.32it/s]
batch:  89%|████████▉ | 48/54 [00:35<00:02,  2.34it/s]
batch:  91%|█████████ | 49/54 [00:36<00:02,  2.34it/s]
batch:  93%|█████████▎| 50/54 [00:36<00:01,  2.18it/s]
batch:  94%|█████████▍| 51/54 [00:37<00:01,  2.30it/s]
batch:  96%|█████████▋| 52/54 [00:37<00:00,  2.32it/s]
batch:  98%|█████████▊| 53/54 [00:37<00:00,  2.61it/s]
batch: 100%|██████████| 54/54 [00:37<00:00,  1.43it/s]

  0%|          | 0/3361 [00:00<?, ?it/s]
  3%|▎         | 99/3361 [00:00<00:03, 974.18it/s]
  6%|▌         | 197/3361 [00:00<00:03, 956.82it/s]
  9%|▊         | 293/3361 [00:00<00:03, 942.23it/s]
 12%|█▏        | 388/3361 [00:00<00:03, 762.51it/s]
 14%|█▍        | 484/3361 [00:00<00:03, 821.20it/s]
 17%|█▋        | 580/3361 [00:00<00:03, 858.43it/s]
 20%|██        | 673/3361 [00:00<00:03, 878.20it/s]
 23%|██▎       | 768/3361 [00:00<00:02, 896.08it/s]
 26%|██▌       | 862/3361 [00:00<00:02, 907.92it/s]
 28%|██▊       | 954/3361 [00:01<00:02, 901.48it/s]
 31%|███       | 1047/3361 [00:01<00:02, 903.75it/s]
 34%|███▍      | 1140/3361 [00:01<00:02, 905.61it/s]
 37%|███▋      | 1234/3361 [00:01<00:02, 915.66it/s]
 39%|███▉      | 1326/3361 [00:01<00:02, 911.10it/s]
 42%|████▏     | 1418/3361 [00:01<00:02, 901.92it/s]
 45%|████▍     | 1509/3361 [00:01<00:02, 903.95it/s]
 48%|████▊     | 1600/3361 [00:01<00:01, 897.71it/s]
 50%|█████     | 1690/3361 [00:01<00:01, 888.10it/s]
 53%|█████▎    | 1780/3361 [00:01<00:01, 890.59it/s]
 56%|█████▌    | 1872/3361 [00:02<00:01, 895.30it/s]
 58%|█████▊    | 1962/3361 [00:02<00:01, 894.72it/s]
 61%|██████    | 2052/3361 [00:02<00:01, 887.48it/s]
 64%|██████▎   | 2141/3361 [00:02<00:01, 887.92it/s]
 66%|██████▋   | 2230/3361 [00:02<00:01, 872.77it/s]
 69%|██████▉   | 2318/3361 [00:02<00:01, 872.48it/s]
 72%|███████▏  | 2406/3361 [00:02<00:01, 873.70it/s]
 74%|███████▍  | 2494/3361 [00:02<00:00, 868.37it/s]
 77%|███████▋  | 2581/3361 [00:02<00:00, 865.91it/s]
 79%|███████▉  | 2668/3361 [00:03<00:00, 865.53it/s]
 82%|████████▏ | 2755/3361 [00:03<00:00, 863.72it/s]
 85%|████████▍ | 2842/3361 [00:03<00:00, 860.53it/s]
 87%|████████▋ | 2929/3361 [00:03<00:00, 849.65it/s]
 90%|████████▉ | 3014/3361 [00:03<00:00, 840.25it/s]
 92%|█████████▏| 3099/3361 [00:03<00:00, 832.90it/s]
 95%|█████████▍| 3183/3361 [00:03<00:00, 830.15it/s]
 97%|█████████▋| 3267/3361 [00:03<00:00, 823.10it/s]
100%|█████████▉| 3350/3361 [00:03<00:00, 817.55it/s]
100%|██████████| 3361/3361 [00:03<00:00, 871.50it/s]

  0%|          | 0/3361 [00:00<?, ?it/s]
  3%|▎         | 96/3361 [00:00<00:03, 954.36it/s]
  6%|▌         | 194/3361 [00:00<00:03, 967.94it/s]
  9%|▊         | 291/3361 [00:00<00:03, 962.90it/s]
 12%|█▏        | 388/3361 [00:00<00:03, 954.55it/s]
 14%|█▍        | 484/3361 [00:00<00:03, 907.95it/s]
 17%|█▋        | 578/3361 [00:00<00:03, 917.99it/s]
 20%|██        | 673/3361 [00:00<00:02, 921.62it/s]
 23%|██▎       | 768/3361 [00:00<00:02, 924.21it/s]
 26%|██▌       | 862/3361 [00:00<00:02, 926.42it/s]
 28%|██▊       | 955/3361 [00:01<00:02, 925.05it/s]
 31%|███       | 1048/3361 [00:01<00:02, 922.45it/s]
 34%|███▍      | 1141/3361 [00:01<00:02, 918.75it/s]
 37%|███▋      | 1233/3361 [00:01<00:02, 918.53it/s]
 39%|███▉      | 1325/3361 [00:01<00:02, 911.41it/s]
 42%|████▏     | 1417/3361 [00:01<00:02, 905.97it/s]
 45%|████▍     | 1508/3361 [00:01<00:02, 898.59it/s]
 48%|████▊     | 1599/3361 [00:01<00:01, 897.28it/s]
 50%|█████     | 1689/3361 [00:01<00:01, 892.46it/s]
 53%|█████▎    | 1779/3361 [00:01<00:01, 888.22it/s]
 56%|█████▌    | 1869/3361 [00:02<00:01, 887.93it/s]
 58%|█████▊    | 1959/3361 [00:02<00:01, 890.66it/s]
 61%|██████    | 2049/3361 [00:02<00:01, 885.37it/s]
 64%|██████▎   | 2141/3361 [00:02<00:01, 890.41it/s]
 66%|██████▋   | 2231/3361 [00:02<00:01, 886.68it/s]
 69%|██████▉   | 2320/3361 [00:02<00:01, 881.54it/s]
 72%|███████▏  | 2409/3361 [00:02<00:01, 877.10it/s]
 74%|███████▍  | 2497/3361 [00:02<00:00, 873.72it/s]
 77%|███████▋  | 2585/3361 [00:02<00:00, 873.64it/s]
 80%|███████▉  | 2673/3361 [00:02<00:00, 869.89it/s]
 82%|████████▏ | 2760/3361 [00:03<00:00, 867.01it/s]
 85%|████████▍ | 2847/3361 [00:03<00:00, 861.11it/s]
 87%|████████▋ | 2934/3361 [00:03<00:00, 854.92it/s]
 90%|████████▉ | 3020/3361 [00:03<00:00, 846.24it/s]
 92%|█████████▏| 3105/3361 [00:03<00:00, 837.11it/s]
 95%|█████████▍| 3189/3361 [00:03<00:00, 819.65it/s]
 97%|█████████▋| 3272/3361 [00:03<00:00, 822.08it/s]
100%|█████████▉| 3355/3361 [00:03<00:00, 819.48it/s]
100%|██████████| 3361/3361 [00:03<00:00, 885.07it/s]
2026-05-28 11:55:08.923388: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:55:19.839056: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-28 11:55:19.842098: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-28 11:55:19.863990: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:02:00.0 name: NVIDIA TITAN Xp computeCapability: 6.1
coreClock: 1.911GHz coreCount: 30 deviceMemorySize: 11.90GiB deviceMemoryBandwidth: 510.07GiB/s
2026-05-28 11:55:19.864117: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:55:19.869205: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-28 11:55:19.869276: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-28 11:55:19.871685: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-28 11:55:19.872926: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-28 11:55:19.876964: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-28 11:55:19.878759: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-28 11:55:19.880180: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-28 11:55:19.880801: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-28 11:55:19.881308: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-28 11:55:19.881452: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-28 11:55:19.882004: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:02:00.0 name: NVIDIA TITAN Xp computeCapability: 6.1
coreClock: 1.911GHz coreCount: 30 deviceMemorySize: 11.90GiB deviceMemoryBandwidth: 510.07GiB/s
2026-05-28 11:55:19.882072: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:55:19.882126: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-28 11:55:19.882161: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-28 11:55:19.882194: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-28 11:55:19.882228: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-28 11:55:19.882261: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-28 11:55:19.882294: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-28 11:55:19.882327: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-28 11:55:19.882788: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-28 11:55:19.882844: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-28 11:55:20.425211: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-28 11:55:20.425319: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-28 11:55:20.425338: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-28 11:55:20.426276: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11209 MB memory) -> physical GPU (device: 0, name: NVIDIA TITAN Xp, pci bus id: 0000:02:00.0, compute capability: 6.1)
2026-05-28 11:55:20.492791: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2026-05-28 11:55:20.512203: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2399915000 Hz
2026-05-28 11:55:24.785086: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-28 11:55:25.024273: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-28 11:55:25.029601: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-28 11:55:25.975819: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-28 11:55:26.091300: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)