Lmod Warning:
-------------------------------------------------------------------------------
The following dependent module(s) are not currently loaded: curl/8.4.0
(required by: htslib/1.16)
-------------------------------------------------------------------------------




The following have been reloaded with a version change:
  1) curl/8.4.0 => curl/8.17.0     2) openssl/3.0.7 => openssl/3.6.0

2026-05-11 03:47:10.102310: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:53:55.398109: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 03:53:55.440742: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-11 03:53:55.537944: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:05:00.0 name: NVIDIA TITAN Xp computeCapability: 6.1
coreClock: 1.911GHz coreCount: 30 deviceMemorySize: 11.90GiB deviceMemoryBandwidth: 510.07GiB/s
2026-05-11 03:53:55.538056: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:53:56.509095: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:53:56.509264: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:53:56.873208: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 03:53:57.548537: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 03:53:58.247122: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 03:53:58.580320: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 03:53:58.956101: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:53:58.964641: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 03:53:58.965449: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-11 03:53:58.965745: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 03:53:58.967109: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:05:00.0 name: NVIDIA TITAN Xp computeCapability: 6.1
coreClock: 1.911GHz coreCount: 30 deviceMemorySize: 11.90GiB deviceMemoryBandwidth: 510.07GiB/s
2026-05-11 03:53:58.967186: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:53:58.967241: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:53:58.967288: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:53:58.967333: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 03:53:58.967385: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 03:53:58.967429: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 03:53:58.967473: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 03:53:58.967544: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:53:58.969463: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 03:53:58.969546: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 03:54:01.769142: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-11 03:54:01.769261: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-11 03:54:01.769283: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-11 03:54:01.772872: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11209 MB memory) -> physical GPU (device: 0, name: NVIDIA TITAN Xp, pci bus id: 0000:05:00.0, compute capability: 6.1)
2026-05-11 03:54:13.599642: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-05-11 03:54:13.600266: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2400255000 Hz
2026-05-11 03:54:17.156145: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 03:54:18.821394: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 03:54:18.828173: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 03:54:54.799680: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-11 03:54:54.868352: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'jitters', 'index', 'status', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])
2026-05-11 04:08:40.602948: W tensorflow/python/util/util.cc:348] Sets are not currently considered sequences, but this may change in the future, so consider avoiding using them.
2026-05-11 04:10:41.448346: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:15:45.059723: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 04:15:45.068287: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-11 04:15:45.129430: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:05:00.0 name: NVIDIA TITAN Xp computeCapability: 6.1
coreClock: 1.911GHz coreCount: 30 deviceMemorySize: 11.90GiB deviceMemoryBandwidth: 510.07GiB/s
2026-05-11 04:15:45.129546: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:15:45.156235: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:15:45.156330: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:15:45.170351: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 04:15:45.183468: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 04:15:45.199336: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 04:15:45.212936: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 04:15:45.225129: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:15:45.226110: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 04:15:45.226462: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-11 04:15:45.226600: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 04:15:45.227596: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:05:00.0 name: NVIDIA TITAN Xp computeCapability: 6.1
coreClock: 1.911GHz coreCount: 30 deviceMemorySize: 11.90GiB deviceMemoryBandwidth: 510.07GiB/s
2026-05-11 04:15:45.227693: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:15:45.227755: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:15:45.227810: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:15:45.227863: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 04:15:45.227915: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 04:15:45.227968: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 04:15:45.228021: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 04:15:45.228100: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:15:45.229484: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 04:15:45.229583: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:15:45.938511: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-11 04:15:45.938616: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-11 04:15:45.938635: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-11 04:15:45.943664: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11209 MB memory) -> physical GPU (device: 0, name: NVIDIA TITAN Xp, pci bus id: 0000:05:00.0, compute capability: 6.1)
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)

batch:   0%|          | 0/48 [00:00<?, ?it/s]2026-05-11 04:15:48.358786: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2026-05-11 04:15:48.359443: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2400255000 Hz
2026-05-11 04:15:49.121945: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:15:49.681224: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:15:49.682916: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:15:51.643239: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-11 04:15:51.797269: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/engine/functional.py:595: UserWarning: Input dict contained keys ['coordinates', 'true_profiles', 'true_logcounts', 'rev_comp'] which did not match any model input. They will be ignored by the model.
  [n for n in tensors.keys() if n not in ref_input_names])

batch:   2%|▏         | 1/48 [00:55<43:08, 55.07s/it]
batch:   4%|▍         | 2/48 [00:55<17:34, 22.91s/it]
batch:   6%|▋         | 3/48 [00:55<09:28, 12.63s/it]
batch:   8%|▊         | 4/48 [00:56<05:43,  7.81s/it]
batch:  10%|█         | 5/48 [00:56<03:41,  5.15s/it]
batch:  12%|█▎        | 6/48 [00:57<02:28,  3.53s/it]
batch:  15%|█▍        | 7/48 [00:57<01:42,  2.50s/it]
batch:  17%|█▋        | 8/48 [00:57<01:13,  1.84s/it]
batch:  19%|█▉        | 9/48 [00:58<00:54,  1.39s/it]
batch:  21%|██        | 10/48 [00:58<00:41,  1.08s/it]
batch:  23%|██▎       | 11/48 [00:59<00:32,  1.13it/s]
batch:  25%|██▌       | 12/48 [00:59<00:26,  1.36it/s]
batch:  27%|██▋       | 13/48 [00:59<00:22,  1.58it/s]
batch:  29%|██▉       | 14/48 [01:00<00:19,  1.79it/s]
batch:  31%|███▏      | 15/48 [01:00<00:16,  1.96it/s]
batch:  33%|███▎      | 16/48 [01:01<00:15,  2.11it/s]
batch:  35%|███▌      | 17/48 [01:01<00:13,  2.22it/s]
batch:  38%|███▊      | 18/48 [01:01<00:13,  2.25it/s]
batch:  40%|███▉      | 19/48 [01:02<00:12,  2.28it/s]
batch:  42%|████▏     | 20/48 [01:02<00:11,  2.35it/s]
batch:  44%|████▍     | 21/48 [01:03<00:11,  2.40it/s]
batch:  46%|████▌     | 22/48 [01:03<00:10,  2.37it/s]
batch:  48%|████▊     | 23/48 [01:03<00:10,  2.40it/s]
batch:  50%|█████     | 24/48 [01:04<00:10,  2.37it/s]
batch:  52%|█████▏    | 25/48 [01:04<00:09,  2.42it/s]
batch:  54%|█████▍    | 26/48 [01:05<00:08,  2.46it/s]
batch:  56%|█████▋    | 27/48 [01:05<00:08,  2.49it/s]
batch:  58%|█████▊    | 28/48 [01:05<00:07,  2.51it/s]
batch:  60%|██████    | 29/48 [01:06<00:07,  2.52it/s]
batch:  62%|██████▎   | 30/48 [01:06<00:07,  2.53it/s]
batch:  65%|██████▍   | 31/48 [01:07<00:06,  2.53it/s]
batch:  67%|██████▋   | 32/48 [01:07<00:06,  2.54it/s]
batch:  69%|██████▉   | 33/48 [01:07<00:05,  2.55it/s]
batch:  71%|███████   | 34/48 [01:08<00:05,  2.48it/s]
batch:  73%|███████▎  | 35/48 [01:08<00:05,  2.52it/s]
batch:  75%|███████▌  | 36/48 [01:09<00:04,  2.46it/s]
batch:  77%|███████▋  | 37/48 [01:09<00:04,  2.38it/s]
batch:  79%|███████▉  | 38/48 [01:10<00:04,  2.40it/s]
batch:  81%|████████▏ | 39/48 [01:10<00:03,  2.37it/s]
batch:  83%|████████▎ | 40/48 [01:10<00:03,  2.34it/s]
batch:  85%|████████▌ | 41/48 [01:11<00:02,  2.34it/s]
batch:  88%|████████▊ | 42/48 [01:11<00:02,  2.40it/s]
batch:  90%|████████▉ | 43/48 [01:12<00:02,  2.45it/s]
batch:  92%|█████████▏| 44/48 [01:12<00:01,  2.41it/s]
batch:  94%|█████████▍| 45/48 [01:12<00:01,  2.45it/s]
batch:  96%|█████████▌| 46/48 [01:13<00:00,  2.49it/s]
batch:  98%|█████████▊| 47/48 [01:13<00:00,  2.47it/s]
batch: 100%|██████████| 48/48 [01:13<00:00,  1.54s/it]

  0%|          | 0/3007 [00:00<?, ?it/s]
  3%|▎         | 82/3007 [00:00<00:03, 814.74it/s]
  6%|▌         | 180/3007 [00:00<00:03, 904.10it/s]
  9%|▉         | 282/3007 [00:00<00:02, 954.77it/s]
 13%|█▎        | 381/3007 [00:00<00:02, 967.89it/s]
 16%|█▌        | 479/3007 [00:00<00:02, 971.79it/s]
 19%|█▉        | 577/3007 [00:00<00:02, 971.06it/s]
 22%|██▏       | 676/3007 [00:00<00:02, 971.50it/s]
 26%|██▌       | 774/3007 [00:00<00:02, 973.94it/s]
 29%|██▉       | 873/3007 [00:00<00:02, 973.41it/s]
 32%|███▏      | 973/3007 [00:01<00:02, 978.28it/s]
 36%|███▌      | 1071/3007 [00:01<00:02, 959.89it/s]
 39%|███▉      | 1168/3007 [00:01<00:01, 960.20it/s]
 42%|████▏     | 1265/3007 [00:01<00:01, 945.55it/s]
 45%|████▌     | 1360/3007 [00:01<00:01, 931.44it/s]
 48%|████▊     | 1454/3007 [00:01<00:01, 933.03it/s]
 51%|█████▏    | 1548/3007 [00:01<00:01, 930.14it/s]
 55%|█████▍    | 1642/3007 [00:01<00:01, 931.87it/s]
 58%|█████▊    | 1736/3007 [00:01<00:01, 925.36it/s]
 61%|██████    | 1829/3007 [00:01<00:01, 921.36it/s]
 64%|██████▍   | 1922/3007 [00:02<00:01, 921.54it/s]
 67%|██████▋   | 2015/3007 [00:02<00:01, 920.18it/s]
 70%|███████   | 2108/3007 [00:02<00:00, 912.80it/s]
 73%|███████▎  | 2200/3007 [00:02<00:00, 906.48it/s]
 76%|███████▋  | 2293/3007 [00:02<00:00, 909.74it/s]
 79%|███████▉  | 2387/3007 [00:02<00:00, 912.62it/s]
 82%|████████▏ | 2479/3007 [00:02<00:00, 911.54it/s]
 86%|████████▌ | 2573/3007 [00:02<00:00, 919.08it/s]
 89%|████████▊ | 2665/3007 [00:02<00:00, 898.35it/s]
 92%|█████████▏| 2755/3007 [00:02<00:00, 892.03it/s]
 95%|█████████▍| 2845/3007 [00:03<00:00, 884.91it/s]
 98%|█████████▊| 2934/3007 [00:03<00:00, 865.48it/s]
100%|██████████| 3007/3007 [00:03<00:00, 921.86it/s]

  0%|          | 0/3007 [00:00<?, ?it/s]
  3%|▎         | 99/3007 [00:00<00:02, 976.95it/s]
  7%|▋         | 198/3007 [00:00<00:02, 984.42it/s]
 10%|█         | 302/3007 [00:00<00:02, 1006.40it/s]
 13%|█▎        | 403/3007 [00:00<00:02, 989.35it/s] 
 17%|█▋        | 502/3007 [00:00<00:02, 981.60it/s]
 20%|█▉        | 601/3007 [00:00<00:02, 973.36it/s]
 23%|██▎       | 701/3007 [00:00<00:02, 978.77it/s]
 27%|██▋       | 799/3007 [00:00<00:02, 978.72it/s]
 30%|██▉       | 898/3007 [00:00<00:02, 978.94it/s]
 33%|███▎      | 996/3007 [00:01<00:02, 974.23it/s]
 36%|███▋      | 1094/3007 [00:01<00:01, 960.15it/s]
 40%|███▉      | 1191/3007 [00:01<00:01, 961.92it/s]
 43%|████▎     | 1288/3007 [00:01<00:01, 944.85it/s]
 46%|████▌     | 1383/3007 [00:01<00:01, 936.28it/s]
 49%|████▉     | 1477/3007 [00:01<00:01, 926.99it/s]
 52%|█████▏    | 1570/3007 [00:01<00:01, 924.40it/s]
 55%|█████▌    | 1664/3007 [00:01<00:01, 924.92it/s]
 58%|█████▊    | 1757/3007 [00:01<00:01, 921.81it/s]
 62%|██████▏   | 1852/3007 [00:01<00:01, 928.44it/s]
 65%|██████▍   | 1945/3007 [00:02<00:01, 917.31it/s]
 68%|██████▊   | 2039/3007 [00:02<00:01, 923.09it/s]
 71%|███████   | 2132/3007 [00:02<00:00, 910.71it/s]
 74%|███████▍  | 2224/3007 [00:02<00:00, 907.71it/s]
 77%|███████▋  | 2315/3007 [00:02<00:00, 908.01it/s]
 80%|████████  | 2407/3007 [00:02<00:00, 906.06it/s]
 83%|████████▎ | 2500/3007 [00:02<00:00, 911.68it/s]
 86%|████████▌ | 2592/3007 [00:02<00:00, 910.55it/s]
 89%|████████▉ | 2684/3007 [00:02<00:00, 895.97it/s]
 92%|█████████▏| 2774/3007 [00:02<00:00, 878.42it/s]
 95%|█████████▌| 2862/3007 [00:03<00:00, 867.32it/s]
 98%|█████████▊| 2949/3007 [00:03<00:00, 841.05it/s]
100%|██████████| 3007/3007 [00:03<00:00, 922.55it/s]
2026-05-11 04:20:15.707332: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:22:52.004740: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 04:22:52.014090: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcuda.so.1
2026-05-11 04:22:52.064216: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:05:00.0 name: NVIDIA TITAN Xp computeCapability: 6.1
coreClock: 1.911GHz coreCount: 30 deviceMemorySize: 11.90GiB deviceMemoryBandwidth: 510.07GiB/s
2026-05-11 04:22:52.064370: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:22:52.076349: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:22:52.076458: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:22:52.081496: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 04:22:52.086288: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 04:22:52.094870: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 04:22:52.100449: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 04:22:52.105327: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:22:52.106462: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 04:22:52.107038: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2026-05-11 04:22:52.107282: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2026-05-11 04:22:52.107881: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1720] Found device 0 with properties: 
pciBusID: 0000:05:00.0 name: NVIDIA TITAN Xp computeCapability: 6.1
coreClock: 1.911GHz coreCount: 30 deviceMemorySize: 11.90GiB deviceMemoryBandwidth: 510.07GiB/s
2026-05-11 04:22:52.107948: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:22:52.107999: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:22:52.108044: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:22:52.108117: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcufft.so.10
2026-05-11 04:22:52.108163: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcurand.so.10
2026-05-11 04:22:52.108207: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusolver.so.10
2026-05-11 04:22:52.108251: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcusparse.so.11
2026-05-11 04:22:52.108321: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:22:52.109360: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1862] Adding visible gpu devices: 0
2026-05-11 04:22:52.109446: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2026-05-11 04:22:52.773402: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1261] Device interconnect StreamExecutor with strength 1 edge matrix:
2026-05-11 04:22:52.773522: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1267]      0 
2026-05-11 04:22:52.773542: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1280] 0:   N 
2026-05-11 04:22:52.778147: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1406] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 11209 MB memory) -> physical GPU (device: 0, name: NVIDIA TITAN Xp, pci bus id: 0000:05:00.0, compute capability: 6.1)
2026-05-11 04:22:52.840642: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:196] None of the MLIR optimization passes are enabled (registered 0 passes)
2026-05-11 04:22:52.862492: I tensorflow/core/platform/profile_utils/cpu_utils.cc:112] CPU Frequency: 2400255000 Hz
2026-05-11 04:22:56.748171: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublas.so.11
2026-05-11 04:22:57.055041: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcublasLt.so.11
2026-05-11 04:22:57.059389: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudnn.so.8
2026-05-11 04:22:57.978548: W tensorflow/stream_executor/gpu/asm_compiler.cc:63] Running ptxas --version returned 256
2026-05-11 04:22:58.046298: W tensorflow/stream_executor/gpu/redzone_allocator.cc:314] Internal: ptxas exited with non-zero error code 256, output: 
Relying on driver to perform ptx compilation. 
Modify $PATH to customize ptxas location.
This message will be only logged once.
RuntimeError: module compiled against API version 0xe but this version of numpy is 0xd
/home/users/shouvikm/miniconda3/envs/bpnet/lib/python3.7/site-packages/tensorflow/python/keras/layers/core.py:1059: UserWarning: bpnet.model.arch is not loaded, but a Lambda layer uses it. It may cause errors.
  , UserWarning)