Goal¶
- for each binding site, get the binding site class: implement Winner-take-all autoencoder: https://arxiv.org/pdf/1409.2752.pdf
TODO¶
- [x] implement Winner-take-all autoencoder: https://arxiv.org/pdf/1409.2752.pdf
- [x] Spatial Sparsity
- for each feature map, consider only the neuron with the highest acctivation accross spatial dimensions
- [x] Lifetime Sparsity
- for each feature map, consider only top k% in the batch (in the paper they used k={20, 5}
- [x] Spatial Sparsity
- [x] Evaluate each on the ChipNexus data
- visualize the deconv filters (also called atoms in the paper)
- visualize the activation layer (with and without the sparsity regularization)
- visualize the predicted track (with and without the sparsity regularization)
- [x] Train seq->chipnexus, and (seq, chipnexus) -> chipnexus models
- simply stack sequence+signal
- [ ] Run the final prediction turning off the regularization
- compare the two cases (on and off)
- [ ] compute the PWM wrt to every output footprint
- can we see any clear binding footprints?
Technical improvements¶
- [ ] use larger window size as input with valid convolutions
Organizatory¶
- [ ] setup the model function with easy hyper-parameters
Potential problems¶
- the width of the footprint could vary, hence a single deconv filter might not be enough to represent it
In [2]:
import keras
import pyBigWig
import matplotlib.pyplot as plt
from concise.preprocessing import encodeDNA
from concise.utils.fasta import read_fasta
from pybedtools import BedTool
from tqdm import tqdm
import pandas as pd
import numpy as np
from basepair.config import get_data_dir, create_tf_session
from basepair.datasets import get_sox2_data, seq_inp_exo_out
from basepair.math import softmax
from basepair.losses import twochannel_multinomial_nll
import basepair as bp
from basepair.layers import SpatialLifetimeSparsity
from basepair.plots import *
ddir = get_data_dir()
In [3]:
import keras.layers as kl
from keras.optimizers import Adam
from keras.callbacks import EarlyStopping, ModelCheckpoint, History
from keras.models import Model, load_model
In [4]:
def get_model(mfn, mkwargs):
"""Get the model"""
import datetime
mdir = f"{ddir}/processed/chipnexus/exp/models/wta-ae"
name = mfn + "_" + \
",".join([f'{k}={v}' for k,v in mkwargs.items()]) + \
"." + str(datetime.datetime.now()).replace(" ", "::")
!mkdir -p {mdir}
ckp_file = f"{mdir}/{name}.h5"
return eval(mfn)(**mkwargs), name, ckp_file
In [5]:
create_tf_session(1)
keras.backend.get_session().list_devices()
Out[5]:
In [5]:
dfc = get_sox2_data()
In [6]:
train, valid, test = seq_inp_exo_out()
In [7]:
[x.shape for x in train]
Out[7]:
In [8]:
dfc.head(2)
Out[8]:
In [9]:
from basepair.models import seq_dense, seq_dense_wta, wta_ae
In [10]:
# hyper-parameters
mfn = "seq_dense_wta"
mkwargs = dict(filters=21,
conv1_kernel_size=25,
tconv_kernel_size=25,
n_dil_layers=6,
wta_lifetime_rate=1.0,
n_bottleneck=10,
#use_seq=True,
lr=0.004)
In [16]:
# best valid so far: 108238.6558
model, name, ckp_file = get_model(mfn, mkwargs)
history = model.fit({"seq": train[0], "chip_nexus": train[1]}, train[1],
batch_size=256,
epochs=5,
validation_split=0.2,
callbacks=[EarlyStopping(patience=20),
History(),
ModelCheckpoint(ckp_file, save_best_only=True)]
)
# get the best model
model = load_model(ckp_file, custom_objects={"twochannel_multinomial_nll": twochannel_multinomial_nll,
"SpatialLifetimeSparsity": SpatialLifetimeSparsity})
In [20]:
dfh = pd.DataFrame(history.history)
In [ ]:
model.history
In [137]:
p = Seq2NexusTrack({"seq": train[0], "chip_nexus": train[1]}, #train[1], #
train[1], model)
In [138]:
p.plot()
In [139]:
p = Seq2NexusTrack({"seq": test[0], "chip_nexus": test[1]},
test[1], model)
In [140]:
p.plot(sort='top')
In [ ]:
p = CAENexus({"seq": test[0], "chip_nexus": test[1]},
test[1], test[2], model)
In [144]:
p.plot(10, sort='top', bottleneck_type='after', show_seq=True)
In [145]:
deconv_filters(model, figsize=(15, 4), ncol=5)
In [153]:
a = model.evaluate(test[0], test[1])
In [159]:
a
Out[159]: