ó ‡ˆ\c@sêdZddlZddlZddljZddlmZddl m Z ddl m Z ddl m Z ddlmZdd lmZdd lmZdd lmZdd lmZdd lmZde e fd„ƒYZdS(sRestricted Boltzmann Machine iÿÿÿÿN(texpiti(t BaseEstimator(tTransformerMixin(txrange(t check_array(tcheck_random_state(tgen_even_slices(tsafe_sparse_dot(t log_logistic(tcheck_is_fittedt BernoulliRBMcBs‰eZdZddddddd„Zd„Zd„Zd„Zd „Zd „Z d „Z dd „Z d „Z d„Z dd„ZRS(s¾ Bernoulli Restricted Boltzmann Machine (RBM). A Restricted Boltzmann Machine with binary visible units and binary hidden units. Parameters are estimated using Stochastic Maximum Likelihood (SML), also known as Persistent Contrastive Divergence (PCD) [2]. The time complexity of this implementation is ``O(d ** 2)`` assuming d ~ n_features ~ n_components. Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, optional Number of binary hidden units. learning_rate : float, optional The learning rate for weight updates. It is *highly* recommended to tune this hyper-parameter. Reasonable values are in the 10**[0., -3.] range. batch_size : int, optional Number of examples per minibatch. n_iter : int, optional Number of iterations/sweeps over the training dataset to perform during training. verbose : int, optional The verbosity level. The default, zero, means silent mode. random_state : integer or RandomState, optional A random number generator instance to define the state of the random permutations generator. If an integer is given, it fixes the seed. Defaults to the global numpy random number generator. Attributes ---------- intercept_hidden_ : array-like, shape (n_components,) Biases of the hidden units. intercept_visible_ : array-like, shape (n_features,) Biases of the visible units. components_ : array-like, shape (n_components, n_features) Weight matrix, where n_features in the number of visible units and n_components is the number of hidden units. Examples -------- >>> import numpy as np >>> from sklearn.neural_network import BernoulliRBM >>> X = np.array([[0, 0, 0], [0, 1, 1], [1, 0, 1], [1, 1, 1]]) >>> model = BernoulliRBM(n_components=2) >>> model.fit(X) BernoulliRBM(batch_size=10, learning_rate=0.1, n_components=2, n_iter=10, random_state=None, verbose=0) References ---------- [1] Hinton, G. E., Osindero, S. and Teh, Y. A fast learning algorithm for deep belief nets. Neural Computation 18, pp 1527-1554. https://www.cs.toronto.edu/~hinton/absps/fastnc.pdf [2] Tieleman, T. Training Restricted Boltzmann Machines using Approximations to the Likelihood Gradient. International Conference on Machine Learning (ICML) 2008 igš™™™™™¹?i icCs:||_||_||_||_||_||_dS(N(t n_componentst learning_ratet batch_sizetn_itertverboset random_state(tselfR R R RRR((s9lib/python2.7/site-packages/sklearn/neural_network/rbm.pyt__init__cs      cCs5t|dƒt|dddtjƒ}|j|ƒS(s`Compute the hidden layer activation probabilities, P(h=1|v=X). Parameters ---------- X : {array-like, sparse matrix} shape (n_samples, n_features) The data to be transformed. Returns ------- h : array, shape (n_samples, n_components) Latent representations of the data. t components_t accept_sparsetcsrtdtype(R Rtnptfloat64t _mean_hiddens(RtX((s9lib/python2.7/site-packages/sklearn/neural_network/rbm.pyt transformls cCs2t||jjƒ}||j7}t|d|ƒS(sNComputes the probabilities P(h=1|v). Parameters ---------- v : array-like, shape (n_samples, n_features) Values of the visible layer. Returns ------- h : array-like, shape (n_samples, n_components) Corresponding mean field values for the hidden layer. tout(RRtTtintercept_hidden_R(Rtvtp((s9lib/python2.7/site-packages/sklearn/neural_network/rbm.pyR~s  cCs(|j|ƒ}|jd|jƒ|kS(sŠSample from the distribution P(h|v). Parameters ---------- v : array-like, shape (n_samples, n_features) Values of the visible layer to sample from. rng : RandomState Random number generator to use. Returns ------- h : array-like, shape (n_samples, n_components) Values of the hidden layer. tsize(Rt random_sampletshape(RRtrngR ((s9lib/python2.7/site-packages/sklearn/neural_network/rbm.pyt_sample_hiddensscCsKtj||jƒ}||j7}t|d|ƒ|jd|jƒ|kS(sŠSample from the distribution P(v|h). Parameters ---------- h : array-like, shape (n_samples, n_components) Values of the hidden layer to sample from. rng : RandomState Random number generator to use. Returns ------- v : array-like, shape (n_samples, n_features) Values of the visible layer. RR!(RtdotRtintercept_visible_RR"R#(RthR$R ((s9lib/python2.7/site-packages/sklearn/neural_network/rbm.pyt_sample_visibles¢s cCsCt||jƒ tjdt||jjƒ|jƒjddƒS(sHComputes the free energy F(v) = - log sum_h exp(-E(v,h)). Parameters ---------- v : array-like, shape (n_samples, n_features) Values of the visible layer. Returns ------- free_energy : array-like, shape (n_samples,) The value of the free energy. itaxisi(RR'Rt logaddexpRRRtsum(RR((s9lib/python2.7/site-packages/sklearn/neural_network/rbm.pyt _free_energy·s cCs_t|dƒt|dƒs1t|jƒ|_n|j||jƒ}|j||jƒ}|S(sVPerform one Gibbs sampling step. Parameters ---------- v : array-like, shape (n_samples, n_features) Values of the visible layer to start from. Returns ------- v_new : array-like, shape (n_samples, n_features) Values of the visible layer after one Gibbs step. Rt random_state_(R thasattrRRR.R%R)(RRth_tv_((s9lib/python2.7/site-packages/sklearn/neural_network/rbm.pytgibbsÈs cCs$t|dddtjƒ}t|dƒs?t|jƒ|_nt|dƒs‹tj|jjdd|j |j dfƒd d ƒ|_ nt|d ƒs²tj |j ƒ|_ nt|d ƒsÝtj |j dƒ|_nt|d ƒs tj |j|j fƒ|_n|j||jƒdS(s3Fit the model to the data X which should contain a partial segment of the data. Parameters ---------- X : array-like, shape (n_samples, n_features) Training data. Returns ------- self : BernoulliRBM The fitted model. RRRR.Rig{®Gáz„?itordertFRR't h_samples_N(RRRR/RRR.tasarraytnormalR R#RtzerosRR'R R5t_fit(RRty((s9lib/python2.7/site-packages/sklearn/neural_network/rbm.pyt partial_fitÝs" !cCs9|j|ƒ}|j|j|ƒ}|j|ƒ}t|jƒ|jd}t|j|dtƒj}|t j |j|ƒ8}|j ||7_ |j ||j ddƒ|j ddƒ7_ |j|t j|j ddƒƒjƒ|j ddƒ7_d||jd|jƒ|ks