ó ‡ˆ\c@ sŒdZddlmZddlZddlmZmZddlZddl m Z ddl j Z ddlmZmZddlmZdd lmZdd lmZdd lmZdd lmZdd lmZmZddlmZdddgZ dd„Z!d„Z"d„Z#dd„Z%ddd„Z&dej'eeeƒfd„ƒYZ(de(fd„ƒYZ)de(fd„ƒYZ*dS(sQRandom Projection transformers Random Projections are a simple and computationally efficient way to reduce the dimensionality of the data by trading a controlled amount of accuracy (as additional variance) for faster processing times and smaller model sizes. The dimensions and distribution of Random Projections matrices are controlled so as to preserve the pairwise distances between any two samples of the dataset. The main theoretical result behind the efficiency of random projection is the `Johnson-Lindenstrauss lemma (quoting Wikipedia) `_: In mathematics, the Johnson-Lindenstrauss lemma is a result concerning low-distortion embeddings of points from high-dimensional into low-dimensional Euclidean space. The lemma states that a small set of points in a high-dimensional space can be embedded into a space of much lower dimension in such a way that distances between the points are nearly preserved. The map used for the embedding is at least Lipschitz, and can even be taken to be an orthogonal projection. iÿÿÿÿ(tdivisionN(tABCMetatabstractmethod(t assert_equali(t BaseEstimatortTransformerMixin(tsix(txrange(tcheck_random_state(tsafe_sparse_dot(tsample_without_replacement(t check_arraytcheck_is_fitted(tDataDimensionalityWarningtSparseRandomProjectiontGaussianRandomProjectiontjohnson_lindenstrauss_min_dimgš™™™™™¹?cC s¾tj|ƒ}tj|ƒ}tj|dkƒsHtj|dkƒr[td|ƒ‚ntj|ƒdkrƒtd|ƒ‚n|dd|dd}dtj|ƒ|jtjƒS( sÿFind a 'safe' number of components to randomly project to The distortion introduced by a random projection `p` only changes the distance between two points by a factor (1 +- eps) in an euclidean space with good probability. The projection `p` is an eps-embedding as defined by: (1 - eps) ||u - v||^2 < ||p(u) - p(v)||^2 < (1 + eps) ||u - v||^2 Where u and v are any rows taken from a dataset of shape [n_samples, n_features], eps is in ]0, 1[ and p is a projection by a random Gaussian N(0, 1) matrix with shape [n_components, n_features] (or a sparse Achlioptas matrix). The minimum number of components to guarantee the eps-embedding is given by: n_components >= 4 log(n_samples) / (eps^2 / 2 - eps^3 / 3) Note that the number of dimensions is independent of the original number of features but instead depends on the size of the dataset: the larger the dataset, the higher is the minimal dimensionality of an eps-embedding. Read more in the :ref:`User Guide `. Parameters ---------- n_samples : int or numpy array of int greater than 0, Number of samples. If an array is given, it will compute a safe number of components array-wise. eps : float or numpy array of float in ]0,1[, optional (default=0.1) Maximum distortion rate as defined by the Johnson-Lindenstrauss lemma. If an array is given, it will compute a safe number of components array-wise. Returns ------- n_components : int or numpy array of int, The minimal number of components to guarantee with good probability an eps-embedding with n_samples. Examples -------- >>> johnson_lindenstrauss_min_dim(1e6, eps=0.5) 663 >>> johnson_lindenstrauss_min_dim(1e6, eps=[0.5, 0.1, 0.01]) array([ 663, 11841, 1112658]) >>> johnson_lindenstrauss_min_dim([1e4, 1e5, 1e6], eps=0.1) array([ 7894, 9868, 11841]) References ---------- .. [1] https://en.wikipedia.org/wiki/Johnson%E2%80%93Lindenstrauss_lemma .. [2] Sanjoy Dasgupta and Anupam Gupta, 1999, "An elementary proof of the Johnson-Lindenstrauss Lemma." http://citeseer.ist.psu.edu/viewdoc/summary?doi=10.1.1.45.3654 gis1The JL bound is defined for eps in ]0, 1[, got %ris?The JL bound is defined for n_samples greater than zero, got %riii(tnptasarraytanyt ValueErrortlogtastypetint(t n_samplestepst denominator((s8lib/python2.7/site-packages/sklearn/random_projection.pyR5sB* cC sQ|dkr"dtj|ƒ}n+|dks:|dkrMtd|ƒ‚n|S(s.Factorize density check according to Li et al.tautoiis)Expected density in range ]0, 1], got: %r(RtsqrtR(tdensityt n_features((s8lib/python2.7/site-packages/sklearn/random_projection.pyt_check_density‡s   cC sB|dkrtd|ƒ‚n|dkr>td|ƒ‚ndS(s8Factorize argument checking for random matrix generationis.n_components must be strictly positive, got %ds,n_features must be strictly positive, got %dN(R(t n_componentsR((s8lib/python2.7/site-packages/sklearn/random_projection.pyt_check_input_size’s    cC sNt||ƒt|ƒ}|jddddtj|ƒd||fƒ}|S(s Generate a dense Gaussian random matrix. The components of the random matrix are drawn from N(0, 1.0 / n_components). Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, Dimensionality of the target projection space. n_features : int, Dimensionality of the original source space. random_state : int, RandomState instance or None, optional (default=None) Control the pseudo random number generator used to generate the matrix at fit time. If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Returns ------- components : numpy array of shape [n_components, n_features] The generated Gaussian random matrix. See Also -------- GaussianRandomProjection sparse_random_matrix tlocgtscalegð?tsize(R!RtnormalRR(R Rt random_statetrngt components((s8lib/python2.7/site-packages/sklearn/random_projection.pytgaussian_random_matrixœs "  Rc C set||ƒt||ƒ}t|ƒ}|dkrl|jdd||fƒdd}dtj|ƒ|Sg}d}|g}x_t|ƒD]Q} |j||ƒ} t|| d|ƒ} |j| ƒ|| 7}|j|ƒqŽWtj |ƒ}|jdddtj |ƒƒdd} t j | ||fd||fƒ}tjd|ƒtj|ƒ|SdS( s^Generalized Achlioptas random sparse matrix for random projection Setting density to 1 / 3 will yield the original matrix by Dimitris Achlioptas while setting a lower value will yield the generalization by Ping Li et al. If we note :math:`s = 1 / density`, the components of the random matrix are drawn from: - -sqrt(s) / sqrt(n_components) with probability 1 / 2s - 0 with probability 1 - 1 / s - +sqrt(s) / sqrt(n_components) with probability 1 / 2s Read more in the :ref:`User Guide `. Parameters ---------- n_components : int, Dimensionality of the target projection space. n_features : int, Dimensionality of the original source space. density : float in range ]0, 1] or 'auto', optional (default='auto') Ratio of non-zero component in the random projection matrix. If density = 'auto', the value is set to the minimum density as recommended by Ping Li et al.: 1 / sqrt(n_features). Use density = 1 / 3.0 if you want to reproduce the results from Achlioptas, 2001. random_state : int, RandomState instance or None, optional (default=None) Control the pseudo random number generator used to generate the matrix at fit time. If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Returns ------- components : array or CSR matrix with shape [n_components, n_features] The generated Gaussian random matrix. See Also -------- SparseRandomProjection gaussian_random_matrix References ---------- .. [1] Ping Li, T. Hastie and K. W. Church, 2006, "Very Sparse Random Projections". http://web.stanford.edu/~hastie/Papers/Ping/KDD06_rp.pdf .. [2] D. Achlioptas, 2001, "Database-friendly random projections", http://www.cs.ucsc.edu/~optas/papers/jl.pdf igà?iiR&R$tshapeN( R!RRtbinomialRRRR tappendt concatenateR$tspt csr_matrix( R RRR&R'R(tindicestoffsettindptrtit n_nonzero_it indices_itdata((s8lib/python2.7/site-packages/sklearn/random_projection.pytsparse_random_matrixÆs*>   #     )tBaseRandomProjectioncB sMeZdZeddedd„ƒZed„ƒZdd„Zd„Z RS(s~Base class for random projections. Warning: This class should not be used directly. Use derived classes instead. Rgš™™™™™¹?cC s(||_||_||_||_dS(N(R Rt dense_outputR&(tselfR RR9R&((s8lib/python2.7/site-packages/sklearn/random_projection.pyt__init__/s   cC sdS(s– Generate the random projection matrix Parameters ---------- n_components : int, Dimensionality of the target projection space. n_features : int, Dimensionality of the original source space. Returns ------- components : numpy array or CSR matrix [n_components, n_features] The generated random matrix. N((R:R R((s8lib/python2.7/site-packages/sklearn/random_projection.pyt_make_random_matrix7scC sWt|dddgƒ}|j\}}|jdkr¹td|d|jƒ|_|jdkr‚td|j||jfƒ‚q|j|krtd |j||j|fƒ‚qn`|jdkrÞtd |jƒ‚n/|j|kr tjd ||jft ƒn|j|_|j |j|ƒ|_ t |j j|j|fd d ƒ|S(s¡Generate a sparse random projection matrix Parameters ---------- X : numpy array or scipy.sparse of shape [n_samples, n_features] Training set: only the shape is used to find optimal random matrix dimensions based on the theory referenced in the afore mentioned papers. y Ignored Returns ------- self t accept_sparsetcsrtcscRRRisIeps=%f and n_samples=%d lead to a target dimension of %d which is invalidsseps=%f and n_samples=%d lead to a target dimension of %d which is larger than the original space with n_features=%ds+n_components must be greater than 0, got %ssšThe number of components is higher than the number of features: n_features < n_components (%s < %s).The dimensionality of the problem will not be reduced.terr_msgsLAn error has occurred the self.components_ matrix has not the proper shape.( R R*R RRt n_components_RtwarningstwarnR R<t components_R(R:tXtyRR((s8lib/python2.7/site-packages/sklearn/random_projection.pytfitJs<      cC sŽt|dddgƒ}t|dƒ|jd|jjdkrltd|jd|jjdfƒ‚nt||jjd|jƒ}|S(sProject the data by using matrix product with the random matrix Parameters ---------- X : numpy array or scipy.sparse of shape [n_samples, n_features] The input data to project into a smaller dimensional space. Returns ------- X_new : numpy array or scipy sparse of shape [n_samples, n_components] Projected array. R=R>R?RDis^Impossible to perform projection:X at fit stage had a different number of features. (%s != %s)R9(R R R*RDRR tTR9(R:REtX_new((s8lib/python2.7/site-packages/sklearn/random_projection.pyt transformŒs  $ N( t__name__t __module__t__doc__RtFalsetNoneR;R<RGRJ(((s8lib/python2.7/site-packages/sklearn/random_projection.pyR8's  BcB s)eZdZdddd„Zd„ZRS(s˜Reduce dimensionality through Gaussian random projection The components of the random matrix are drawn from N(0, 1 / n_components). Read more in the :ref:`User Guide `. Parameters ---------- n_components : int or 'auto', optional (default = 'auto') Dimensionality of the target projection space. n_components can be automatically adjusted according to the number of samples in the dataset and the bound given by the Johnson-Lindenstrauss lemma. In that case the quality of the embedding is controlled by the ``eps`` parameter. It should be noted that Johnson-Lindenstrauss lemma can yield very conservative estimated of the required number of components as it makes no assumption on the structure of the dataset. eps : strictly positive float, optional (default=0.1) Parameter to control the quality of the embedding according to the Johnson-Lindenstrauss lemma when n_components is set to 'auto'. Smaller values lead to better embedding and higher number of dimensions (n_components) in the target projection space. random_state : int, RandomState instance or None, optional (default=None) Control the pseudo random number generator used to generate the matrix at fit time. If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Attributes ---------- n_component_ : int Concrete number of components computed when n_components="auto". components_ : numpy array of shape [n_components, n_features] Random matrix used for the projection. Examples -------- >>> import numpy as np >>> from sklearn.random_projection import GaussianRandomProjection >>> X = np.random.rand(100, 10000) >>> transformer = GaussianRandomProjection() >>> X_new = transformer.fit_transform(X) >>> X_new.shape (100, 3947) See Also -------- SparseRandomProjection Rgš™™™™™¹?c C s/tt|ƒjd|d|dtd|ƒdS(NR RR9R&(tsuperRR;tTrue(R:R RR&((s8lib/python2.7/site-packages/sklearn/random_projection.pyR;ãs cC s"t|jƒ}t||d|ƒS(s– Generate the random projection matrix Parameters ---------- n_components : int, Dimensionality of the target projection space. n_features : int, Dimensionality of the original source space. Returns ------- components : numpy array or CSR matrix [n_components, n_features] The generated random matrix. R&(RR&R)(R:R RR&((s8lib/python2.7/site-packages/sklearn/random_projection.pyR<êsN(RKRLRMROR;R<(((s8lib/python2.7/site-packages/sklearn/random_projection.pyR¨s:cB s/eZdZdddedd„Zd„ZRS(sReduce dimensionality through sparse random projection Sparse random matrix is an alternative to dense random projection matrix that guarantees similar embedding quality while being much more memory efficient and allowing faster computation of the projected data. If we note `s = 1 / density` the components of the random matrix are drawn from: - -sqrt(s) / sqrt(n_components) with probability 1 / 2s - 0 with probability 1 - 1 / s - +sqrt(s) / sqrt(n_components) with probability 1 / 2s Read more in the :ref:`User Guide `. Parameters ---------- n_components : int or 'auto', optional (default = 'auto') Dimensionality of the target projection space. n_components can be automatically adjusted according to the number of samples in the dataset and the bound given by the Johnson-Lindenstrauss lemma. In that case the quality of the embedding is controlled by the ``eps`` parameter. It should be noted that Johnson-Lindenstrauss lemma can yield very conservative estimated of the required number of components as it makes no assumption on the structure of the dataset. density : float in range ]0, 1], optional (default='auto') Ratio of non-zero component in the random projection matrix. If density = 'auto', the value is set to the minimum density as recommended by Ping Li et al.: 1 / sqrt(n_features). Use density = 1 / 3.0 if you want to reproduce the results from Achlioptas, 2001. eps : strictly positive float, optional, (default=0.1) Parameter to control the quality of the embedding according to the Johnson-Lindenstrauss lemma when n_components is set to 'auto'. Smaller values lead to better embedding and higher number of dimensions (n_components) in the target projection space. dense_output : boolean, optional (default=False) If True, ensure that the output of the random projection is a dense numpy array even if the input and random projection matrix are both sparse. In practice, if the number of components is small the number of zero components in the projected data will be very small and it will be more CPU and memory efficient to use a dense representation. If False, the projected data uses a sparse representation if the input is sparse. random_state : int, RandomState instance or None, optional (default=None) Control the pseudo random number generator used to generate the matrix at fit time. If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Attributes ---------- n_component_ : int Concrete number of components computed when n_components="auto". components_ : CSR matrix with shape [n_components, n_features] Random matrix used for the projection. density_ : float in range 0.0 - 1.0 Concrete density computed from when density = "auto". Examples -------- >>> import numpy as np >>> from sklearn.random_projection import SparseRandomProjection >>> np.random.seed(42) >>> X = np.random.rand(100, 10000) >>> transformer = SparseRandomProjection() >>> X_new = transformer.fit_transform(X) >>> X_new.shape (100, 3947) >>> # very few components are non-zero >>> np.mean(transformer.components_ != 0) # doctest: +ELLIPSIS 0.0100... See Also -------- GaussianRandomProjection References ---------- .. [1] Ping Li, T. Hastie and K. W. Church, 2006, "Very Sparse Random Projections". http://web.stanford.edu/~hastie/Papers/Ping/KDD06_rp.pdf .. [2] D. Achlioptas, 2001, "Database-friendly random projections", https://users.soe.ucsc.edu/~optas/papers/jl.pdf Rgš™™™™™¹?c C s8tt|ƒjd|d|d|d|ƒ||_dS(NR RR9R&(RPRR;R(R:R RRR9R&((s8lib/python2.7/site-packages/sklearn/random_projection.pyR;ks cC s@t|jƒ}t|j|ƒ|_t||d|jd|ƒS(s– Generate the random projection matrix Parameters ---------- n_components : int, Dimensionality of the target projection space. n_features : int, Dimensionality of the original source space. Returns ------- components : numpy array or CSR matrix [n_components, n_features] The generated random matrix. RR&(RR&RRtdensity_R7(R:R RR&((s8lib/python2.7/site-packages/sklearn/random_projection.pyR<us  N(RKRLRMRNROR;R<(((s8lib/python2.7/site-packages/sklearn/random_projection.pyRsi  (+RMt __future__RRBtabcRRtnumpyRt numpy.testingRt scipy.sparsetsparseR.tbaseRRt externalsRtexternals.six.movesRtutilsRt utils.extmathR t utils.randomR tutils.validationR R t exceptionsR t__all__RRR!ROR)R7twith_metaclassR8RR(((s8lib/python2.7/site-packages/sklearn/random_projection.pyts6    R * `€Y