B \uj @sdZddlZddlmZmZmZmZddlm Z m Z ddl m Z ddl mZmZmZddlmZmZdd lmZdd lmZdd lmZdd lmZd ddZd!ddZd"ddZd#ddZGdddeeeZdS)$zLocally Linear EmbeddingN)eighsvdqrsolve)eye csr_matrix)eigsh) BaseEstimatorTransformerMixin_UnstableArchMixin)check_random_state check_array) stable_cumsum)check_is_fitted) FLOAT_DTYPES)NearestNeighborsMbP?cCst|td}t|tdd}|jd|jd}}tj||f|jd}tj||jd}xt|dddD]\}}|j ||} t | | j } t | } | dkr|| } n|} | j dd|jdd| 7<t | |dd} | t| ||ddf<qhW|S) abCompute barycenter weights of X from Y along the first axis We estimate the weights to assign to each point in Y[i] to recover the point X[i]. The barycenter weights sum to 1. Parameters ---------- X : array-like, shape (n_samples, n_dim) Z : array-like, shape (n_samples, n_neighbors, n_dim) reg : float, optional amount of regularization to add for the problem to be well-posed in the case of n_neighbors > n_dim Returns ------- B : array-like, shape (n_samples, n_neighbors) Notes ----- See developers note for more information. )dtypeT)rZallow_ndrr N)Zsym_pos)rrshapenpemptyrones enumerate transposeTdottraceflatrsum)XZreg n_samples n_neighborsBviACGrRwr.>lib/python3.7/site-packages/sklearn/manifold/locally_linear.pybarycenter_weightss    "r0c Cst|d|d|}|j}|jd}|j|ddddddf}t||||d}td||d|}t| | |f||fdS) a4Computes the barycenter weighted graph of k-Neighbors for points in X Parameters ---------- X : {array-like, NearestNeighbors} Sample data, shape = (n_samples, n_features), in the form of a numpy array or a NearestNeighbors object. n_neighbors : int Number of neighbors for each sample. reg : float, optional Amount of regularization when solving the least-squares problem. Only relevant if mode='barycenter'. If None, use the default. n_jobs : int or None, optional (default=None) The number of parallel jobs to run for neighbors search. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. Returns ------- A : sparse matrix in CSR format, shape = [n_samples, n_samples] A[i, j] is assigned the weight of edge that connects i to j. See also -------- sklearn.neighbors.kneighbors_graph sklearn.neighbors.radius_neighbors_graph r)n_jobsrF)return_distanceN)r#)r) rfit_fit_Xr kneighborsr0rZarangerZravel) r!r%r#r1Zknnr$inddataZindptrr.r.r/barycenter_kneighbors_graphCs! r8rarpackư>dc Cs:|dkr,|jddkr(||dkr(d}nd}|dkrt|}|dd|jd}y t|||d |||d \}} Wn.tk r} ztd | Wd d } ~ XYnX| d d |d ft||d fS|dkr*t|d r| }t ||||dfdd\}} t t |} | d d | ft|fStd|d S)a Find the null space of a matrix M. Parameters ---------- M : {array, matrix, sparse matrix, LinearOperator} Input covariance matrix: should be symmetric positive semi-definite k : integer Number of eigenvalues/vectors to return k_skip : integer, optional Number of low eigenvalues to skip. eigen_solver : string, {'auto', 'arpack', 'dense'} auto : algorithm will attempt to choose the best method for input data arpack : use arnoldi iteration in shift-invert mode. For this method, M may be a dense matrix, sparse matrix, or general linear operator. Warning: ARPACK can be unstable for some problems. It is best to try several random seeds in order to check results. dense : use standard dense matrix operations for the eigenvalue decomposition. For this method, M must be an array or matrix type. This method should be avoided for large problems. tol : float, optional Tolerance for 'arpack' method. Not used if eigen_solver=='dense'. max_iter : int Maximum number of iterations for 'arpack' method. Not used if eigen_solver=='dense' random_state : int, RandomState instance or None, optional (default=None) If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Used when ``solver`` == 'arpack'. autor r9denserg)Zsigmatolmaxiterv0zError in determining null-space with ARPACK. Error message: '%s'. Note that method='arpack' can fail when the weight matrix is singular or otherwise ill-behaved. method='dense' is recommended. See online documentation for more information.NtoarrayT)ZeigvalsZ overwrite_azUnrecognized eigen_solver '%s') rr Zuniformr RuntimeError ValueErrorrr hasattrrDrZargsortabs) Mkk_skip eigen_solverrAmax_iter random_staterCZ eigen_valuesZ eigen_vectorsmsgindexr.r.r/ null_spacens.+ &  rQr<standard-C6?-q=c ; Cs |dkrtd||dkr(td|t|d| d} | || j}|j\} }||krbtd|| krztd| |f|d krtd |d k}|d krt| ||| d }|rt|jd|ji|}|j| }n:|j||j| }|j dd|jd dd7<n|dkrR||dd}|||krDtd| j ||ddd}|ddddf}t j|d||ft jd}d|ddd f<t j| | ft jd}||k}xt| D]z}|||}||d 8}|rt|d dd }n,t ||j}t|ddddddf}|ddd|f|dddd|f<d|}xbt|D]V}|dd||df|dd||f|dd||||f<|||7}q^Wt|\}}|dd|ddf}|d }d|t t||k<||}t ||||\} }!|| |!ft ||j7<qW|rt|}n|dkr||krntd| j ||ddd}|ddddf}t | ||f}"t||}#t | |#g}$||k}|rx@t| D]4}|||||}%t|%dd\|"|<|$|<}&qW|$dC}$nnxlt| D]`}|||||}%t |%|%j}'t|'\}(})|(ddd|$|<|)dddddf|"|<q"Wd|$d}t |"d ddt |}*|*ddd|#f|$|dddf<|*dd|#df|dddf<t | |f}+x*t| D]}t |"||*||+|<qW|+|+ddddf}+|$dd|dfd|$ddd|fd},t |,}-t j| t d}.t!|$d}/|/ddddf|/ddddfd}0x0t| D]$}t "|0|dddf|-|.|<qW|.||#7}.t j| | ft jd}xTt| D]F}|.|}1|"|dd||1df}2t j#$|2d t %|1}3t &|1|3t |2jt |}4t j#$|4}5|5| kr|4d 9}4n|4|5}4|2dt 't |2|4|4d|3|+|dddf}6t ||||\} }!|| |!ft |6|6j7<|6d}7||||f|78<||||f|78<|||f|17<q8W|rt|}nd|dkr| j ||ddd}|ddddf}t | | f}||k}xt| D] }|||}8|8|8d 8}8|r"t|8ddd }9n,t |8|8j}t|ddddddf}9t ||df}|9ddd|f|ddddf<dt %||ddd f<t ||j}:t ||||\} }!|| |!f|:8<|||||fd7<qWt(||d|||| dS)aPerform a Locally Linear Embedding analysis on the data. Read more in the :ref:`User Guide `. Parameters ---------- X : {array-like, NearestNeighbors} Sample data, shape = (n_samples, n_features), in the form of a numpy array or a NearestNeighbors object. n_neighbors : integer number of neighbors to consider for each point. n_components : integer number of coordinates for the manifold. reg : float regularization constant, multiplies the trace of the local covariance matrix of the distances. eigen_solver : string, {'auto', 'arpack', 'dense'} auto : algorithm will attempt to choose the best method for input data arpack : use arnoldi iteration in shift-invert mode. For this method, M may be a dense matrix, sparse matrix, or general linear operator. Warning: ARPACK can be unstable for some problems. It is best to try several random seeds in order to check results. dense : use standard dense matrix operations for the eigenvalue decomposition. For this method, M must be an array or matrix type. This method should be avoided for large problems. tol : float, optional Tolerance for 'arpack' method Not used if eigen_solver=='dense'. max_iter : integer maximum number of iterations for the arpack solver. method : {'standard', 'hessian', 'modified', 'ltsa'} standard : use the standard locally linear embedding algorithm. see reference [1]_ hessian : use the Hessian eigenmap method. This method requires n_neighbors > n_components * (1 + (n_components + 1) / 2. see reference [2]_ modified : use the modified locally linear embedding algorithm. see reference [3]_ ltsa : use local tangent space alignment algorithm see reference [4]_ hessian_tol : float, optional Tolerance for Hessian eigenmapping method. Only used if method == 'hessian' modified_tol : float, optional Tolerance for modified LLE method. Only used if method == 'modified' random_state : int, RandomState instance or None, optional (default=None) If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Used when ``solver`` == 'arpack'. n_jobs : int or None, optional (default=None) The number of parallel jobs to run for neighbors search. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. Returns ------- Y : array-like, shape [n_samples, n_components] Embedding vectors. squared_error : float Reconstruction error for the embedding vectors. Equivalent to ``norm(Y - W Y, 'fro')**2``, where W are the reconstruction weights. References ---------- .. [1] `Roweis, S. & Saul, L. Nonlinear dimensionality reduction by locally linear embedding. Science 290:2323 (2000).` .. [2] `Donoho, D. & Grimes, C. Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data. Proc Natl Acad Sci U S A. 100:5591 (2003).` .. [3] `Zhang, Z. & Wang, J. MLLE: Modified Locally Linear Embedding Using Multiple Weights.` http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.70.382 .. [4] `Zhang, Z. & Zha, H. Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. Journal of Shanghai Univ. 8:406 (2004)` )r<r9r?zunrecognized eigen_solver '%s')rRhessianmodifiedltsazunrecognized method '%s'r)r%r1z>output dimension must be less than or equal to input dimensionzHExpected n_neighbors <= n_samples, but n_samples = %d, n_neighbors = %drzn_neighbors must be positiver?rR)r%r#r1formatNrUr z^for method='hessian', n_neighbors must be greater than [n_components * (n_components + 3) / 2]F)r%r2)r)Z full_matricesr@rVz1modified LLE requires n_neighbors >= n_componentsTgMbP?rWg?)rKrLrArMrN))rFrr3r4rr8rrXrZtocsrrDrr5rrZfloat64ZzerosrangeZmeanrrrrr whererHZmeshgridrminrrZmedianintrZ searchsortedZlinalgZnormZsqrtZfullZouterrQ);r!r% n_componentsr#rLrArMmethod hessian_tol modified_tolrNr1ZnbrsNZd_inZM_sparseWrIZdp neighborsZYiZuse_svdr(ZGiUZCijrJQr,r-SZnbrs_xZnbrs_yVZnevZevalsZX_nbrs_ZC_nbrsZeviZviZtmpZw_regZrhoZetaZs_rangeZ evals_cumsumZ eta_rangeZs_iZViZalpha_ihZnorm_hZWiZWi_sum1ZXir'ZGiGiTr.r.r/locally_linear_embeddingsd      &   (.  $        ,(4  ,$          $" rkc @s>eZdZdZdd d ZddZdddZdddZddZd S)LocallyLinearEmbeddingaoLocally Linear Embedding Read more in the :ref:`User Guide `. Parameters ---------- n_neighbors : integer number of neighbors to consider for each point. n_components : integer number of coordinates for the manifold reg : float regularization constant, multiplies the trace of the local covariance matrix of the distances. eigen_solver : string, {'auto', 'arpack', 'dense'} auto : algorithm will attempt to choose the best method for input data arpack : use arnoldi iteration in shift-invert mode. For this method, M may be a dense matrix, sparse matrix, or general linear operator. Warning: ARPACK can be unstable for some problems. It is best to try several random seeds in order to check results. dense : use standard dense matrix operations for the eigenvalue decomposition. For this method, M must be an array or matrix type. This method should be avoided for large problems. tol : float, optional Tolerance for 'arpack' method Not used if eigen_solver=='dense'. max_iter : integer maximum number of iterations for the arpack solver. Not used if eigen_solver=='dense'. method : string ('standard', 'hessian', 'modified' or 'ltsa') standard : use the standard locally linear embedding algorithm. see reference [1] hessian : use the Hessian eigenmap method. This method requires ``n_neighbors > n_components * (1 + (n_components + 1) / 2`` see reference [2] modified : use the modified locally linear embedding algorithm. see reference [3] ltsa : use local tangent space alignment algorithm see reference [4] hessian_tol : float, optional Tolerance for Hessian eigenmapping method. Only used if ``method == 'hessian'`` modified_tol : float, optional Tolerance for modified LLE method. Only used if ``method == 'modified'`` neighbors_algorithm : string ['auto'|'brute'|'kd_tree'|'ball_tree'] algorithm to use for nearest neighbors search, passed to neighbors.NearestNeighbors instance random_state : int, RandomState instance or None, optional (default=None) If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Used when ``eigen_solver`` == 'arpack'. n_jobs : int or None, optional (default=None) The number of parallel jobs to run. ``None`` means 1 unless in a :obj:`joblib.parallel_backend` context. ``-1`` means using all processors. See :term:`Glossary ` for more details. Attributes ---------- embedding_ : array-like, shape [n_samples, n_components] Stores the embedding vectors reconstruction_error_ : float Reconstruction error associated with `embedding_` nbrs_ : NearestNeighbors object Stores nearest neighbors instance, including BallTree or KDtree if applicable. Examples -------- >>> from sklearn.datasets import load_digits >>> from sklearn.manifold import LocallyLinearEmbedding >>> X, _ = load_digits(return_X_y=True) >>> X.shape (1797, 64) >>> embedding = LocallyLinearEmbedding(n_components=2) >>> X_transformed = embedding.fit_transform(X[:100]) >>> X_transformed.shape (100, 2) References ---------- .. [1] `Roweis, S. & Saul, L. Nonlinear dimensionality reduction by locally linear embedding. Science 290:2323 (2000).` .. [2] `Donoho, D. & Grimes, C. Hessian eigenmaps: Locally linear embedding techniques for high-dimensional data. Proc Natl Acad Sci U S A. 100:5591 (2003).` .. [3] `Zhang, Z. & Wang, J. MLLE: Modified Locally Linear Embedding Using Multiple Weights.` http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.70.382 .. [4] `Zhang, Z. & Zha, H. Principal manifolds and nonlinear dimensionality reduction via tangent space alignment. Journal of Shanghai Univ. 8:406 (2004)` r MbP?r<ư>r;rR-C6?-q=Nc CsL||_||_||_||_||_||_||_||_| |_| |_ | |_ | |_ dS)N) r%r]r#rLrArMr^r_r`rNneighbors_algorithmr1) selfr%r]r#rLrArMr^r_r`rrrNr1r.r.r/__init__|szLocallyLinearEmbedding.__init__cCszt|j|j|jd|_t|j}t|td}|j |t |j|j|j |j |j |j|j|j|j||j|jd \|_|_dS)N) algorithmr1)r) rLrArMr^r_r`rNr#r1)rr%rrr1nbrs_r rNrfloatr3rkr]rLrArMr^r_r`r# embedding_Zreconstruction_error_)rsr!rNr.r.r/_fit_transforms     z%LocallyLinearEmbedding._fit_transformcCs|||S)a Compute the embedding vectors for data X Parameters ---------- X : array-like of shape [n_samples, n_features] training set. y : Ignored Returns ------- self : returns an instance of self. )ry)rsr!yr.r.r/r3s zLocallyLinearEmbedding.fitcCs|||jS)a-Compute the embedding vectors for data X and transform X. Parameters ---------- X : array-like of shape [n_samples, n_features] training set. y : Ignored Returns ------- X_new : array-like, shape (n_samples, n_components) )ryrx)rsr!rzr.r.r/ fit_transforms z$LocallyLinearEmbedding.fit_transformcCst|dt|}|jj||jdd}t||jj||jd}t |j d|j f}x6t |j dD]$}t |j||j||||<qdW|S)a Transform new points into embedding space. Parameters ---------- X : array-like, shape = [n_samples, n_features] Returns ------- X_new : array, shape = [n_samples, n_components] Notes ----- Because of scaling performed by this method, it is discouraged to use it together with methods that are not scale-invariant (like SVMs) rvF)r%r2)r#r)rrrvr5r%r0r4r#rrrr]rYrrxr)rsr!r6ZweightsZX_newr(r.r.r/ transforms   $z LocallyLinearEmbedding.transform) rmr rnr<ror;rRrprqr<NN)N)N) __name__ __module__ __qualname____doc__rtryr3r{r|r.r.r.r/rl sq   rl)r)rN)rr9r:r;N) rr<r:r;rRrSrTNN) rZnumpyrZ scipy.linalgrrrrZ scipy.sparserrZscipy.sparse.linalgrbaser r r Zutilsr rZ utils.extmathrZutils.validationrrrcrr0r8rQrkrlr.r.r.r/s*      / + N L