ó ‡ˆ\c@ sˆdZddlmZddlZddlmZddlmZddl m Z m Z dd l m Z mZd efd „ƒYZdS( s*Incremental Principal Components Analysis.i’’’’(tdivisionN(tlinalgi(t_BasePCAi(t check_arrayt gen_batches(tsvd_flipt_incremental_mean_and_vartIncrementalPCAcB s>eZdZdeedd„Zdd„Zded„ZRS(s}Incremental principal components analysis (IPCA). Linear dimensionality reduction using Singular Value Decomposition of centered data, keeping only the most significant singular vectors to project the data to a lower dimensional space. Depending on the size of the input data, this algorithm can be much more memory efficient than a PCA. This algorithm has constant memory complexity, on the order of ``batch_size``, enabling use of np.memmap files without loading the entire file into memory. The computational overhead of each SVD is ``O(batch_size * n_features ** 2)``, but only 2 * batch_size samples remain in memory at a time. There will be ``n_samples / batch_size`` SVD computations to get the principal components, versus 1 large SVD of complexity ``O(n_samples * n_features ** 2)`` for PCA. Read more in the :ref:`User Guide `. Parameters ---------- n_components : int or None, (default=None) Number of components to keep. If ``n_components `` is ``None``, then ``n_components`` is set to ``min(n_samples, n_features)``. whiten : bool, optional When True (False by default) the ``components_`` vectors are divided by ``n_samples`` times ``components_`` to ensure uncorrelated outputs with unit component-wise variances. Whitening will remove some information from the transformed signal (the relative variance scales of the components) but can sometimes improve the predictive accuracy of the downstream estimators by making data respect some hard-wired assumptions. copy : bool, (default=True) If False, X will be overwritten. ``copy=False`` can be used to save memory but is unsafe for general use. batch_size : int or None, (default=None) The number of samples to use for each batch. Only used when calling ``fit``. If ``batch_size`` is ``None``, then ``batch_size`` is inferred from the data and set to ``5 * n_features``, to provide a balance between approximation accuracy and memory consumption. Attributes ---------- components_ : array, shape (n_components, n_features) Components with maximum variance. explained_variance_ : array, shape (n_components,) Variance explained by each of the selected components. explained_variance_ratio_ : array, shape (n_components,) Percentage of variance explained by each of the selected components. If all components are stored, the sum of explained variances is equal to 1.0. singular_values_ : array, shape (n_components,) The singular values corresponding to each of the selected components. The singular values are equal to the 2-norms of the ``n_components`` variables in the lower-dimensional space. mean_ : array, shape (n_features,) Per-feature empirical mean, aggregate over calls to ``partial_fit``. var_ : array, shape (n_features,) Per-feature empirical variance, aggregate over calls to ``partial_fit``. noise_variance_ : float The estimated noise covariance following the Probabilistic PCA model from Tipping and Bishop 1999. See "Pattern Recognition and Machine Learning" by C. Bishop, 12.2.1 p. 574 or http://www.miketipping.com/papers/met-mppca.pdf. n_components_ : int The estimated number of components. Relevant when ``n_components=None``. n_samples_seen_ : int The number of samples processed by the estimator. Will be reset on new calls to fit, but increments across ``partial_fit`` calls. Examples -------- >>> from sklearn.datasets import load_digits >>> from sklearn.decomposition import IncrementalPCA >>> X, _ = load_digits(return_X_y=True) >>> transformer = IncrementalPCA(n_components=7, batch_size=200) >>> # either partially fit on smaller batches of data >>> transformer.partial_fit(X[:100, :]) IncrementalPCA(batch_size=200, copy=True, n_components=7, whiten=False) >>> # or let the fit function itself divide the data into batches >>> X_transformed = transformer.fit_transform(X) >>> X_transformed.shape (1797, 7) Notes ----- Implements the incremental PCA model from: `D. Ross, J. Lim, R. Lin, M. Yang, Incremental Learning for Robust Visual Tracking, International Journal of Computer Vision, Volume 77, Issue 1-3, pp. 125-141, May 2008.` See http://www.cs.toronto.edu/~dross/ivt/RossLimLinYang_ijcv.pdf This model is an extension of the Sequential Karhunen-Loeve Transform from: `A. Levy and M. Lindenbaum, Sequential Karhunen-Loeve Basis Extraction and its Application to Images, IEEE Transactions on Image Processing, Volume 9, Number 8, pp. 1371-1374, August 2000.` See http://www.cs.technion.ac.il/~mic/doc/skl-ip.pdf We have specifically abstained from an optimization used by authors of both papers, a QR decomposition used in specific situations to reduce the algorithmic complexity of the SVD. The source for this technique is `Matrix Computations, Third Edition, G. Holub and C. Van Loan, Chapter 5, section 5.4.4, pp 252-253.`. This technique has been omitted because it is advantageous only when decomposing a matrix with ``n_samples`` (rows) >= 5/3 * ``n_features`` (columns), and hurts the readability of the implemented algorithm. This would be a good opportunity for future optimization, if it is deemed necessary. References ---------- D. Ross, J. Lim, R. Lin, M. Yang. Incremental Learning for Robust Visual Tracking, International Journal of Computer Vision, Volume 77, Issue 1-3, pp. 125-141, May 2008. G. Golub and C. Van Loan. Matrix Computations, Third Edition, Chapter 5, Section 5.4.4, pp. 252-253. See also -------- PCA KernelPCA SparsePCA TruncatedSVD cC s(||_||_||_||_dS(N(t n_componentstwhitentcopyt batch_size(tselfRR R R ((sDlib/python2.7/site-packages/sklearn/decomposition/incremental_pca.pyt__init__žs   cC słd|_d|_d|_d|_d|_d|_d|_d|_d|_t |d|j dt j t j gƒ}|j\}}|jdkr¦d||_n |j|_x@t||jd|jpŠdƒD]}|j||dtƒqŌW|S( sFit the model with X, using minibatches of size batch_size. Parameters ---------- X : array-like, shape (n_samples, n_features) Training data, where n_samples is the number of samples and n_features is the number of features. y : Ignored Returns ------- self : object Returns the instance itself. igR tdtypeitmin_batch_sizet check_inputN(tNonet components_tn_samples_seen_tmean_tvar_tsingular_values_texplained_variance_texplained_variance_ratio_tnoise_variance_RR tnptfloat64tfloat32tshapeR t batch_size_RRt partial_fittFalse(R tXtyt n_samplest n_featurestbatch((sDlib/python2.7/site-packages/sklearn/decomposition/incremental_pca.pytfit„s$         ' c C s^|r0t|d|jdtjtjgƒ}n|j\}}t|dƒsZd|_n|j dkr£|jdkrt ||ƒ|_ q|jjd|_ nrd|j ko½|knsŽt d|j |fƒ‚n7|j |ks t d|j |fƒ‚n |j |_ |jdk rc|jjd|j krct d|jjd|j fƒ‚nt|d ƒsd|_ d |_d |_nt|d |jd |jd tj|j |jdƒƒ\}}}|d}|j dkrł||8}notj|ddƒ} || 8}tj|j ||ƒ|j| } tj|jjdƒ|j|| fƒ}tj|dtƒ\} } } t| | dtƒ\} } | d|d}| dtj||ƒ}||_ | |j |_| |j |_||_||_||j |_||j |_|j |krQ||j jƒ|_n d |_|S(sŅIncremental fit with X. All of X is processed as a single batch. Parameters ---------- X : array-like, shape (n_samples, n_features) Training data, where n_samples is the number of samples and n_features is the number of features. check_input : bool Run check_array on X. y : Ignored Returns ------- self : object Returns the instance itself. R RRiisdn_components=%r invalid for n_features=%d, need more rows than columns for IncrementalPCA processingsHn_components=%r must be less or equal to the batch number of samples %d.s{Number of input features has changed from %i to %i between calls to partial_fit! Try setting n_components to a fixed value.Rgt last_meant last_variancetlast_sample_counttaxisi’’’’t full_matricestu_based_decisioniN(i’’’’i(RR RRRRthasattrRRRtmint n_components_t ValueErrorRRRRtrepeattmeantsqrttvstackRtreshapeRtsvdR RtsumRRR(R R!R"RR#R$tcol_meantcol_vartn_total_samplestcol_batch_meantmean_correctiontUtStVtexplained_variancetexplained_variance_ratio((sDlib/python2.7/site-packages/sklearn/decomposition/incremental_pca.pyRĶsf*       (       N( t__name__t __module__t__doc__RR tTrueR R&R(((sDlib/python2.7/site-packages/sklearn/decomposition/incremental_pca.pyRs Œ   ((RDt __future__RtnumpyRtscipyRtbaseRtutilsRRt utils.extmathRRR(((sDlib/python2.7/site-packages/sklearn/decomposition/incremental_pca.pyts