ó ‡ˆ\c@ südZddlmZddlZddlZddlmZmZddl m Z m Z ddl m Z ddl mZdd lmZd d lmZdd lmZdd lmZe dƒd„ƒZd„Zed„Zd„Zd„Ze dƒdd„ƒZd„Z ed„Z!ddd„Z"dddde#dd„Z$e dƒdd„ƒZdd„Z%e d ƒdde#d!„ƒZ&dd"„Z'e#d#„Z(dd$„Z)e#d%„Z*d&„Z+dd'„Z,d(„Z-d)„Z.dd*d+d,„Z/dS(-s Extended math utilities. iÿÿÿÿ(tdivisionN(tlinalgtsparsei(tcheck_random_statet deprecated(t np_version(t logsumexp(t_log_logistic_sigmoidi(txrange(t csr_row_norms(t check_arraysusklearn.utils.extmath.norm was deprecated in version 0.19 and will be removed in 0.21. Use scipy.linalg.norm instead.cC s tj|ƒS(sÆCompute the Euclidean or Frobenius norm of x. Returns the Euclidean norm when x is a vector, the Frobenius norm when x is a matrix (2-d array). More precise than sqrt(squared_norm(x)). (Rtnorm(tx((s4lib/python2.7/site-packages/sklearn/utils/extmath.pyR scC sPtj|ddƒ}tj|jtjƒr@tjdtƒntj||ƒS(s Squared Euclidean or Frobenius norm of x. Faster than norm(x) ** 2. Parameters ---------- x : array_like Returns ------- float The Euclidean norm when x is a vector, the Frobenius norm when x is a matrix (2-d array). tordertKsYArray type is integer, np.dot may overflow. Data should be float type to avoid this issue( tnptravelt issubdtypetdtypetintegertwarningstwarnt UserWarningtdot(R ((s4lib/python2.7/site-packages/sklearn/utils/extmath.pyt squared_norm(s   cC sttj|ƒrBt|tjƒs3tj|ƒ}nt|ƒ}ntjd||ƒ}|sptj||ƒn|S(sÒRow-wise (squared) Euclidean norm of X. Equivalent to np.sqrt((X * X).sum(axis=1)), but also supports sparse matrices and does not create an X.shape-sized temporary. Performs no input validation. Parameters ---------- X : array_like The input array squared : bool, optional (default = False) If True, return squared norms. Returns ------- array_like The row-wise (squared) Euclidean norm of X. sij,ij->i(Rtissparset isinstancet csr_matrixR Rteinsumtsqrt(tXtsquaredtnorms((s4lib/python2.7/site-packages/sklearn/utils/extmath.pyt row_norms?scC s0tjj|ƒ\}}|dks,tj S|S(sæCompute log(det(A)) for A symmetric Equivalent to : np.log(nl.det(A)) but more robust. It returns -Inf if det(A) is non positive or is not defined. Parameters ---------- A : array_like The matrix i(RRtslogdettinf(tAtsigntld((s4lib/python2.7/site-packages/sklearn/utils/extmath.pyt fast_logdet_s  cC sK|jjr+t|jdtddƒtfSt|dtddƒtfSdS(sHelper FunctiontcopyR tFN(tflagst c_contiguousR tTtFalsetTrue(R((s4lib/python2.7/site-packages/sklearn/utils/extmath.pyt_impose_f_orderps s}sklearn.utils.extmath.fast_dot was deprecated in version 0.19 and will be removed in 0.21. Use the equivalent np.dot instead.cC stj|||ƒS(N(RR(tatbtout((s4lib/python2.7/site-packages/sklearn/utils/extmath.pytfast_dotzscK slt|dƒr7t|jƒ|jd|jd}n1|dkrIdnt|dkjƒƒ|j}|S(sÀCompute density of a sparse vector Parameters ---------- w : array_like The sparse vector Returns ------- float The density of w, between 0 and 1 ttoarrayiiN(thasattrtfloattnnztshapetNonetsumtsize(twtkwargstd((s4lib/python2.7/site-packages/sklearn/utils/extmath.pytdensity€s (1cC sdtj|ƒstj|ƒrP||}|rLt|dƒrL|jƒ}n|Stj||ƒSdS(s2Dot product that handle the sparse matrix case correctly Uses BLAS GEMM as replacement for numpy.dot where possible to avoid unnecessary copies. Parameters ---------- a : array or sparse matrix b : array or sparse matrix dense_output : boolean, default False When False, either ``a`` or ``b`` being sparse will yield sparse output. When True, output will always be an array. Returns ------- dot_product : array or sparse matrix sparse if ``a`` or ``b`` is sparse and ``dense_output=False``. R4N(RRR5R4RR(R0R1t dense_outputtret((s4lib/python2.7/site-packages/sklearn/utils/extmath.pytsafe_sparse_dot”s  tautocC s¥t|ƒ}|jd|jd|fƒ}|jjdkrX|j|jdtƒ}n|dkr‚|dkryd}q‚d}nxøt|ƒD]ê}|dkrÅt||ƒ}t|j |ƒ}q|dkrt j t||ƒd t ƒ\}}t j t|j |ƒd t ƒ\}}q|d krt j t||ƒd d ƒ\}}t j t|j |ƒd d ƒ\}}qqWt j t||ƒd d ƒ\}}|S( sÞComputes an orthonormal matrix whose range approximates the range of A. Parameters ---------- A : 2D array The input data matrix size : integer Size of the return array n_iter : integer Number of power iterations used to stabilize the result power_iteration_normalizer : 'auto' (default), 'QR', 'LU', 'none' Whether the power iterations are normalized with step-by-step QR factorization (the slowest but most accurate), 'none' (the fastest but numerically unstable when `n_iter` is large, e.g. typically 5 or larger), or 'LU' factorization (numerically stable but can lose slightly in accuracy). The 'auto' mode applies no normalization if `n_iter` <= 2 and switches to LU otherwise. .. versionadded:: 0.18 random_state : int, RandomState instance or None, optional (default=None) The seed of the pseudo random number generator to use when shuffling the data. If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Returns ------- Q : 2D array A (size x size) projection matrix, the range of which approximates well the range of the input matrix A. Notes ----- Follows Algorithm 4.3 of Finding structure with randomness: Stochastic algorithms for constructing approximate matrix decompositions Halko, et al., 2009 (arXiv:909) https://arxiv.org/pdf/0909.4061.pdf An implementation of a randomized algorithm for principal component analysis A. Szlam et al. 2014 R;itfR(RCitnonetLUt permute_ltQRtmodeteconomic(RtnormalR8RtkindtastypeR-trangeRBR,RtluR.tqr(R$R;tn_itertpower_iteration_normalizert random_statetQtit_((s4lib/python2.7/site-packages/sklearn/utils/extmath.pytrandomized_range_finder°s(3       $* $.$i icC sòt|tjtjfƒrCtjdjt|ƒjƒtj ƒnt |ƒ}||}|j \} } |dkrœ|dt |j ƒkr“dnd}n|dkr·| | k}n|rÉ|j }nt|||||ƒ} t| j |ƒ} tj| dtƒ\} }}~ tj| | ƒ}|rh|sJt||ƒ\}}qht||dtƒ\}}n|r±|d|…dd…fj || |dd…d|…fj fS|dd…d|…f|| |d|…dd…ffSdS( s¯ Computes a truncated randomized SVD Parameters ---------- M : ndarray or sparse matrix Matrix to decompose n_components : int Number of singular values and vectors to extract. n_oversamples : int (default is 10) Additional number of random vectors to sample the range of M so as to ensure proper conditioning. The total number of random vectors used to find the range of M is n_components + n_oversamples. Smaller number can improve speed but can negatively impact the quality of approximation of singular vectors and singular values. n_iter : int or 'auto' (default is 'auto') Number of power iterations. It can be used to deal with very noisy problems. When 'auto', it is set to 4, unless `n_components` is small (< .1 * min(X.shape)) `n_iter` in which case is set to 7. This improves precision with few components. .. versionchanged:: 0.18 power_iteration_normalizer : 'auto' (default), 'QR', 'LU', 'none' Whether the power iterations are normalized with step-by-step QR factorization (the slowest but most accurate), 'none' (the fastest but numerically unstable when `n_iter` is large, e.g. typically 5 or larger), or 'LU' factorization (numerically stable but can lose slightly in accuracy). The 'auto' mode applies no normalization if `n_iter` <= 2 and switches to LU otherwise. .. versionadded:: 0.18 transpose : True, False or 'auto' (default) Whether the algorithm should be applied to M.T instead of M. The result should approximately be the same. The 'auto' mode will trigger the transposition if M.shape[1] > M.shape[0] since this implementation of randomized SVD tend to be a little faster in that case. .. versionchanged:: 0.18 flip_sign : boolean, (True by default) The output of a singular value decomposition is only unique up to a permutation of the signs of the singular vectors. If `flip_sign` is set to `True`, the sign ambiguity is resolved by making the largest loadings for each component in the left singular vectors positive. random_state : int, RandomState instance or None, optional (default=None) The seed of the pseudo random number generator to use when shuffling the data. If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Notes ----- This algorithm finds a (usually very good) approximate truncated singular value decomposition using randomization to speed up the computations. It is particularly fast on large matrices on which you wish to extract only a small number of components. In order to obtain further speed up, `n_iter` can be set <=2 (at the cost of loss of precision). References ---------- * Finding structure with randomness: Stochastic algorithms for constructing approximate matrix decompositions Halko, et al., 2009 https://arxiv.org/abs/0909.4061 * A randomized algorithm for the decomposition of matrices Per-Gunnar Martinsson, Vladimir Rokhlin and Mark Tygert * An implementation of a randomized algorithm for principal component analysis A. Szlam et al. 2014 sCCalculating SVD of a {} is expensive. csr_matrix is more efficient.RCgš™™™™™¹?iit full_matricestu_based_decisionN(RRt lil_matrixt dok_matrixRRtformatttypet__name__tSparseEfficiencyWarningRR8tminR,RWRBRtsvdR-RRtsvd_flip(tMt n_componentst n_oversamplesRQRRt transposet flip_signRStn_randomt n_samplest n_featuresRTtBtUhattstVtU((s4lib/python2.7/site-packages/sklearn/utils/extmath.pytrandomized_svds4R     (    Cs}sklearn.utils.extmath.logsumexp was deprecated in version 0.19 and will be removed in 0.21. Use scipy.misc.logsumexp instead.cC s t||ƒS(s|Computes the sum of arr assuming arr is in the log domain. Returns log(sum(exp(arr))) while minimizing the possibility of over/underflow. Examples -------- >>> import numpy as np >>> from sklearn.utils.extmath import logsumexp >>> a = np.arange(10) >>> np.log(np.sum(np.exp(a))) 9.458... >>> logsumexp(a) # doctest: +SKIP 9.458... (tscipy_logsumexp(tarrtaxis((s4lib/python2.7/site-packages/sklearn/utils/extmath.pyR‡sc C sk|dkr3tj|ƒ}tj|ƒ}d}ntj|ƒ}tj|ƒ}|j|jkr„tj|j|d|jƒ}ntjtj|ƒƒ}t|jƒ}d||>> from sklearn.utils.extmath import weighted_mode >>> x = [4, 1, 4, 2, 4, 2] >>> weights = [1, 1, 1, 1, 1, 1] >>> weighted_mode(x, weights) (array([4.]), array([3.])) The value 4 appears three times: with uniform weights, the result is simply the mode of the distribution. >>> weights = [1, 3, 0.5, 1.5, 1, 2] # deweight the 4's >>> weighted_mode(x, weights) (array([2.]), array([3.5])) The value 2 has the highest score: it appears twice with weights of 1.5 and 2: the sum of these is 3. See Also -------- scipy.stats.mode iRiN(R9RRtasarrayR8tfullRtuniquetlisttzerost expand_dimsR:twheretmaximum( R0R<Rstscorest testshapet oldmostfreqt oldcountstscorettemplatetindtcountst mostfrequent((s4lib/python2.7/site-packages/sklearn/utils/extmath.pyt weighted_modešs,.  !    swsklearn.utils.extmath.pinvh was deprecated in version 0.19 and will be removed in 0.21. Use scipy.linalg.pinvh instead.cC stj||||ƒS(N(Rtpinvh(R0tcondtrcondtlower((s4lib/python2.7/site-packages/sklearn/utils/extmath.pyR†ãscC sÜg|D]}tj|ƒ^q}d„|Dƒ}|dj}tj|ƒ}|jt|ƒdƒj}|dkrtj|d|ƒ}nxHt |ƒD]:\}}|||dd…|f|dd…|f>> cartesian(([1, 2, 3], [4, 5], [6, 7])) array([[1, 4, 6], [1, 4, 7], [1, 5, 6], [1, 5, 7], [2, 4, 6], [2, 4, 7], [2, 5, 6], [2, 5, 7], [3, 4, 6], [3, 4, 7], [3, 5, 6], [3, 5, 7]]) cs s|]}t|ƒVqdS(N(tlen(t.0R ((s4lib/python2.7/site-packages/sklearn/utils/extmath.pys siiÿÿÿÿRN( RRtRtindicestreshapeRŠR,R9t empty_liket enumerate(tarraysR2R R8RtixtnRr((s4lib/python2.7/site-packages/sklearn/utils/extmath.pyt cartesianés!"  2cC sé|rttjtj|ƒddƒ}tj||t|jdƒfƒ}||9}||dd…tjf9}nktjtj|ƒddƒ}tj|t|jdƒ|fƒ}||9}||dd…tjf9}||fS(sÕSign correction to ensure deterministic output from SVD. Adjusts the columns of u and the rows of v such that the loadings in the columns in u that are largest in absolute value are always positive. Parameters ---------- u : ndarray u and v are the output of `linalg.svd` or `sklearn.utils.extmath.randomized_svd`, with matching inner dimensions so one can compute `np.dot(u * s, v)`. v : ndarray u and v are the output of `linalg.svd` or `sklearn.utils.extmath.randomized_svd`, with matching inner dimensions so one can compute `np.dot(u * s, v)`. u_based_decision : boolean, (default=True) If True, use the columns of u as the basis for sign flipping. Otherwise, use the rows of v. The choice of which variable to base the decision on is generally algorithm dependent. Returns ------- u_adjusted, v_adjusted : arrays with the same dimensions as the input. RsiiN(RtargmaxtabsR%RR8tnewaxis(tutvRYt max_abs_colstsignst max_abs_rows((s4lib/python2.7/site-packages/sklearn/utils/extmath.pyRbs&  & cC sŠ|jdk}tj|ƒ}t|dtjƒ}|j\}}|dkr`tj|ƒ}nt||||ƒ|r†tj |ƒS|S(s9Compute the log of the logistic function, ``log(1 / (1 + e ** -x))``. This implementation is numerically stable because it splits positive and negative values:: -log(1 + exp(-x_i)) if x_i > 0 x_i - log(1 + exp(x_i)) if x_i <= 0 For the ordinary logistic function, use ``scipy.special.expit``. Parameters ---------- X : array-like, shape (M, N) or (M, ) Argument to the logistic function out : array-like, shape: (M, N) or (M, ), optional: Preallocated output array. Returns ------- out : array, shape (M, N) or (M, ) Log of the logistic function evaluated at every point in x Notes ----- See the blog post describing this implementation: http://fa.bianp.net/blog/2013/numerical-optimizers-for-logistic-regression/ iRN( tndimRt atleast_2dR tfloat64R8R9RŽRtsqueeze(RR2tis_1dRiRj((s4lib/python2.7/site-packages/sklearn/utils/extmath.pyt log_logisticFs  cC s||rtj|ƒ}ntj|ddƒjdƒ}||8}tj||ƒtj|ddƒjdƒ}||}|S(s5 Calculate the softmax function. The softmax function is calculated by np.exp(X) / np.sum(np.exp(X), axis=1) This will cause overflow when large values are exponentiated. Hence the largest value in each row is subtracted from each data point to prevent this. Parameters ---------- X : array-like of floats, shape (M, N) Argument to the logistic function copy : bool, optional Copy X or not. Returns ------- out : array, shape (M, N) Softmax function evaluated at every point in x Rsiiÿÿÿÿ(iÿÿÿÿi(iÿÿÿÿi(RR(tmaxRtexpR:(RR(tmax_probtsum_prob((s4lib/python2.7/site-packages/sklearn/utils/extmath.pytsoftmaxss  cC sktj|ƒr]t|jƒdkr(dS|jjƒ}|jƒ|jkrP|St|dƒS|jƒSdS(sReturns the minimum value of a dense or a CSR/CSC matrix. Adapated from https://stackoverflow.com/q/13426580 Parameters ---------- X : array_like The input array or sparse matrix Returns ------- Float The min value of X iN(RRRŠtdataR`tgetnnzR;(Rtm((s4lib/python2.7/site-packages/sklearn/utils/extmath.pytsafe_min•s &cC sKt|ƒ}||krGtj|ƒr6tdƒ‚n|||}n|S(s6Ensure `X.min()` >= `min_value`. Parameters ---------- X : array_like The matrix to make non-negative min_value : float The threshold value Returns ------- array_like The thresholded array Raises ------ ValueError When X is sparse s{Cannot make the data matrix nonnegative because it is sparse. Adding a value to every entry would make it no longer sparse.(RªRRt ValueError(Rt min_valuetmin_((s4lib/python2.7/site-packages/sklearn/utils/extmath.pytmake_nonnegative­s   cC sa||}tj|jtjƒra|jjdkratj|dddtjƒj|jƒ}ntj|ddƒ}tjtj |ƒddƒ}||}|||}|d krÂd } n’tj |ddƒ|} ||} tj ddddƒ1||} | | | ||| |d} Wd QX|dk}| || |<| |} || |fS( sÈCalculate mean update and a Youngs and Cramer variance update. last_mean and last_variance are statistics computed at the last step by the function. Both must be initialized to 0.0. In case no scaling is required last_variance can be None. The mean is always required and returned because necessary for the calculation of the variance. last_n_samples_seen is the number of samples encountered until now. From the paper "Algorithms for computing the sample variance: analysis and recommendations", by Chan, Golub, and LeVeque. Parameters ---------- X : array-like, shape (n_samples, n_features) Data to use for variance update last_mean : array-like, shape: (n_features,) last_variance : array-like, shape: (n_features,) last_sample_count : array-like, shape (n_features,) Returns ------- updated_mean : array, shape (n_features,) updated_variance : array, shape (n_features,) If None, only mean is computed updated_sample_count : array, shape (n_features,) Notes ----- NaNs are ignored during the algorithm. References ---------- T. Chan, G. Golub, R. LeVeque. Algorithms for computing the sample variance: recommendations, The American Statistician, Vol. 37, No. 3, pp. 242-247 Also, see the sparse implementation of this in `utils.sparsefuncs.incr_mean_variance_axis` and `utils.sparsefuncs_fast.incr_mean_variance_axis0` iRsiRtdividetignoretinvalidiN( RRRtfloatingtitemsizetnansumRžRMR:tisnanR9tnanvarterrstate(Rt last_meant last_variancetlast_sample_counttlast_sumtnew_sumtnew_sample_counttupdated_sample_countt updated_meantupdated_variancetnew_unnormalized_variancetlast_unnormalized_variancetlast_over_new_counttupdated_unnormalized_varianceRx((s4lib/python2.7/site-packages/sklearn/utils/extmath.pyt_incremental_mean_and_varÌs(1 *-       cC setjtj|ƒddƒ}tj|t|jdƒ|fƒ}||dd…tjf9}|S(sModify the sign of vectors for reproducibility Flips the sign of elements of all the vectors (rows of u) such that the absolute maximum element of each vector is positive. Parameters ---------- u : ndarray Array with vectors as its rows. Returns ------- u_flipped : ndarray with same shape as u Array with the sign flipped vectors as its rows. RsiiN(RR”R•R%RNR8R–(R—R›Rš((s4lib/python2.7/site-packages/sklearn/utils/extmath.pyt_deterministic_vector_sign_flips&gñh㈵øä>g:Œ0âŽyE>c C s·td kr(tj|d|dtjƒStj|d|dtjƒ}tj|d|dtjƒ}tjtj|jdd|ƒ|d|d|dtƒƒs³t j d t ƒn|S( sÉUse high precision for cumsum and check that final value matches sum Parameters ---------- arr : array-like To be cumulatively summed as flat axis : int, optional Axis along which the cumulative sum is computed. The default (None) is to compute the cumsum over the flattened array. rtol : float Relative tolerance, see ``np.allclose`` atol : float Absolute tolerance, see ``np.allclose`` ii RsRiÿÿÿÿtrtoltatolt equal_nansLcumsum was found to be unstable: its last element does not correspond to sum(ii ( RRtcumsumRžR:talltisclosettakeR.RRtRuntimeWarning(RrRsRÇRÈR2texpected((s4lib/python2.7/site-packages/sklearn/utils/extmath.pyt stable_cumsum4s *  (0t__doc__t __future__RRtnumpyRtscipyRRtRRtfixesRRRqt_logistic_sigmoidRtexternals.six.movesRtsparsefuncs_fastR t validationR R RR-R!R'R/R9R3R?RBRWR.RpR…R†R“RbR¡R¦RªR®RÅRÆRÐ(((s4lib/python2.7/site-packages/sklearn/utils/extmath.pytsN         S€  I  1 , - "   R