ó ‡ˆ\c@sdZddlZddlZddljZddlmZddl m Z ddl m Z ddl m Z mZmZddlmZdd lmZdd lmZmZd e e fd „ƒYZd e e fd„ƒYZde e fd„ƒYZde e fd„ƒYZdS(s The :mod:`sklearn.kernel_approximation` module implements several approximate kernel feature maps base on Fourier transforms. iÿÿÿÿN(tsvdi(t BaseEstimator(tTransformerMixin(t check_arraytcheck_random_statetas_float_array(tsafe_sparse_dot(tcheck_is_fitted(tpairwise_kernelst KERNEL_PARAMSt RBFSamplercBs5eZdZdddd„Zdd„Zd„ZRS(sApproximates feature map of an RBF kernel by Monte Carlo approximation of its Fourier transform. It implements a variant of Random Kitchen Sinks.[1] Read more in the :ref:`User Guide `. Parameters ---------- gamma : float Parameter of RBF kernel: exp(-gamma * x^2) n_components : int Number of Monte Carlo samples per original feature. Equals the dimensionality of the computed feature space. random_state : int, RandomState instance or None, optional (default=None) If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Examples -------- >>> from sklearn.kernel_approximation import RBFSampler >>> from sklearn.linear_model import SGDClassifier >>> X = [[0, 0], [1, 1], [1, 0], [0, 1]] >>> y = [0, 0, 1, 1] >>> rbf_feature = RBFSampler(gamma=1, random_state=1) >>> X_features = rbf_feature.fit_transform(X) >>> clf = SGDClassifier(max_iter=5, tol=1e-3) >>> clf.fit(X_features, y) ... # doctest: +NORMALIZE_WHITESPACE SGDClassifier(alpha=0.0001, average=False, class_weight=None, early_stopping=False, epsilon=0.1, eta0=0.0, fit_intercept=True, l1_ratio=0.15, learning_rate='optimal', loss='hinge', max_iter=5, n_iter=None, n_iter_no_change=5, n_jobs=None, penalty='l2', power_t=0.5, random_state=None, shuffle=True, tol=0.001, validation_fraction=0.1, verbose=0, warm_start=False) >>> clf.score(X_features, y) 1.0 Notes ----- See "Random Features for Large-Scale Kernel Machines" by A. Rahimi and Benjamin Recht. [1] "Weighted Sums of Random Kitchen Sinks: Replacing minimization with randomization in learning" by A. Rahimi and Benjamin Recht. (http://people.eecs.berkeley.edu/~brecht/papers/08.rah.rec.nips.pdf) gð?idcCs||_||_||_dS(N(tgammat n_componentst random_state(tselfR R R ((s;lib/python2.7/site-packages/sklearn/kernel_approximation.pyt__init__Ns  cCs‰t|ddƒ}t|jƒ}|jd}tjd|jƒ|jd||jfƒ|_ |j ddtj d|jƒ|_ |S(sžFit the model with X. Samples random projection according to n_features. Parameters ---------- X : {array-like, sparse matrix}, shape (n_samples, n_features) Training data, where n_samples in the number of samples and n_features is the number of features. Returns ------- self : object Returns the transformer. t accept_sparsetcsriitsizei( RRR tshapetnptsqrtR tnormalR trandom_weights_tuniformtpitrandom_offset_(RtXtyR t n_features((s;lib/python2.7/site-packages/sklearn/kernel_approximation.pytfitSs cCsut|dƒt|ddƒ}t||jƒ}||j7}tj||ƒ|tjdƒtj|jƒ9}|S(spApply the approximate feature map to X. Parameters ---------- X : {array-like, sparse matrix}, shape (n_samples, n_features) New data, where n_samples in the number of samples and n_features is the number of features. Returns ------- X_new : array-like, shape (n_samples, n_components) RRRg@( RRRRRRtcosRR (RRt projection((s;lib/python2.7/site-packages/sklearn/kernel_approximation.pyt transformos  #N(t__name__t __module__t__doc__tNoneRRR!(((s;lib/python2.7/site-packages/sklearn/kernel_approximation.pyR s4 tSkewedChi2SamplercBs5eZdZdddd„Zdd„Zd„ZRS(s’Approximates feature map of the "skewed chi-squared" kernel by Monte Carlo approximation of its Fourier transform. Read more in the :ref:`User Guide `. Parameters ---------- skewedness : float "skewedness" parameter of the kernel. Needs to be cross-validated. n_components : int number of Monte Carlo samples per original feature. Equals the dimensionality of the computed feature space. random_state : int, RandomState instance or None, optional (default=None) If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Examples -------- >>> from sklearn.kernel_approximation import SkewedChi2Sampler >>> from sklearn.linear_model import SGDClassifier >>> X = [[0, 0], [1, 1], [1, 0], [0, 1]] >>> y = [0, 0, 1, 1] >>> chi2_feature = SkewedChi2Sampler(skewedness=.01, ... n_components=10, ... random_state=0) >>> X_features = chi2_feature.fit_transform(X, y) >>> clf = SGDClassifier(max_iter=10, tol=1e-3) >>> clf.fit(X_features, y) SGDClassifier(alpha=0.0001, average=False, class_weight=None, early_stopping=False, epsilon=0.1, eta0=0.0, fit_intercept=True, l1_ratio=0.15, learning_rate='optimal', loss='hinge', max_iter=10, n_iter=None, n_iter_no_change=5, n_jobs=None, penalty='l2', power_t=0.5, random_state=None, shuffle=True, tol=0.001, validation_fraction=0.1, verbose=0, warm_start=False) >>> clf.score(X_features, y) 1.0 References ---------- See "Random Fourier Approximations for Skewed Multiplicative Histogram Kernels" by Fuxin Li, Catalin Ionescu and Cristian Sminchisescu. See also -------- AdditiveChi2Sampler : A different approach for approximating an additive variant of the chi squared kernel. sklearn.metrics.pairwise.chi2_kernel : The exact chi squared kernel. gð?idcCs||_||_||_dS(N(t skewednessR R (RR'R R ((s;lib/python2.7/site-packages/sklearn/kernel_approximation.pyR½s  cCst|ƒ}t|jƒ}|jd}|jd||jfƒ}dtjtjtj tjd|ƒƒ|_ |jddtjd|jƒ|_ |S(sFit the model with X. Samples random projection according to n_features. Parameters ---------- X : array-like, shape (n_samples, n_features) Training data, where n_samples in the number of samples and n_features is the number of features. Returns ------- self : object Returns the transformer. iRgð?g@ii( RRR RRR RRtlogttanRR(RRRR RR((s;lib/python2.7/site-packages/sklearn/kernel_approximation.pyRÂs  1cCsÉt|dƒt|dtƒ}t|dtƒ}||j kjƒrVtdƒ‚n||j7}tj ||ƒt ||j ƒ}||j 7}tj ||ƒ|tjdƒtj|jƒ9}|S(s¨Apply the approximate feature map to X. Parameters ---------- X : array-like, shape (n_samples, n_features) New data, where n_samples in the number of samples and n_features is the number of features. All values of X must be strictly greater than "-skewedness". Returns ------- X_new : array-like, shape (n_samples, n_components) Rtcopys3X may not contain entries smaller than -skewedness.g@(RRtTrueRtFalseR'tanyt ValueErrorRR(RRRRRR (RRR ((s;lib/python2.7/site-packages/sklearn/kernel_approximation.pyR!Þs   #N(R"R#R$R%RRR!(((s;lib/python2.7/site-packages/sklearn/kernel_approximation.pyR&†s5 tAdditiveChi2SamplercBsDeZdZddd„Zdd„Zd„Zd„Zd„ZRS(s2 Approximate feature map for additive chi2 kernel. Uses sampling the fourier transform of the kernel characteristic at regular intervals. Since the kernel that is to be approximated is additive, the components of the input vectors can be treated separately. Each entry in the original space is transformed into 2*sample_steps+1 features, where sample_steps is a parameter of the method. Typical values of sample_steps include 1, 2 and 3. Optimal choices for the sampling interval for certain data ranges can be computed (see the reference). The default values should be reasonable. Read more in the :ref:`User Guide `. Parameters ---------- sample_steps : int, optional Gives the number of (complex) sampling points. sample_interval : float, optional Sampling interval. Must be specified when sample_steps not in {1,2,3}. Examples -------- >>> from sklearn.datasets import load_digits >>> from sklearn.linear_model import SGDClassifier >>> from sklearn.kernel_approximation import AdditiveChi2Sampler >>> X, y = load_digits(return_X_y=True) >>> chi2sampler = AdditiveChi2Sampler(sample_steps=2) >>> X_transformed = chi2sampler.fit_transform(X, y) >>> clf = SGDClassifier(max_iter=5, random_state=0, tol=1e-3) >>> clf.fit(X_transformed, y) SGDClassifier(alpha=0.0001, average=False, class_weight=None, early_stopping=False, epsilon=0.1, eta0=0.0, fit_intercept=True, l1_ratio=0.15, learning_rate='optimal', loss='hinge', max_iter=5, n_iter=None, n_iter_no_change=5, n_jobs=None, penalty='l2', power_t=0.5, random_state=0, shuffle=True, tol=0.001, validation_fraction=0.1, verbose=0, warm_start=False) >>> clf.score(X_transformed, y) # doctest: +ELLIPSIS 0.9543... Notes ----- This estimator approximates a slightly different version of the additive chi squared kernel then ``metric.additive_chi2`` computes. See also -------- SkewedChi2Sampler : A Fourier-approximation to a non-additive variant of the chi squared kernel. sklearn.metrics.pairwise.chi2_kernel : The exact chi squared kernel. sklearn.metrics.pairwise.additive_chi2_kernel : The exact additive chi squared kernel. References ---------- See `"Efficient additive kernels via explicit feature maps" `_ A. Vedaldi and A. Zisserman, Pattern Analysis and Machine Intelligence, 2011 icCs||_||_dS(N(t sample_stepstsample_interval(RR0R1((s;lib/python2.7/site-packages/sklearn/kernel_approximation.pyR?s cCs‘t|ddƒ}|jd kr|jdkr<d|_q|jdkrWd|_q|jdkrrd|_qtd ƒ‚n |j|_|S( sNSet the parameters Parameters ---------- X : array-like, shape (n_samples, n_features) Training data, where n_samples in the number of samples and n_features is the number of features. Returns ------- self : object Returns the transformer. RRigš™™™™™é?igà?igš™™™™™Ù?sHIf sample_steps is not in [1, 2, 3], you need to provide sample_intervalN(RR1R%R0tsample_interval_R.(RRR((s;lib/python2.7/site-packages/sklearn/kernel_approximation.pyRCs    cCsŒd}t|dd|ƒt|ddƒ}tj|ƒ}|rI|jn|dkjƒrjtdƒ‚n|ry|jn|j}||ƒS(sApply approximate feature map to X. Parameters ---------- X : {array-like, sparse matrix}, shape = (n_samples, n_features) Returns ------- X_new : {array, sparse matrix}, shape = (n_samples, n_features * (2*sample_steps + 1)) Whether the return value is an array of sparse matrix depends on the type of the input X. sO%(name)s is not fitted. Call fit to set the parameters before calling transformR2tmsgRRis"Entries of X must be non-negative.( RRtsptissparsetdataR-R.t_transform_sparset_transform_dense(RRR3tsparsettransf((s;lib/python2.7/site-packages/sklearn/kernel_approximation.pyR!as!c Cs1|dk}||}tj|ƒ}tj||jƒ||<|g}|jtj|ƒ}d||j}x²td|jƒD]ž}tj|tjtj||jƒƒ} tj|ƒ}| tj ||ƒ||<|j |ƒtj|ƒ}| tj ||ƒ||<|j |ƒq‚Wtj |ƒS(Ngii( Rt zeros_likeRR2R(trangeR0tcoshRRtappendtsinthstack( RRtnon_zerotX_nztX_steptX_newt log_step_nztstep_nztjt factor_nz((s;lib/python2.7/site-packages/sklearn/kernel_approximation.pyR8€s"    ! c Cs™|jjƒ}|jjƒ}tj|j|jƒ}tj|||fd|j d|j dt ƒ}|g}|jtj |jƒ}d|j|j}xìt d|jƒD]Ø} tj|tjtj| |jƒƒ} | tj| |ƒ}tj|||fd|j d|j dt ƒ}|j|ƒ| tj| |ƒ}tj|||fd|j d|j dt ƒ}|j|ƒq°Wtj|ƒS(NRtdtypeR*ii(tindicesR*tindptrRRR6R2R4t csr_matrixRRIR,R(R<R0R=RRR>R?R@( RRRJRKt data_stepRCRDRERFRGRH((s;lib/python2.7/site-packages/sklearn/kernel_approximation.pyR7šs(  ! N( R"R#R$R%RRR!R8R7(((s;lib/python2.7/site-packages/sklearn/kernel_approximation.pyR/ýs @   tNystroemcBsJeZdZdddddddd„Zdd„Zd„Zd„ZRS(sApproximate a kernel map using a subset of the training data. Constructs an approximate feature map for an arbitrary kernel using a subset of the data as basis. Read more in the :ref:`User Guide `. Parameters ---------- kernel : string or callable, default="rbf" Kernel map to be approximated. A callable should accept two arguments and the keyword arguments passed to this object as kernel_params, and should return a floating point number. gamma : float, default=None Gamma parameter for the RBF, laplacian, polynomial, exponential chi2 and sigmoid kernels. Interpretation of the default value is left to the kernel; see the documentation for sklearn.metrics.pairwise. Ignored by other kernels. coef0 : float, default=None Zero coefficient for polynomial and sigmoid kernels. Ignored by other kernels. degree : float, default=None Degree of the polynomial kernel. Ignored by other kernels. kernel_params : mapping of string to any, optional Additional parameters (keyword arguments) for kernel function passed as callable object. n_components : int Number of features to construct. How many data points will be used to construct the mapping. random_state : int, RandomState instance or None, optional (default=None) If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Attributes ---------- components_ : array, shape (n_components, n_features) Subset of training points used to construct the feature map. component_indices_ : array, shape (n_components) Indices of ``components_`` in the training set. normalization_ : array, shape (n_components, n_components) Normalization matrix needed for embedding. Square root of the kernel matrix on ``components_``. Examples -------- >>> from sklearn import datasets, svm >>> from sklearn.kernel_approximation import Nystroem >>> digits = datasets.load_digits(n_class=9) >>> data = digits.data / 16. >>> clf = svm.LinearSVC() >>> feature_map_nystroem = Nystroem(gamma=.2, ... random_state=1, ... n_components=300) >>> data_transformed = feature_map_nystroem.fit_transform(data) >>> clf.fit(data_transformed, digits.target) ... # doctest: +NORMALIZE_WHITESPACE LinearSVC(C=1.0, class_weight=None, dual=True, fit_intercept=True, intercept_scaling=1, loss='squared_hinge', max_iter=1000, multi_class='ovr', penalty='l2', random_state=None, tol=0.0001, verbose=0) >>> clf.score(data_transformed, digits.target) # doctest: +ELLIPSIS 0.9987... References ---------- * Williams, C.K.I. and Seeger, M. "Using the Nystroem method to speed up kernel machines", Advances in neural information processing systems 2001 * T. Yang, Y. Li, M. Mahdavi, R. Jin and Z. Zhou "Nystroem Method vs Random Fourier Features: A Theoretical and Empirical Comparison", Advances in Neural Information Processing Systems 2012 See also -------- RBFSampler : An approximation to the RBF kernel using random Fourier features. sklearn.metrics.pairwise.kernel_metrics : List of built-in kernels. trbfidcCsC||_||_||_||_||_||_||_dS(N(tkernelR tcoef0tdegreet kernel_paramsR R (RRPR RQRRRSR R ((s;lib/python2.7/site-packages/sklearn/kernel_approximation.pyRs      c Cst|ddƒ}t|jƒ}|jd}|j|krS|}tjdƒn |j}t||ƒ}|j|ƒ}|| }||}t |d|j dt |j ƒ} t | ƒ\} } } tj| dƒ} tj| tj| ƒ| ƒ|_||_||_|S(sFit estimator to data. Samples a subset of training points, computes kernel on these and computes normalization matrix. Parameters ---------- X : array-like, shape=(n_samples, n_feature) Training data. RRisŽn_components > n_samples. This is not possible. n_components was set to n_samples, which results in inefficient evaluation of the full kernel.tmetrict filter_paramsgê-™—q=(RRR RR twarningstwarntmint permutationRRPR+t_get_kernel_paramsRRtmaximumtdotRtnormalization_t components_tcomponent_indices_( RRRtrndt n_samplesR tindst basis_indstbasist basis_kerneltUtStV((s;lib/python2.7/site-packages/sklearn/kernel_approximation.pyRs(     "  cCset|dƒt|ddƒ}|jƒ}t||jd|jdt|}tj||j j ƒS(s„Apply feature map to X. Computes an approximate feature map using the kernel between some training points and X. Parameters ---------- X : array-like, shape=(n_samples, n_features) Data to transform. Returns ------- X_transformed : array, shape=(n_samples, n_components) Transformed data. R^RRRTRU( RRRZRR^RPR+RR\R]tT(RRRStembedded((s;lib/python2.7/site-packages/sklearn/kernel_approximation.pyR!Hs    cCs·|j}|dkri}nt|jƒssxƒt|jD]1}t||ƒdk r;t||ƒ||s  nwº