ó ‡ˆ\c@s•dZddlmZmZddlZddlmZddlmZm Z m Z m Z ddl m Z dd lmZdd lmZdd lmZdd lmZmZdd lmZddlmZddlmZmZmZddlm Z ddl!m"Z"m#Z#ddl$m%Z%m&Z&ddgZ'de j(eeƒfd„ƒYZ)d„Z*de)efd„ƒYZ+de)e fd„ƒYZ,dS(s/Weight Boosting This module contains weight boosting estimators for both classification and regression. The module structure is the following: - The ``BaseWeightBoosting`` base class implements a common ``fit`` method for all the estimators in the module. Regression and classification only differ from each other in the loss function that is optimized. - ``AdaBoostClassifier`` implements adaptive boosting (AdaBoost-SAMME) for classification problems. - ``AdaBoostRegressor`` implements adaptive boosting (AdaBoost.R2) for regression problems. iÿÿÿÿ(tABCMetatabstractmethodNi(t BaseEnsemblei(tClassifierMixintRegressorMixint is_regressort is_classifier(tsix(tzip(txrange(t BaseForest(tDecisionTreeClassifiertDecisionTreeRegressor(tBaseDecisionTree(tDTYPE(t check_arrayt check_X_ytcheck_random_state(t stable_cumsum(taccuracy_scoretr2_score(thas_fit_parametertcheck_is_fittedtAdaBoostClassifiertAdaBoostRegressortBaseWeightBoostingcBsneZdZed deƒdd d„ƒZd d„Zed„ƒZd d„Z e d„ƒZ d„Z RS( sBase class for AdaBoost estimators. Warning: This class should not be used directly. Use derived classes instead. i2gð?cCs;tt|ƒjd|d|d|ƒ||_||_dS(Ntbase_estimatort n_estimatorstestimator_params(tsuperRt__init__t learning_ratet random_state(tselfRRRRR ((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyR9s  c Cs8|jdkrtdƒ‚n|jd ksEt|jttfƒrTt}d}nd }ddg}t||d|d|dt |ƒƒ\}}|d krÓt j |j ddt j ƒ}d|j d|(nLt|d tƒ}||jdt j ƒ}|jƒdkrtd ƒ‚n|jƒg|_t j|jdt j ƒ|_t j|jdt j ƒ|_t|jƒ}x´t|jƒD]£}|j|||||ƒ\}}} |d krÇPn||j|<| |j|<| dkrñPnt j|ƒ} | dkrPn||jd kr|| :}qqW|S( sbBuild a boosted classifier/regressor from the training set (X, y). Parameters ---------- X : {array-like, sparse matrix} of shape = [n_samples, n_features] The training input samples. Sparse matrix can be CSC, CSR, COO, DOK, or LIL. COO, DOK, and LIL are converted to CSR. The dtype is forced to DTYPE from tree._tree if the base classifier of this ensemble weighted boosting classifier is a tree or forest. y : array-like of shape = [n_samples] The target values (class labels in classification, real numbers in regression). sample_weight : array-like of shape = [n_samples], optional Sample weights. If None, the sample weights are initialized to 1 / n_samples. Returns ------- self : object is'learning_rate must be greater than zerotcsctcsrt accept_sparsetdtypet y_numericgð?t ensure_2dsAAttempting to fit with a non-positive weighted number of samples.iN(Rt ValueErrorRtNonet isinstanceR R RRRtnptemptytshapetfloat64RtFalsetsumt_validate_estimatort estimators_tzerosRtestimator_weights_tonestestimator_errors_RR tranget_boost( R!tXtyt sample_weightR%R$R tiboosttestimator_weighttestimator_errortsample_weight_sum((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pytfitIsT             cCsdS(sžImplement a single boost. Warning: This method needs to be overridden by subclasses. Parameters ---------- iboost : int The index of the current boost iteration. X : {array-like, sparse matrix} of shape = [n_samples, n_features] The training input samples. Sparse matrix can be CSC, CSR, COO, DOK, or LIL. COO, DOK, and LIL are converted to CSR. y : array-like of shape = [n_samples] The target values (class labels). sample_weight : array-like of shape = [n_samples] The current sample weights. random_state : RandomState The current random number generator Returns ------- sample_weight : array-like of shape = [n_samples] or None The reweighted sample weights. If None then boosting has terminated early. estimator_weight : float The weight for the current boost. If None then boosting has terminated early. error : float The classification error for the current boost. If None then boosting has terminated early. N((R!R<R9R:R;R ((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyR8¨s&ccsUxN|j|ƒD]=}t|ƒr9t||d|ƒVqt||d|ƒVqWdS(sÃReturn staged scores for X, y. This generator method yields the ensemble score after each iteration of boosting and therefore allows monitoring, such as to determine the score on a test set after each boost. Parameters ---------- X : {array-like, sparse matrix} of shape = [n_samples, n_features] The training input samples. Sparse matrix can be CSC, CSR, COO, DOK, or LIL. DOK and LIL are converted to CSR. y : array-like, shape = [n_samples] Labels for X. sample_weight : array-like, shape = [n_samples], optional Sample weights. Returns ------- z : float R;N(tstaged_predictRRR(R!R9R:R;ty_pred((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyt staged_scoreÐs cCs‘|jdks$t|jƒdkr3tdƒ‚ny:|jjƒ}td„t|j|jƒDƒƒ|SWntk rŒtdƒ‚nXdS(s¼Return the feature importances (the higher, the more important the feature). Returns ------- feature_importances_ : array, shape = [n_features] is?Estimator not fitted, call `fit` before `feature_importances_`.css"|]\}}||jVqdS(N(tfeature_importances_(t.0tweighttclf((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pys üssiUnable to compute feature importances since base_estimator does not have a feature_importances_ attributeN(R2R)tlenR(R4R0RtAttributeError(R!tnorm((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyRDís $   cCsa|jdks't|jttfƒrBt|dddtƒ}nt|ddddgƒ}|S(s%Ensure that X is in the proper formatR$R#R%R"tcooN(RR)R*R R RR(R!R9((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyt_validate_X_predicts  N( t__name__t __module__t__doc__RR)ttupleRR@R8RCtpropertyRDRL(((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyR2s _( cCs}|j|ƒ}tj|tj|jƒjdd|ƒtj|ƒ}|d|d||jddƒdd…tj fS(s°Calculate algorithm 4, step 2, equation c) of Zhu et al [1]. References ---------- .. [1] J. Zhu, H. Zou, S. Rosset, T. Hastie, "Multi-class AdaBoost", 2009. toutigð?taxisN( t predict_probaR+tcliptfinfoR%tepsR)tlogR0tnewaxis(t estimatort n_classesR9tprobat log_proba((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyt _samme_probas (cBs•eZdZdddddd„Zdd„Zd„Zd„Zd„Zd „Z d „Z d „Z d „Z d „Z d„Zd„Zd„ZRS(s¬ An AdaBoost classifier. An AdaBoost [1] classifier is a meta-estimator that begins by fitting a classifier on the original dataset and then fits additional copies of the classifier on the same dataset but where the weights of incorrectly classified instances are adjusted such that subsequent classifiers focus more on difficult cases. This class implements the algorithm known as AdaBoost-SAMME [2]. Read more in the :ref:`User Guide `. Parameters ---------- base_estimator : object, optional (default=None) The base estimator from which the boosted ensemble is built. Support for sample weighting is required, as well as proper ``classes_`` and ``n_classes_`` attributes. If ``None``, then the base estimator is ``DecisionTreeClassifier(max_depth=1)`` n_estimators : integer, optional (default=50) The maximum number of estimators at which boosting is terminated. In case of perfect fit, the learning procedure is stopped early. learning_rate : float, optional (default=1.) Learning rate shrinks the contribution of each classifier by ``learning_rate``. There is a trade-off between ``learning_rate`` and ``n_estimators``. algorithm : {'SAMME', 'SAMME.R'}, optional (default='SAMME.R') If 'SAMME.R' then use the SAMME.R real boosting algorithm. ``base_estimator`` must support calculation of class probabilities. If 'SAMME' then use the SAMME discrete boosting algorithm. The SAMME.R algorithm typically converges faster than SAMME, achieving a lower test error with fewer boosting iterations. random_state : int, RandomState instance or None, optional (default=None) If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Attributes ---------- estimators_ : list of classifiers The collection of fitted sub-estimators. classes_ : array of shape = [n_classes] The classes labels. n_classes_ : int The number of classes. estimator_weights_ : array of floats Weights for each estimator in the boosted ensemble. estimator_errors_ : array of floats Classification error for each estimator in the boosted ensemble. feature_importances_ : array of shape = [n_features] The feature importances if supported by the ``base_estimator``. See also -------- AdaBoostRegressor, GradientBoostingClassifier, sklearn.tree.DecisionTreeClassifier References ---------- .. [1] Y. Freund, R. Schapire, "A Decision-Theoretic Generalization of on-Line Learning and an Application to Boosting", 1995. .. [2] J. Zhu, H. Zou, S. Rosset, T. Hastie, "Multi-class AdaBoost", 2009. i2gð?sSAMME.Rc Cs8tt|ƒjd|d|d|d|ƒ||_dS(NRRRR (RRRt algorithm(R!RRRR_R ((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyRts cCsA|jdkr%td|jƒ‚ntt|ƒj|||ƒS(s}Build a boosted classifier from the training set (X, y). Parameters ---------- X : {array-like, sparse matrix} of shape = [n_samples, n_features] The training input samples. Sparse matrix can be CSC, CSR, COO, DOK, or LIL. DOK and LIL are converted to CSR. y : array-like of shape = [n_samples] The target values (class labels). sample_weight : array-like of shape = [n_samples], optional Sample weights. If None, the sample weights are initialized to ``1 / n_samples``. Returns ------- self : object tSAMMEsSAMME.Rsalgorithm %s is not supported(R`sSAMME.R(R_R(RRR@(R!R9R:R;((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyR@ƒscCs‡tt|ƒjdtddƒƒ|jdkrUt|jdƒsUtdƒ‚qUnt|jdƒsƒt d|jj j ƒ‚nd S( s:Check the estimator and set the base_estimator_ attribute.tdefaultt max_depthisSAMME.RRTsÚAdaBoostClassifier with algorithm='SAMME.R' requires that the weak learner supports the calculation of class probabilities with a predict_proba method. Please change the base estimator or set algorithm='SAMME' instead.R;s!%s doesn't support sample_weight.N( RRR1R R_thasattrtbase_estimator_t TypeErrorRR(t __class__RM(R!((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyR1žscCsE|jdkr(|j|||||ƒS|j|||||ƒSdS(s Implement a single boost. Perform a single boost according to the real multi-class SAMME.R algorithm or to the discrete SAMME algorithm and return the updated sample weights. Parameters ---------- iboost : int The index of the current boost iteration. X : {array-like, sparse matrix} of shape = [n_samples, n_features] The training input samples. Sparse matrix can be CSC, CSR, COO, DOK, or LIL. DOK and LIL are converted to CSR. y : array-like of shape = [n_samples] The target values (class labels). sample_weight : array-like of shape = [n_samples] The current sample weights. random_state : RandomState The current random number generator Returns ------- sample_weight : array-like of shape = [n_samples] or None The reweighted sample weights. If None then boosting has terminated early. estimator_weight : float The weight for the current boost. If None then boosting has terminated early. estimator_error : float The classification error for the current boost. If None then boosting has terminated early. sSAMME.RN(R_t _boost_realt_boost_discrete(R!R<R9R:R;R ((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyR8°s'cCsâ|jd|ƒ}|j||d|ƒ|j|ƒ}|dkrmt|dd ƒ|_t|jƒ|_n|jjt j |ddƒddƒ}||k} t j t j | d|ddƒƒ} | dkrÝ|dd fS|j} |j} t j d | ddgƒ} | j| |d d …t jfkƒ}|}t j|t j|jƒjd d |ƒd |j| d| |t j|ƒjddƒ}||jdksÕ|t j||dk|dkBƒ9}n|d| fS( s:Implement a single boost using the SAMME.R real algorithm.R R;itclasses_RSitweightsgð?ggð¿NRR(t_make_estimatorR@RTtgetattrR)RiRHt n_classes_ttakeR+targmaxtmeantaveragetarrayRYRURVR%RWRRXR0Rtexp(R!R<R9R:R;R RZty_predict_probat y_predictt incorrectR>R[tclassesty_codesty_codingR\R=((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyRgÞs2      ((6  c Csˆ|jd|ƒ}|j||d|ƒ|j|ƒ}|dkrmt|dd ƒ|_t|jƒ|_n||k}tj tj |d|ddƒƒ} | dkr¶|ddfS|j} | dd| kr |j j d ƒt|j ƒdkrt d ƒ‚nd S|jtjd| | ƒtj| dƒ} ||jd ks{|tj| ||dk| dkBƒ9}n|| | fS(s<Implement a single boost using the SAMME discrete algorithm.R R;iRiRjRSgð?giÿÿÿÿs\BaseClassifier in AdaBoostClassifier ensemble is worse than random, ensemble can not be fit.iN(NNN(RkR@tpredictRlR)RiRHRmR+RpRqR2tpopR(RRXRRs( R!R<R9R:R;R RZRuRvR>R[R=((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyRhs2      cCs_|j|ƒ}|jdkr:|jj|dkddƒS|jjtj|ddƒddƒS(s Predict classes for X. The predicted class of an input sample is computed as the weighted mean prediction of the classifiers in the ensemble. Parameters ---------- X : {array-like, sparse matrix} of shape = [n_samples, n_features] The training input samples. Sparse matrix can be CSC, CSR, COO, DOK, or LIL. DOK and LIL are converted to CSR. Returns ------- y : array of shape = [n_samples] The predicted classes. iiRSi(tdecision_functionRmRiRnR+Ro(R!R9tpred((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyRzHsccs¨|j}|j}|dkr^xƒ|j|ƒD])}tj|j|dkddƒƒVq.WnFxC|j|ƒD]2}tj|jtj|ddƒddƒƒVqnWdS(s}Return staged predictions for X. The predicted class of an input sample is computed as the weighted mean prediction of the classifiers in the ensemble. This generator method yields the ensemble prediction after each iteration of boosting and therefore allows monitoring, such as to determine the prediction on a test set after each boost. Parameters ---------- X : array-like of shape = [n_samples, n_features] The input samples. Returns ------- y : generator of array, shape = [n_samples] The predicted classes. iiRSiN(RmRitstaged_decision_functionR+RrRnRo(R!R9R[RwR}((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyRA`s   * csøt|dƒ|jˆƒ‰|j‰|jdd…tjf‰d}|jdkr{t‡‡fd†|j Dƒƒ}n.t‡‡fd†t |j |j ƒDƒƒ}||j jƒ:}ˆdkrô|dd…dfcd9<|jd d ƒS|S( sCompute the decision function of ``X``. Parameters ---------- X : {array-like, sparse matrix} of shape = [n_samples, n_features] The training input samples. Sparse matrix can be CSC, CSR, COO, DOK, or LIL. DOK and LIL are converted to CSR. Returns ------- score : array, shape = [n_samples, k] The decision function of the input samples. The order of outputs is the same of that of the `classes_` attribute. Binary classification is a special cases with ``k == 1``, otherwise ``k==n_classes``. For binary classification, values closer to -1 or 1 mean more like the first or second class in ``classes_``, respectively. RmNsSAMME.Rc3s!|]}t|ˆˆƒVqdS(N(R^(RERZ(R9R[(s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pys œsc3s1|]'\}}|jˆƒˆkj|VqdS(N(RztT(RERZtw(R9Rw(s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pys ŸsiiiÿÿÿÿRSi( RRLRmRiR+RYR)R_R0R2RR4(R!R9R}((R9RwR[s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyR|€s     c cs>t|dƒ|j|ƒ}|j}|jdd…tjf}d}d}xêt|j|j ƒD]Ó\}}||7}|j dkrt |||ƒ}n"|j |ƒ}||kj |}|dkrÔ|}n ||7}|dkr-tj|ƒ} | dd…dfcd9<| |jdd ƒVqc||VqcWdS( sCompute decision function of ``X`` for each boosting iteration. This method allows monitoring (i.e. determine error on testing set) after each boosting iteration. Parameters ---------- X : {array-like, sparse matrix} of shape = [n_samples, n_features] The training input samples. Sparse matrix can be CSC, CSR, COO, DOK, or LIL. DOK and LIL are converted to CSR. Returns ------- score : generator of array, shape = [n_samples, k] The decision function of the input samples. The order of outputs is the same of that of the `classes_` attribute. Binary classification is a special cases with ``k == 1``, otherwise ``k==n_classes``. For binary classification, values closer to -1 or 1 mean more like the first or second class in ``classes_``, respectively. RmNgsSAMME.RiiiÿÿÿÿRSi(RRLRmRiR+RYR)RR4R2R_R^RzRtcopyR0( R!R9R[RwR}RJRFRZt current_predttmp_pred((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyR~©s*        cst|dƒ|j‰|jˆƒ‰ˆdkrKtjˆjddfƒS|jdkrt‡‡fd†|jDƒƒ}n+t‡fd†t |j|j ƒDƒƒ}||j jƒ:}tj dˆd|ƒ}|jddƒd d …tj f}d||d k<||:}|S( s¶Predict class probabilities for X. The predicted class probabilities of an input sample is computed as the weighted mean predicted class probabilities of the classifiers in the ensemble. Parameters ---------- X : {array-like, sparse matrix} of shape = [n_samples, n_features] The training input samples. Sparse matrix can be CSC, CSR, COO, DOK, or LIL. DOK and LIL are converted to CSR. Returns ------- p : array of shape = [n_samples, n_classes] The class probabilities of the input samples. The order of outputs is the same of that of the `classes_` attribute. RmiisSAMME.Rc3s!|]}t|ˆˆƒVqdS(N(R^(RERZ(R9R[(s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pys ûsc3s(|]\}}|jˆƒ|VqdS(N(RT(RERZR€(R9(s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pys þsgð?RSNg( RRmRLR+R5R-R_R0R2RR4RsRY(R!R9R\t normalizer((R9R[s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyRTÞs"    % c cs|j|ƒ}|j}d}d}xæt|j|jƒD]Ï\}}||7}|jdkrtt|||ƒ}n|j|ƒ|}|dkrœ|}n ||7}t j d|d||ƒ}|j ddƒdd…t j f} d| | dk<|| :}|Vq:WdS(s¸Predict class probabilities for X. The predicted class probabilities of an input sample is computed as the weighted mean predicted class probabilities of the classifiers in the ensemble. This generator method yields the ensemble predicted class probabilities after each iteration of boosting and therefore allows monitoring, such as to determine the predicted class probabilities on a test set after each boost. Parameters ---------- X : {array-like, sparse matrix} of shape = [n_samples, n_features] The training input samples. Sparse matrix can be CSC, CSR, COO, DOK, or LIL. DOK and LIL are converted to CSR. Returns ------- p : generator of array, shape = [n_samples] The class probabilities of the input samples. The order of outputs is the same of that of the `classes_` attribute. gsSAMME.Rgð?iRSN( RLRmR)RR4R2R_R^RTR+RsR0RY( R!R9R[R\RJRFRZt current_probat real_probaR„((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pytstaged_predict_proba s$      % cCstj|j|ƒƒS(sÂPredict class log-probabilities for X. The predicted class log-probabilities of an input sample is computed as the weighted mean predicted class log-probabilities of the classifiers in the ensemble. Parameters ---------- X : {array-like, sparse matrix} of shape = [n_samples, n_features] The training input samples. Sparse matrix can be CSC, CSR, COO, DOK, or LIL. DOK and LIL are converted to CSR. Returns ------- p : array of shape = [n_samples, n_classes] The class probabilities of the input samples. The order of outputs is the same of that of the `classes_` attribute. (R+RXRT(R!R9((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pytpredict_log_proba>sN(RMRNROR)RR@R1R8RgRhRzRAR|R~RTR‡Rˆ(((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyR's$L   . : 0  ) 5 , 4cBs_eZdZd dddd d„Zd d„Zd„Zd„Zd„Zd „Z d „Z RS( sú An AdaBoost regressor. An AdaBoost [1] regressor is a meta-estimator that begins by fitting a regressor on the original dataset and then fits additional copies of the regressor on the same dataset but where the weights of instances are adjusted according to the error of the current prediction. As such, subsequent regressors focus more on difficult cases. This class implements the algorithm known as AdaBoost.R2 [2]. Read more in the :ref:`User Guide `. Parameters ---------- base_estimator : object, optional (default=None) The base estimator from which the boosted ensemble is built. Support for sample weighting is required. If ``None``, then the base estimator is ``DecisionTreeRegressor(max_depth=3)`` n_estimators : integer, optional (default=50) The maximum number of estimators at which boosting is terminated. In case of perfect fit, the learning procedure is stopped early. learning_rate : float, optional (default=1.) Learning rate shrinks the contribution of each regressor by ``learning_rate``. There is a trade-off between ``learning_rate`` and ``n_estimators``. loss : {'linear', 'square', 'exponential'}, optional (default='linear') The loss function to use when updating the weights after each boosting iteration. random_state : int, RandomState instance or None, optional (default=None) If int, random_state is the seed used by the random number generator; If RandomState instance, random_state is the random number generator; If None, the random number generator is the RandomState instance used by `np.random`. Attributes ---------- estimators_ : list of classifiers The collection of fitted sub-estimators. estimator_weights_ : array of floats Weights for each estimator in the boosted ensemble. estimator_errors_ : array of floats Regression error for each estimator in the boosted ensemble. feature_importances_ : array of shape = [n_features] The feature importances if supported by the ``base_estimator``. See also -------- AdaBoostClassifier, GradientBoostingRegressor, sklearn.tree.DecisionTreeRegressor References ---------- .. [1] Y. Freund, R. Schapire, "A Decision-Theoretic Generalization of on-Line Learning and an Application to Boosting", 1995. .. [2] H. Drucker, "Improving Regressors using Boosting Techniques", 1997. i2gð?tlinearc CsAtt|ƒjd|d|d|d|ƒ||_||_dS(NRRRR (RRRtlossR (R!RRRRŠR ((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyR–s cCs:|jdkrtdƒ‚ntt|ƒj|||ƒS(sxBuild a boosted regressor from the training set (X, y). Parameters ---------- X : {array-like, sparse matrix} of shape = [n_samples, n_features] The training input samples. Sparse matrix can be CSC, CSR, COO, DOK, or LIL. DOK and LIL are converted to CSR. y : array-like of shape = [n_samples] The target values (real numbers). sample_weight : array-like of shape = [n_samples], optional Sample weights. If None, the sample weights are initialized to 1 / n_samples. Returns ------- self : object R‰tsquaret exponentials1loss must be 'linear', 'square', or 'exponential'(R‰R‹RŒ(RŠR(RRR@(R!R9R:R;((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyR@¦s cCs&tt|ƒjdtddƒƒdS(s:Check the estimator and set the base_estimator_ attribute.RaRbiN(RRR1R (R!((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyR1ÂscCsÚ|jd|ƒ}t|ƒ}||d:}|j|jdƒ}|j|ddƒ} tj| dtƒ} |j|| || ƒ|j |ƒ} tj | |ƒ} | j ƒ} | dkrË| | :} n|j dkrç| d C} n&|j d kr d tj | ƒ} n|| jƒ} | dkr6|d dfS| d krnt|jƒd krj|jjdƒndS| d | }|jtjd |ƒ}||jd ksÍ|tj|d | |jƒ9}n||| fS(sImplement a single boost for regression Perform a single boost according to the AdaBoost.R2 algorithm and return the updated sample weights. Parameters ---------- iboost : int The index of the current boost iteration. X : {array-like, sparse matrix} of shape = [n_samples, n_features] The training input samples. Sparse matrix can be CSC, CSR, COO, DOK, or LIL. DOK and LIL are converted to CSR. y : array-like of shape = [n_samples] The target values (class labels in classification, real numbers in regression). sample_weight : array-like of shape = [n_samples] The current sample weights. random_state : RandomState The current random number generator Returns ------- sample_weight : array-like of shape = [n_samples] or None The reweighted sample weights. If None then boosting has terminated early. estimator_weight : float The weight for the current boost. If None then boosting has terminated early. estimator_error : float The regression error for the current boost. If None then boosting has terminated early. R iÿÿÿÿitsidetrightRgR‹iRŒgð?gà?iN(NNN(RkRt random_sampleR-t searchsortedR+RrR/R@RztabstmaxRŠRsR0RHR2R{R)RRXRtpower(R!R<R9R:R;R RZtcdftuniform_samplest bootstrap_idxRut error_vectt error_maxR>tbetaR=((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyR8Çs<'         c Csætjg|j| D]}|j|ƒ^qƒj}tj|ddƒ}t|j|ddƒ}|d|dd…dfdd…tjfk}|j ddƒ}|tj |j dƒ|f} |tj |j dƒ| fS(NRSigà?iÿÿÿÿi( R+RrR2RzRtargsortRR4RYRotarangeR-( R!R9tlimittestt predictionst sorted_idxt weight_cdftmedian_or_abovet median_idxtmedian_estimators((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyt_get_median_predict"s/3 cCs5t|dƒ|j|ƒ}|j|t|jƒƒS(s-Predict regression value for X. The predicted regression value of an input sample is computed as the weighted median prediction of the classifiers in the ensemble. Parameters ---------- X : {array-like, sparse matrix} of shape = [n_samples, n_features] The training input samples. Sparse matrix can be CSC, CSR, COO, DOK, or LIL. DOK and LIL are converted to CSR. Returns ------- y : array of shape = [n_samples] The predicted regression values. R4(RRLR¤RHR2(R!R9((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyRz4s ccsWt|dƒ|j|ƒ}x4t|jdƒD] \}}|j|d|ƒVq/WdS(s Return staged predictions for X. The predicted regression value of an input sample is computed as the weighted median prediction of the classifiers in the ensemble. This generator method yields the ensemble prediction after each iteration of boosting and therefore allows monitoring, such as to determine the prediction on a test set after each boost. Parameters ---------- X : {array-like, sparse matrix} of shape = [n_samples, n_features] The training input samples. Sparse matrix can be CSC, CSR, COO, DOK, or LIL. DOK and LIL are converted to CSR. Returns ------- y : generator of array, shape = [n_samples] The predicted regression values. R4iRœN(RRLt enumerateR2R¤(R!R9tit_((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyRAJs N( RMRNROR)RR@R1R8R¤RzRA(((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyRTsA   [  (-ROtabcRRtnumpyR+tbaseRRRRRt externalsRtexternals.six.movesRR R7tforestR ttreeR R t tree.treeR t tree._treeRtutilsRRRt utils.extmathRtmetricsRRtsklearn.utils.validationRRt__all__twith_metaclassRR^RR(((s?lib/python2.7/site-packages/sklearn/ensemble/weight_boosting.pyts.  " "á ÿÿ/