ó ‡ˆ\c @sÒdZddlZddlmZddlmZmZmZddlm Z ddl Z ddl j ZddlmZddlmZdd lmZdd lmZd d lmZd d lmZddlmZd dlmZd dlmZeddddddƒeddddddƒeddddddƒeddddddƒedd dd!dd"ƒfZedd#dd$dd%ƒZeje ƒZ!dd&e#de$e$d'„Z%d(„Z&d)„Z'dS(*shRCV1 dataset. The dataset page is available at http://jmlr.csail.mit.edu/papers/volume5/lewis04a/ iÿÿÿÿN(tremove(tdirnametexiststjoin(tGzipFilei(t get_data_home(t _pkl_filepath(t _fetch_remote(tRemoteFileMetadatai(tmakedirs(t_joblib(tload_svmlight_files(tshuffle(tBunchturls.https://ndownloader.figshare.com/files/5976069tchecksumt@ed40f7e418d10484091b059703eeb95ae3199fe042891dcec4be6696b9968374tfilenames lyrl2004_vectors_test_pt0.dat.gzs.https://ndownloader.figshare.com/files/5976066t@87700668ae45d45d5ca1ef6ae9bd81ab0f5ec88cc95dcef9ae7838f727a13aa6s lyrl2004_vectors_test_pt1.dat.gzs.https://ndownloader.figshare.com/files/5976063t@48143ac703cbe33299f7ae9f4995db49a258690f60e5debbff8995c34841c7f5s lyrl2004_vectors_test_pt2.dat.gzs.https://ndownloader.figshare.com/files/5976060t@dfcb0d658311481523c6e6ca0c3f5a3e1d3d12cde5d7a8ce629a9006ec7dbb39s lyrl2004_vectors_test_pt3.dat.gzs.https://ndownloader.figshare.com/files/5976057t@5468f656d0ba7a83afc7ad44841cf9a53048a5c083eedc005dcdb5cc768924aeslyrl2004_vectors_train.dat.gzs.https://ndownloader.figshare.com/files/5976048t@2a98e5e5d8b770bded93afc8930d88299474317fe14181aee1466cc754d0d1c1srcv1v2.topics.qrels.gztallc) Cs¼d}d}d}d} td|ƒ}t|dƒ} |rXt| ƒsXt| ƒqXnt| dƒ} t| dƒ} t| d ƒ} t| d ƒ}|rìt| ƒ s´t| ƒ rìg}xJtD]B}tjd |jƒt |d | ƒ}|j t d |ƒƒqÁWt |d|ƒ}t j|d|d|d|d|dgƒjƒ}tj|d|d|d|d|dfƒ}|jtjƒ}tj|| ddƒtj|| ddƒxF|D]}|jƒt|jƒqÈWntj| ƒ}tj| ƒ}|rEt| ƒ s*t|ƒ rEtjd tjƒt td | ƒ}d}d}d}tj||fdtjƒ}tj|dtjƒ}i}t d |ddƒÀ}x¶|D]®}|jdƒjdƒ}t |ƒdkr½|\}} }!||kr|d7}|||`. .. versionadded:: 0.17 Parameters ---------- data_home : string, optional Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in '~/scikit_learn_data' subfolders. subset : string, 'train', 'test', or 'all', default='all' Select the dataset to load: 'train' for the training set (23149 samples), 'test' for the test set (781265 samples), 'all' for both, with the training samples first if shuffle is False. This follows the official LYRL2004 chronological split. download_if_missing : boolean, default=True If False, raise a IOError if the data is not locally available instead of trying to download the data from the source site. random_state : int, RandomState instance or None (default) Determines random number generation for dataset shuffling. Pass an int for reproducible output across multiple function calls. See :term:`Glossary `. shuffle : bool, default=False Whether to shuffle dataset. return_X_y : boolean, default=False. If True, returns ``(dataset.data, dataset.target)`` instead of a Bunch object. See below for more information about the `dataset.data` and `dataset.target` object. .. versionadded:: 0.20 Returns ------- dataset : dict-like object with the following attributes: dataset.data : scipy csr array, dtype np.float64, shape (804414, 47236) The array has 0.16% of non zero values. dataset.target : scipy csr array, dtype np.uint8, shape (804414, 103) Each sample has a value of 1 in its categories, and 0 in others. The array has 3.15% of non zero values. dataset.sample_id : numpy array, dtype np.uint32, shape (804414,) Identification number of each sample, as ordered in dataset.data. dataset.target_names : numpy array, dtype object, length (103) Names of each target (RCV1 topics), as ordered in dataset.target. dataset.DESCR : string Description of the RCV1 dataset. (data, target) : tuple if ``return_X_y`` is True .. versionadded:: 0.20 i>F i„¸igimZt data_hometRCV1s samples.pkls sample_id.pklssample_topics.pklstopics_names.pklsDownloading %sRRt n_featuresiiiiii iiiitcompressiÿÿÿÿtdtypetmodetrbtasciiu NRttrainttestsLUnknown subset parameter. Got '%s' instead of one of ('all', 'train', test')t random_statetdescrsrcv1.rsttdatattargett sample_idt target_namestDESCR(/RRRR Rt XY_METADATAtloggertinfoRRtappendRR tsptvstackttocsrtnpthstacktastypetuint32R tdumptcloseRtnametloadtTOPICS_METADATAtzerostuint8tint32tdecodetsplittlentintt_find_permutationtemptytobjecttkeystargsortt csr_matrixt ValueErrortshuffle_Rt__file__topentreadR ()Rtsubsettdownload_if_missingR"R t return_X_yt N_SAMPLESt N_FEATURESt N_CATEGORIEStN_TRAINtrcv1_dirt samples_pathtsample_id_pathtsample_topics_patht topics_pathtfilesteacht file_pathtXytXR&tfttopics_archive_pathtn_cattn_doct doc_previoustyt sample_id_bistcategory_namestlinetline_componentstcattdoct_t permutationt categoriestktordert module_pathtrst_filetfdescr((s4lib/python2.7/site-packages/sklearn/datasets/rcv1.pyt fetch_rcv1Os´J   82             !        $  cCsP|j}tj|dtjƒ}tj|dtjƒ}tj|||ƒ|S(sinverse permutation pR(tsizeR0R9R;tarangetput(tptntsti((s4lib/python2.7/site-packages/sklearn/datasets/rcv1.pyt_inverse_permutations  cCs2tj|ƒ}tj|ƒ}t|ƒ}||S(s find the permutation from a to b(R0RDRx(tatbtttutu_((s4lib/python2.7/site-packages/sklearn/datasets/rcv1.pyR@s ((t__doc__tloggingtosRtos.pathRRRtgzipRtnumpyR0t scipy.sparsetsparseR-tbaseRRRRt utils.fixesR tutilsR tsvmlight_formatR R RGR R)R8t getLoggert__name__R*tNonetTruetFalseRpRxR@(((s4lib/python2.7/site-packages/sklearn/datasets/rcv1.pytsX      Æ