B \+@sdZddlZddlZddlmZddlZddlZddlmZm Z m Z ddl Z ddl mZddl mZddl mZd d lmZd d lmZd d lmZd d lmZd dlmZeddddZeddddZeeZdddZdddZddZ dS) zKDDCUP 99 dataset. A classic dataset for anomaly detection. The dataset page is available from UCI Machine Learning Repository https://archive.ics.uci.edu/ml/machine-learning-databases/kddcup99-mld/kddcup.data.gz N)GzipFile)dirnameexistsjoin) _fetch_remote) get_data_home)RemoteFileMetadata)six)Bunch)_joblib)check_random_state)shuffleZ kddcup99_dataz.https://ndownloader.figshare.com/files/5976045Z@3b6c942aa0356c0ca35b7b595a26c89d343652c9db428893e7494f837b274292)filenameurlZchecksumZkddcup99_10_dataz.https://ndownloader.figshare.com/files/5976042Z@8045aca0d84e70e622d1148d7df782496f6333bf6eb979a1b0837c42a9fd9561FTc Cs t|d}t|||d}|j}|j} |dkr| dk} t| } || ddf} | | } || ddf}| | }|jd}t|}|d|d}||}||}tj | |f}tj | |f} |dks|d ks|d kr|ddd fd k} tj || dd f|| d dff}| | } t |dddfd t |dddf<t |dddfd t |dddf<t |dddfd t |dddf<|d kr|dddfdk} || }| | } tj |dddf|dddf|dddff}|d kr^|dddfdk} || }| | } tj |dddf|dddf|dddff}|dkrtj |dddf|dddf|dddf|dddff}|rt|| |d\}} tt}tt|dd}|}WdQRX|r|| fSt|| |dS)aLoad the kddcup99 dataset (classification). Download it if necessary. ================= ==================================== Classes 23 Samples total 4898431 Dimensionality 41 Features discrete (int) or continuous (float) ================= ==================================== Read more in the :ref:`User Guide `. .. versionadded:: 0.18 Parameters ---------- subset : None, 'SA', 'SF', 'http', 'smtp' To return the corresponding classical subsets of kddcup 99. If None, return the entire kddcup 99 dataset. data_home : string, optional Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in '~/scikit_learn_data' subfolders. .. versionadded:: 0.19 shuffle : bool, default=False Whether to shuffle dataset. random_state : int, RandomState instance or None (default) Determines random number generation for dataset shuffling and for selection of abnormal samples if `subset='SA'`. Pass an int for reproducible output across multiple function calls. See :term:`Glossary `. percent10 : bool, default=True Whether to load only 10 percent of the data. download_if_missing : bool, default=True If False, raise a IOError if the data is not locally available instead of trying to download the data from the source site. return_X_y : boolean, default=False. If True, returns ``(data, target)`` instead of a Bunch object. See below for more information about the `data` and `target` object. .. versionadded:: 0.20 Returns ------- data : Bunch Dictionary-like object, the interesting attributes are: - 'data', the data to learn. - 'target', the regression target for each sample. - 'DESCR', a description of the dataset. (data, target) : tuple if ``return_X_y`` is True .. versionadded:: 0.20 ) data_home)r percent10download_if_missingZSAsnormal.Nri1 ZSFZhttpZsmtp r g?r shttpssmtp) random_stateZdescrz kddcup99.rst)datatargetZDESCR)r_fetch_brute_kddcup99rrnpZ logical_notshaperZrandintZr_Zc_logastypefloatshuffle_methodr__file__openrreadr )ZsubsetrrrrrZ return_X_ykddcup99rrstZnormal_samplesZnormal_targetsZabnormal_samplesZabnormal_targetsZn_samples_abnormalrZ module_pathZrst_fileZfdescrr*8lib/python3.7/site-packages/sklearn/datasets/kddcup99.pyfetch_kddcup991s^?   &,,, 4 4 Br,c*Cst|d}tjddkrd}nd}|r:t|d|}t}nt|d|}t}t|d}t|d }t|}|r|st|t d |j t ||d d t fd dddt fdt fdt fdt fdt fdt fdt fdt fdt fdt fdt fdt fdt fdt fdt fdt fd t fd!t fd"t fd#t fd$t fd%t fd&t fd't fd(t fd)t fd*t fd+t fd,t fd-t fd.t fd/t fd0t fd1t fd2t fd3t fd4t fd5g*} t| } td6t||j} t| d7d8} g} x:| D].}tjr|}| |d9dd:qW| td;t| tj| td<} x8td=D],}| d>d>|f | || d>d>|f<q(W| d>d>d>d?f}| d>d>d?f}t!j"||dd@t!j"||dd@n|s|st#dAy ||fWn*t$k rt!%|}t!%|}YnXt&||dBS)CaLoad the kddcup99 dataset, downloading it if necessary. Parameters ---------- data_home : string, optional Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in '~/scikit_learn_data' subfolders. download_if_missing : boolean, default=True If False, raise a IOError if the data is not locally available instead of trying to download the data from the source site. percent10 : bool, default=True Whether to load only 10 percent of the data. Returns ------- dataset : dict-like object with the following attributes: dataset.data : numpy array of shape (494021, 41) Each row corresponds to the 41 features in the dataset. dataset.target : numpy array of shape (494021,) Each value corresponds to one of the 21 attack types or to the label 'normal.'. dataset.DESCR : string Description of the kddcup99 dataset. )rrz-py3Z kddcup99_10r&ZsamplesZtargetszDownloading %s)rZduration)Z protocol_typeZS4)ZserviceZS11)flagZS6Z src_bytesZ dst_bytesZlandZwrong_fragmentZurgentZhotZnum_failed_loginsZ logged_inZnum_compromisedZ root_shellZ su_attemptedZnum_rootZnum_file_creationsZ num_shellsZnum_access_filesZnum_outbound_cmdsZ is_host_loginZis_guest_logincountZ srv_countZ serror_rateZsrv_serror_rateZ rerror_rateZsrv_rerror_rateZ same_srv_rateZ diff_srv_rateZsrv_diff_host_rateZdst_host_countZdst_host_srv_countZdst_host_same_srv_rateZdst_host_diff_srv_rateZdst_host_same_src_port_rateZdst_host_srv_diff_host_rateZdst_host_serror_rateZdst_host_srv_serror_rateZdst_host_rerror_rateZdst_host_srv_rerror_rate)labelsZS16zextracting archiver))rmode ,zextraction done)dtype*N)compressz1Data not found and `download_if_missing` is False)rr)'rsys version_inforARCHIVE_10_PERCENTARCHIVEr_mkdirploggerinforrintr!rr5debugrr readlinesr ZPY3decodeappendreplacesplitcloseosremoveZasarrayobjectranger r dumpIOError NameErrorloadr )rrrZ dir_suffixZ kddcup_dirarchiveZ samples_pathZ targets_pathZ availableZdtZDTZ archive_pathZfile_ZXylinejXyr*r*r+rs           ,  rc CsDyt|Wn0tk r>}z|jtjkr.Wdd}~XYnXdS)zgEnsure directory d exists (like mkdir -p on Unix) No guarantee that the directory is writable. N)rHmakedirsOSErrorerrnoZEEXIST)der*r*r+r=8s  r=)NNFNTTF)NTT)!__doc__r9rWZgziprZloggingrHos.pathrrrZnumpyrbaserrr Z externalsr Zutilsr r rrr"r<r;Z getLogger__name__r>r,rr=r*r*r*r+ s>           }