B \L @svdZddlmZmZmZddlmZmZmZm Z ddl Z ddl m Z ddl ZddlmZmZmZdd lmZdd lmZdd lmZdd lmZdd lmZe eZeddddZeddddZeddddeddddeddddfZ eddd Z!d5d"d#Z"d$d%Z#d6d'd(Z$dd!d)dd&e%d*d+e%d,d-fd!d&fd.d/Z&d7d0d1Z'd2dd!d)d&e%d*d+e%d,d-fd!fd3d4Z(dS)8zLabeled Faces in the Wild (LFW) dataset This dataset is a collection of JPEG pictures of famous people collected over the internet, all details are available on the official website: http://vis-www.cs.umass.edu/lfw/ )listdirmakedirsremove)dirnamejoinexistsisdirN) LooseVersion) get_data_home _fetch_remoteRemoteFileMetadata) deprecated)Bunch)Memory)b)_joblibzlfw.tgzz.https://ndownloader.figshare.com/files/5976018Z@055f7d9c632d7370e6fb4afc7468d40f970c34a80d4c6f50ffec63f5a8d536c0)filenameurlZchecksumzlfw-funneled.tgzz.https://ndownloader.figshare.com/files/5976015Z@b47c8422c8cded889dc5a13418c4bc2abbda121092b3533a83306f90d900100azpairsDevTrain.txtz.https://ndownloader.figshare.com/files/5976012Z@1d454dada7dfeca0e7eab6f65dc4e97a6312d44cf142207be28d688be92aabfazpairsDevTest.txtz.https://ndownloader.figshare.com/files/5976009Z@7cb06600ea8b2814ac26e946201cdb304296262aad67d046a16a7ec85d0ff87cz pairs.txtz.https://ndownloader.figshare.com/files/5976006Z@ea42330c62c92989f9d7c03237ed5d591365e89b3e649747777b70e692dc1592zIThis function was deprecated in version 0.20 and will be removed in 0.22.cCs||}||}|S)a=Scale back to 0-1 range in case of normalization for plotting. .. deprecated:: 0.20 This function was deprecated in version 0.20 and will be removed in 0.22. Parameters ---------- face : array_like The array to scale Returns ------- array_like The scaled array )minmax)faceZscaledr3lib/python3.7/site-packages/sklearn/datasets/lfw.py scale_faceEs  rTc Cst|d}t|d}t|s$t|xLtD]D}t||j}t|s*|rbtd|jt ||dq*t d|q*W|rt|d}t }nt|d}t }t|st||j}t|s|rtd|jt ||dn t d|d d l } td || |d j|d t|||fS)z0Helper function to download any missing LFW data) data_homelfw_homezDownloading LFW metadata: %s)rz %s is missingZ lfw_funneledZlfwz!Downloading LFW data (~200MB): %srNz$Decompressing the data archive to %szr:gz)path)r rrrTARGETSrloggerinforr IOErrorFUNNELED_ARCHIVEARCHIVEtarfiledebugopenZ extractallr) rfunneleddownload_if_missingrtargetZtarget_filepathdata_folder_patharchiveZ archive_pathr%rrr_check_fetch_lfwbs:          r-cCsddlm}m}tddtddf}|dkr2|}ntddt||D}|\}}|j|j|jpdd} |j|j|jpzd} |dk rt |}t || } t || } t |} |st j | | | ft jd } nt j | | | d ft jd } xt|D]\} }| d dkrtd | d| ||}|jdkr4td |t j||t jd }|d}|dk rd|||}|sv|jdd}|| | df<qW| S)zInternally used to load imagesr)imreadimresizerNcss|]\}}|p|VqdS)Nr).0sZdsrrr sz_load_imgs..r )dtypeizLoading face #%05d / %05dzLFailed to read the image file %s, Please make sure that libjpeg is installedgo@)Zaxis.)Zexternals._pilutilr.r/slicetuplezipstopstartstepfloatintlennpzerosZfloat32 enumerater r&ndim RuntimeErrorZasarrayZmean) file_pathsslice_colorresizer.r/Z default_sliceZh_sliceZw_slicehwn_facesfacesi file_pathZimgrrrr _load_imgss>      rNFcsgg}}xxtt|D]h}t||ts0qfddttD}t|} | |kr|dd}||g| ||qWt|} | dkrtd|t |} t | |} t ||||} t | }tj d|| || |} } | | | fS)z~Perform the actual data loading for the lfw people dataset This operation is meant to be cached by a joblib wrapper. csg|]}t|qSr)r)r1f) folder_pathrr sz%_fetch_lfw_people.._ rz*min_faces_per_person=%d is too restrictive*)sortedrrrr>replaceextend ValueErrorr?uniqueZ searchsortedrNZarangeZrandomZ RandomStateZshuffle)r+rErFrGmin_faces_per_personZ person_namesrDZ person_namepathsZ n_picturesrJ target_namesr*rKindicesr)rPr_fetch_lfw_peoples,      r^g?FNc Cst|||d\}} td|ttjtdkr@t|ddd} nt|ddd} | t} | | ||||d\} } }| t | d }t t }t t|d d }|}Wd QRX|r|| fSt|| | ||d S)ar Load the Labeled Faces in the Wild (LFW) people dataset (classification). Download it if necessary. ================= ======================= Classes 5749 Samples total 13233 Dimensionality 5828 Features real, between 0 and 255 ================= ======================= Read more in the :ref:`User Guide `. Parameters ---------- data_home : optional, default: None Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in '~/scikit_learn_data' subfolders. funneled : boolean, optional, default: True Download and use the funneled variant of the dataset. resize : float, optional, default 0.5 Ratio used to resize the each face picture. min_faces_per_person : int, optional, default None The extracted dataset will only retain pictures of people that have at least `min_faces_per_person` different pictures. color : boolean, optional, default False Keep the 3 RGB channels instead of averaging them to a single gray level channel. If color is True the shape of the data has one more dimension than the shape with color = False. slice_ : optional Provide a custom 2D slice (height, width) to extract the 'interesting' part of the jpeg files and avoid use statistical correlation from the background download_if_missing : optional, True by default If False, raise a IOError if the data is not locally available instead of trying to download the data from the source site. return_X_y : boolean, default=False. If True, returns ``(dataset.data, dataset.target)`` instead of a Bunch object. See below for more information about the `dataset.data` and `dataset.target` object. .. versionadded:: 0.20 Returns ------- dataset : dict-like object with the following attributes: dataset.data : numpy array of shape (13233, 2914) Each row corresponds to a ravelled face image of original size 62 x 47 pixels. Changing the ``slice_`` or resize parameters will change the shape of the output. dataset.images : numpy array of shape (13233, 62, 47) Each row is a face image corresponding to one of the 5749 people in the dataset. Changing the ``slice_`` or resize parameters will change the shape of the output. dataset.target : numpy array of shape (13233,) Labels associated to each face image. Those labels range from 0-5748 and correspond to the person IDs. dataset.DESCR : string Description of the Labeled Faces in the Wild (LFW) dataset. (data, target) : tuple if ``return_X_y`` is True .. versionadded:: 0.20 )rr(r)z Loading LFW people faces from %sz0.12r)cachedircompressverbose)locationrerf)rGrZrFrEdescrzlfw.rstN)dataZimagesr*r\DESCR)r-r r&r r __version__rcacher^reshaper>r__file__r'rreadr)rr(rGrZrFrEr)Z return_X_yrr+m load_funcrKr*r\X module_pathrst_filefdescrrrrfetch_lfw_peoples(Q   rwc Cst|d}dd|D}WdQRXdd|D}t|}tj|tjd} t} x0t|D]"\} } t| dkrd| | <| d t| ddf| d t| d dff} nZt| d krd | | <| d t| ddf| d t| ddff} ntd | d| fxxt| D]l\}\}}yt||}Wn&t k rPt|t |d }YnXtt t |}t|||}| |qWq\Wt| |||}t|j}|d }|d d |d |d ||_|| tddgfS)z}Perform the actual data loading for the LFW pairs dataset This operation is meant to be cached by a joblib wrapper. rbcSsg|]}|tdqS) )stripsplitr)r1ZlnrrrrQtsz$_fetch_lfw_pairs..NcSsg|]}t|dkr|qS)r)r>)r1ZslrrrrQus)r4r5r rrzinvalid line %d: %rzUTF-8zDifferent personsz Same person)r'r>r?r@r=listrArXr TypeErrorstrrUrappendrNshapepopinsertZarray)index_file_pathr+rErFrGZ index_fileZ split_linesZ pair_specsZn_pairsr*rDrLZ componentsZpairjnameidxZ person_folder filenamesrMpairsrrJrrr_fetch_lfw_pairsks>      rtrainc Cst|||d\}}td||ttjtdkrBt|ddd} nt|ddd} | t} dd d d } || krt d |t t | ft || |} | | ||||d \} }}tt}tt |dd}|}WdQRXt| t| d| |||dS)a Load the Labeled Faces in the Wild (LFW) pairs dataset (classification). Download it if necessary. ================= ======================= Classes 5749 Samples total 13233 Dimensionality 5828 Features real, between 0 and 255 ================= ======================= In the official `README.txt`_ this task is described as the "Restricted" task. As I am not sure as to implement the "Unrestricted" variant correctly, I left it as unsupported for now. .. _`README.txt`: http://vis-www.cs.umass.edu/lfw/README.txt The original images are 250 x 250 pixels, but the default slice and resize arguments reduce them to 62 x 47. Read more in the :ref:`User Guide `. Parameters ---------- subset : optional, default: 'train' Select the dataset to load: 'train' for the development training set, 'test' for the development test set, and '10_folds' for the official evaluation set that is meant to be used with a 10-folds cross validation. data_home : optional, default: None Specify another download and cache folder for the datasets. By default all scikit-learn data is stored in '~/scikit_learn_data' subfolders. funneled : boolean, optional, default: True Download and use the funneled variant of the dataset. resize : float, optional, default 0.5 Ratio used to resize the each face picture. color : boolean, optional, default False Keep the 3 RGB channels instead of averaging them to a single gray level channel. If color is True the shape of the data has one more dimension than the shape with color = False. slice_ : optional Provide a custom 2D slice (height, width) to extract the 'interesting' part of the jpeg files and avoid use statistical correlation from the background download_if_missing : optional, True by default If False, raise a IOError if the data is not locally available instead of trying to download the data from the source site. Returns ------- The data is returned as a Bunch object with the following attributes: data : numpy array of shape (2200, 5828). Shape depends on ``subset``. Each row corresponds to 2 ravel'd face images of original size 62 x 47 pixels. Changing the ``slice_``, ``resize`` or ``subset`` parameters will change the shape of the output. pairs : numpy array of shape (2200, 2, 62, 47). Shape depends on ``subset`` Each row has 2 face images corresponding to same or different person from the dataset containing 5749 people. Changing the ``slice_``, ``resize`` or ``subset`` parameters will change the shape of the output. target : numpy array of shape (2200,). Shape depends on ``subset``. Labels associated to each pair of images. The two label values being different persons or the same person. DESCR : string Description of the Labeled Faces in the Wild (LFW) dataset. )rr(r)zLoading %s LFW pairs from %sz0.12rcr)rdrerf)rgrerfzpairsDevTrain.txtzpairsDevTest.txtz pairs.txt)rZtestZ10_foldsz+subset='%s' is invalid: should be one of %r)rGrFrErizlfw.rstNrh)rjrr*r\rk)r-r r&r rrlrrmrrXr}rUkeysrrror'rprrnr>)Zsubsetrr(rGrFrEr)rr+rqrrZlabel_filenamesrrr*r\rtrurvrrrfetch_lfw_pairss0Q  r)NTT)NFNr)NFN))__doc__osrrros.pathrrrrZloggingZdistutils.versionr Znumpyr?baser r r ZutilsrrZ utils._joblibrZ externals.sixrrZ getLogger__name__r r$r#rrr-rNr^r6rwrrrrrrs\         += (u 2