ó ‡ˆ\c@smdZddlZddlZddlZddlZddlmZddlmZmZddl m Z ddl m Z erej Z ejZeZnej Z ejZyddlZWnek rØdZnXyddlZWnek rdZnXdZd „Zd „Zd „Zd „Zd„Zejdd„ƒZdd„ZdZ dd„Z!dS(sFUtilities for fast persistence of big data, with optional compression.iÿÿÿÿN(tclosingi(t PY3_OR_LATERtPY27(t _ZFILE_PREFIX(t _COMPRESSORSiicCs9tr(t|d|ƒ}t|tjƒSt|tƒSdS(s=Check if fileobj is a raw file object, e.g created with open.trawN(Rtgetattrt isinstancetiotFileIOtfile(tfileobj((sJlib/python2.7/site-packages/sklearn/externals/joblib/numpy_pickle_utils.pyt _is_raw_file+scCsEgtjƒD]}t|jƒ^q }|ttƒg7}t|ƒS(N(RtvaluestlentprefixRtmax(t compressortprefixes((sJlib/python2.7/site-packages/sklearn/externals/joblib/numpy_pickle_utils.pyt_get_prefixes_max_len4s(cCstƒ}t|dƒr*|j|ƒ}n|j|ƒ}|jdƒ|jtƒrYdSx0tjƒD]"\}}|j|j ƒrf|SqfWdS(sÊReturn the compressor matching fileobj. Parameters ---------- fileobj: file object Returns ------- str in {'zlib', 'gzip', 'bz2', 'lzma', 'xz', 'compat', 'not-compressed'} tpeekitcompatsnot-compressed( RthasattrRtreadtseekt startswithRRtitemsR(R tmax_prefix_lent first_bytestnameR((sJlib/python2.7/site-packages/sklearn/externals/joblib/numpy_pickle_utils.pyt_detect_compressor=s  cCs?tr(tdk r(t|tjƒr(|Stj|dtƒSdS(s0Return a buffered version of a read file object.t buffer_sizeN(Rtbz2tNoneRtBZ2FileRtBufferedReadert_IO_BUFFER_SIZE(tfobj((sJlib/python2.7/site-packages/sklearn/externals/joblib/numpy_pickle_utils.pyt_buffered_read_file]s$cCsEtr.tdk r.t|tjƒr.t|ƒStj|dtƒSdS(s1Return a buffered version of a write file object.RN( RR R!RR"RRtBufferedWriterR$(R%((sJlib/python2.7/site-packages/sklearn/externals/joblib/numpy_pickle_utils.pyt_buffered_write_filegs$ ccst|ƒ}|dkr:tjd|tddƒ|VnÉ|tkrnt|}|j|ƒ}t|ƒ}n|d k rþt|t j ƒr©tjdt ƒddƒqþ|dkrÒtjdt ƒddƒqþt |ƒsþtjdt ƒddƒqþn|Vd S( s-Utility function opening the right fileobject from a filename. The magic number is used to choose between the type of file object to open: * regular file object (default) * zlib file object * gzip file object * bz2 file object * lzma file object (for xz and lzma compressor) Parameters ---------- fileobj: file object compressor: str in {'zlib', 'gzip', 'bz2', 'lzma', 'xz', 'compat', 'not-compressed'} filename: str filename path corresponding to the fileobj parameter. mmap_mode: str memory map mode that should be used to open the pickle file. This parameter is useful to verify that the user is not trying to one with compression. Default: None. Returns ------- a file like object RsjThe file '%s' has been generated with a joblib version less than 0.10. Please regenerate this pickle file.t stacklevelisuIn memory persistence is not compatible with mmap_mode "%(mmap_mode)s" flag passed. mmap_mode option will be ignored.snot-compressedstmmap_mode "%(mmap_mode)s" is not compatible with compressed file %(filename)s. "%(mmap_mode)s" flag will be ignored.sP"%(fileobj)r" is not a raw file, mmap_mode "%(mmap_mode)s" flag will be ignored.N( RtwarningstwarntDeprecationWarningRtdecompressor_fileR&R!RRtBytesIOtlocalsR (R tfilenamet mmap_modeRtcompressor_wrappertinst((sJlib/python2.7/site-packages/sklearn/externals/joblib/numpy_pickle_utils.pyt_read_fileobjectss*            tzlibicCsp|d}|d}|tjƒkrIt|j|d|ƒ}t|ƒStdj|d|ƒ}t|ƒSdS(s6Return the right compressor file object in write mode.iit compresslevelR5N(Rtkeystcompressor_fileR((R0tcompresstcompressmethodR6t file_instance((sJlib/python2.7/site-packages/sklearn/externals/joblib/numpy_pickle_utils.pyt_write_fileobject¹s     isran out of datacCs¾tƒ}xptr{yO|j|t|ƒƒ}||7}t|ƒdks\t|ƒ|kr`PnWq tjk rwq Xq Wt|ƒ|kr¶d}t|||t|ƒfƒ‚n|SdS(sRead from file-like object until size bytes are read. Raises ValueError if not EOF is encountered before size bytes are read. Non-blocking objects only supported if they derive from io objects. Required as e.g. ZipExtFile in python 2.6 can return less data than requested. This function was taken from numpy/lib/format.py in version 1.10.2. Parameters ---------- fp: file-like object size: int error_template: str Returns ------- a bytes object The data read in bytes. is)EOF: reading %s, expected %d bytes got %dN(tbytestTrueRRRtBlockingIOErrort ValueError(tfptsizeterror_templatetdatatrtmsg((sJlib/python2.7/site-packages/sklearn/externals/joblib/numpy_pickle_utils.pyt _read_bytesÎs   $"i(R5ii("t__doc__tpickleRR*t contextlibRt_compatRRRRRt _Unpicklert Unpicklert_PicklertPicklertrangetxrangetnumpytnpt ImportErrorR!R R$R RRR&R(tcontextmanagerR4R<t BUFFER_SIZERG(((sJlib/python2.7/site-packages/sklearn/externals/joblib/numpy_pickle_utils.pytsB              E