o QfX#@sdZddlZddlZddlZddlZddlZddlZddlmZddl Z ej ddkZ e r1ej Z nej Z GdddeZGdd d eZGd d d e ZGd d d eZdddZdS)zb Fast cryptographic hash of Python objects, with a special case for fast hashing of numpy arrays. N)_bytes_or_unicode3c@eZdZdZddZdS)_ConsistentSetza Class used to ensure the hash of Sets is preserved whatever the order of its items. cCst||_dSN)sortedZ _sequence)selfZ set_sequencer c/oak/stanford/groups/akundaje/marinovg/programs/SPAdes-3.15.4-Linux/share/spades/joblib3/hashing.py__init__ sz_ConsistentSet.__init__N__name__ __module__ __qualname____doc__r r r r r rs rc@r)_MyHashz7 Class used to hash objects that won't normally pickle cGs ||_dSr)args)r rr r r r 's z_MyHash.__init__Nr r r r r r$s rc@seZdZdZdddZdddZdd Zd d Zd ej fd dZ e j Z e e ee<e e ee<e e ee <e e eej<ddZddZee ee<d S)HasherzW A subclass of pickler, to do cryptographic hashing, rather than pickling. md5cCs<t|_tr tjntj}tj||j|dt ||_ dS)N)protocol) ioBytesIOstreamPY3pickleDEFAULT_PROTOCOLHIGHEST_PROTOCOLPicklerr hashlibnew_hash)r hash_namerr r r r 0s  zHasher.__init__Tc Cspz||Wntjy#}ztd||fWYd}~nd}~ww|j}|j||r6|j SdS)Nz"PicklingError while hashing %r: %r) dumpr PicklingErrorwarningswarnrgetvaluer!update hexdigest)r objZ return_digestedumpsr r r hash:s   z Hasher.hashcCst|tjtijfr>t|dr|jj}n|j}|j}t|tt kr*t ||j}n|dur4t ||}n |jj }t |||}t ||dS)N__func__) isinstancetypes MethodTypetypepophasattrr.r__self__rr __class__rsave)r r* func_nameinstclsr r r r7Ds    z Hasher.savecCst|trdSt||dSr)r/rrmemoize)r r*r r r r;Ws zHasher.memoizeNcCst||d}tjdkr|d=z tj||fi|WdStjyYtj||fi|t|dd}|dkrS|}|dur@|j}tj |}t ||sVt |||YdSYdSYdSw)N)namepack)r=r__main__) dictsys version_infor save_globalrr$getattrrmodulesr4setattr)r r*r<r=kwargsmoduleZmy_namemodr r r rDbs$     zHasher.save_globalcCst|tt|dSr)r_batch_setitemsiterr)r itemsr r r rKszHasher._batch_setitemscCst|t|dSr)rr7r)r Z set_itemsr r r save_setszHasher.save_set)r)T)rrrrr r-r7r;structr=rDrdispatchcopyr2lenobjectrr#rKrNsetr r r r r+s       rc@s"eZdZdZd ddZddZdS) NumpyHasherz7 Special case the hasher for when numpy is loaded. rFcCsB||_tj||dddl}||_t|dr|j|_dSt|_dS)a Parameters ---------- hash_name: string The hash algorithm to be used coerce_mmap: boolean Make no difference between np.memmap and np.ndarray objects. r"rN getbuffer) coerce_mmaprr numpynpr4rW _getbuffer memoryview)r r"rXrZr r r r s    zNumpyHasher.__init__c Cst||jjrY|jjsYz||jj}|j| |Wnt t fy:| |jj}|j| |Ynw|j rJt||jjrJ|jj}n|j}|d|j|j|jff}nt||jjrj|j}|d|jff}t||dS)z Subclass the save method, to hash ndarray subclass, rather than pickling them. Off course, this is a total abuse of the Pickler class. ZHASHEDN)r/rZZndarrayZdtypeZ hasobjectviewZuint8r!r(r[ ValueError BufferErrorflattenrXZmemmapr6shapestridesdescrrr7)r r*Zobj_bytes_viewklassr r r r7s   zNumpyHasher.saveNrF)rrrrr r7r r r r rUs  rUrFcCs,dtjvr t||d}nt|d}||S)al Quick calculation of a hash to identify uniquely Python objects containing numpy arrays. Parameters ----------- hash_name: 'md5' or 'sha1' Hashing algorithm used. sha1 is supposedly safer, but md5 is faster. coerce_mmap: boolean Make no difference between np.memmap and np.ndarray rY)r"rXrV)rBrFrUrr-)r*r"rXZhasherr r r r-s   r-re)rr%rrrBr0rO_compatrrversionr_PicklerrrSrrrrUr-r r r r s$  bQ