B Z\ˀ@sdZddlmZmZmZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddl Z ddlmZeeZdZdddeDZe jdd kZereZeZeZeZeZn&e e!Ze"Ze!Zee#fZee j$fZe j%Z&e j'Z(d d Z)d d Z*ddZ+ddZ,Gddde-Z.Gddde-Z/Gddde/Z0Gddde-Z1d)ddZ2d*ddZ3d+ddZ4d,d d!Z5e4Z6e2Z7Gd"d#d#e-Z8Gd$d%d%e8Z9Gd&d'd'e8Z:e9e:gZ;ed(krddl0srcCs"|dkrtd|Stdd|SdS)zE Encode an unsigned integer into a variable sized blob of bytes. z Use of str.isidentifier() for Legacy Python, but slower. z^\w+$z^[0-9]N) isinstance string_typesrematchUNICODE)sr r r _isidentifierms r(c@sdeZdZdZdddZddd Zd d Zd d ZddZddZ ddZ ddZ ddZ ddZ dS)BsdfSerializera{ Instances of this class represent a BSDF encoder/decoder. It acts as a placeholder for a set of extensions and encoding/decoding options. Use this to predefine extensions and options for high performance encoding/decoding. For general use, see the functions `save()`, `encode()`, `load()`, and `decode()`. This implementation of BSDF supports streaming lists (keep adding to a list after writing the main file), lazy loading of blobs, and in-place editing of blobs (for streams opened with a+). Options for encoding: * compression (int or str): ``0`` or "no" for no compression (default), ``1`` or "zlib" for Zlib compression (same as zip files and PNG), and ``2`` or "bz2" for Bz2 compression (more compact but slower writing). Note that some BSDF implementations (e.g. JavaScript) may not support compression. * use_checksum (bool): whether to include a checksum with binary blobs. * float64 (bool): Whether to write floats as 64 bit (default) or 32 bit. Options for decoding: * load_streaming (bool): if True, and the final object in the structure was a stream, will make it available as a stream in the decoded object. * lazy_blob (bool): if True, bytes are represented as Blob objects that can be used to lazily access the data, and also overwrite the data if the file is open in a+ mode. NcKs@i|_i|_|dkrt}x|D]}||qW|jf|dS)N) _extensions_extensions_by_clsstandard_extensions add_extension_parse_options)self extensionsoptions extensionr r r __init__s zBsdfSerializer.__init__rFTcCsht|tr&dddd}|||}|dkr6td||_t||_t||_t||_ t||_ dS)Nrrr)Znozlibbz2)rrrz3Compression must be 0, 1, 2, "no", "zlib", or "bz2") r"r#getlower TypeError _compressionbool _use_checksum_float64_load_streaming _lazy_blob)r/ compression use_checksumZfloat64Zload_streamingZ lazy_blobmr r r r.s     zBsdfSerializer._parse_optionscCst|trt|tstd|}|j}t|ts:tdt|dksRt|dkrZtd||j krrt d||j }|sg}nt|t tfr|}n|g}x|D]}t|tstdqWx|D]}||jf|j|<qW||j |<|S)z Add an extension to this serializer instance, which must be a subclass of Extension. Can be used as a decorator. z*add_extension() expects a Extension class.zExtension name must be str.rrz|d}||d|t t |||nt|t t fr||d|t t |x|D]}| |||dqpWn2t|tr,||d|t t |xp|D]d\}}tr|stnt|st|d} |t t | || | |||dqWnt|trh||d|t||j|jd} | |nXt|tr||d|||n.t|tr|jdkrtdn2t|tr||d|tdddntdt |dkrtd|||||j |j n|dk r0td||j!"|j#d} | dkrx6|j$D]$\} } | %||rV| | jf} PqVWd} | dk r| \}}| ||||||nd}t||j#j&dS) z Main encoder function. NvTyFniihhiztddd}|d}n|dkrVtddd}|dkr|dk}tddd}jrt|r|nd}|jj nX|rfddt |D}n8g}yx|  qWWntk rYnXn6|dkrzsz*BsdfSerializer._decode..csg|]}qSr )rq)r r )rr/r r rsrZr[zParse error %rz'BSDF warning: no extension found for %r)rr7EOFErrorrdecoder=rnrprbrqrangerorcrgr>ri get_bytes RuntimeErrorr*r6rGrH)r/rcharrzrrrtZn_sclosedr Zn_namerErxr2r )rr/r rqKs                zBsdfSerializer._decodecCst}||||S)z) Save the given object to bytes. )rsavegetvalue)r/obrr r r rs zBsdfSerializer.encodecCsx|d|tdtd|tdtdg}||||dt|dkrt|d}|j|krttddS)z: Write the given object to the given file object. sBSDFztd|j|j|dS)zB Seek to the given position (relative to the blob start). Nz>Cannot seek in a blob that is not created by the BSDF decoder.rzSeek beyond blob boundaries.)rrrrrr)r/pr r r rs  z Blob.seekcCs"|jdkrtd|j|jS)zM Get the current file pointer position (relative to the blob start). Nz>Cannot tell in a blob that is not created by the BSDF decoder.)rrrr)r/r r r rs z Blob.tellcCsR|jdkrtd|jr td|jt||jkr@tdd|_|j|S)z" Write bytes to the blob. Nz?Cannot write in a blob that is not created by the BSDF decoder.z,Cannot arbitrarily write in compressed blob.zWrite beyond blob boundaries.T) rrr?rrrrrr^)r/r r r r r^s z Blob.writecCsH|jdkrtd|jr td|j||jkrCannot read in a blob that is not created by the BSDF decoder.z+Cannot arbitrarily read in compressed blob.zRead beyond blob boundaries.)rrr?rrrr)r/rr r r rs z Blob.readcCs|jdk r|j}n.|j}|d|j|j}|j||jdkrP|}n:|jdkrft|}n$|jdkr|t |}nt d|j|S)z0 Get the contents of the blob as bytes. NrrrzInvalid compression %i) rrrrrrr?r4Z decompressr5r)r/rr rtr r r rs         zBlob.get_bytescCsZ|jrV|jrV|d|j|j}|j|j|jdd|jt | dS)z[ Reset the blob's checksum if present. Call this after modifying the data. rrrN) r@rrrrrrrr^rrr)r/rr r r update_checksums   zBlob.update_checksumN)rrF)rsrrrr3rrjrrrr^rrrr r r r ri]s (   ricKst|f|}||S)zs Save (BSDF-encode) the given object to bytes. See `BSDFSerializer` for details on extensions and options. )r)r)rr0r1r'r r r rs rc KsFt|f|}t|tr6t|d}|||SQRXn |||SdS)z Save (BSDF-encode) the given object to the given filename or file object. See` BSDFSerializer` for details on extensions and options. wbN)r)r"r#openr)rrr0r1r'fpr r r r's    rcKst|f|}||S)zq Load a (BSDF-encoded) structure from bytes. See `BSDFSerializer` for details on extensions and options. )r)r)r r0r1r'r r r r3s rc KsXt|f|}t|trJ|dr,tj|}t|d }||SQRXn ||SdS)z Load a (BSDF-encoded) structure from the given filename or file object. See `BSDFSerializer` for details on extensions and options. )z~/z~\rbN) r)r"r# startswithospath expanduserrr)rr0r1r'rr r r r;s     rc@s8eZdZdZdZdZddZddZdd Zd d Z d S) rDaE Base class to implement BSDF extensions for special data types. Extension classes are provided to the BSDF serializer, which instantiates the class. That way, the extension can be somewhat dynamic: e.g. the NDArrayExtension exposes the ndarray class only when numpy is imported. A extension instance must have two attributes. These can be attribiutes of the class, or of the instance set in ``__init__()``: * name (str): the name by which encoded values will be identified. * cls (type): the type (or list of types) to match values with. This is optional, but it makes the encoder select extensions faster. Further, it needs 3 methods: * `match(serializer, value) -> bool`: return whether the extension can convert the given value. The default is ``isinstance(value, self.cls)``. * `encode(serializer, value) -> encoded_value`: the function to encode a value to more basic data types. * `decode(serializer, encoded_value) -> value`: the function to decode an encoded value back to its intended representation. r cCsd|jtt|fS)Nz)rEhexid)r/r r r __repr__rszExtension.__repr__cCs t||jS)N)r"rI)r/r'rvr r r r%uszExtension.matchcCs tdS)N)NotImplementedError)r/r'rvr r r rxszExtension.encodecCs tdS)N)r)r/r'rvr r r r{szExtension.decodeN) rsrrrrErIrr%rrr r r r rDUsrDc@s$eZdZdZeZddZddZdS)ComplexExtensionrzcCs |j|jfS)N)realimag)r/r'rvr r r rszComplexExtension.encodecCst|d|dS)Nrr)complex)r/r'rvr r r rszComplexExtension.decodeN)rsrrrErrIrrr r r r rsrc@s0eZdZdZddZddZddZdd Zd S) NDArrayExtensionndarraycCsdtjkrddl}|j|_dS)Nnumpyr)sysmodulesrrrI)r/npr r r r3s zNDArrayExtension.__init__cCst|dot|dot|dS)Nshapedtypetobytes)r)r/r'rvr r r r%szNDArrayExtension.matchcCst|jt|j|dS)N)rrdata)rcr text_typerr)r/r'rvr r r rszNDArrayExtension.encodecCsFy ddl}Wntk r |SX|j|d|dd}|d|_|S)Nrrr)rr)r ImportErrorZ frombufferr)r/r'rvrar r r rs  zNDArrayExtension.decodeN)rsrrrEr3r%rrr r r r rs r__main__)N)N)N)N)>rZ __future__rrrr5rZloggingrr$rrtypesr4iorZ getLoggerrsrGrjoinr version_inforer rr#rarr_rBrLZ basicConfigZunicodeZ basestringZlongZ ClassTyperrZunpackrrrr!r(objectr)rkrnrirrrrloadsdumpsrDrrr,Zbsdf_climainr r r r sj      o XC    *