B q\Q[@sdZddlZddlZddlZddlZddlZddlmZddlm Z ddl m Z ddl m Z ddlmZddlZdd lmZd d d d ddgZeeddfZdddddZeddZdd Zdd ZddZd"ddZGdd d ZGdd d eZGd ddeZ Gd!dde Z!dS)#aThis module contains functions and methods that relate to the DataInfo class which provides a container for informational attributes as well as summary info methods. A DataInfo object is attached to the Quantity, SkyCoord, and Time classes in astropy. Here it allows those classes to be used in Tables and uniformly carry table column attributes such as name, format, dtype, meta, and description. N)StringIO)deepcopy)partial) OrderedDict)contextmanager)metadatadata_info_factorydtype_info_nameBaseColumnInfoDataInfo MixinInfoParentDtypeInfozUAll-NaN|Mean of empty slice|Degrees of freedom <= 0|invalid value encountered in sqrt)categorymessagestrZunicodebytes))FS)FU)Tr)Trccstj}|t_dV|t_dS)akSet context for serialization. This will allow downstream code to understand the context in which a column is being serialized. Objects like Time or SkyCoord will have different default serialization representations depending on context. Parameters ---------- context : str Context name, e.g. 'fits', 'hdf5', 'ecsv', 'yaml' N)r _serialize_context)contextZ old_contextr6lib/python3.7/site-packages/astropy/utils/data_info.pyserialize_context_as0s rcCsJt|}|jdkr@td|jd}td|jf}||}n|j}|S)abReturn a human-oriented string name of the ``dtype`` arg. This can be use by astropy methods that present type information about a data object. The output is mostly equivalent to ``dtype.name`` which takes the form [B] where is like ``int`` or ``bool`` and [B] is an optional number of bits which gets included only for numeric types. For bytes, string and unicode types, the output is shown below, where is the number of characters. This representation corresponds to the Python type that matches the dtype:: Numpy S U Python bytes str Parameters ---------- dtype : str, np.dtype, type Input dtype as an object that can be converted via np.dtype() Returns ------- dtype_info_name : str String name of ``dtype`` )rrz(\d+)rT) npdtypekindresearchrgroupSTRING_TYPE_NAMESname)rlengthZ type_nameoutrrrr Cs   csfdd}|S)a8 Factory to create a function that can be used as an ``option`` for outputting data object summary information. Examples -------- >>> from astropy.utils.data_info import data_info_factory >>> from astropy.table import Column >>> c = Column([4., 3., 2., 1.]) >>> mystats = data_info_factory(names=['min', 'median', 'max'], ... funcs=[np.min, np.median, np.max]) >>> c.info(option=mystats) min = 1.0 median = 2.5 max = 4.0 n_bad = 0 length = 4 Parameters ---------- names : list List of information attribute names funcs : list List of functions that compute the corresponding information attribute Returns ------- func : function Function that can be used as a data info option c s|g}xhtD]Z\}}y$t|tr2t||}n||}Wntk rZ|dYqX|t|qWtt|S)Nz--)zip isinstancergetattr Exceptionappendr)datZoutsr!funcr#)funcsnamesrrr*s  zdata_info_factory..funcr)r,r+r*r)r+r,rr hscCs2i}x(|D] }t||d}|dk r |||<q W|S)a[ Get the values for object ``attrs`` and return as a dict. This ignores any attributes that are None and in Py2 converts any unicode attribute names or values to str. In the context of serializing the supported core astropy classes this conversion will succeed and results in more succinct and less python-specific YAML. N)r&)objattrsr#attrvalrrr_get_obj_attrs_maps    r1cCsn|dkrt|j}nF|dkr*t|jj}n0|dkrN|jdd}|rH|nd}n t|j|}|dkrfd}t|S)zP Get a data object attribute for the ``attributes`` info summary method classrshaperN)type__name__r inforr3r&r)r)r/r0Zdatshaperrr_get_data_attributes  r8cs eZdZdZddddgZeZedddd d d gZeZd Z d Z dZ dZ d Z d/ddZeddZejddZddZddZddZddZfddZfdd Zd!d"Zd#d$Zeee d%d&e Dd'Zeeed(d&eDd'Zd0d+d,Zfd-d.ZZ S)1r a Descriptor that data classes use to add an ``info`` attribute for storing data attributes in a uniform and portable way. Note that it *must* be called ``info`` so that the DataInfo() object can be stored in the ``instance`` using the ``info`` key. Because owner_cls.x is a descriptor, Python doesn't use __dict__['x'] normally, and the descriptor can safely store stuff there. Thanks to http://nbviewer.ipython.org/urls/gist.github.com/ChrisBeaumont/5758381/raw/descriptor_writeup.ipynb for this trick that works for non-hashable classes. Parameters ---------- bound : bool If True this is a descriptor attribute in a class definition, else it is a DataInfo() object that is bound to a data object instance. Default is False. ZmeanZstdminmaxr!unitrformat descriptionmeta)rr3r;r<r=r2NrFcCs|rtdd|jD|_dS)Ncss|]}|dfVqdS)Nr).0r/rrr sz$DataInfo.__init__..)dict attr_names_attrs)selfboundrrr__init__szDataInfo.__init__cCs.|jdkrdS|}|dk r"|StddS)Naxfailed access "info" attribute on a temporary object. It looks like you have done something like ``col[3:5].info``, i.e. you accessed ``info`` from a temporary slice object ``col[3:5]`` that only exists momentarily. This has failed because the reference to that temporary object is now lost. Instead force a permanent reference with ``c = col[3:5]`` followed by ``c.info``.) _parent_refAttributeError)rDparentrrr_parents zDataInfo._parentcCs |dkrd|_n t||_dS)N)rGweakrefref)rDvaluerrrrJscCsH|dkr|}||_n0|jd}|dkr>|jdd}|jd<||_|S)Nr7T)rE) _parent_cls__dict__get __class__rJ)rDinstanceZ owner_clsr7rrr__get__s zDataInfo.__get__cCsn|dkrtdt|trb|jdd}|jd<x8|j|j|jD]}tt |||j |<qDWnt ddS)Nzcannot set unbound descriptorT)rEr7z)info must be set with a DataInfo instance) ValueErrorr%r rQrOrBattrs_from_parent_attrs_no_copyrr&rC TypeError)rDrRrMr7r/rrr__set__s zDataInfo.__set__cCs|jS)N)rC)rDrrr __getstate__!szDataInfo.__getstate__cCs ||_dS)N)rC)rDstaterrr __setstate__$szDataInfo.__setstate__cs|drt|S||jkr,t|j|Sy|j|}Wn tk rZt|YnX|dkrrt|rr|}|dkr|dkrt d}|S)N_ parent_tablerO) startswithsuper__getattribute__rUr&rJrCKeyErrorcallablerr)rDr/rM)rQrr __getattr__'s     zDataInfo.__getattr__cst|j|d}||jkr4t|ts4t|j||dSt|tr`|jdkrPtd|||dS| dr|t ||dS||j krtd |j |dkr|dkrdnt|}||j|<dS)Nzcan't set attributer\zattribute must be one of {0}r])r&rQrUr%propertysetattrrJfsetrHr_r` __setattr__rBr<rKrLrC)rDr/rMZpropobj)rQrrrh=s"     zDataInfo.__setattr__cCst|j|jS)z=Get the values for the parent ``attrs`` and return as a dict.)r1rJ_represent_as_dict_attrs)rDrrr_represent_as_dict\szDataInfo._represent_as_dictcs fdd|jD}|j|S)Ncsg|]}|qSr)pop)r?r/)maprr asz1DataInfo._construct_from_dict..)_construct_from_dict_argsrN)rDrlargsr)rlr_construct_from_dict`szDataInfo._construct_from_dictcCsg|]}tt|dqS))r/)rr8)r?r/rrrrmfszDataInfo.)r,r+cCsg|]}ttd|qS)nan)r&r)r?statrrrrmls attributesr4c Cs|dkrtj}|j}t}|jj}|dk r2||d<t|ttfrD|n|g}x~|D]v}t|t rt |d|r|t |d|}nt d |t.xtD]}tjd |qW|||WdQRXqPWt |drt|j}n:ytt|t|B}Wntk rd}YnX||d <yt||d <Wntk rJYnX|dkrZ|Sx8|D],\} } | dkrd|d | | tjqdWdS) a Write summary information about data object to the ``out`` filehandle. By default this prints to standard output via sys.stdout. The ``option`` argument specifies what type of information to include. This can be a string, a function, or a list of strings or functions. Built-in options are: - ``attributes``: data object attributes like ``dtype`` and ``format`` - ``stats``: basic statistics: min, mean, and max If a function is specified then that function will be called with the data object as its single argument. The function must return an OrderedDict containing the information attributes. If a list is provided then the information attributes will be appended for each of the options, in order. Examples -------- >>> from astropy.table import Column >>> c = Column([1, 2], unit='m', dtype='int32') >>> c.info() dtype = int32 unit = m class = Column n_bad = 0 length = 2 >>> c.info(['attributes', 'stats']) dtype = int32 unit = m class = Column mean = 1.5 std = 0.5 min = 1 max = 2 n_bad = 0 length = 2 Parameters ---------- option : str, function, list of (str or function) Info option, defaults to 'attributes'. out : file-like object, None Output destination, defaults to sys.stdout. If None then the OrderedDict with information attributes is returned Returns ------- info : OrderedDict if out==None else None r4Nr!Z info_summary_z-option={0} is not an allowed information typeignoremaskrn_badr"z {0} = {1})rt)sysstdoutrJrr7r!r%listtuplerhasattrr&rTr<warningscatch_warningsIGNORE_WARNINGSfilterwarningsupdaterZ count_nonzeroruZisinfZisnanr'lenrWitemswriteoslinesep) rDZoptionr#r)r7r!ZoptionsZ ignore_kwargsrvkeyr0rrr__call__osD6        zDataInfo.__call__cs.|jdkrtSt}|j|d|S)N)r#)rJr`__repr__rrgetvalue)rDr#)rQrrrs    zDataInfo.__repr__)F)rsr4)!r6 __module__ __qualname____doc__Z_statssetrUrBrVZ_info_summary_attrsrGrirnZ_represent_as_dict_primary_datarFrerJsetterrSrXrYr[rdrhrjrp staticmethodr Zinfo_summary_attributesZinfo_summary_statsrr __classcell__rr)rQrr s>         bcsdeZdZdZejddgZedgZdZ dfdd Z dd Z d d Z d d Z eddZZS)r ag Base info class for anything that can be a column in an astropy Table. There are at least two classes that inherit from this: ColumnInfo: for native astropy Column / MaskedColumn objects MixinInfo: for mixin column objects Note that this class is defined here so that mixins can use it without importing the table package. r]indicesNFcstj|d|ri|_dS)N)rE)r`rFZ _format_funcs)rDrE)rQrrrFszBaseColumnInfo.__init__ccsP|j}|jdkrddlm}n|jj}|j}x||dddiD] }|Vq>WdS)zG This is a mixin-safe version of Column.iter_str_vals. Nr) FORMATTERF)rJr]Zastropy.table.columnr formatter_pformat_col_iter)rDcolrrZstr_valrrr iter_str_valss zBaseColumnInfo.iter_str_valsc Cs|js dSt|tr,||}tt|}n.t|tjrT|jjdkrTt |d}n|g}t |}|j dkr~t|t |}x6t ||D](\}}x|jD]}|||j|qWqWdS)a Adjust info indices after column modification. Parameters ---------- index : slice, int, list, or ndarray Element(s) of column to modify. This parameter can be a single row number, a list of row numbers, an ndarray of row numbers, a boolean ndarray (a mask), or a column slice. value : int, list, or ndarray New value(s) to insert col_len : int Length of the column Nbrr)rr%sliceryrangerndarrayrrwhereZ atleast_1dsizerr$replacer!) rDindexrMcol_lentkeysrr0Z col_indexrrradjust_indicess     zBaseColumnInfo.adjust_indicesc sddlm}t|dds$g|j_|SttrHfdd|jD|j_n|jrttjrtj j dkrtt dd}t d|k}g|j_xJ|jD]@}|st||r| |}nt|}||jj|qW|S) aX Given a sliced object, modify its indices to correctly represent the slice. Parameters ---------- col_slice : Column or mixin Sliced object item : slice, list, or ndarray Slice used to create col_slice col_len : int Length of original object r) SortedArrayZ _copy_indicesTcsg|] }|qSrr)r?x)itemrrrm@sz0BaseColumnInfo.slice_indices..rg333333?)Zastropy.table.sorted_arrayrr&r7rr%rrrrrrrZ get_slicerZ replace_rowsr() rDZ col_slicerrrZ thresholdZsmallrZ new_indexr)rr slice_indices,s&     zBaseColumnInfo.slice_indicesc sddlm}fdd}fdd}||d}x*|ddD]}tj|||||d }q>Wt||d <td d |D} t| dkr|d | |d<|S)a Utility method to merge and validate the attributes ``attrs`` for the input table columns ``cols``. Note that ``dtype`` and ``shape`` attributes are handled specially. These should not be passed in ``attrs`` but will always be in the returned dict of merged attributes. Parameters ---------- cols : list List of input Table column objects metadata_conflicts : str ('warn'|'error'|'silent') How to handle metadata conflicts name : str Output column name attrs : list List of attribute names to be merged Returns ------- attrs : dict of merged attributes r)TableMergeErrorcsd||||}|S)Nz_In merged column '{}' the '{}' attribute does not match ({} != {}). Using {} for merged output)r<)rleftrightr#)r!rr warn_str_funcrsz;BaseColumnInfo.merge_cols_attributes..warn_str_funccsfddDS)Ncs,i|]$}tj|ddk rtj||qS)N)r&r7)r?r/)rrr yszJBaseColumnInfo.merge_cols_attributes..getattrs..r)r)r.)rrgetattrsxsz6BaseColumnInfo.merge_cols_attributes..getattrsrN)metadata_conflictsrrcss|]}|jddVqdS)rN)r3)r?rrrrr@sz7BaseColumnInfo.merge_cols_attributes..zcolumns have different shapesr3)astropy.table.np_utilsrrmergeZ common_dtyperrrk) Zcolsrr!r.rrrr#rZ uniq_shapesr)r.r!rmerge_cols_attributesVs       z$BaseColumnInfo.merge_cols_attributes)F)r6rrrr rBunionrrVrrFrrrrrrrr)rQrr s  &*cseZdZfddZZS)r csJ|dkr8|jdk r8ddlm}||}|jj|j|t||dS)Nr!r)fix_column_name)r]rrcolumnsZ_rename_columnr!r`rh)rDr/rMrnew_name)rQrrrhs  zMixinInfo.__setattr__)r6rrrhrrr)rQrr sc@seZdZdZedgZdS)rz&Mixin that gets info.dtype from parentrN)r6rrrrrUrrrrrs)N)"rrrrwrKr|iorcopyr functoolsr collectionsr contextlibrZnumpyrr4r__all__rARuntimeWarningr~r rr r r1r8r r r rrrrr s>        %0 !4