B q\@sdZddlmZddlZddlmZmZddlmZddlZ ddl m Z ddl m Z dgZGd ddeZdd d Zd dZddZddZddZddZdS)z High-level operations for numpy structured arrays. Some code and inspiration taken from numpy.lib.recfunctions.join_by(). Redistribution license restrictions apply. )chainN) OrderedDictCounter)Sequence) _np_utilsTableMergeErrorc@s eZdZdS)rN)__name__ __module__ __qualname__r r 5lib/python3.7/site-packages/astropy/table/np_utils.pyrs{col_name}_{table_name}c stfddg}|dkr4ddttD}xtD]\}}||}x||jjD]p}|kr||kr|n@t} | |t fdd| Dr|j |d}||||<qXWq>Wt |} d d| D} | rtd | tfd d|DS) a Find the column names mapping when merging the list of structured ndarrays ``arrays``. It is assumed that col names in ``common_names`` are to be merged into a single column while the rest will be uniquely represented in the output. The args ``uniq_col_name`` and ``table_names`` specify how to rename columns in case of conflicts. Returns a dict mapping each output column name to the input(s). This takes the form {outname : (col_name_0, col_name_1, ...), ... }. For key columns all of input names will be present, while for the other non-key columns the value will be (col_name_0, None, ..) or (None, col_name_1, ..) etc. csdgtS)N)lenr )arraysr r (sz"get_col_name_map..NcSsg|]}t|dqS)r)str).0Ziir r r ,sz$get_col_name_map..c3s|]}|jjkVqdS)N)dtypenames)rother)namer r =sz#get_col_name_map..) table_nameZcol_namecSsg|]\}}|dkr|qS)rr )rrcountr r r rEszhMerging column names resulted in duplicates: {0}. Change uniq_col_name or table_names args to fix this.c3s|]}||fVqdS)Nr )rr) col_name_mapr r rLs) collections defaultdictranger enumeraterrappendlistpopanyformatritemsrr) rZ common_namesZ uniq_col_nameZ table_namesZ col_name_listidxarrayrout_nameZothersZcol_name_countZrepeated_namesr )rrrr get_col_name_maps0   r*c Csg}x|D]\}}ddt||D}dd|D}y t|}Wn8tk r|}ztd|d|jWdd}~XYnXtdd|D} t| d krtd t| } | t ||| fqW|S) z Find the dtypes descrs resulting from merging the list of arrays' dtypes, using the column name mapping ``col_name_map``. Return a list of descrs for the output. cSs g|]\}}|dk r||qS)Nr )rarrrr r r r]szget_descrs..cSsg|]}|dk r|qS)Nr )rrr r r r`sz.The '{0}' columns have incompatible types: {1}rNcss|]}|jddVqdS)rN)shape)rcolr r r rlszget_descrs..rz&Key columns {0!r} have different shape) r&zip common_dtyperr%_incompat_typessetrrr#r!fix_column_name) rrZ out_descrsr)Zin_namesZin_colsrrtmeZ uniq_shapesr,r r r get_descrsQs " r4cstjtjtjtjtjftfdd|D}t|dkr`dd|D}td |}||_ |dd|D}x&|D]}|j j dkrtd |j |d <qtWtd d|D}|j jS) z Use numpy to find the common dtype for a list of structured ndarray columns. Only allow columns within the following fundamental numpy data types: np.bool_, np.object_, np.number, np.character, np.void c3s$|]tfddDVqdS)c3s|]}tjj|VqdS)N) issubclassrtype)rZnp_type)r-r r r~sz)common_dtype...N)tuple)r)np_types)r-r r~szcommon_dtype..rcSsg|] }|jjqSr )rr)rr-r r r rsz common_dtype..z#Columns have incompatible types {0}cSsg|]}tjd|jdqS)r)r)npemptyr)rr-r r r rs)SU0rcSsg|] }|dqS)rr )rr+r r r rs)r9Zbool_Zobject_Znumber characterZvoidr1rrr%r0rZkinditemsizer(r)ZcolsZ uniq_typesZincompat_typesr3Zarrsr+Z arr_commonr )r8r r/vs     r/cCs\d}t|tst|x,|D]$}t|tjr8|jjdkrt|qWt|dkrXtddS)Nz@`arrays` arg must be a sequence (e.g. list) of structured arraysrz,`arrays` arg must include at least one array) isinstancer TypeErrorr9Zndarrayrrr ValueError)rerrr(r r r (_check_for_sequence_of_structured_arrayss    rDcCs0|dk r,y t|}Wntk r*YnX|S)z Fixes column names so that they are compatible with Numpy on Python 2. Raises a ValueError exception if the column name contains Unicode characters, which can not reasonably be used as a column name. N)rUnicodeEncodeError)valr r r r2s  r2csjt|d}tj|tdfddt|D}g}x|D]|jjq:Wd|}tj j ||dS)aT Partial replacement for `~numpy.core.records.fromrecords` which includes a workaround for the bug with unicode arrays described at: https://github.com/astropy/astropy/issues/3052 This should not serve as a full replacement for the original function; this only does enough to fulfill the needs of the table module. r)rcs"g|]}td|fqS).)r9r(tolist)ri)objr r rsz(recarray_fromrecords..,)formats) rr9r(objectrr!rrjoinZrecZ fromarrays)Zrec_listZnfieldsZ array_listrKr )rIr recarray_fromrecordss   rN)rN)__doc__ itertoolsrrrrZcollections.abcrZnumpyr9Znumpy.maZmar__all__rBrr*r4r/rDr2rNr r r r s      7%