ó ¦–Õ\c@`s¥dZddlmZmZmZddlmZddlmZddl Z ddl m Z dd lmZmZmZdd lmZdd lmZdd lmZydd l mZWnek rÚe jZnXd„Ze jZe jZe jZe jZee jƒZee j ƒZ ee j!ƒZ!ee j"ƒZ"e j#Z#e j$Z$e j%Z%e j&Z&yddl m'Z'm(Z(Wn#ek r§e j'Z'e j(Z(nXe j)Z)e j*Z*e j+Z+ee,ƒe j-Z-WdQXe j.Z.ee,ƒe j/Z/WdQXe j0Z0ee,ƒe j1Z1WdQXe2d„Z3dd„Z5d„Z6d„Z7d„Z8d„Z9d„Z:d„Z;dd„Z<dd„Z=d„Z>d„Z?dS(s- A set of NumPy functions to apply per chunk i(tabsolute_importtdivisiontprint_function(twraps(tconcatNi(t numpy_compati(t ContainertIterabletSequence(tflatten(tignoring(tIntegral(ttake_along_axisc`s%tˆƒdd‡fd†ƒ}|S(sU A wrapper for functions that don't provide keepdims to ensure that they do. c `sLj|d|||Ž}|s"|S|}|dkrFt|jƒ}nt|tttfƒsj|g}ntƒ}xCt|jƒD]2}||kr¢|d7}qƒ|tdƒf7}qƒW||}|S(Ntaxis(N( tNonetrangetndimt isinstanceRRRttupletslice( txR tkeepdimstargstkwargstrtaxestr_slicet each_axis(t a_callable(s/lib/python2.7/site-packages/dask/array/chunk.pytkeepdims_wrapped_callables      N(RR(RR((Rs/lib/python2.7/site-packages/dask/array/chunk.pytkeepdims_wrappers (t nancumprodt nancumsumc`säx0t|jƒD]}|ˆkrdˆ|>> x = np.array([1, 2, 3, 4, 5, 6]) >>> coarsen(np.sum, x, {0: 2}) array([ 3, 7, 11]) >>> coarsen(np.max, x, {0: 3}) array([3, 6]) Provide dictionary of scale per dimension >>> x = np.arange(24).reshape((4, 6)) >>> x array([[ 0, 1, 2, 3, 4, 5], [ 6, 7, 8, 9, 10, 11], [12, 13, 14, 15, 16, 17], [18, 19, 20, 21, 22, 23]]) >>> coarsen(np.min, x, {0: 2, 1: 3}) array([[ 0, 3], [12, 15]]) You must avoid excess elements explicitly >>> x = np.array([1, 2, 3, 4, 5, 6, 7, 8]) >>> coarsen(np.min, x, {0: 3}, trim_excess=True) array([1, 4]) ic3`sJ|]@\}}|ˆ|r5td|ˆ| ƒn tddƒVqdS(iN(RR(t.0titd(R(s/lib/python2.7/site-packages/dask/array/chunk.pys ‹sR i(RRRt enumeratetshapeRtreshape(t reductionRRt trim_excessR"tindtnewshape((Rs/lib/python2.7/site-packages/dask/array/chunk.pytcoarsen^s(   >cC`szt|tƒr"|g|j}nt|tƒrbgt|jƒD]}|j|dƒ^qA}n|td„|DƒƒS(sD Trim boundaries off of array >>> x = np.arange(24).reshape((4, 6)) >>> trim(x, axes={0: 0, 1: 1}) array([[ 1, 2, 3, 4], [ 7, 8, 9, 10], [13, 14, 15, 16], [19, 20, 21, 22]]) >>> trim(x, axes={0: 1, 1: 1}) array([[ 7, 8, 9, 10], [13, 14, 15, 16]]) ics`s+|]!}t||r| ndƒVqdS(N(RR(R!tax((s/lib/python2.7/site-packages/dask/array/chunk.pys ªs(RR RtdictRtgetR(RRR"((s/lib/python2.7/site-packages/dask/array/chunk.pyttrim—s 1c`s¥|tkst‚ˆd‰t|ƒ|jˆkr9|Stj|| dˆƒ}|dkrnt| dƒn t| ƒ‰|t‡‡fd†t |j ƒDƒƒS(s  Chunk and combine function of topk Extract the k largest elements from a on the given axis. If k is negative, extract the -k smallest elements instead. Note that, unlike in the parent function, the returned elements are not sorted internally. iR c3`s-|]#}|ˆkrˆn tdƒVqdS(N(RR(R!R"(R tk_slice(s/lib/python2.7/site-packages/dask/array/chunk.pys ¼sN( tTruetAssertionErrortabsR%tnpt partitionRRRRR(tatkR R((R R0s/lib/python2.7/site-packages/dask/array/chunk.pyttopk­s )c`s}|tkst‚t||ˆ|ƒ}ˆd‰tj|dˆƒ}|dkrV|S|t‡fd†t|jƒDƒƒS(sn Final aggregation function of topk Invoke topk one final time and then sort the results internally. iR c3`s9|]/}|ˆkr'tdddƒn tdƒVqdS(iÿÿÿÿN(RR(R!R"(R (s/lib/python2.7/site-packages/dask/array/chunk.pys Ës(R1R2R8R4tsortRRR(R6R7R R((R s/lib/python2.7/site-packages/dask/array/chunk.pyttopk_aggregateÀs  cC`s ||fS(s_ Preparatory step for argtopk Put data together with its original indices in a tuple. ((R6tidx((s/lib/python2.7/site-packages/dask/array/chunk.pytargtopk_preprocessÏsc `s^|tkst‚ˆd‰t|tƒr¥tt|ƒƒ}tjg|D]\}}|^qJˆƒ}tjg|D]!\}}tj||jƒ^quˆƒ}n |\}}t |ƒ|jˆkrÎ|Stj || dˆƒ} |dkrt | dƒn t | ƒ‰| t ‡‡fd†t|jƒDƒƒ} t|| ˆƒt|| ˆƒfS(s* Chunk and combine function of argtopk Extract the indices of the k largest elements from a on the given axis. If k is negative, extract the indices of the -k smallest elements instead. Note that, unlike in the parent function, the returned elements are not sorted internally. iR c3`s-|]#}|ˆkrˆn tdƒVqdS(N(RR(R!R"(R R0(s/lib/python2.7/site-packages/dask/array/chunk.pys ïsN(R1R2RtlistR R4t concatenatet broadcast_toR%R3t argpartitionRRRRRR ( t a_plus_idxR7R Rtait_R6tidxiR;tidx2((R R0s/lib/python2.7/site-packages/dask/array/chunk.pytargtopk×s + 4 )c`s•|tkst‚t||ˆ|ƒ\}}ˆd‰tj|dˆƒ}t||ˆƒ}|dkrn|S|t‡fd†t|jƒDƒƒS(s™ Final aggregation function of argtopk Invoke argtopk one final time, sort the results internally, drop the data and return the index only. iR c3`s9|]/}|ˆkr'tdddƒn tdƒVqdS(iÿÿÿÿN(RR(R!R"(R (s/lib/python2.7/site-packages/dask/array/chunk.pys s( R1R2RFR4targsortR RRR(RAR7R RR6R;RE((R s/lib/python2.7/site-packages/dask/array/chunk.pytargtopk_aggregateôs  cC`s6tj||||ƒ}t|ƒ|kr2|d S|S(Niÿÿÿÿ(R4tarangetlen(tstarttstoptsteptlengthtdtypetres((s/lib/python2.7/site-packages/dask/array/chunk.pyRIscK`s|j||S(N(tastype(Rt astype_dtypeR((s/lib/python2.7/site-packages/dask/array/chunk.pyRQ stCcC`sN|dkr(tj|ƒ}|j|ƒStj|ƒ}|jj|ƒjSdS(NRS(R4tascontiguousarraytviewtasfortranarraytT(RROtorder((s/lib/python2.7/site-packages/dask/array/chunk.pyRUs   c`sŒˆjtjƒ‰tjˆdkˆ|ˆƒ‰ˆ|‰ˆdkˆ|jˆk@}ˆ|‰|t‡‡fd†t|jƒDƒƒS(sµ Chunk function of `slice_with_int_dask_array_on_axis`. Slice one chunk of x by one chunk of idx. Parameters ---------- x: ndarray, any dtype, any shape i-th chunk of x idx: ndarray, ndim=1, dtype=any integer j-th chunk of idx (cartesian product with the chunks of x) offset: ndarray, shape=(1, ), dtype=int64 Index of the first element along axis of the current chunk of x x_size: int Total size of the x da.Array along axis axis: int normalized axis to take elements from (0 <= axis < x.ndim) Returns ------- x sliced along axis, using only the elements of idx that fall inside the current chunk. ic3`s-|]#}|ˆkrˆn tdƒVqdS(N(RR(R!R"(R R;(s/lib/python2.7/site-packages/dask/array/chunk.pys @s(RQR4tint64twhereR%RRR(RR;toffsettx_sizeR t idx_filter((R R;s/lib/python2.7/site-packages/dask/array/chunk.pytslice_with_int_dask_arrays  c `s|jtjƒ}tj|dk|t|ƒ|ƒ}d}d}tj|ƒ‰x‚|D]z}||k|||k@}tj|ƒ}ˆtj||d|dƒ7‰||7}|jdkrY||d7}qYqYW|t‡‡fd†t |j ƒDƒƒS(sT Final aggregation function of `slice_with_int_dask_array_on_axis`. Aggregate all chunks of x by one chunk of idx, reordering the output of `slice_with_int_dask_array`. Note that there is no combine function, as a recursive aggregation (e.g. with split_every) would not give any benefit. Parameters ---------- idx: ndarray, ndim=1, dtype=any integer j-th chunk of idx chunk_outputs: ndarray concatenation along axis of the outputs of `slice_with_int_dask_array` for all chunks of x and the j-th chunk of idx x_chunks: tuple dask chunks of the x da.Array along axis, e.g. ``(3, 3, 2)`` axis: int normalized axis to take elements from (0 <= axis < x.ndim) Returns ------- Selection from all chunks of x for the j-th chunk of idx, in the correct order iiiÿÿÿÿc3`s-|]#}|ˆkrˆn tdƒVqdS(N(RR(R!R"(R t idx_final(s/lib/python2.7/site-packages/dask/array/chunk.pys ws( RQR4RYRZtsumt zeros_liketcumsumtsizeRRR( R;t chunk_outputstx_chunksR tx_chunk_offsettchunk_output_offsettx_chunkR]tidx_cum((R R_s/lib/python2.7/site-packages/dask/array/chunk.pyt#slice_with_int_dask_array_aggregateEs% ! (@t__doc__t __future__RRRt functoolsRttoolzRtnumpyR4tRtnpcompatt compatibilityRRRtcoreR tutilsR tnumbersR R t ImportErrorRR`tprodtmintmaxtargmint nanargmintargmaxt nanargmaxtanytalltnansumtnanprodRR tnanmintnanmaxtmeantAttributeErrortnanmeantvartnanvartstdtnanstdtFalseR+RR/R8R:R<RFRHRIRQRUR^Rj(((s/lib/python2.7/site-packages/dask/array/chunk.pytsl    #                    9         -