ó ¦–Õ\c@s€ddlZddlZddlZddlmZmZddlmZddlm Z ddl m Z d„Z d„Z dS( iÿÿÿÿNi(tbasetutils(tunpack_collections(tHighLevelGraph(t blockwisec sq|jdd ƒ}|jdd ƒ}|jdd ƒ}|jdd ƒ‰|jdiƒ}|jdtƒ}tt|ƒƒt|ƒkrÐtdgtj|ƒjƒD]\} } | dkr¦| ^q¦ƒ‚nt|ƒd „|dd d …Dƒt|pdƒ} | rtd | ƒ‚ndd l m } m } m }|d krVtdƒ‚n|rq| |Œ\}}n’gtj d |ƒD]$\}}|d k r„||f^q„}|rêt|dd„ƒ\}‰ttˆ|jƒƒ}ni}|d d d …}x?|jƒD]1\} } t| tƒs7| f} n| || >> z = blockwise(operator.add, 'ij', x, 'ij', y, 'ij', dtype='f8') # z = x + y # doctest: +SKIP Outer product multiplying x by y, two 1-d vectors >>> z = blockwise(operator.mul, 'ij', x, 'i', y, 'j', dtype='f8') # doctest: +SKIP z = x.T >>> z = blockwise(np.transpose, 'ji', x, 'ij', dtype=x.dtype) # doctest: +SKIP The transpose case above is illustrative because it does same transposition both on each in-memory block by calling ``np.transpose`` and on the order of the blocks themselves, by switching the order of the index ``ij -> ji``. We can compose these same patterns with more variables and more complex in-memory functions z = X + Y.T >>> z = blockwise(lambda x, y: x + y.T, 'ij', x, 'ij', y, 'ji', dtype='f8') # doctest: +SKIP Any index, like ``i`` missing from the output index is interpreted as a contraction (note that this differs from Einstein convention; repeated indices do not imply contraction.) In the case of a contraction the passed function should expect an iterable of blocks on any array that holds that index. To receive arrays concatenated along contracted dimensions instead pass ``concatenate=True``. Inner product multiplying x by y, two 1-d vectors >>> def sequence_dot(x_blocks, y_blocks): ... result = 0 ... for x, y in zip(x_blocks, y_blocks): ... result += x.dot(y) ... return result >>> z = blockwise(sequence_dot, '', x, 'i', y, 'i', dtype='f8') # doctest: +SKIP Add new single-chunk dimensions with the ``new_axes=`` keyword, including the length of the new dimension. New dimensions will always be in a single chunk. >>> def f(x): ... return x[:, None] * np.ones((1, 5)) >>> z = blockwise(f, 'az', x, 'a', new_axes={'z': 5}, dtype=x.dtype) # doctest: +SKIP New dimensions can also be multi-chunk by specifying a tuple of chunk sizes. This has limited utility as is (because the chunks are all the same), but the resulting graph can be modified to achieve more useful results (see ``da.map_blocks``). >>> z = blockwise(f, 'az', x, 'a', new_axes={'z': (5, 5)}, dtype=x.dtype) # doctest: +SKIP If the applied function changes the size of each chunk you can specify this with a ``adjust_chunks={...}`` dictionary holding a function for each index that modifies the dimension size in that index. >>> def double(x): ... return np.concatenate([x, x]) >>> y = blockwise(double, 'ij', x, 'ij', ... adjust_chunks={'i': lambda n: 2 * n}, dtype=x.dtype) # doctest: +SKIP Include literals by indexing with None >>> y = blockwise(add, 'ij', x, 'ij', 1234, None, dtype=x.dtype) # doctest: +SKIP tnamettokentdtypet adjust_chunkstnew_axest align_arrayss-Repeated elements not allowed in output indexicSs/h|]%}|dk r|D] }|’qqS(N(tNone(t.0targta((s3lib/python2.7/site-packages/dask/array/blockwise.pys |s NisUnknown dimension(tArrayt unify_chunkst normalize_args"Must specify dtype of output arraytkeycSst|dƒS(Ni(tlen(tai((s3lib/python2.7/site-packages/dask/array/blockwise.pyt‹ttndimt__len__s1Index string %s does not match array dimension %dcSs1i|]'\}}|dk r|j|j“qS(N(R t numblocksR(R Rtind((s3lib/python2.7/site-packages/dask/array/blockwise.pys œs s%s-%st_Rt dependenciesc3s|]}ˆˆVqdS(N((R R(RR(s3lib/python2.7/site-packages/dask/array/blockwise.pys Æss4adjust_chunks values must be callable, int, or tuple(()tpopR tTrueRtsett ValueErrorttoolzt frequenciestitemstcoreRRRt partitiontmaxtdicttziptchunkst isinstancettupletlistthasattrRRtextendtappendRRtfuncnametstripRttokenizetcore_blockwiseRtfrom_collectionst enumeratetcallabletmaptnumberstIntegraltNotImplementedError(tfunctout_indtargstkwargstoutRRR R tktvtnewRRRtchunksstarraysRtitargindsR RRt argindsstrt collectionstkwargs2tgraphR)((RRs3lib/python2.7/site-packages/dask/array/blockwise.pyR sŽd@7 = "3      "  $* cOstjdƒt||ŽS(Ns.The da.atop function has moved to da.blockwise(twarningstwarnR(R=R>((s3lib/python2.7/site-packages/dask/array/blockwise.pytatopÑs ( R8RKR!RRRtdelayedRthighlevelgraphRRR3RM(((s3lib/python2.7/site-packages/dask/array/blockwise.pyts    Å