ó ¦–Õ\c @`séddlmZmZmZddlZddlZddlmZddlm Z ddl m Z m Z ddl m Z ddlmZmZd d lmZe d eƒZ dd e d ejeddd„Zd„Zd„ZdS(i(tprint_functiontdivisiontabsolute_importN(tconcati(tunicode(tsystem_encodingt parse_bytes(tdelayed(t open_filest read_bytesi(t from_delayedtpuretinfertstrictc  C`sÉ|d k r'|d k r'tdƒ‚nt|ttfƒrKt|ƒ}nt|ddd|d|d||pri} |d kr/|d krÄg| D]$} ttƒtt ƒ| ƒƒ^q—} q™g} xÌt dt | ƒ|ƒD]E} | | | |!} tt ƒtt ƒt | ƒƒ}| j|ƒqãWnjt|d|jƒd |d td||p\i\}} gt | ƒD]}ttƒ|||ƒ^qu} | s±td |ƒ‚n|s»| St| ƒSd S( sª Read lines from text files Parameters ---------- urlpath : string or list Absolute or relative filepath(s). Prefix with a protocol like ``s3://`` to read from alternative filesystems. To read from multiple files you can pass a globstring or a list of paths, with the caveat that they must all have the same protocol. blocksize: None, int, or str Size (in bytes) to cut up larger files. Streams by default. Can be ``None`` for streaming, an integer number of bytes, or a string like "128MiB" compression: string Compression format like 'gzip' or 'xz'. Defaults to 'infer' encoding: string errors: string linedelimiter: string collection: bool, optional Return dask.bag if True, or list of delayed values if false storage_options: dict Extra options that make sense to a particular storage connection, e.g. host, port, username, password, etc. files_per_partition: None or int If set, group input files into partitions of the requested size, instead of one partition per file. Mutually exclusive with blocksize. Examples -------- >>> b = read_text('myfiles.1.txt') # doctest: +SKIP >>> b = read_text('myfiles.*.txt') # doctest: +SKIP >>> b = read_text('myfiles.*.txt.gz') # doctest: +SKIP >>> b = read_text('s3://bucket/myfiles.*.txt') # doctest: +SKIP >>> b = read_text('s3://key:secret@bucket/myfiles.*.txt') # doctest: +SKIP >>> b = read_text('hdfs://namenode.example.com/myfiles.*.txt') # doctest: +SKIP Parallelize a large file by providing the number of uncompressed bytes to load into each partition. >>> b = read_text('largefile.txt', blocksize='10MB') # doctest: +SKIP Returns ------- dask.bag.Bag if collection is True or list of Delayed lists otherwise See Also -------- from_sequence: Build bag from Python sequence s7Only one of blocksize or files_per_partition can be settmodetrttencodingterrorst compressionit delimitert blocksizetsamplesNo files foundN(tNonet ValueErrort isinstancetstrRRRRtlisttfile_to_blockstrangetlenRtmaptappendR tencodetFalsetdecodeR (turlpathRRRRt linedelimitert collectiontstorage_optionstfiles_per_partitiontfilestfiltblockststartt block_filest block_linest_tb((s,lib/python2.7/site-packages/dask/bag/text.pyt read_texts25   4! 1cc`s)|}x|D] }|VqWWdQXdS(N((t lazy_filetftline((s,lib/python2.7/site-packages/dask/bag/text.pyRgs  cC`s+|j||ƒ}tj|ƒ}t|ƒS(N(R"tiotStringIOR(tblockRRttexttlines((s,lib/python2.7/site-packages/dask/bag/text.pyR"ms(t __future__RRRR4tosttoolzRt compatibilityRtutilsRRRtbytesRR tcoreR tTrueRtlinesepR0RR"(((s,lib/python2.7/site-packages/dask/bag/text.pyts   S