o UݢgP/@sDdZddlmZddlZddlZddlmZmZmZm Z ddl m Z m Z m Z ddlZddlmZddlmZddlmZd]d d Zd^d_ddZd`ddZe daddZe dbddZdcddZddd$d%Zded*d+Z ,dfdgd1d2Zdhd5d6Zdidjd:d;Z dkd?d@Z!dldFdGZ"dmdIdJZ#dndNdOZ$dodSdTZ% d^dpdWdXZ&dqd[d\Z'dS)rzAssorted grab-bag of miscellaneous helper methods. Do not add to this module. Instead, find or create a module with a name that indicates to a potential user what sorts of methods they might find in that module. ) annotationsN) Collection GeneratorIterableSequence)IOAnyStroverload)FilteredBarcodesbarcode bytes | Nonereturn int | NonecCs6|durdS|d}|dkrt||ddSdS)zMethod to get just the gem group from a barcode. Args: barcode (bytes): a barcode, a la "ACGTACTAGAC-1" Returns: The Gem group as an int or None if not present. N-r)indexintr gem_groupr_/oak/stanford/groups/akundaje/marinovg/programs/cellranger-9.0.1/lib/python/cellranger/utils.pyget_gem_group_from_barcodes   rbytesrcCs|dur |d|7}|S)Ns-%drrrrrformat_barcode_seq,s r barcode_seqsCollection[bytes] gem_groupsIterable[int] | None list[bytes]cs&|durSfddtt|DS)zFormat a sequence of barcodes as seqs. Args: barcode_seqs (list[bytes]): _description_ gem_groups (Optional[Iterable[int]]): _description_ Returns: list[bytes]: _description_ Ncs g|] }D]}t||qqSr)r).0Zggbcrrr @s z'format_barcode_seqs..)sortedset)rrrr!rformat_barcode_seqs2s r%Nonetuple[None, None]cCdSNrr rrrsplit_barcode_seqCr+tuple[bytes, int | None]cCr(r)rr*rrrr+Ir,cCs\|durdSt|ts J|jddd}|d}t|dkr(t|d}||fSd}||fS)zSplit a barcode-gem_group. Args: barcode (bytes): A barcode with an optional gem group suffix. Returns: _type_: _description_ N)NNr)maxsplitrr) isinstancersplitlenr)r Z barcode_partsrrrrr+Os   bcsIterable[bytes] gg_id_mapdict[int, str | bytes]list[str | bytes]csfdd|D}|S)aTurn list of barcodes into corresponding list of mapped values. Args: bcs (list): aggr barcodes with suffix corresponding to gem group id gg_id_map (dict): mapping gem-group-ids/barcode-suffixes (int) to a desired named value (typically `aggr_id` for the library id of that one sample, or `batch_id` for the name of a batch it is a part of.) csg|] }t|dqS)r)r+)rr r5rrr"rsz)bcs_suffices_to_names..r)r3r5Z mapped_idsrr8rbcs_suffices_to_namesgs r9search AnyStr | NonegenomesSequence[AnyStr]cCsZt|dksJ|durdSt|dkr|dS|D] }||r%|Sqt|d)zReturn the first genome which is a prefix for the given search string. Args: search: _description_ genomes: _description_ Raises: Exception: _description_ Returns: _type_: _description_ rNr& does not have valid associated genome)r2 startswith ValueError)r:r<genomerrrget_genome_from_strvs   rBFseqrprefixes_as_genomesboolcCst|dksJ|durdSt|dkr|Stdd|D}|D]}||r@|r6|dt|dS|d|dSq!t|d)a_summary_. Args: seq (_type_): _description_ genomes (_type_): _description_ prefixes_as_genomes (bool, optional): _description_. Defaults to False. Raises: ValueError: _description_ Returns: _type_: _description_ rNrcss|]}t|VqdSr))r2)rgrrr z)remove_genome_from_str..r>)r2maxr?r@)rCr<rDmax_lenrArrrremove_genome_from_strs  rKreadtGenerator[tuple[bytes | None, bytes, int, bytes | None] | tuple[bytes | None, None, None, bytes | None], None, None]ccs|sdS|dD]V}t|dkrq |d}t|dkrq |dr'|dnd}|dr@|ddd}t|ddd}nd}d}|drL|dnd}|tjvsY|dusYJ||||fVq dS)aIterate over all transcripts compatible with the given read. We do this by iterating over the TX tag entries that are of the form `TX:,,`. Note: When intronic alignment is turned on, the `TX` tag is set to `TX:,` for intronic reads or exonic reads that are not compatible with annotated splice junctions. We ignore these. N;r,rr.)r1r2r cr_constantsSTRANDS)rLxpartschromstrandposZ cigarstringrrrget_read_transcripts_iters&   rX@bitsrcCsbt||ddks Jd}tt|D]}|||d}|tjvs%J|d>tj|B}q|S)zPack a DNA sequence (no Ns!) into a 2-bit format, into a python int. Args: seq (str): A DNA sequence. bits: the bit size of the integer to pack into. Returns: int: The sequence packed into the bits of an integer. r.rr)r2rangetk_seq NUCS_INVERSE)rCrZresultinucrrr compress_seqs ra in_filenames#Iterable[str | bytes | os.PathLike]dictc Csni}|D]0}|dur qz t|}t|}||Wdn1s%wYWqty4Yqw|S)zAMerge a list of json files and return the result as a dictionary.N)openjsonloadupdateOSError)rbr^filenamefdatarrrmerge_jsons_as_dicts    rmlibrary_prefixstrrAregion read_typecCs$||||g}|dd|}|S)a#Formats the barcode summary into a key to access the `barcode_summary.h5` outs. Here, we need to accommodate both accessing old and new versions of the h5 file compatible with both v2 and v3 of the matrix. Args: library_prefix: Name of the library, used as a prefix genome: Name of the genome used for alignment region: Name of the subset of the genome we are looking at (e.g. transcriptome, regulome, epigenome, ...). This should be a controlled vocabulary in `cellranger.constants` read_type: Name of the read types we are trying to extract (e.g. `conf_mapped_deduped_barcoded`, ...). It should be also controlled in `cellranger.constants`. Returns: output_key: A string constant with the suffix `reads` appended. Zreads_)appendjoin)rnrArprqZstr_listZ output_keyrrrformat_barcode_summary_h5_keys   ru*Generator[tuple[tuple, tuple], None, None]c#st|dkr dSt|ddkrdS|D]}t|t|dks#Jq|D]}t|t|dks4Jq&tjtdd|D}t|}t|d}t|d|d<t||D]\tfdd|Dtfdd|DfVqZdS)aGroup a collection of numpy arrays by key arrays. Yields `(key_tuple, view_tuple)` where `key_tuple` is the key grouped on and `view_tuple` is a tuple of views into the value arrays. Args: values (tuple of arrays): tuple of arrays to group. keys (tuple): tuple of sorted, numeric arrays to group by. Returns: sequence of tuple: Sequence of (`key_tuple`, `view_tuple`). rNcss*|]}tdgt|fdkVqdS)rrN)np concatenatediffrkeyrrrrG=s(z numpy_groupby..c3s|]}|VqdSr)rrz) group_startrrrGDrHc3s|] }|VqdSr)r)rvalueZ group_endr}rrrGDs  )r2rw logical_orreducetuple flatnonzerorollzip)valueskeysZ key_arrayZ value_arrayZkey_change_indicesZ group_startsZ group_endsrrr numpy_groupby#s(   "rrk IO[bytes]*Generator[tuple[bytes, bytes], None, None]ccsdd}d}|D]}|}|dr"|r||fV|dd}d}q||7}q|r0||fVdSdS)zIterate through sequences in a fasta file. Args: f (IO[bytes]): The input file object. Yields: tuple[bytes, bytes]: _description_ >rN)stripr?)rkZhdrrClinerrrget_fasta_iterIs     r barcode_csvstr | bytes | os.PathLikedict[bytes, list[bytes]]cCs,t|tr |}ddt|DS)z$Load a csv file of (genome,barcode).cSsi|] \}}||qSr)encode)rrAbarcodesrrr esz$load_barcode_csv..)r0rdecoder Zper_genome_barcodesitems)rrrrload_barcode_csvas rbarcode_csv_filename set[bytes]cCsPt|tr |}t|}t}|D]\}}|dus ||kr%||q|S)aGet set of cell-associated barcode strings. Args: barcode_csv_filename: TODO genome (bytes): Only get cell-assoc barcodes for this genome. If None, disregard genome. Returns: set of bytes: Cell-associated barcode strings (seq and gem-group). N)r0rorrr$rrh)rrAZcell_bcs_per_genomeZcell_bcsrFr3rrrget_cell_associated_barcode_setks  r input_string bytes | strcCsht|trz|jdd}Wn tyYdSwt|tr2z |jdddWdSty1YdSwdS)zReturns true if the string can be encoded as ascii. Input strings are often stored as ascii in numpy arrays, and we need to check that this conversion works. strict)errorsFasciiT)r0rrUnicodeDecodeErrorrorUnicodeEncodeError)rrrrstring_is_asciis    r)r r r rr))r rrrr r)rrrrr r)r r&r r')r rr r-)r r )r3r4r5r6r r7)r:r;r<r=r r;)F)rCrr<r=rDrEr r;)rLr r rM)rY)rCrrZrr r)rbrcr rd) rnrorArorprorqror ro)r rv)rkrr r)rrr r)rrrAr r r)rrr rE)(__doc__ __future__rrfoscollections.abcrrrrtypingrrr numpyrwcellranger.constants constantsrQ tenkit.seqrCr\cellranger.fast_utilsr rrr%r+r9rBrKrXrarmrurrrrrrrrrs@             % -    &