o Uݢg@s\dZddlmZddlZddlmZGdddeZ ddZ dd Z dd dZ dddZ dS)zSimple Good-Turing estimator. Based on S implementation in:: William A. Gale & Geoffrey Sampson (1995) Good-turing frequency estimation without tears, Journal of Quantitative Linguistics, 2:3, 217-237, DOI: 10.1080/09296179508590051 ) annotationsNc@s eZdZdS)SimpleGoodTuringErrorN)__name__ __module__ __qualname__rr]/oak/stanford/groups/akundaje/marinovg/programs/cellranger-9.0.1/lib/python/cellranger/sgt.pyrsrcCsbttjdtdt|f}td|dd|ddtj|dftdf}|t|S)N)dtypeg?r)np concatenateonesintdiffarrayfloatastype)rnrddrrrr_averaging_transforms6rcCs|tdd|d|S)Nr g?)r power)rcoefrrr_rstestsrxr np.ndarrayxnrc Cs&|t}|t}t||}t||}tt|t|\}}}}}|dkr4td|dnd}tdt ||}||}|t |dddt dfk}t t |} ||d||t |ddt df|||| |<t t |} tt |D]'} || rt| d|| t|| dd|| d|| | | <qt t |} d} tt |D]/}| s||| |<qt||| |d|| |d kr| || |<qd } ||| |<qt| |||}| d|d ||} |d |}|| |fS) aZMake a Simple Good-Turing estimate of the frequencies. Args: xr (np.array(int)): Non-zero item frequencies xnr (np.array(int)): Non-zero frequencies of frequencies Returns: (rstar (np.array(float)), p0 (float)): rstar: The adjusted non-zero frequencies p0: The total probability of unobserved items r zThe SGT slope is .zQThe SGT slope {slope} is > -1. The SGT estimator is not applicable to these data.r NTgffffff?Fr)rrr sumrsp_stats linregresslogprintrr zeroslenrrangesqrtabs)rrZxNZxnrzslope_ZxrstZxrstrelZxrtryZxrstarelZtursdiZ xrstcmbrelZ useturingrZsumprawp0rrrsimple_good_turing"sB  " $:B*  r/ frequenciescst|dkr tdt|t|krtdt|}|ddks$Jt|}t|dkr7tdt|t|||\}}tt ||t |||}tj fdd|Dt t|d}d |||}t |t |d svJ||fS) aUse Simple Good-Turing estimate to adjust for unobserved items. Args: frequencies (np.array(int)): Nonzero frequencies of items Returns: pstar (np.array[float]): The adjusted non-zero proportions p0 (float): The total probability of unobserved items rzInput frequency vector is emptyz%Frequencies must be greater than zero z4Too few non-zero frequency items (%d). Aborting SGT.c3s|]}|VqdS)Nr).0fZ rstar_dictrr sz"sgt_proportions..)r countr )r' ValueErrorr count_nonzerobincount flatnonzerorr/dictzipr!fromiterrisclose)r0Z freqfreqsZ use_freqsrstarr.Z rstar_sumZrstar_iZpstarrr4rsgt_proportionscs$     "r@)rrrr)r0r)__doc__ __future__rnumpyr scipy.statsstatsr" Exceptionrrrr/r@rrrrs   A