ó áp7]c @s dZddlmZddlmZddlmZmZmZddl Z e j dƒZ de fd„ƒYZd e fd „ƒYZd e fd „ƒYZe je jƒd Zde fd„ƒYZdd„Zdd„Zddd„Zde fd„ƒYZddd„Zedkr dddgZdZddljZ ddl!m"Z"m#Z#e$dd d!d"ƒe$dd#d!d$ƒfZ%e"d#d%d d%gd&ed'ej&ej&gd(e%ƒZ'e#ƒZ(d)ekrˆe'e'd*ke'd k@dZ'ee'ed+d,d-dƒ\Z)Z*Z+Z,e)Z-ej.e)e*ƒZ/e0d.e)j1ƒƒe)e)j1ƒZ)ej.e)e*ƒZ2e0d/e/e2ƒe)e2:Z)dZ3e3rðe j4e'd0d1d2e5d3d4ƒe j6e*e)d5d d3d6ƒe j6e*e-d5d d3d7ƒe j7ƒnxee8e,d ƒD]S\Z9Z:xDe8e,d ƒD]2\Z;Z<e0e9e;ej=d8„dd#ƒdƒqWqWx-e,D]"Z:e0ej=d9„dd#ƒƒq_Wndekr·eƒj>e'ed+d,ƒZ?e j@e'j1ƒe'jAƒƒZ*e?e*ƒZBej=e?e?jCŒdZDe0d:eDƒe(jEe*d#d%d d%gd'ej&ej&gd(e%ƒZFd#Z3e3r·e jGƒe j4e'd0d1d2e5d3d4ƒe j6e*eBd5d d3d6ƒe j6e*eFd5d d3d;ƒe jHd<ƒq·ndekr5eƒZ?d"e?_Ie?j>e'ed+d=ƒZ?e j@e'j1ƒe'jAƒƒZ*e?e*ƒZBej=e?e?jCŒdZDe0d:eDƒe(jEe*d#d%d d%gd'ej&ej&gd(e%ƒZFd#Z3e3ròe jGƒe j4e'd0d1d2e5d3d4ƒe jHd>ƒe j6e*eBd5d d3d6ƒe j6e*eFd5d d3d;ƒne0e jAe jJee?j,d dd#ƒde jKdƒƒƒƒndekr½eƒZ?de?_Ie?j>e'ed+d,ƒZ?e j@e'j1ƒe'jAƒƒZ*e?e*ƒZBej=e?e?jCŒdZDe0d:eDƒe(jEe*d#d%d d%gd'ej&ej&gd(e%ƒZFd#Z3e3rze jGƒe j4e'd0d1d2e5d3d4ƒe j6e*eBd5d d3d6ƒe j6e*eFd5d d3d;ƒe jHd?ƒe j7ƒne0e jAe jJee?j,d dd#ƒde jKdƒƒƒƒngeLdƒD]Z9ee9ƒ^qÊZMeeMd@dAƒdZNe0e jAe jJeNe jKdƒƒƒƒe0eNdBjOePƒƒddClQmRZRmSZSgeLdƒD]Z9eRe9ƒ^q[ZTeeTdDdEƒdZUe0eUdBjOePƒƒgeLdFƒD]Z9eSe9ƒ^q­ZVeeVdd#dGdH„ƒ\ZWZXe0e jAe jJeWe jYe jYeWƒƒƒƒƒndS(IsŠdensity estimation based on orthogonal polynomials Author: Josef Perktold Created: 2011-05017 License: BSD 2 versions work: based on Fourier, FPoly, and chebychev T, ChebyTPoly also hermite polynomials, HPoly, works other versions need normalization TODO: * check fourier case again: base is orthonormal, but needs offsetfact = 0 and doesn't integrate to 1, rescaled looks good * hermite: works but DensityOrthoPoly requires currently finite bounds I use it with offsettfactor 0.5 in example * not implemented methods: - add bonafide density correction - add transformation to domain of polynomial base - DONE possible problem: what is the behavior at the boundary, offsetfact requires more work, check different cases, add as option moved to polynomial class by default, as attribute * convert examples to test cases * need examples with large density on boundary, beta ? * organize poly classes in separate module, check new numpy.polynomials, polyvander * MISE measures, order selection, ... enhancements: * other polynomial bases: especially for open and half open support * wavelets * local or piecewise approximations iÿÿÿÿ(tprint_function(tzip(tstatst integratetspecialNg@tFPolycBs eZdZd„Zd„ZRS(sAOrthonormal (for weight=1) Fourier Polynomial on [0,1] orthonormal polynomial but density needs corfactor that I don't see what it is analytically parameterization on [0,1] from Sam Efromovich: Orthogonal series density estimation, 2010 John Wiley & Sons, Inc. WIREs Comp Stat 2010 2 467-476 cCs"||_d|_|j|_dS(Nii(ii(tordertdomaint intdomain(tselfR((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyt__init__?s  cCs?|jdkrtj|ƒSttjtj|j|ƒSdS(Ni(Rtnpt ones_liketsqr2tcostpi(R tx((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyt__call__Ds (t__name__t __module__t__doc__R R(((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyR1s  tF2PolycBs eZdZd„Zd„ZRS(s²Orthogonal (for weight=1) Fourier Polynomial on [0,pi] is orthogonal but first component doesn't square-integrate to 1 final result seems to need a correction factor of sqrt(pi) _corfactor = sqrt(pi) from integrating the density Parameterization on [0, pi] from Peter Hall, Cross-Validation and the Smoothing of Orthogonal Series Density Estimators, JOURNAL OF MULTIVARIATE ANALYSIS 21, 189-206 (1987) cCs4||_dtjf|_|j|_d|_dS(Ni(RR RRRt offsetfactor(R R((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyR Xs  cCsX|jdkr,tj|ƒtjtjƒSttj|j|ƒtjtjƒSdS(Ni(RR R tsqrtRR R(R R((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyR^s(RRRR R(((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyRJs  t ChebyTPolycBs eZdZd„Zd„ZRS(sLOrthonormal (for weight=1) Chebychev Polynomial on (-1,1) Notes ----- integration requires to stay away from boundary, offsetfactor > 0 maybe this implies that we cannot use it for densities that are > 0 at boundary ??? or maybe there is a mistake close to the boundary, sometimes integration works. cCsM||_ddlm}||ƒ|_d|_ddf|_d|_dS( Niÿÿÿÿ(tchebytigíµ ÷Æ°>g{®Gáz„?(iÿÿÿÿigé !çýÿï¿gé !çýÿï?(Rt scipy.specialRtpolyRRR(R RR((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyR rs   cCs‚|jdkr@tj|ƒd|dddtjtjƒS|j|ƒd|dddtjtjƒtjdƒSdS(Niiig@(RR R RRR(R R((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyR|s1(RRRR R(((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyRds  itHPolycBs eZdZd„Zd„ZRS(sŽOrthonormal (for weight=1) Hermite Polynomial, uses finite bounds for current use with DensityOrthoPoly domain is defined as [-6,6] cCsE||_ddlm}||ƒ|_dd f|_d|_dS(Niÿÿÿÿ(thermiteiúÿÿÿigà?(RRRRRR(R RR((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyR Œs  cCsf|j}dd |tjdƒtj|dƒt||d}tj|ƒ}|j|ƒ|S(Ngð?ig@i(RR tlogRtgammalntlogpi2texpR(R Rtktlnfacttfact((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyR“s =(RRRR R(((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyR†s icCs8tjgt|ƒD]}||ƒ|ƒ^qƒ}|S(N(R t column_stacktrange(RtpolybaseRtitpolyarr((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyt polyvander›s4c sEt|ƒ}tj||fƒ}|jtjƒtj||fƒ}xòt|ƒD]ä}xÛt|dƒD]É}||‰||‰ˆdk r½tj ‡‡‡fd†||ƒ\} } n'tj ‡‡fd†||ƒ\} } | |||f<| |||f<||ksj| |||f<| |||f>> from scipy.special import chebyt >>> polys = [chebyt(i) for i in range(4)] >>> r, e = inner_cont(polys, -1, 1) >>> r array([[ 2. , 0. , -0.66666667, 0. ], [ 0. , 0.66666667, 0. , -0.4 ], [-0.66666667, 0. , 0.93333333, 0. ], [ 0. , -0.4 , 0. , 0.97142857]]) icsˆ|ƒˆ|ƒˆ|ƒS(N((R(tp1tp2tweight(sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pytÊtcsˆ|ƒˆ|ƒS(N((R(R+R,(sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyR.ÍR/N( tlenR temptytfilltnantzerosR&tNoneRtquad( tpolystlowertupperR-tn_polyst innerprodtinterrR(tjtinnpterr((R+R,R-sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyt inner_contŸs$!    ' ig:Œ0âŽyE>c sx–tt|ƒƒD]‚}xyt|dƒD]g}||‰||‰tj‡‡fd†||ƒd}tj|||kd|d|ƒs*tSq*WqWtS(sZcheck whether functions are orthonormal Parameters ---------- polys : list of polynomials or function Returns ------- is_orthonormal : bool is False if the innerproducts are not close to 0 or 1 Notes ----- this stops as soon as the first deviation from orthonormality is found. Examples -------- >>> from scipy.special import chebyt >>> polys = [chebyt(i) for i in range(4)] >>> r, e = inner_cont(polys, -1, 1) >>> r array([[ 2. , 0. , -0.66666667, 0. ], [ 0. , 0.66666667, 0. , -0.4 ], [-0.66666667, 0. , 0.93333333, 0. ], [ 0. , -0.4 , 0. , 0.97142857]]) >>> is_orthonormal_cont(polys, -1, 1, atol=1e-6) False >>> polys = [ChebyTPoly(i) for i in range(4)] >>> r, e = inner_cont(polys, -1, 1) >>> r array([[ 1.00000000e+00, 0.00000000e+00, -9.31270888e-14, 0.00000000e+00], [ 0.00000000e+00, 1.00000000e+00, 0.00000000e+00, -9.47850712e-15], [ -9.31270888e-14, 0.00000000e+00, 1.00000000e+00, 0.00000000e+00], [ 0.00000000e+00, -9.47850712e-15, 0.00000000e+00, 1.00000000e+00]]) >>> is_orthonormal_cont(polys, -1, 1, atol=1e-6) True icsˆ|ƒˆ|ƒS(N((R(R+R,(sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyR.R/itrtoltatol(R&R0RR6R tallclosetFalsetTrue(R7R8R9RARBR(R=R;((R+R,sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pytis_orthonormal_cont×s,  %$ tDensityOrthoPolycBs_eZdZd dd„Zd dd d„Zd d„Zd„Zd„Zd„Z d„Z RS( sUnivariate density estimation by orthonormal series expansion Uses an orthonormal polynomial basis to approximate a univariate density. currently all arguments can be given to fit, I might change it to requiring arguments in __init__ instead. icCsZ|dk rD||_gt|ƒD]}||ƒ^q"|_}nd|_d|_dS(Nii(R5R'R&R7t _corfactort _corshift(R R'RR(R7((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyR s   / cCsm|dkr|j| }n5||_gt|ƒD]}||ƒ^q2|_}t|dƒss|dj|_n|jƒ|jƒ}}|dkrÎ|||j|_ } || || f}|_ n|d|d} ||} d| |_ | | d} || |_ |j |ƒ|_}g|D]} | |ƒjƒ^q/} | |_||_|jƒ|S(sIestimate the orthogonal polynomial approximation to the density t offsetfaciigð?g@N(R5R7R'R&thasattrRRJtmintmaxtoffsettlimitstshrinktshiftt _transformRtmeantcoeffst_verify(R RR'RROR7R(txmintxmaxRNtinterval_lengtht xintervaltpRT((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pytfit(s*  ,    %   csu|jˆƒ‰|dkr-t|jƒ}nt‡fd†tt|j|jƒƒ| Dƒƒ}|j|ƒ}|S(Nc3s%|]\}}||ˆƒVqdS(N((t.0tcRZ(txeval(sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pys Ps( RRR5R0R7tsumtlistRRTt _correction(R R^Rtres((R^sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pytevaluateLs  5cCs |j|ƒS(s,alias for evaluate, except no order argument(Rc(R R^((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyRTscCs0|j}dtj|j|Œd|_|jS(s—check for bona fide density correction currently only checks that density integrates to 1 ` non-negativity - NotImplementedYet gð?i(RORR6RcRH(R R((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyRUXs  cCsB|jdkr||j9}n|jdkr>||j7}n|S(shbona fide density correction affine shift of density to make it into a proper density ii(RHRI(R R((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyRajs cCsW|jdj}|d|d}|j|d|j|}|j|}|||S(s„transform observation to the domain of the density uses shrink and shift attribute which are set in fit to stay ii(R7RRQRP(R RRtilenRQRP((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyRRxs  N( RRRR5R R[RcRRURaRR(((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyRGs  $    c s¯ˆdkr0tj|jƒ|jƒdƒ‰ngt|ƒD]}||ƒ^q=}g|D]}||ƒjƒ^q\}t‡fd†t||ƒDƒƒ}|ˆ||fS(Ni2c3s%|]\}}||ˆƒVqdS(N((R\R]RZ(R^(sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pys s( R5R tlinspaceRLRMR&RSR_R( RR'RR^R(R7RZRTRb((R^sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pytdensity_orthopolys  $%%%t__main__RtfourierRi'(t mixture_rvstMixtureDistributiontlocgà¿tscalegà?igš™™™™™É?g@tsizetdisttkwargstchebyt_iþÿÿÿRiR^s f_hat.min()tfint2tbinsi2tnormedtcolortredtlwtblacktgcCst|ƒt|ƒS(N(RZR,(R((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyR.ØR/cCst|ƒdS(Ni(RZ(R((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyR.ÛR/sdop F integraltgreensusing Chebychev polynomialsisusing Fourier polynomialssusing Hermite polynomialsiúÿÿÿii †(RRiöÿÿÿi iR-cCsd||ddS(Niiÿÿÿÿg@((R((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyR.2R/(ZRt __future__Rtstatsmodels.compat.pythonRtscipyRRRtnumpyR RR tobjectRRRRRR RR*R5R@RFRGRfRtexamplestnobstmatplotlib.pyplottpyplottpltt%statsmodels.distributions.mixture_rvsRiRjtdicttmix_kwdstnormtobs_disttmixtf_hattgridRTR7tf_hat0ttrapztfinttprintRLRqtdoplotthistREtplottshowt enumerateR(RZR=R,R6R[tdopReRMtxfROtdopinttpdftmpdftfigurettitleRJtabsteyeR&thpolystinntastypetintRRRthtpolystinnttpolysctrtetdiag(((sQlib/python2.7/site-packages/statsmodels/sandbox/nonparametric/densityorthopoly.pyt'sØ    8;{ */   '  . #   /       /   C     /    C%)%%!