B Z4 @sdZddlmZddlZddlmZddlmZddZ dd Z d d Z d d Z ddZ ddZddZddZddZddZddZddZedkrdZerddZedZejd d!Zeddd"dfZeeeeedddfZd"d"d"gZe e eeeeeeedddfeZ!e e!e eeeeed"ee eeeedd#lm"Z"m#Z#d$d%Z$d&d'Z%d(d)Z&d*d+Z'd,d-Z(d.d/Z)ere d0e e#j*e$d"d1d"d2d!d3e d4ed"dd"ed e d5ed"dd"ed e e#j*e$d6d7d"d8d!d3e d5ed6dd"ed9e e#j*e$d6d7d"d:d!d3e d5ed6dded9e e#j*e$d6d7d"d;d!d3e d5ed6d"ded9e e#j*e%d"d7d"d\ZZ+Z,Z-e d5eee+e,ed9e e#j*e%e+d7d"ee,e-fd!d3e e#j*e&e,d7d"ee+e-fd!d3e d?e e#j*e'd"d1d"d@d!d3e ed"dd"edA\ZZ+Z,e d5eee+e,ee e#j*e(e+d7d"ee,fd!d3e e#j*e)e,d7d"ee+fd!d3dB\ZZ+Z,e d5eee+e,ee e#j*e(e+d7d"ee,fd!d3e e#j*e)e,d7d"ee+fd!d3e dCe e d"e.dd"gdDe$d"dd"dDdEe e d"e.dd"gd e$d"dd"d dEe ed"e.dd"gd e$d"dd"d e e d"e.ddFgd e$dGddGd e ed"e.dd"gdDe$d"dd"dDe e d"e.dd"gd e$d"ddGe/dHd ddIl0m1Z1d@dJdKdLgZe2dMdNdOZ3x|eD]t\Z+Z,e#j*e(e+d7d"e3e,fd!d3Z4e#j*e)e,d7d"e3e+fd!d3Z5ee3e+e,eZ6e1e4e6ddPdQdRe1e5e6d"dPdSdRq,WxdTD]Z-xeD]\Z+Z,e#j*e%e+d7d"e3e,e-fd!d3Z4e#j*e&e,d7d"e3e+e-fd!d3Z5ee3e+e,ee-Z6e1e4e6ddUdQdRe1e5e6d"dUdSdRe1ee3e.e+e,dge-e$e3e+e,e-dPdVdRe1e e3e.e+e,dge-e$e3e+e,e/e-dNe-e-dPdVdRqWqWdS)Wagradient/Jacobian of normal and t loglikelihood use chain rule normal derivative wrt mu, sigma and beta new version: loc-scale distributions, derivative wrt loc, scale also includes "standardized" t distribution (for use in GARCH) TODO: * use sympy for derivative of loglike wrt shape parameters it works for df of t distribution dlog(gamma(a))da = polygamma(0,a) check polygamma is available in scipy.special * get loc-scale example to work with mean = X*b * write some full unit test examples A: josef-pktd )print_functionN)special)gammalncCs<|j\}}dtdtjt|||d|}|S)arnormal loglikelihood given observations and mean mu and variance sigma2 Parameters ---------- y : array, 1d normally distributed random variable params: array, (nobs, 2) array of mean, variance (mu, sigma2) with observations in rows Returns ------- lls : array contribution to loglikelihood for each observation g)Tnplogpi)yparamsmusigma2llsrClib/python3.7/site-packages/statsmodels/sandbox/regression/tools.pynorm_llss .rcCsB|j\}}|||}||d|dt|}t||fS)a=Jacobian of normal loglikelihood wrt mean mu and variance sigma2 Parameters ---------- y : array, 1d normally distributed random variable params: array, (nobs, 2) array of mean, variance (mu, sigma2) with observations in rows Returns ------- grad : array (nobs, 2) derivative of loglikelihood for each observation wrt mean in first column, and wrt variance in second column Notes ----- this is actually the derivative wrt sigma not sigma**2, but evaluated with parameter sigma2 = sigma**2 r)rrsqrt column_stack)r r r r ZdllsdmuZ dllsdsigma2rrr norm_lls_grad0s  rcCs|S)z-gradient/Jacobian for d (x*beta)/ d beta r)xbetarrr mean_gradLsrc Cs|dd}|dtt|df}t||}t||}t||f}t||}t|ddddf||ddddff} | S)aJacobian of normal loglikelihood wrt mean mu and variance sigma2 Parameters ---------- y : array, 1d normally distributed random variable with mean x*beta, and variance sigma2 x : array, 2d explanatory variables, observation in rows, variables in columns params: array_like, (nvars + 1) array of coefficients and variance (beta, sigma2) Returns ------- grad : array (nobs, 2) derivative of loglikelihood for each observation wrt mean in first column, and wrt scale (sigma) in second column assume params = (beta, sigma2) Notes ----- TODO: for heteroscedasticity need sigma to be a 1d array Nr)roneslenrdotrr) r rr rr Zdmudbetar Zparams2ZdllsdmsZgradrrrnormgradQs    2rcCs|j\}}|d}t|ddt|ddt|dtj}||ddtd||d|d|dt|8}|S)at loglikelihood given observations and mean mu and variance sigma2 = 1 Parameters ---------- y : array, 1d normally distributed random variable params: array, (nobs, 2) array of mean, variance (mu, sigma2) with observations in rows df : integer degrees of freedom of the t distribution Returns ------- lls : array contribution to loglikelihood for each observation Notes ----- parameterized for garch g?rg@g?r)rrrrr )r r dfr r rrrrtstd_llsus  4@rcCs| S)z?derivative of log pdf of standard normal with respect to y r)r rrr norm_dlldysr cCs"|d |d|d||S)zderivative of log pdf of standardized (?) t with respect to y Notes ----- parameterized for garch, with mean 0 and variance 1 rrr)r rrrrts_dlldys r!cCsnt|d}tt|ddt|dt|dt}|d|d|d|dd}|S)zIpdf for standardized (not standard) t distribution, variance is one g?rg@r)rarrayZexprrrr )rrrZPxrrrtstd_pdfs8$r$cCst||||j\}}|d}t|ddt|ddt|tj}||ddtd||d||dt|8}|S)at loglikelihood given observations and mean mu and variance sigma2 = 1 Parameters ---------- y : array, 1d normally distributed random variable params: array, (nobs, 2) array of mean, variance (mu, sigma2) with observations in rows df : integer degrees of freedom of the t distribution Returns ------- lls : array contribution to loglikelihood for each observation Notes ----- parameterized for garch normalized/rescaled so that sigma2 is the variance >>> df = 10; sigma = 1. >>> stats.t.stats(df, loc=0., scale=sigma.*np.sqrt((df-2.)/df)) (array(0.0), array(1.0)) >>> sigma = np.sqrt(2.) >>> stats.t.stats(df, loc=0., scale=sigma*np.sqrt((df-2.)/df)) (array(0.0), array(2.0)) g?rg@g?r)printrrrrr )r r rr r rrrrts_llss   0<r&cCs*|d}|d |d|d||S)aderivative of log pdf of standard t with respect to y Parameters ---------- y : array_like data points of random variable at which loglike is evaluated df : array_like degrees of freedom,shape parameters of log-likelihood function of t distribution Returns ------- dlldy : array derivative of loglikelihood wrt random variable y evaluated at the points given in y Notes ----- with mean 0 and scale 1, but variance is df/(df-2) g?rrr)r rrrrr!scCs*|d |dd|d|d|S)a derivative of log pdf of standardized t with respect to y Parameters ---------- y : array_like data points of random variable at which loglike is evaluated df : array_like degrees of freedom,shape parameters of log-likelihood function of t distribution Returns ------- dlldy : array derivative of loglikelihood wrt random variable y evaluated at the points given in y Notes ----- parameterized for garch, standardized to variance=1 rg@rr)r rrrr tstd_dlldysr'cGsN|||}||f| |}d|||f||||d}||fS)aderivative of log-likelihood with respect to location and scale Parameters ---------- y : array_like data points of random variable at which loglike is evaluated loc : float location parameter of distribution scale : float scale parameter of distribution dlldy : function derivative of loglikelihood fuction wrt. random variable x args : array_like shape parameters of log-likelihood function Returns ------- dlldloc : array derivative of loglikelihood wrt location evaluated at the points given in y dlldscale : array derivative of loglikelihood wrt scale evaluated at the points given in y grr)r locscaleZdlldyargsZystZdlldlocZ dlldscalerrr locscale_grad s &r+__main__g?r r)statsmisccCsttjj||||dS)N)r(r))rrr/tpdf)r r(r)rrrrllt=sr3cCsttjj||||dS)N)r(r))rrr/r1r2)r(r r)rrrrlltloc?sr4cCsttjj||||dS)N)r(r))rrr/r1r2)r)r r(rrrrlltscaleAsr5cCsttjj|||dS)N)r(r))rrr/normr2)r r(r)rrrllnormDsr7cCsttjj|||dS)N)r(r))rrr/r6r2)r(r r)rrr llnormlocFsr8cCsttjj|||dS)N)r(r))rrr/r6r2)r)r r(rrr llnormscaleHsr9z gradient of tgư>)rrr-)Zdxnr*orderzt Ztsg?g|=)rrr<)rrr<)rrr<)g?rr<)g?rr<)g?rrr<z gradient of norm)rr)g?rr)g?rrz loglike of tdzdifferently standardizedg?g?g?)assert_almost_equal)g?g?)gg@)g?g@gg@ z deriv loc)Zerr_msgz deriv scale)r.r-r=Zloglike)7__doc__Z __future__rZnumpyrZscipyrZ scipy.specialrrrrrrr r!r$r&r'r+__name__verboseZsigrrZrandomZrandnZrvsrrr r r%Z dllfdbetar/r0r3r4r5r7r8r9Z derivativer(r)rr"rZ numpy.testingr>ZlinspaceZytZdlldloZdlldscZgrrrrrs   $  '          ((&&&0