B &]\\@sdZddlmZddlZddlmZddlmZddl m Z m Z m Z m Z mZdd lmZmZeejjZd d Zed ed ed dZddZddZd"ddZdddej ejfdddifddZddZddZddZej ejfdifd d!Z dS)#z'Routines for numerical differentiation.)divisionN)norm)LinearOperator)issparse csc_matrix csr_matrix coo_matrixfind) group_dense group_sparsecCs|dkrtj|td}n*|dkr:t|}tj|td}ntdt|tj k|tjk@rf||fS||}|}||} ||} |dkr||} | |k| |kB} t|t | | k} || | @d9<| | k| @}| ||||<| | k| @}| | |||<n|dkr| |k| |k@}| | k|@}t ||d| ||||<d||<| | k|@}t ||d| || ||<d||<t | | |}|t||k@}||||<d||<||fS) aAdjust final difference scheme to the presence of bounds. Parameters ---------- x0 : ndarray, shape (n,) Point at which we wish to estimate derivative. h : ndarray, shape (n,) Desired finite difference steps. num_steps : int Number of `h` steps in one direction required to implement finite difference scheme. For example, 2 means that we need to evaluate f(x0 + 2 * h) or f(x0 - 2 * h) scheme : {'1-sided', '2-sided'} Whether steps in one or both directions are required. In other words '1-sided' applies to forward and backward schemes, '2-sided' applies to center schemes. lb : ndarray, shape (n,) Lower bounds on independent variables. ub : ndarray, shape (n,) Upper bounds on independent variables. Returns ------- h_adjusted : ndarray, shape (n,) Adjusted step sizes. Step size decreases only if a sign flip or switching to one-sided scheme doesn't allow to take a full step. use_one_sided : ndarray of bool, shape (n,) Whether to switch to one-sided scheme. Informative only for ``scheme='2-sided'``. z1-sided)dtypez2-sidedz(`scheme` must be '1-sided' or '2-sided'.g?TF) npZ ones_likeboolabs zeros_like ValueErrorallinfcopymaximumZminimum)x0hZ num_stepsZschemelbub use_one_sidedZh_totalZ h_adjustedZ lower_distZ upper_distxZviolatedZfittingZforwardZbackwardZcentralZmin_distZadjusted_centralr6lib/python3.7/site-packages/scipy/optimize/_numdiff.py_adjust_scheme_to_boundssH    r!g?gUUUUUU?)z2-pointz3-pointcscCs@|dkrt|}|dktdd}||tdt|S)Nrrr g?) relative_stepastypefloatrrr)rel_steprmethodZsign_x0rrr _compute_absolute_stepcsr(cCsJdd|D\}}|jdkr*t||j}|jdkrBt||j}||fS)NcSsg|]}tj|tdqS))r)rasarrayr%).0brrr ksz#_prepare_bounds..r)ndimrZresizeshape)boundsrrrrrr _prepare_boundsjs   r0cCst|rt|}nt|}|dktj}|jdkr>td|j\}}|dksZt |rrtj |}| |}nt |}|j|fkrtd|dd|f}t|rt|||j|j}n t|||}|||<|S)aGroup columns of a 2-d matrix for sparse finite differencing [1]_. Two columns are in the same group if in each row at least one of them has zero. A greedy sequential algorithm is used to construct groups. Parameters ---------- A : array_like or sparse matrix, shape (m, n) Matrix of which to group columns. order : int, iterable of int with shape (n,) or None Permutation array which defines the order of columns enumeration. If int or None, a random permutation is used with `order` used as a random seed. Default is 0, that is use a random permutation but guarantee repeatability. Returns ------- groups : ndarray of int, shape (n,) Contains values from 0 to n_groups-1, where n_groups is the number of found groups. Each value ``groups[i]`` is an index of a group to which i-th column assigned. The procedure was helpful only if n_groups is significantly less than n. References ---------- .. [1] A. Curtis, M. J. D. Powell, and J. Reid, "On the estimation of sparse Jacobian matrices", Journal of the Institute of Mathematics and its Applications, 13 (1974), pp. 117-120. rrz`A` must be 2-dimensional.Nz`order` has incorrect shape.)rrr atleast_2dr$Zint32r-rr.ZisscalarZrandomZ RandomStateZ permutationr)r indicesZindptrr r)Aordermnrnggroupsrrr group_columnsus&          r9z3-pointFrc  s|dkrtd|t|}|jdkr0tdt||\} } | j|jksV| j|jkr^td|rtt| rtt| stdfdd} |d kr| |}nt|}|jdkrtd t|| k|| kBrtd |r |d krt |}t | ||||St |||} |d kr:t || dd | | \} }n0|dkr\t || dd| | \} }n|dkrjd}|d krt | ||| ||St|st|dkr|\}}n |}t|}t|rt|}n t|}t|}t| ||| ||||Sd S)aVCompute finite difference approximation of the derivatives of a vector-valued function. If a function maps from R^n to R^m, its derivatives form m-by-n matrix called the Jacobian, where an element (i, j) is a partial derivative of f[i] with respect to x[j]. Parameters ---------- fun : callable Function of which to estimate the derivatives. The argument x passed to this function is ndarray of shape (n,) (never a scalar even if n=1). It must return 1-d array_like of shape (m,) or a scalar. x0 : array_like of shape (n,) or float Point at which to estimate the derivatives. Float will be converted to a 1-d array. method : {'3-point', '2-point', 'cs'}, optional Finite difference method to use: - '2-point' - use the first order accuracy forward or backward difference. - '3-point' - use central difference in interior points and the second order accuracy forward or backward difference near the boundary. - 'cs' - use a complex-step finite difference scheme. This assumes that the user function is real-valued and can be analytically continued to the complex plane. Otherwise, produces bogus results. rel_step : None or array_like, optional Relative step size to use. The absolute step size is computed as ``h = rel_step * sign(x0) * max(1, abs(x0))``, possibly adjusted to fit into the bounds. For ``method='3-point'`` the sign of `h` is ignored. If None (default) then step is selected automatically, see Notes. f0 : None or array_like, optional If not None it is assumed to be equal to ``fun(x0)``, in this case the ``fun(x0)`` is not called. Default is None. bounds : tuple of array_like, optional Lower and upper bounds on independent variables. Defaults to no bounds. Each bound must match the size of `x0` or be a scalar, in the latter case the bound will be the same for all variables. Use it to limit the range of function evaluation. Bounds checking is not implemented when `as_linear_operator` is True. sparsity : {None, array_like, sparse matrix, 2-tuple}, optional Defines a sparsity structure of the Jacobian matrix. If the Jacobian matrix is known to have only few non-zero elements in each row, then it's possible to estimate its several columns by a single function evaluation [3]_. To perform such economic computations two ingredients are required: * structure : array_like or sparse matrix of shape (m, n). A zero element means that a corresponding element of the Jacobian identically equals to zero. * groups : array_like of shape (n,). A column grouping for a given sparsity structure, use `group_columns` to obtain it. A single array or a sparse matrix is interpreted as a sparsity structure, and groups are computed inside the function. A tuple is interpreted as (structure, groups). If None (default), a standard dense differencing will be used. Note, that sparse differencing makes sense only for large Jacobian matrices where each row contains few non-zero elements. as_linear_operator : bool, optional When True the function returns an `scipy.sparse.linalg.LinearOperator`. Otherwise it returns a dense array or a sparse matrix depending on `sparsity`. The linear operator provides an efficient way of computing ``J.dot(p)`` for any vector ``p`` of shape (n,), but does not allow direct access to individual elements of the matrix. By default `as_linear_operator` is False. args, kwargs : tuple and dict, optional Additional arguments passed to `fun`. Both empty by default. The calling signature is ``fun(x, *args, **kwargs)``. Returns ------- J : {ndarray, sparse matrix, LinearOperator} Finite difference approximation of the Jacobian matrix. If `as_linear_operator` is True returns a LinearOperator with shape (m, n). Otherwise it returns a dense array or sparse matrix depending on how `sparsity` is defined. If `sparsity` is None then a ndarray with shape (m, n) is returned. If `sparsity` is not None returns a csr_matrix with shape (m, n). For sparse matrices and linear operators it is always returned as a 2-dimensional structure, for ndarrays, if m=1 it is returned as a 1-dimensional gradient array with shape (n,). See Also -------- check_derivative : Check correctness of a function computing derivatives. Notes ----- If `rel_step` is not provided, it assigned to ``EPS**(1/s)``, where EPS is machine epsilon for float64 numbers, s=2 for '2-point' method and s=3 for '3-point' method. Such relative step approximately minimizes a sum of truncation and round-off errors, see [1]_. A finite difference scheme for '3-point' method is selected automatically. The well-known central difference scheme is used for points sufficiently far from the boundary, and 3-point forward or backward scheme is used for points near the boundary. Both schemes have the second-order accuracy in terms of Taylor expansion. Refer to [2]_ for the formulas of 3-point forward and backward difference schemes. For dense differencing when m=1 Jacobian is returned with a shape (n,), on the other hand when n=1 Jacobian is returned with a shape (m, 1). Our motivation is the following: a) It handles a case of gradient computation (m=1) in a conventional way. b) It clearly separates these two different cases. b) In all cases np.atleast_2d can be called to get 2-d Jacobian with correct dimensions. References ---------- .. [1] W. H. Press et. al. "Numerical Recipes. The Art of Scientific Computing. 3rd edition", sec. 5.7. .. [2] A. Curtis, M. J. D. Powell, and J. Reid, "On the estimation of sparse Jacobian matrices", Journal of the Institute of Mathematics and its Applications, 13 (1974), pp. 117-120. .. [3] B. Fornberg, "Generation of Finite Difference Formulas on Arbitrarily Spaced Grids", Mathematics of Computation 51, 1988. Examples -------- >>> import numpy as np >>> from scipy.optimize import approx_derivative >>> >>> def f(x, c1, c2): ... return np.array([x[0] * np.sin(c1 * x[1]), ... x[0] * np.cos(c2 * x[1])]) ... >>> x0 = np.array([1.0, 0.5 * np.pi]) >>> approx_derivative(f, x0, args=(1, 2)) array([[ 1., 0.], [-1., 0.]]) Bounds can be used to limit the region of function evaluation. In the example below we compute left and right derivative at point 1.0. >>> def g(x): ... return x**2 if x >= 1 else x ... >>> x0 = 1.0 >>> approx_derivative(g, x0, bounds=(-np.inf, 1.0)) array([ 1.]) >>> approx_derivative(g, x0, bounds=(1.0, np.inf)) array([ 2.]) )z2-pointz3-pointr"zUnknown method '%s'. r z#`x0` must have at most 1 dimension.z,Inconsistent shapes between bounds and `x0`.z7Bounds not supported when `as_linear_operator` is True.cs,t|f}|jdkr(td|S)Nr z-`fun` return value has more than 1 dimension.)r atleast_1dr- RuntimeError)rf)argsfunkwargsrr fun_wrapped[s z&approx_derivative..fun_wrappedNz&`f0` passed has more than 1 dimension.z `x0` violates bound constraints.z2-pointz1-sidedz3-pointz2-sidedr"Fr)rrr:r-r0r.rZisinfanyr#_linear_operator_differencer(r!_dense_differencerlenr9rr1_sparse_difference)r>rr'r&f0r/sparsityZas_linear_operatorr=r?rrr@rr structurer8r)r=r>r?r approx_derivatives`                   rIcsxjj}|dkr*fdd}n@|dkrFfdd}n$|dkrbfdd}ntdt|f|S) Nz2-pointcsHt|t|rtSt|}||}|}||S)N)r array_equalrzerosr)pdxrdf)rFr>rr5rrr matvecs     z+_linear_operator_difference..matvecz3-pointcslt|t|rtSdt|}|d|}|d|}|}|}||}||S)Nr)rrJrrKr)rLrMx1x2f1f2rN)r>rr5rrr rOs r"csNt|t|rtSt|}||d}|}|j}||S)Ny?)rrJrrKrimag)rLrMrrRrN)r>rr5rrr rOs  zNever be here.)sizer;r)r>rrFrr'r6rOr)rFr>rr5rr rBs  rBcCs|j}|j}t||f}t|} xDt|jD]4} |dkrj|| | } | | || } || |} n|dkr|| r|| | }|d| | }|| || } ||}||}d|d||} n|dkr"|| s"|| | }|| | }|| || } ||}||}||} n:|dkrT||| | d}|j} | | | f} ntd| | || <q2W|d krt|}|jS) Nz2-pointz3-pointrgr"y?zNever be here.r ) rUremptyZdiagrangerTr;ravelT)r>rrFrrr'r5r6Z J_transposedZh_vecsirrMrNrPrQrRrSrrr rCs@         rCc!Cs|j}|j} g} g} g} t|d} xPt| D]B}t||}||}|dkr||}||}|||}t|\}t|dd|f\}}}||}n|dkr|}|}||@}||||7<||d||7<||@}||||8<||||7<t| }||||||<||||||<||}||}t|\}t|dd|f\}}}||}||}t |}||}d||d||||||<||}||||||<n\|dkrD|||d}|j }|}t|\}t|dd|f\}}}||}nt d | || || ||||q2Wt | } t | } t | } t| | | ff|| fd } t| S) Nr z2-pointz3-pointrrVr"y?zNever be here.)r.)rUrmaxrXZequalZnonzeror rrKrWrTrappendZhstackr r)!r>rrFrrrHr8r'r5r6Z row_indicesZ col_indicesZ fractionsZn_groupsgroupeZh_vecrrMrNZcolsr[j_rPrQZmask_1Zmask_2rRrSmaskZrowsJrrr rEsn         $         rEc Cs||f||}t|rt||||||d}t|}||}t|\} } } t|| | f} tt| t dt| St|||||d}t||}t|t dt|SdS)aK Check correctness of a function computing derivatives (Jacobian or gradient) by comparison with a finite difference approximation. Parameters ---------- fun : callable Function of which to estimate the derivatives. The argument x passed to this function is ndarray of shape (n,) (never a scalar even if n=1). It must return 1-d array_like of shape (m,) or a scalar. jac : callable Function which computes Jacobian matrix of `fun`. It must work with argument x the same way as `fun`. The return value must be array_like or sparse matrix with an appropriate shape. x0 : array_like of shape (n,) or float Point at which to estimate the derivatives. Float will be converted to 1-d array. bounds : 2-tuple of array_like, optional Lower and upper bounds on independent variables. Defaults to no bounds. Each bound must match the size of `x0` or be a scalar, in the latter case the bound will be the same for all variables. Use it to limit the range of function evaluation. args, kwargs : tuple and dict, optional Additional arguments passed to `fun` and `jac`. Both empty by default. The calling signature is ``fun(x, *args, **kwargs)`` and the same for `jac`. Returns ------- accuracy : float The maximum among all relative errors for elements with absolute values higher than 1 and absolute errors for elements with absolute values less or equal than 1. If `accuracy` is on the order of 1e-6 or lower, then it is likely that your `jac` implementation is correct. See Also -------- approx_derivative : Compute finite difference approximation of derivative. Examples -------- >>> import numpy as np >>> from scipy.optimize import check_derivative >>> >>> >>> def f(x, c1, c2): ... return np.array([x[0] * np.sin(c1 * x[1]), ... x[0] * np.cos(c2 * x[1])]) ... >>> def jac(x, c1, c2): ... return np.array([ ... [np.sin(c1 * x[1]), c1 * x[0] * np.cos(c1 * x[1])], ... [np.cos(c2 * x[1]), -c2 * x[0] * np.sin(c2 * x[1])] ... ]) ... >>> >>> x0 = np.array([1.0, 0.5 * np.pi]) >>> check_derivative(f, jac, x0, args=(1, 2)) 2.4492935982947064e-16 )r/rGr=r?r )r/r=r?N) rrIrr rr)rYr]rr) r>Zjacrr/r=r?Z J_to_testZJ_diffZabs_errr[raZ abs_err_dataZ J_diff_datarrr check_derivative4s=    re)r)!__doc__Z __future__rZnumpyrZ numpy.linalgrZscipy.sparse.linalgrZsparserrrr r Z_group_columnsr r ZfinfoZfloat64ZepsZEPSr!r#r(r0r9rrIrBrCrErerrrr s.   O  =`)(P