U fJ@sddlZddlZddlmZddlmZddlmZda GdddeZ dd Z d d Z d d Z dgZddZgZddZddZddZddZiZeed<eed<eed<eed<eed<eed<eed<eed <eed!<eed"<eed#<eed$<eed%<eed&<eed'<eed(<eed)<eed*<eed+<eed,<eed-<eed.<eed/<eed0<eed1<eed2<eed3<eed4<eed<eed5<eed6<dS)7N) Explainer)!standard_combine_mult_and_diffref) LooseVersionc@sBeZdZefddZddZddZddZd d Zdd dZ d S)PyTorchDeepExplainerc Cstdkr,ddlattjtdkr,tdd|_t|drF|d}n|}t|tkr\|g}t|tkrnd|_t|tkr|g}||_ ||_ d|_ d|_ d|_ d|_d|_t|tkr>d|_ |\}}|}||_ ||j t@||}|j j}t|tkrdd |D|_n |jg|_W5QRX|j|j `||_d|_d |_t\||}|j|_|jd d krd|_|jd |_t|ddkr|d|_W5QRXdS) a Parameters ---------- model : an nn.Module object (model), or a tuple (model, layer), where both are nn.Module objects. The model is an nn.Module object which takes as input a tensor (or list of tensors) of shape data, and returns an output of shape (B, 1), where B is the batch dimension. Note that the second dimension must be 1. Otherwise, the gradient cannot be computed (since the output won't be a scalar). If the input is a tuple, the returned shap values will be for the input of the layer argument. layer must be a layer in the model, i.e. model.conv2 data : torch tensor or function If a function is supplied, it must be a function that takes a particular input example and generates the background dataset for that example. When None is supplied as an input, the function should return values that are in the same format as what it would return if an actual input were supplied. combine_mult_and_diffref : function This function determines how to combine the multipliers, the original input and the reference input to get the final attributions. Defaults to standard_combine_mult_and_diffref, which just multiplies the multipliers with the difference-from-reference (in accordance with the standard DeepLIFT formulation) and then averages the importance scores across the different references. However, different approaches may be applied depending on the use case (e.g. for computing hypothetical contributions in genomic data) Nrz0.4z9Your PyTorch version is older than 0.4 and not supported.F__call__TcSsg|] }|jqSshape.0irrH/tmp/pip-target-lpfmz8o1/lib/python/shap/explainers/deep/deep_pytorch.py Usz1PyTorchDeepExplainer.__init__..r)torchr __version__warningswarn multi_inputhasattrtypelistdatacombine_mult_and_diffreflayer input_handleinteriminterim_inputs_shapeZexpected_valuetupleevaladd_target_handleno_grad target_inputr target_handleremovemodel multi_output num_outputsdeviceZmeancpunumpy) selfr%rrZ sample_datar_interim_inputsoutputsrrr__init__ s\$            zPyTorchDeepExplainer.__init__cCs|t}||_dSN)register_forward_hookget_target_inputr#)r+rrrrrr ns z&PyTorchDeepExplainer.add_target_handlecCs\g}|D]J}dtt|kr6|||||q ||||||q |S)zt Add handles to all non-container layers in the model. Recursively for non-container layers nn.modules.container)childrenstrrextend add_handlesappendr1Zregister_backward_hook)r+r%Zforward_handleZbackward_handleZ handles_listchildrrrr7rs z PyTorchDeepExplainer.add_handlesc Csj|D]\}dtt|kr(||qz|`Wntk rDYnXz|`Wqtk rbYqXqdS)z Removes the x and y attributes which were added by the forward handles Recursively searches for non-container layers r3N)r4r5rremove_attributesxAttributeErrory)r+r%r9rrrr:s  z&PyTorchDeepExplainer.remove_attributescs|jdd|D}|j|}dd|dd|fD|jrt|jj}fdd|D}|j`|dd|DfSfdd|D}|SdS)NcSsg|] }|qSr)Zrequires_grad_r r;rrrrsz1PyTorchDeepExplainer.gradient..cSsg|]}|qSrr)r valrrrrscs*g|]"}tjj|dddqST)Z retain_graphrrZautogradgradr)r*)r inputselectedrrrscSsg|]}|qSrdetachr)r*r rrrrscs*g|]"}tjj|dddqSr@rAr>rDrrrs)r%Z zero_gradrrr")r+idxinputsXr.r-gradsrrDrgradients  zPyTorchDeepExplainer.gradientNmaxc s\js"ttkstdgnttks6tdfddD|dk rjrtj}W5QRX|dkrtj|dd\}}nJ|dkrtj|d d\}}n.|d krtjt |dd\}}n d std |ddd|f}n0t d j d j f td j  }jtt}jrDjg} t|j d D]} g} jrttjD]4} | td j d fj| d dqvn*ttD]} | t| j qtd j d D]|dk r*|d kr*tddd j d tjtj drp fddttDttkrvgnj fddttDfddttD} || f}!|| jrp\}gg}}tt|D].} t"|| d\}}||||qj#fddttjD||d}ttjD]}||| |<qVnnj#fddttDfddttDddDd}ttD]}||| |<qƐq| js| d n| qV|D]}|$q%jjr2j&$jsB| d S|dk rT| |fS| SdS)Nz%Expected a single tensor model input!z Expected a list of model inputs!csg|]}|jqSr)tor(r>)r+rrrsz4PyTorchDeepExplainer.shap_values..rMT)Z descendingminFZmax_absz/output_rank_order must be max, min, or max_abs!rrDonez examples ofrcsg|]}|qSrrr lrJjrrrsc sVg|]N}|d|jdftddtt|jdDqS)rrcSsg|]}dqSrr)r krrrrsz?PyTorchDeepExplainer.shap_values...)repeatr rrangelenrQ)rJbg_datarTrrrs2cs&g|]}tj||fddqS)r)dim)rcatrQ)rZtiled_Xrrrscs(g|] }|d|jd qSNrr rQrZ sample_phisrrrs)ZmultZorig_inprZcs(g|] }|d|jd qSr_r rQr`rrrscs$g|]}|qSrrFrQrSrrrscSsg|]}|qSrrFr>rrrrs)'rrrAssertionErrorr&rr!r%sortabsZonesr r'intZaranger7add_interim_values deeplift_gradrr rrXrYrr8npZzerosprintsysstdoutflushrrrLsplitrr$r:r#)r+rJZranked_outputsZoutput_rank_orderZprogress_messageZmodel_output_valuesr,Zmodel_output_ranksZhandlesZ output_phisr ZphisrVZjoint_xZ feature_indoutputr;rZx_tempZ data_tempZphis_jrRhandler)rJrZrTrar+r]r shap_valuess   4   "                z PyTorchDeepExplainer.shap_values)NrMN) __name__ __module__ __qualname__rr/r r7r:rLrprrrrr s crcCsJ|jj}|tkr0t|jdkrFt||||Sntd|d|SdS)zPThe backward hook which computes the deeplift gradient for an nn.Module ) passthrough linear_1dz#Warning: unrecognized nn.Module: {}z; using regular gradientsN) __class__rq op_handlerriformat)module grad_input grad_output module_typerrrrgs rgcCsLz|`Wntk rYnXz|`Wntk r:YnX|jj}|tkrHt|j}|dkrbntt|D]0}|dkrnt|t krn||||ksnt dqn|dkr0t|t krt |dt j |dnt |dt j |t|t krt |dt j |dnt |dt j ||tkrH|dtdS)zrThe forward hook used to save interim tensors, detached from the graph. Used to calculate the multipliers rtrzOnly the 0th input may vary!)maxpool nonlinear_1dr;r=N)r;r<r=rvrqrwrXrYrrrbsetattrrnn ParameterrGfailure_case_modules register_hookdeeplift_tensor_grad)ryrCrnr| func_namer rrrrf!s2     rfcCs.z|`Wntk rYnXt|d|dS)zA forward hook which saves the tensor - attached to its graph. Used if we want to explain the interim outputs of a model r"N)r"r<r)ryrCrnrrrr2Hs r2 MaxPool1dcCstd}td=|S)N)complex_module_gradients)rBZ return_gradrrrr]srcCsdS)zNo change made to gradientsNrryrzr{rrrrtfsrtc Cstjjjtjjjtjjjd}tjjjtjjjtjjjd}|j dt |j j dd|j t |j j ddd}dgdd|j ddD}t |j d\}}t||} t| ||| gd} tn||jj|j |j|j|j|j|jd\} } t ||jj|d| | |j|j|jt|j j d\} }W5QRXdd|D}tt|d kt|| ||||d<|jjd krt|dt|dd |  d|d<|` |` t!|S) N)r MaxPool2d MaxPool3drr^cSsg|]}dqSrUrr rrrrwszmaxpool..rTcSsg|]}dqSr0rr r,rrrrsgHz>rr)"rrZ functionalZ max_unpool1dZ max_unpool2dZ max_unpool3dZ max_pool1dZ max_pool2dZ max_pool3dr;rer chunkr=rMr\r!rvrqZ kernel_sizeZstridepaddingZdilationZ ceil_moderwhererdZ zeros_likerWrr8gatherZ unsqueezer)ryrzr{Zpool_to_unpoolZpool_to_functiondelta_indup0r=Z ref_outputZ cross_maxZdiffsr,indicesZxmax_posZrmax_posrrrr}ksZ<      r}cCsdS)zNo change made to gradients.NrrrrrrusrucCs|jdt|jjdd|jt|jjddd}|jdt|jjdd|jt|jjddd}dgdd|jddD}dd|D}tt||dk|d|d||||d<|`|`t|S)Nrr^cSsg|]}dqSrUrr rrrrsz nonlinear_1d..rcSsg|]}dqSr0rrrrrrsgư>) r=rer r;rrrdrWr)ryrzr{Z delta_outrrrKrrrr~s<<r~Z Dropout3dZ Dropout2dZDropoutZ AlphaDropoutZConv1dZConv2dZConv3dZConvTranspose1dZConvTranspose2dZConvTranspose3dZLinearZ AvgPool1dZ AvgPool2dZ AvgPool3dZAdaptiveAvgPool1dZAdaptiveAvgPool2dZAdaptiveAvgPool3dZ BatchNorm1dZ BatchNorm2dZ BatchNorm3dZ LeakyReLUZReLUZ ThresholdZELUZSigmoidZTanhZSoftplusZSoftmaxrr)r*rhrZshap.explainers.explainerrmiscrZdistutils.versionrrrrgrfr2rrrrtr}rur~rwrrrrsd    '&