h4_dZddlZddlZddlmZmZmZmZm Z m Z m Z m Z ddl Zddl mZddlmZddlmZddlmZddlmZddlmZdd lmZdd lmZddlZ dd l!m"Z"m#Z#d d l$m%Z%de&de&fdZ'de j(dee&de&ddfdZ)de j*dee&de&ddfdZ+de#edfdee&de&ddfdZ,de#edfdee&de&ddfdZ-GddeZ. d=ded e"ed!fd"e e&d#ee e&ee&fd$eed%ee e/e/fddfd&Z0d'Z1d(d)d*d+d,Z2ed-.Z3e1e2e3fd/ee&ee&e"ed0fffd1ee&ee&e e/e/fffd2e&d3e&d#ee&e&fd4eddfd5Z4d6ee&ee&e e/e5fffde&ddfd7Z6d8e j*dee&d9e&d:e7d;e7ddf d<Z8dS)>a{Visualization module for generating plots and reports for Fi-NeMo results. This module provides functions for: - Plotting motif contribution weight matrices (CWMs) as sequence logos - Generating distribution plots for hit statistics - Creating co-occurrence heatmaps - Producing HTML reports with interactive visualizations - Plotting confusion matrices and performance metrics N)ListOptionalDictAnyTupleUnionMappingIterable)ndarray)Axes)AbstractPathEffect)TextPath)Affine2D)FontProperties)Template)FloatInt) templatesnamereturnc |d\}}|dkrd}n|dkrd}nt|dd}|d|S#t$r|cYSwxYw) a`Convert TF-MoDISco motif names to abbreviated format. Converts full TF-MoDISco pattern names to shorter, more readable format for display in plots and reports. Parameters ---------- name : str Full motif name (e.g., 'pos_patterns.pattern_0'). Returns ------- str Abbreviated name (e.g., '+/0') or original name if parsing fails. Examples -------- >>> abbreviate_motif_name('pos_patterns.pattern_0') '+/0' >>> abbreviate_motif_name('neg_patterns.pattern_1') '-/1' >>> abbreviate_motif_name('invalid_name') 'invalid_name' . pos_patterns+ neg_patterns-_r/)split Exception)rgroupmotif group_short motif_nums ;/srv/www/kundaje/kobbad/Fi-NeMo/src/finemo/visualization.pyabbreviate_motif_namer's2 zz# u N " "KK n $ $KKOKK$$Q' ++ +++  sAA A$#A$hits_df motif_namesplot_dircJ|}|dd}|}tj|d}tj|d|D])}||f|}|d } |d } |d } tj d \} } | | d d n(#t$r| | d d YnwxYwtj||d}tj|dtj||d}tj|tj| tj d \} } | | d d n(#t$r| | d d YnwxYwtj||d}tj|dtj||d}tj|tj| tj d \} } | | d d n(#t$r| | d d YnwxYwtj||d}tj|dtj||d}tj|tj| +dS)aPlot distributions of hit statistics for each motif. Creates separate histogram plots for coefficient, similarity, and importance score distributions for each motif. Saves plots in both PNG (high-res) and SVG (vector) formats. Parameters ---------- hits_df : pl.LazyFrame Lazy DataFrame containing hit data with required columns: - motif_name : str, name of the motif - hit_coefficient_global : float, global coefficient values - hit_similarity : float, similarity scores to motif CWM - hit_importance : float, importance scores from attribution motif_names : List[str] List of motif names to generate plots for. Motifs not present in hits_df will result in empty histograms. plot_dir : str Directory path where plots will be saved. Creates subdirectory 'motif_stat_distributions' if it doesn't exist. Notes ----- For each motif, creates three separate plots: - {motif_name}_coefficients.{png,svg} : coefficient distribution - {motif_name}_similarities.{png,svg} : similarity distribution - {motif_name}_importances.{png,svg} : importance distribution motif_nameT)as_dictmotif_stat_distributionsexist_okhit_coefficient_globalhit_similarityhit_importancefigsize2)binsdensityrz_coefficients.png,dpiz_coefficients.svgz_similarities.pngz_similarities.svgz_importances.pngz_importances.svgN)collect partition_byclearospathjoinmakedirsget get_columnto_numpypltsubplotshist ValueErrorsavefigclose)r(r)r*hits_df_collected hits_by_motifdummy_df motifs_dirmhits coefficients similarities importancesfigaxoutput_path_pngoutput_path_svgs r&plot_hit_stat_distributionsr\Es> ))%22<2NNM &&((Hh(BCCJK T**** ..  !x00'?@@IIKK '788AACC oo&677@@BB ,v...R 8 GGLr4G 8 8 8 8 8 8 8 GGLq$G 7 7 7 7 7 8',,za3J3J3JKK O----',,za3J3J3JKK O$$$ #,v...R 8 GGLr4G 8 8 8 8 8 8 8 GGLq$G 7 7 7 7 7 8',,za3J3J3JKK O----',,za3J3J3JKK O$$$ #,v...R 7 GGKb$G 7 7 7 7 7 7 7 GGKaG 6 6 6 6 6 7',,za3I3I3IJJ O----',,za3I3I3IJJ O$$$ #]..s6 D99"EE>H"H<;H<K55"LLocc_dfctj|d}tj|d|D]>}t jd\}}t j||d\}}|| z } t j |dd z} t j | } t j | } | | |<| | | tj||d } t j| d tj||d }t j|t j|@t jd\}}t j|dd\}}|| z } t j |dd z} t j | } t j | } | | |<| | | |d|dtj|d} t j| d tj|d}t j|d t j|dS)aPlot distribution of hits per peak for each motif. Creates bar plots showing the frequency distribution of hit counts per peak for each motif, plus an overall distribution of total hits per peak. Parameters ---------- occ_df : pl.DataFrame DataFrame containing motif occurrence counts per peak. Expected to have: - One column per motif name with integer hit counts - 'total' column with sum of all motif hits per peak - Each row represents a peak/genomic region motif_names : List[str] List of motif names corresponding to columns in occ_df. plot_dir : str Directory to save plots. Creates 'motif_hit_distributions' subdirectory. Notes ----- Generates the following plots: - Individual motif hit distributions: {motif_name}.{png,svg} - Overall hit distribution: total_hit_distribution.{png,svg} Bar plots show frequency (proportion) on y-axis and hit count on x-axis. motif_hit_distributionsTr/r4r7) return_countsr)initialr.pngr<r=.svg)totalzTotal hits per region Frequencyztotal_hit_distribution.pngztotal_hit_distribution.svgN)rBrCrDrErIrJnpuniquerGsumamaxarangezerosbarrMrN set_xlabel set_ylabel)r]r)r*rRrSrXrYricountsfreqnum_binsxyrZr[s r&plot_hit_peak_distributionsrvsr8h(ABBJK T**** ,v...R6#4#4Q#7#7tLLL $761---1 Ih   HX  &  q! ',,za:::>> O----',,za:::>> O$$$ #l6***GCYv0099NNNNFF FJJLL Dwvq)))A-H (A AAfIFF1aLLLMM)***MM+gll8-IJJOKS))))gll8-IJJOKS))))IcNNNNNpeak_hit_countszM M output_dircdtjtj|z }||dddfz|dddfz}d|D}tjdd\}}||ddd }|tjt|| || tjt|| |d | d | d|dd|||ddd} | jdt"j|d} tj| dt"j|d} tj| dtjdS)a=Plot co-occurrence heatmap showing motif associations across peaks. Creates a normalized correlation heatmap showing how frequently pairs of motifs co-occur within the same genomic peaks. Values are normalized by the geometric mean of individual motif frequencies. Parameters ---------- peak_hit_counts : Int[ndarray, "M M"] Co-occurrence matrix where M = len(motif_names). Entry (i,j) represents the number of peaks containing both motif i and j. Diagonal entries represent total peaks containing each individual motif. motif_names : List[str] List of motif names for axis labels. Order must match matrix dimensions. output_dir : str Directory path where the heatmap plots will be saved. Notes ----- Saves plots as: - motif_cooocurrence.png : High-resolution raster format - motif_cooocurrence.svg : Vector format The heatmap uses correlation normalization: matrix[i,j] / sqrt(matrix[i,i] * matrix[j,j]) Colors use the 'Greens' colormap with values typically in [0, 1] range. rNc,g|]}t|Sr'.0rSs r& z5plot_peak_motif_indicator_heatmap..!@@@q'**@@@rwrdrd constrainedr8layoutnearestequalGreens interpolationaspectcmapZrotationzMotif izMotif jbothrdaxis labelsizevertical333333?rY orientationshrinkrrzmotif_cooocurrence.pngr<r=zmotif_cooocurrence.svg)rhsqrtdiagrIrJimshow set_yticksrllenset_yticklabels set_xticksset_xticklabelsrorp tick_paramscolorbarrYrBrCrDrMrN) rxr)rycov_normmatrix motif_keysrXrYcaxcbarrZr[s r&!plot_peak_motif_indicator_heatmaprs:2727?33444H x40 08D!!!G3D DF@@K@@@Jl6-@@@GC ))F)G() S SCMM")C OO,,---z"""MM")C OO,,---zB///MM)MM)NN!N,,, << 3r< R RDG!$$$gll:/GHHOKS))))gll:/GHHOKS))))IKKKKKrwseqlet_confusioncd|D}tjdd\}}||ddd}|t jt ||||t jt || |d | d | d | d d| ||ddd}|j dtj|d}tj|dtj|d} tj| dtjdS)aPlot confusion matrix heatmap comparing seqlets to hit calls. Creates a heatmap showing the overlap between TF-MoDISco seqlets and Fi-NeMo hit calls. Rows represent seqlet motifs, columns represent hit motifs. Parameters ---------- seqlet_confusion : Int[ndarray, "M M"] Confusion matrix where M = len(motif_names). Entry (i,j) represents the number of seqlets of motif i that overlap with hits called for motif j. motif_names : List[str] List of motif names for axis labels. Order must match matrix dimensions. output_dir : str Directory path where the confusion matrix plots will be saved. Notes ----- Saves plots as: - seqlet_confusion.png : High-resolution raster format - seqlet_confusion.svg : Vector format The heatmap uses 'Blues' colormap. Perfect agreement would show a diagonal pattern with high values along the diagonal and low off-diagonal values. c,g|]}t|Sr|r}r~s r&rz1plot_seqlet_confusion_heatmap..8rrwrrrrrBluesrrrz Hit motifz Seqlet motifrrdrrrrrrzseqlet_confusion.pngr<r=zseqlet_confusion.svgN)rIrJrrrhrlrrrrrorprrrYrBrCrDrMrN) rr)ryrrXrYrrrZr[s r&plot_seqlet_confusion_heatmaprs8A@K@@@Jl6-@@@GC )) '   C MM")C OO,,---z"""MM")C OO,,---zB///MM+MM.!!!NN!N,,, << 3r< R RDG!$$$gll:/EFFOKS))))gll:/EFFOKS))))IKKKKKrwc heZdZdZ d dededeedeeefd df fd Z d e fd Z xZ S) LogoGlyphaPath effect for creating sequence logo glyphs with normalized dimensions. This class creates properly scaled and positioned text glyphs for sequence logos by normalizing character dimensions and applying appropriate transforms. Parameters ---------- glyph : str Single character to render (e.g., 'A', 'C', 'G', 'T'). ref_glyph : str, default 'E' Reference character used for width normalization. font_props : FontProperties, optional Font properties for the glyph rendering. offset : Tuple[float, float], default (0., 0.) Offset for glyph positioning. **kwargs Additional graphics collection parameters. ENrglyph ref_glyph font_propsoffsetrc Pt|td|d|}|}td|d|}d|jz } t |j|j} d| z } d|j| z z dz } |j } |j }t | | | |  | d}| ||_ ||_dS)Nrrr)sizepropr6)txty)sxsyr)super__init__r get_extentsheightmaxwidthx0y0r translatescaletransform_pathrC_gc)selfrrrrkwargs path_origdimsref_dimsh_scale ref_widthw_scalew_shiftx_shifty_shiftstretch __class__s r&rzLogoGlyph.__init__js    VUDDD $$&&FIAJGGGSSUUdk/ HN33 i-tzI--27(7( JJ Y'gY . . Ug'U * * Y'aY ( ( **955 rwc<|||j||S)a,Draw the glyph path using the renderer. Parameters ---------- renderer : matplotlib renderer The renderer to draw with. gc : GraphicsContext Graphics context for drawing properties. tpath : Path Original text path (unused, using self.path instead). affine : Transform Affine transformation to apply. rgbFace : color Face color for the glyph. Returns ------- Any Result from renderer.draw_path. ) draw_pathrC)rrenderergctpathaffinergbFaces r&rzLogoGlyph.draw_paths *!!"diAAArw)rNr) __name__ __module__ __qualname____doc__strrrrfloatrrr __classcell__)rs@r&rrVs,/3&0 ^,  eUl#  @BBBBBBBBBrwrrYheightszB Wglyphscolorsr shade_boundsc | d|D}|ddtj|dd}tj|dd}tj|d}tj|ddddddf} tj|d} tj| d} tjtjtj||dd| d} tjtjtj|| dd| d} | | z|z }tj|jd}t|||D]7\}}}| ||d|t|| g|| 8|$|\}}| |d z |d z d d | dd ddS)a}Plot sequence logo from contribution weight matrix. Creates a sequence logo visualization where letter heights represent the contribution or information content at each position. Supports both positive and negative contributions with proper stacking. Parameters ---------- ax : Axes Matplotlib axes object to plot on. heights : Float[ndarray, "B W"] Height matrix where B = len(glyphs) and W = motif width. Entry (i,j) represents the height/contribution of base i at position j. Can contain both positive and negative values. glyphs : Iterable[str] Sequence of base characters corresponding to rows in heights matrix. Typically ['A', 'C', 'G', 'T'] for DNA. colors : Dict[str, str], optional Color mapping for each base. Keys should match glyphs. If None, all bases will use default matplotlib colors. font_props : FontProperties, optional Font properties for letter rendering. If None, uses default font. shade_bounds : Tuple[int, int], optional (start, end) position indices to shade in background. Useful for highlighting core motif regions. Notes ----- Positive and negative contributions are handled separately: - Positive values are stacked above zero line in order of descending absolute value - Negative values are stacked below zero line in order of descending absolute value - A horizontal line is drawn at y=0 for reference The resulting plot has: - X-axis: Position in motif (0-indexed) - Y-axis: Contribution magnitude - Bar width: 0.95 (small gaps between positions) Nci|]}|dS)Nr|)rgs r& zplot_logo..s***a!T***rwr)rtru)rrgffffff?)r)bottom path_effectscolorg?z0.9)rzorderblack)r linewidthr) marginsrhclipargsorttake_along_axiscumsumrlshapeziprnraxvspanaxhline)rYrrrrr pos_values neg_values pos_order neg_order pos_reorder neg_reorder pos_offsets neg_offsetsbottomsrtrrrstartends r& plot_logor s \~**6***JJaJ!T**J$**J :A...I :A...tttQQQw7I*YQ///K*YQ///K$ "$ZCCC!LLL K $ "$ZCCC!LLL K K''1G '-"##A!$VWg!>!>  vv   #EjAAAB-     ! s 53;c r BBBJJbCwJ77777rwACGTz#109648z#255C99z#F7B32Bz#D62839)ACGTbold)weightmotifsz4 W trim_boundsout_diralphabetfontc |D]G\}}tj||}tj|d|D]\} } t jd\} } t| | |||||| | jD]\} }| dtj|| d}t j |d tj|| d }t j |t j | Id S) uPlot contribution weight matrices as sequence logos. Creates sequence logo plots for all motifs and CWM types, with optional shading to highlight trimmed regions. Saves plots in both PNG and SVG formats. Parameters ---------- motifs : Dict[str, Dict[str, Float[ndarray, "4 W"]]] Nested dictionary structure: {motif_name: {motif_type: motif_array}}. Each motif_array has shape (4, W) where W is motif width. Rows correspond to bases in alphabet order. trim_bounds : Dict[str, Dict[str, Tuple[int, int]]] Nested dictionary: {motif_name: {motif_type: (start, end)}}. Defines regions to shade in the sequence logos. out_dir : str Output directory where motif subdirectories will be created. alphabet : str, default LOGO_ALPHABET DNA alphabet string, typically 'ACGT'. colors : Dict[str, str], default LOGO_COLORS Color mapping for DNA bases. Keys should match alphabet characters. font : FontProperties, default LOGO_FONT Font properties for sequence logo rendering. Notes ----- Directory structure created: ``` out_dir/ ├── motif1/ │ ├── motif_type1.png │ ├── motif_type1.svg │ └── ... └── motif2/ └── ... ``` Each plot is 10x2 inches with trimmed regions shaded if specified. Spines (plot borders) are hidden for cleaner appearance. Tr/) r6r7)rrrFrbdr=rcN) itemsrBrCrDrErIrJr spines set_visiblerMrN)rrrrrrrSv motif_dir motif_typer#rXrYrspinerZr[s r& plot_motifsr"se^ 1GLL!,,  I----!"   Jl7333GC (^J7      "y00 ) ) e!!%(((( gll96I6I6IJJO KS 1 1 1 1 gll96I6I6IJJO K ( ( ( IcNNNN) rw recall_datacg}g}g}|D]P\}}||d||d||Qttj|tj|}t jdd\}} | d||fddd | ||d t|D];\} } t| } | | || || fd d<| d| d| d| dt j|d} t j| dt j|d}t j|t jdS)aPlot scatter plot comparing hit counts to seqlet counts per motif. Creates a log-log scatter plot showing the relationship between the number of hits called by Fi-NeMo and the number of seqlets identified by TF-MoDISco for each motif. Includes diagonal reference line and motif annotations. Parameters ---------- recall_data : Dict[str, Dict[str, Union[int, float]]] Dictionary with motif names as keys and metrics dictionaries as values. Each metrics dictionary must contain: - 'num_hits_total' : int, total number of hits for the motif - 'num_seqlets' : int, total number of seqlets for the motif output_dir : str Directory path where the scatter plot will be saved. Notes ----- Saves plots as: - hit_vs_seqlet_counts.png : High-resolution raster format - hit_vs_seqlet_counts.svg : Vector format Plot features: - Log-log scale on both axes - Diagonal reference line (y = x) as dashed line - Points annotated with abbreviated motif names num_hits_total num_seqletsrrrrz0.3gffffff?)r)r5r5)rr linestyler5)srdr)fontsizerlogzHits per motifzSeqlets per motifzhit_vs_seqlet_counts.pngr<r=zhit_vs_seqlet_counts.svgN)rappendrrhrkrIrJaxlinescatter enumerater'annotate set_yscale set_xscalerorprBrCrDrMrN)r#ryrtrurSkrlimrXrYitxtshortrZr[s r&plot_hit_vs_seqlet_countsr7Hs< A A A!!##1 #$%%% =!"""  bgajj"'!** % %Cl6-@@@GCIIfsCj ITTTJJq!qJA,,DD3%c** EAaD!A$rds MMMMMMMMMMMMMMMMMMMM 555555((((((******222222 $$$$$$NS \S(,S S=@S SSSSlC LC'+CyC "U8 SMU8 WS(3-/0 1 U8 ( U8 5c?+ U8 U8U8U8U8p IIINN N& ) ) ) "($ FF d3gun 5566 7Fc4U38_ 4556FF F cN F  F FFFFR:c4U3:%6 6778:FI: ::::z7|7c77 7  7  777777rw