API Reference

This page provides the technical documentation for all public components of TrustLens.

Core API

trustlens.analyze

trustlens.api.analyze(model: Any, X: ndarray, y_true: ndarray, y_pred: ndarray | None = None, y_prob: ndarray | None = None, *, framework: str | None = None, embeddings: ndarray | None = None, sensitive_features: dict[str, ndarray] | None = None, modules: list[str] | None = None, plugins: list[str] | None = None, verbose: bool = True) → TrustReport[source]

Run a full TrustLens analysis on a trained model.

Parameters:

  • model (Any, optional) – Trained machine learning model. Can be None if y_pred or y_prob are provided manually.

  • X (np.ndarray) – Validation feature matrix, shape (n_samples, n_features).

  • y_true (np.ndarray) – Ground-truth labels, shape (n_samples,).

  • y_pred (np.ndarray, optional) – Predicted class labels, shape (n_samples,). If None, TrustLens will automatically resolve predictions via the backend system.

  • y_prob (np.ndarray, optional) – Predicted class probabilities, shape (n_samples, n_classes). If None, TrustLens will automatically resolve probabilities via the backend system.

  • framework (str, optional) – Explicitly specify the model framework (e.g., ‘sklearn’, ‘xgboost’). If None, TrustLens will attempt to auto-detect the framework.

  • embeddings (np.ndarray, optional) – Latent representations / embeddings for representation analysis, shape (n_samples, embedding_dim).

  • sensitive_features (dict, optional) – Mapping of feature name → 1-D array for bias/subgroup analysis.

  • modules (list[str], optional) – Subset of analysis modules to run.

  • plugins (list[str], optional) – Names of registered plugins to activate.

  • verbose (bool) – Print progress updates. Default True.

Returns:

Populated report object with metrics, plots, and narrative summaries.

Return type:

TrustReport

trustlens.TrustReport

class trustlens.report.TrustReport(results: dict[str, Any], model: Any, X: ndarray, y_true: ndarray, y_pred: ndarray, y_prob: ndarray | None, embeddings: ndarray | None = None, framework: str | None = None, backend_metadata: dict[str, Any] | None = None)[source]

Bases: object

Container for all TrustLens analysis results.

results

Nested dictionary keyed by module name.

Type:

dict

trust_score

Composite 0–100 trust score with sub-scores and grade.

Type:

TrustScoreResult

metadata

Run-level metadata (timestamp, library version, etc.).

Type:

dict

model

Reference to the analysed model (not serialized to JSON).

Type:

Any

property patterns: list[str]

Detected behavioral patterns (e.g. ‘Confidently Wrong’).

show(verbose: bool = False) → None[source]

Print a rich, structured summary of all analysis results to stdout.

Displays the Trust Score prominently at the top, followed by key insights and then delimited per-module metric summaries.

summary_plot(save_path: str | None = None, show: bool = True)[source]

Render the TrustLens Summary Dashboard — a single-figure overview of the model’s trustworthiness.

Layout (2×3 grid):

Trust Score Gauge Reliability Diag Confidence Gap

Error Rate Dist. Class Dist. Sub-score Bars

Parameters:

  • save_path (str, optional) – If provided, saves the figure to this path (PNG or PDF).

  • show (bool) – If True, calls plt.show() for interactive display. Default True. Set to False in non-interactive environments.

Return type:

matplotlib.figure.Figure

show_failures(top_k: int = 10, images: ndarray | None = None, feature_names: list | None = None, save_path: str | None = None) → None[source]

Display the most alarming model failures — high-confidence wrong predictions that deserve immediate attention.

For each failure, reports: * Predicted class and confidence level * True class * A “danger rating” based on confidence level * Feature values (if feature_names provided)

Parameters:

  • top_k (int) – Number of top failures to show. Default 10.

  • images (np.ndarray, optional) – Image array shape (n_samples, H, W, C) or (n_samples, H, W). If provided, renders a grid of the most-confident wrong predictions.

  • feature_names (list[str], optional) – Column names for tabular features in self.X.

  • save_path (str, optional) – If provided and images is given, saves the failure grid as PNG.

Examples

>>> report.show_failures(top_k=10)
>>> report.show_failures(top_k=5, images=X_images)
plot(module: str | None = None, save_dir: str | None = None) → None[source]

Render per-module visualisations.

Parameters:

  • module (str, optional) – Which module to plot (e.g., "calibration"). If None, all available modules are plotted.

  • save_dir (str, optional) – Directory path where figures are saved as PNG files.

plot_embedding_2d(method: str = 'umap', n_max: int = 5000, save_path: str | None = None, show: bool = True)[source]

Project stored embeddings to 2D and render a class-colored scatter plot.

Delegates to the underlying plot_embedding_2d in the visualization sub-package, forwarding silhouette score (when available) so the plot is annotated automatically.

Guarantees: - Returns matplotlib.figure.Figure - Raises ValueError when embeddings were not supplied to analyze() - Projection falls back gracefully (UMAP -> t-SNE -> PCA) when optional libraries are missing

Parameters:

  • method (str) – Projection algorithm: "umap" (default), "tsne", or "pca".

  • n_max (int) – Max samples plotted. Subsampled randomly when the embedding matrix exceeds this limit.

  • save_path (str, optional) – File path to save the figure (e.g. "clusters.png").

  • show (bool) – Whether to display the figure interactively. Default True.

Return type:

matplotlib.figure.Figure

Raises:

ValueError – If embeddings are not available (i.e. not passed to analyze()).

plot_bias(mode: str = 'summary', show: bool = True, save_path: str | None = None, multi_feature: bool = False)[source]

Generate fairness/bias visualizations from report results.

mode and multi_feature are independent dimensions; the return shape is fully determined by their combination, with no flattening.

Guarantees

  • Return shape is fixed by the (mode, multi_feature) combination (see the table below). The structure never collapses across calls.

  • Multi-feature outputs never contain None values; missing components are represented by empty dicts {}.

  • Raises ValueError for invalid mode or unusable data, regardless of multi_feature.

Return shape by (mode, multi_feature):

mode

multi_feature

Return type

single mode (summary, subgroup, equalized_odds, gap)

False

Figure

summary

True

Figure (same as False)

subgroup, equalized_odds, gap

True

dict[str, Figure] keyed by feature name (sorted)

all

False

dict[str, Figure | None] keyed by mode (existing behavior)

all

True

dict[str, dict[str, Figure]] keyed by mode, then feature. Always contains exactly the keys ‘subgroup’, ‘equalized_odds’, ‘gap’ (in that order). Components with no data return {}.

Returned dict for mode="all" (regardless of multi_feature) ALWAYS contains exactly three keys in order: 'subgroup' -> 'equalized_odds' -> 'gap'. Within each multi-feature dict, feature order follows sorted(feature_names) for determinism.

Note: mode="all" returns a structured dict and does NOT display figures unless show=True.

Parameters:

  • mode (str, optional) – Visualization mode. One of {“summary”, “all”, “subgroup”, “equalized_odds”, “gap”}. Default “summary”.

  • show (bool) – Whether to display the figure interactively. Default True.

  • save_path (str, optional) – If provided, saves the figure(s) to this path. Only honored when multi_feature=False (single-feature behavior). - Single modes: Treated as full file path. Defaults to .png if extension missing. - mode="all": Treated as base name. Appends suffixes and .png. When multi_feature=True, save_path is ignored (per-feature saving is intentionally not exposed here; use the lower-level plot_*_multi helpers if you need on-disk output).

  • multi_feature (bool, optional) – If True, return per-feature figures rather than only the first sensitive feature. Default False (backward compatible).

Returns:

The return type depends on the (mode, multi_feature) combination. Possible shapes: - Figure - dict[str, Figure | None] - dict[str, Figure] - dict[str, dict[str, Figure]] See the return-shape table above.

Return type:

matplotlib.figure.Figure | dict

Notes

New modes can be added by extending ALLOWED_MODES and dispatch logic without breaking existing behavior. Function behavior MUST be deterministic given identical inputs (no randomness, no state mutation). This includes consistent plot ordering, consistent key ordering in dicts, and no randomness in visualization. Each returned Figure must be independent (no shared axes, state, or references between plots). Each plot must be independently renderable and savable.

Caution: Figures are returned open. If calling this method repeatedly in a loop, ensure you call plt.close(fig) on the returned figures to avoid memory accumulation.

Raises:

ValueError – If mode is invalid, "bias" is not present in self.results, or the data is unusable.

save(path: str = 'trust_report', **kwargs) → Path[source]

Save the analysis report.

If path ends with ‘.json’ or ‘.txt’, saves a single file. Otherwise, treats path as a directory and saves a full report bundle (JSON, metadata, plots).

Parameters:

  • path (str) – Target file path (e.g., “report.json”) or directory path.

  • **kwargs (Any) – Backward compatibility for directory argument.

Returns:

Resolved path to the saved file or directory.

Return type:

Path

to_dict() → dict[str, Any][source]

Return all results as a flat, JSON-serializable dictionary.

Useful for logging to MLflow, W&B, or any experiment tracker.

Returns:

Flat dict with keys like "calibration.brier_score".

Return type:

dict

trustlens.TrustScoreResult

class trustlens.trust_score.TrustScoreResult(score: int, grade: str, verdict: str, sub_scores: dict[str, float]=<factory>, weights_used: dict[str, float]=<factory>, breakdown: dict[str, float]=<factory>, penalties_applied: dict[str, float]=<factory>, base_score: int = 0, is_blocked: bool = False)[source]

Bases: object

Structured result from the Trust Score computation.

score

Overall Trust Score in [0, 100].

Type:

int

grade

Letter grade: A / B / C / D.

Type:

str

verdict

Plain-English deployment recommendation.

Type:

str

sub_scores

Per-dimension scores in [0, 100].

Type:

dict

weights_used

Actual weights used (after redistribution for missing dimensions).

Type:

dict

breakdown

Weighted contribution of each dimension to the final score.

Type:

dict

Visualization

trustlens.visualization.plot_module

trustlens.visualization.plot_module(module_name: str, data: dict, save_dir: str | None = None, *, embeddings: ndarray | None = None, labels: ndarray | None = None) → None[source]

Dispatch a module’s result data to the appropriate visualization function.

Parameters:

  • module_name (str) – Name of the analysis module (e.g., "calibration").

  • data (dict) – Module result data from TrustReport.results[module_name].

  • save_dir (str, optional) – Directory to save the resulting PNG file(s).

  • embeddings (np.ndarray, optional) – Embedding matrix (only used by "representation" module).

  • labels (np.ndarray, optional) – Ground-truth labels (only used by "representation" module).

trustlens.visualization.fairness

Fairness Visualizations

Visualizations for fairness diagnostics across subgroups.

Built on top of equalized_odds() and subgroup_performance() outputs. Inputs are expected to come from report.results["bias"].

trustlens.visualization.fairness.plot_subgroup_performance(subgroup_data: dict, feature_name: str, metric: str = 'accuracy', save_path: str | None = None, show: bool = True) → Figure[source]

Bar chart of model performance broken down by subgroup.

Parameters:

  • subgroup_data (dict) – Output from subgroup_performance() for a single feature. Example: results["gender"].

  • feature_name (str) – Name of the sensitive feature (used as axis label and title).

  • metric (str, optional) – Which metric to plot. Supports "accuracy" and "f1". Default: "accuracy".

  • save_path (str, optional) – If provided, saves figure to this path.

  • show (bool, optional) – Whether to display the figure interactively. Default: True.

Return type:

matplotlib.figure.Figure

Notes

subgroup_data may optionally contain a "__summary__" key (as returned by subgroup_performance()) for gap annotation.

Examples

>>> results = subgroup_performance(y_true, y_pred, {"gender": gender})
>>> fig = plot_subgroup_performance(results["gender"], "gender")
trustlens.visualization.fairness.plot_equalized_odds(equalized_odds_data: dict, feature_name: str, save_path: str | None = None, show: bool = True) → Figure[source]

Grouped bar chart of TPR and FPR per subgroup.

Displays TPR and FPR side by side for each group, making disparities between subgroups immediately visible.

Parameters:

  • equalized_odds_data (dict) – Output from equalized_odds() for a single feature. Example: results["gender"].

  • feature_name (str) – Name of the sensitive feature (used as axis label and title).

  • save_path (str, optional) – If provided, saves figure to this path.

  • show (bool, optional) – Whether to display the figure interactively. Default: True.

Return type:

matplotlib.figure.Figure

Notes

equalized_odds_data must contain a "__summary__" key (as returned by equalized_odds()). Missing summary data will result in annotations being silently skipped.

Examples

>>> results = equalized_odds(y_true, y_pred, {"gender": gender})
>>> fig = plot_equalized_odds(results["gender"], "gender")
trustlens.visualization.fairness.plot_fairness_gap(equalized_odds_data: dict, feature_name: str, save_path: str | None = None, show: bool = True) → Figure[source]

Bar chart showing TPR gap and FPR gap between best and worst subgroups.

Parameters:

  • equalized_odds_data (dict) – Output from equalized_odds() for a single feature. Example: results["gender"].

  • feature_name (str) – Name of the sensitive feature (used as axis label and title).

  • save_path (str, optional) – If provided, saves figure to this path.

  • show (bool, optional) – Whether to display the figure interactively. Default: True.

Return type:

matplotlib.figure.Figure

Notes

equalized_odds_data must contain a "__summary__" key (as returned by equalized_odds()). Missing summary data will result in annotations being silently skipped.

Examples

>>> results = equalized_odds(y_true, y_pred, {"gender": gender})
>>> fig = plot_fairness_gap(results["gender"], "gender")
trustlens.visualization.fairness.plot_subgroup_performance_multi(subgroup_data: dict, metric: str = 'accuracy', save_dir: str | None = None, show: bool = True) → dict[str, Figure][source]

Plot subgroup performance for each sensitive feature.

Iterates over all features in subgroup_data and delegates to plot_subgroup_performance() for each one. No feature is silently dropped. Features are processed in sorted order for deterministic output.

Parameters:

  • subgroup_data (dict) – Mapping of {feature_name: feature_data} as returned by subgroup_performance() when multiple sensitive features are passed. Example: results["bias"]["subgroup_performance"].

  • metric (str, optional) – Which metric to plot. Supports "accuracy" and "f1". Default: "accuracy".

  • save_dir (str, optional) – Directory in which per-feature PNG files are saved. Files are named subgroup_performance_<feature_name>.png. If None, no files are written.

  • show (bool, optional) – Whether to display each figure interactively. Default: True.

Returns:

Mapping of {feature_name: Figure} for every feature processed.

Return type:

dict[str, matplotlib.figure.Figure]

Examples

>>> results = subgroup_performance(y_true, y_pred, sensitive_features)
>>> figs = plot_subgroup_performance_multi(results)
>>> fig_gender = figs["gender"]
trustlens.visualization.fairness.plot_equalized_odds_multi(equalized_odds_data: dict, save_dir: str | None = None, show: bool = True) → dict[str, Figure][source]

Plot equalized odds for each sensitive feature.

Iterates over all features in equalized_odds_data and delegates to plot_equalized_odds() for each one. No feature is silently dropped. Features are processed in sorted order for deterministic output.

Parameters:

  • equalized_odds_data (dict) – Mapping of {feature_name: feature_data} as returned by equalized_odds() when multiple sensitive features are passed. Example: results["bias"]["equalized_odds"].

  • save_dir (str, optional) – Directory in which per-feature PNG files are saved. Files are named equalized_odds_<feature_name>.png. If None, no files are written.

  • show (bool, optional) – Whether to display each figure interactively. Default: True.

Returns:

Mapping of {feature_name: Figure} for every feature processed.

Return type:

dict[str, matplotlib.figure.Figure]

Examples

>>> results = equalized_odds(y_true, y_pred, sensitive_features)
>>> figs = plot_equalized_odds_multi(results)
>>> fig_age = figs["age"]
trustlens.visualization.fairness.plot_fairness_gap_multi(equalized_odds_data: dict, save_dir: str | None = None, show: bool = True) → dict[str, Figure][source]

Plot fairness gap for each sensitive feature.

Iterates over all features in equalized_odds_data and delegates to plot_fairness_gap() for each one. No feature is silently dropped. Features are processed in sorted order for deterministic output.

Parameters:

  • equalized_odds_data (dict) – Mapping of {feature_name: feature_data} as returned by equalized_odds() when multiple sensitive features are passed. Example: results["bias"]["equalized_odds"].

  • save_dir (str, optional) – Directory in which per-feature PNG files are saved. Files are named fairness_gap_<feature_name>.png. If None, no files are written.

  • show (bool, optional) – Whether to display each figure interactively. Default: True.

Returns:

Mapping of {feature_name: Figure} for every feature processed.

Return type:

dict[str, matplotlib.figure.Figure]

Examples

>>> results = equalized_odds(y_true, y_pred, sensitive_features)
>>> figs = plot_fairness_gap_multi(results)
>>> fig_gender = figs["gender"]