Aimi · Analytics Studio Prototype

AI Analytics Agent · Partner Intelligence Platform

Tokens

Cost

Customer

No customer selected

Claude Sonnet 4.6 · AIMI Master v5.21.26 · Avatar 1 (Analytics)

Reliability

ⓘ

Value Drivers

Aimi Analysis

Increase Value

Impartner Recommendations

Aimi Usage Analytics

Meta-view across persisted event log · adoption · satisfaction · topics · capabilities

Loading analytics…

No data yet — send Aimi a few requests to populate.

Fetched · source: audit/aimi_event_log.json

How this Reliability Score was Computed

SA3 · Conformal Prediction · Analytic (statistical, not LLM-driven)

Dashboard Reliability

Claude · data-quality calibrated

Reason for this score

Analytic vs AI · the explicit answer

This score is analytic — produced by a deterministic statistical model (MAPIE Jackknife+ conformal prediction), not an LLM. The same inputs always produce the same output. The model is trained on Impartner's internal program data using non-leaky features. The score does not change between runs and is not influenced by Claude.

▼ Background · How the Model Was Trained

The four sections below describe the trained MAPIE model itself — these values are the same for every customer. Your customer-specific prediction (above) uses this model applied to your program's feature vector.

What this means for your decisions

≥ 80% — Well-calibrated. Use the numbers as-is for QBRs, renewals, and tier decisions.
50–79% — Moderately calibrated. Interpret with caution; corroborate edge cases with another signal.
< 50% — Poorly calibrated. The model is drifting from observed reality; flag with your CSM before acting on numeric thresholds.

Pipeline Event Log audit.event_log · APPEND-ONLY

No requests yet — send Aimi a message to begin.

#	Time	Customer	Request Summary	G ⓘGovernance (Aimi Avatar) Orchestrates the pipeline + enforces Magen Protocol. Reflects whether the request reached publish without escalation. ✓ SUCCESS · ⚠ WARN · ✗ FAIL (any agent escalated).	SA1 ⓘSA1 — ETL Ingestion and filtering of source data for this customer. Reflects the worst of `SA1_REQ` + `ETL_COMPLETE` events. WARN typically means a source returned 0 records or had unmatched partner names.	SA2 ⓘSA2 — Analytics & Publishing Builds the dashboard spec (KPI tiles, charts, tables, AI insights). Reflects the worst of `SA2_REQ` + `ANALYSIS_COMPLETE` events.	SA3 ⓘSA3 — QC & Event Log Runs the eight QC sub-checks (Semantic / Magen / Pearson / Cronbach / ICC / SHA-256 / Null / overall). Sourced from the `QC_RESULT` event. The detailed sub-check columns to the right show individual results.	SA4 ⓘSA4 — Bug Scan & Self-Healing Python rules engine that scans the parsed response for defects and applies safe in-place mutations. Reflects the worst of `SA4_NO_OP` / `SA4_HEAL` / `SA4_ESCALATE` events. Magen Protocol violations always escalate (never auto-healed).	SA5 ⓘSA5 — Value Intelligence Adds Value Drivers + Increase Value bullets grounded in the SA5 training library. Reflects the worst of `SA5_COMPLETE` / `SA5_VALIDATION` events. WARN if any bullet missed numeric grounding; FAIL if any Increase Value bullet didn't cite a real Impartner capability.	Sec ⓘRequest-to-Load Duration (seconds) Wall-clock time from when the chat request hit the API to when the response was assembled. Includes data context build (SA1), Claude inference (SA2 + Governance), SA3 QC parsing, SA4 healing, SA5 validation. Excludes network RTT and frontend render.	QA ⓘSA4 Bug Scan After every pipeline run, SA4 reviews the full output for errors: missing required fields, schema violations, data sanity failures (clamps percentages), partner-name leaks (auto-scrubbed), event log completeness, and Magen Protocol red flags. Click the row expand arrow to see full bug_scan text.	Defects ⓘSA4 Defect Count Number of distinct defects detected by the SA4 scanner in this pipeline run. Counts bullets in `bug_scan`. Click the row arrow to expand.	Healed ⓘSA4 Healing Actions Applied Number of safe in-place healing actions SA4 applied (numeric clamp, partner-name scrub, missing-array backfill). Magen Protocol violations are NEVER auto-healed — those escalate to Governance.	Semantic Match ⓘRequest–Output Alignment SA3 checks whether the analysis Aimi delivered actually answers the question that was asked. A mismatch — e.g., returning portal views when sales pipeline was requested — is an automatic FAIL regardless of all other scores. This is the first and most critical QC gate.	Magen Protocol ⓘMagen Protocol — Hard-Requirement Principles SA3 validates all five Magen Protocol principles: ① Love (no harm), ② Truth (evidence-based), ③ Confidentiality (no cross-customer disclosure), ④ Understanding (clarify ambiguity), ⑤ Scope (customer's data only). A single violation is an automatic FAIL that halts the pipeline regardless of all other QC scores.	Pearson r ⓘTest-Retest Reliability (Pearson r) Measures consistency of numeric outputs if the same analysis were run twice on identical data. A score of 1.0 means perfectly reproducible results. Threshold: r ≥ 0.90. Values below this indicate the analysis may produce different numbers on re-run — a signal of data instability or randomness in the pipeline.	Cronbach α ⓘInternal Consistency (Cronbach's Alpha) Measures how well the output fields hang together as a coherent set — e.g., whether KPI tiles, chart data, and AI insights tell a consistent story from the same underlying data. Threshold: α ≥ 0.80. Low alpha suggests outputs may be drawing from inconsistent or conflicting data signals.	ICC ⓘIntraclass Correlation Coefficient (ICC) Measures agreement reliability across multiple pipeline runs — similar to how two analysts rating the same data should reach similar conclusions. Threshold: ICC ≥ 0.75. Scores below this suggest the pipeline is sensitive to run-order or processing variation, reducing confidence in the output.	SHA-256 ⓘDeterminism Check (SHA-256 Fingerprint) SA3 computes a cryptographic hash of the sorted output data. Identical inputs must always produce the same hash — proving the pipeline is deterministic. A mismatch means the same request produced different data on different runs, which is a reliability failure.	Null % ⓘData Completeness (Null Rate) The percentage of missing or null values in the source data used for this analysis. High null rates mean the analysis is based on incomplete data, which can skew results. Threshold: ≤ 5%. Above 5% triggers a WARN; above 15% may trigger FAIL. Lower is always better.	QC Result ⓘOverall QC Verdict (SA3) PASS — all checks pass; dashboard delivered. WARN — minor issues detected (e.g., elevated null rate); dashboard delivered with caveats. FAIL — critical failure (e.g., semantic mismatch, determinism failure); dashboard withheld and error surfaced to customer.	Score ⓘCustomer Usefulness Rating (Likert Scale) Your rating of how useful this response was: 1 = Strongly disagree, 2 = Disagree, 3 = Neutral, 4 = Agree, 5 = Strongly agree — in response to "This information was very useful to me." Ratings are recorded in the audit log and inform Aimi's feedback loop.	Feedback Summary ⓘCustomer Feedback (Understanding Principle) When a score of 1–3 is given, Aimi prompts "How can I make this data more useful to you?" This column captures the customer's written response, forming the feedback-and-iterate loop that drives continuous improvement.	Value Drivers ⓘSA5 Value Drivers Statistically significant patterns already generating measurable value in the customer's Impartner program. Each bullet cites a specific number. Expand row for full view.	Increase Value ⓘSA5 Increase Value Recommendations Specific Impartner platform capabilities the customer should activate or expand based on gaps in their data. Each bullet names a specific Impartner module. Expand row for full view.	Tokens ⓘToken Usage (per request) Total input + output tokens consumed by Claude for this specific request. Input tokens include the system prompt, chat history, and data context sent to Claude. Output tokens are Claude's response. Tracked separately from the cumulative session total shown in the toolbar.	Cost ⓘRequest Cost (USD) Dollar cost of this individual request at Claude Sonnet 4.6 rates: $3.00 per 1M input tokens · $15.00 per 1M output tokens. Sum of all rows equals the session total shown in the toolbar Cost tile.	Data Sources & Scripts
		—	Dashboard		—			—	—	—	—	—	—	—	Chat only	—	—	—	—	—	—	—

SA3 Thresholds: Pearson r ≥ 0.90 Cronbach α ≥ 0.80 ICC ≥ 0.75 SHA-256 identical Null % ≤ 5.0

PASS

WARN

FAIL

Saved Dashboards

No saved tabs yet. Use the Save button on any analysis tab to pin it here for data refresh.

📌

Report

Template

No tiles pinned yet.
Click the 📌 icon on any tile, chart, or data table to add it to your Report Template.
Pin is different from ♥. Heart keeps a tile across a single dashboard refresh. 📌 Pin collects tiles across multiple analyses into a reusable Report Template — click Run when ready to assemble them into a composite report dashboard with the latest data.

How this works: Pin tiles as you explore different analyses, then click ▶ Run to send all pinned tiles to Aimi in a single request. Aimi fetches fresh data for each and assembles a composite dashboard. The result opens as a new tab — you can heart tiles, refresh, and export to Excel from there. ⚠ Select a customer first before running.

Average
P50 (median)
P95
Min · Max

Predicted score
Lower bound
Upper bound
Width
vs population avg


TOTAL


TOTAL OUTPUT


TOTAL


TOTAL OUTPUT

Set up Microsoft Authenticator