QA

The QA node runs an LLM quality review on the call transcript after each conversation ends. Use it to score agent performance, flag compliance issues, and surface patterns across calls, without listening to recordings manually.

When it runs

QA Node runs post-call only. It is not part of the live call flow - it reads the completed transcript after the conversation ends. It has no edges and cannot connect to any other node in the workflow.

This means QA does not affect call behavior. It only measures it.

Where it sits on the canvas

Add QA Node anywhere on the canvas. It does not need to connect to other nodes - most builders place it off to the side to make clear it is a standalone evaluator. The node runs automatically for every eligible call once the workflow is published.

Configuring the System Prompt

The core configuration is a single System Prompt: instructions to the reviewer LLM that describe how to evaluate each call. The LLM reads the transcript and returns a structured JSON result.

The default prompt already covers common quality dimensions out of the box:

• Tags: named issues detected in the transcript. The default tag set includes DEAD_AIR, USER_FRUSTRATED, ASSISTANT_IN_LOOP, ASSISTANT_REPLY_IMPROPER, USER_NOT_UNDERSTANDING, HEARING_ISSUES, UNCLEAR_CONVERSATION, USER_REQUESTING_FEATURE, ASSISTANT_LACKS_EMPATHY, USER_DETECTS_AI, and three Guardrails-related tags: INSTRUCTION_LEAK (the agent revealed internal instructions/configuration), OFF_TOPIC_DRIFT (the agent engaged with something outside its purpose instead of redirecting), and UNGROUNDED_CLAIM (the agent stated a specific fact without clear support)
• Call quality score: 1-10 rating
• Overall sentiment: positive, neutral, or negative
• Summary: 1-2 sentence description of the call segment

You can replace the default System Prompt with your own instructions when you need domain-specific evaluation. Write the prompt as instructions to the QA reviewer LLM, and specify the JSON structure you want it to return.

For compliance use cases, for example:

You are a compliance reviewer for a debt collection workflow.

Review the transcript and return a JSON object with the following fields:
- "identity_verified": true or false — did the agent confirm the caller's identity before discussing the account?
- "options_presented": true or false — did the agent offer all three payment options before accepting a decision?
- "tone_score": 1-5 — how calm and professional was the agent throughout?
- "summary": one sentence describing the call outcome.

Tiered Scoring

Freeform scoring works well for open-ended review, but it produces one number and one sentiment per node — there's no way to say "this specific compliance step matters more than that one" or "flag the call if this one thing is wrong, but let minor issues just get noted." Scoring Mode: Tiered solves that.

Instead of a System Prompt, you define a rubric — a list of yes/no questions, each assigned a tier:

• Critical — any failed question flags the call for review, no matter what else happened
• Important — failures count toward a pass-rate; you decide how strict that bar is
• Advisory — reported for visibility, never gates anything on its own

For example, a debt-collection compliance rubric might mark "Did the agent verify identity before discussing the account?" as Critical, "Did the agent offer all payment options?" as Important, and "Was the tone professional throughout?" as Advisory.

To use it, switch Scoring Mode to Tiered, then add rows under Rubric Items — each with a Tier, a Question, and an optional short Metric Name for reporting. The reviewer LLM judges every question independently against the transcript and returns a pass/fail with a reason for each one.

Switching Scoring Mode back to Freeform at any time restores the System Prompt view — your rubric items aren't lost, just hidden until you switch back.

Additional settings

Viewing QA Results

After a call completes, the QA analysis runs automatically. You can view the results two ways:

• Per call — open the run's Run Detail page and scroll to the QA Results card. For the default System Prompt, each node shows a score badge, a sentiment badge, and any tags with their reasons. A node running in Tiered mode instead shows each rubric question grouped by tier, with a pass/fail mark and the LLM's reason. If you replaced the System Prompt with custom freeform instructions that return a different JSON shape, the card falls back to showing that node's raw output instead of dropping it.
• Across calls — the agent's Runs list (Agents & Runs) shows a QA Summary card with the average quality score and the most common tags across every QA-scored call, so you can spot recurring issues without opening runs one at a time. You can also filter the run list itself by QA Status (Flagged / Not Flagged / All) — see QA Alerts below.

You can also leave a manual Good / Bad / Flag verdict and note on any individual call from its recording/transcript preview — a human check on top of (or instead of) the automated QA score. See Agents & Runs.

QA Alerts

QA analysis can flag a call and notify your organization's admins automatically, instead of results just sitting in the run history. This is configured under Settings → QA Alerts, off by default — turning it on requires organization admin/owner access.

For Tiered QA nodes, any failed Critical question flags the call automatically — this isn't a toggle, it always applies, the same way a compliance failure should never depend on remembering to turn on an alert. Important-tier failures only flag if you've set the pass-rate threshold above; Advisory-tier never flags.

When a call is flagged, Talkr sets a Needs Attention indicator on the run (visible as a Flagged badge in the run list and on the run detail page) and sends a notification — in-app always, plus email to organization owners/admins — the same way other organization notifications work (see Notifications to toggle the Call flagged by QA category on or off).

There's no platform-wide default threshold — an organization that never configures QA Alerts gets no alerting behavior, the same way usage alerts work.

Suggested Fixes

On the QA Results card, any Freeform-mode node with a recognized score/summary has a Suggest Improvement button. It generates an AI-rewritten version of that node's prompt from the QA feedback — the score, sentiment, tags, and summary for that specific node — and shows it next to the current prompt for comparison.

Tiered-mode nodes don't have a System Prompt to rewrite, so this button isn't available for them yet — review a failed rubric question's reason directly and adjust the question wording or the node's own agent prompt instead.

Nothing changes until you act on it:

• Generate Suggestion is a read-only preview. It doesn't touch your workflow.
• Apply as Draft patches only that one node's prompt into a draft version of the workflow (creating one if none exists yet, or updating your current draft in place — it never overwrites in-progress draft edits on other nodes). The rest of the graph is untouched.
• The draft is never auto-published. You review and publish it yourself through the normal workflow editor, exactly like any other manual edit.

This is scoped to a single node's prompt text, not a full redesign of the conversation flow — QA feedback is always about how one stage of the call went, so that's what gets rewritten. Suggestions are capped at 20 per organization per day.

Common mistakes

Expecting QA to affect the live call. QA Node is read-only at runtime. It evaluates what happened; it does not change what happens.

Connecting QA Node to other nodes. QA has no edges. Drawing a connection from QA to another node is not possible - edges will snap back when you try.

Writing a vague System Prompt. "Was the agent good?" produces unhelpful scores. Define specific, observable behaviors and the exact JSON fields you want back.

Not specifying output format. The LLM follows your instructions, but if you do not specify a return format, the output may be inconsistent across calls. Always describe the exact JSON structure in the System Prompt.

Next Steps