A flowchart is not a substitute for real utterances

What it is

Arrows on a flowchart name user-act types (“provide destination”). What arrives at the ear is a sentence (“uh, my mom's place — that compound from last time, what's it called”). The diagram can guarantee that nodes have in- and out-edges; it cannot guarantee that this sentence fires that arrow. Shipping when the boxes are pretty substitutes type labels for speech. The thing to validate is talk, not boxes.

Why it happens

The diagram compresses speech into event names and throws away hesitations, mid-utterance repairs, anaphora, extra slots in one breath, and phrasings that were never in the vocabulary. Recognition and understanding live entirely in the hop sentence → event; the diagram assumes that hop already succeeded. In a design walkthrough everyone nods at the boxes because the boxes are written in the designers' language, not the users'.

First utterances diverge most. People open from their own task model, not from slot order, not from the grammar of “please say a date.” If the first sentence maps to no arrow, none of the later boxes will be reached. Diagrams also imply mutually exclusive branches (once a date is chosen, the place will not change in the same breath); real sentences often splice a second act before the first is done.

Studying it

Before freezing the diagram, collect spontaneous first utterances from target users: give the goal, give no example prompts, at least one sentence each. Label whether each sentence maps uniquely onto one user-side arrow. Dependents: mappable rate, share of sentences that carry more than one event, share that contain repair or anaphora the diagram has no handler for.

After launch, do the same mapping on logs: sample real sentences on each user-side arrow and see whether NLU hits the intent that arrow assumed. In Wizard-of-Oz, people complete the task without seeing the flowchart; compare that against “the user will provide X.” Internal walkthroughs test self-consistency of the diagram, not coverage of sentences.

Where it stops holding

On a keypad menu (“press 1”) the user-side event is a keypress; diagram and interface nearly coincide, so utterance evidence drops in priority. Trained operators will volunteer closer to the prompt grammar. Diagrams remain useful as internal communication; they fail as acceptance evidence. Skills with few slots and a closed vocabulary (on/off for one lamp) vary little, so the harm is small. Once logs are large, the diagram can be rewritten as a summary of the corpus — data constraining the diagram, not the diagram standing in for data.

Applying it

• Under every user-side arrow, paste at least five real sentences: a clean fill, a vague anaphor, a mid-sentence repair, “I don't know,” and one that names later slots in the same breath. If five cannot be pasted, the arrow is still a wish.
• Build a first-utterance corpus on its own; do not back-solve what users will say from the system's question.
• In review, listen to recordings or read sentences before looking at the diagram. A path that walks on paper but whose sentences never hit NLU is uncovered.
• How to check: have people who did not draw the diagram do the task; map every sentence onto it. Unmapped sentences are gaps — fix understanding or add a branch, rather than tidy the boxes.

Related

• Same group: M2.07.1 Happy-path-only flows leave failure and walk-away unspecified · M2.07.2 Every dialogue state needs a way out · M2.07.4 Slot filling should accept scrambled order · M2.07.5 Cancel, restart, and help must work from every state · M2.07.6 State count grows past what you can test
• Nearby: C7.14 Command syntax versus free expression · M2.02 Open and closed questions · M1.06 Repair strategies
• Search terms: utterance validation · dialogue flowchart · first-utterance corpus