Rules need a rehearsal before going live, without touching real devices

What it is

Dry-run (rehearsal): before a new rule is activated, run it without touching real devices — replaying historical data or simulating a trigger — and show "what this rule would do", leaving the activation decision to the user. Software delivery has the tradition of a staging environment; home automation has none: the home is the production environment, and a rule, once active, executes on real people and real things. Rehearsal is a deliberately built read-only execution layer for a field that otherwise has no test environment.

What it prevents is not syntax errors — those are caught at save time — but semantic errors: a rule running perfectly legally, doing something other than what the user wanted. The only chance a semantic error has to show itself before activation is the rehearsal.

Why it happens

Why does home automation in particular need rehearsal? Because its rules' side effects are invisible, irreversible, and spilling:

Rule-writing and rule-firing are separated in time. Creating "turn off the humidifier at 3 a.m." in the afternoon gives no way to watch it compete at 3 a.m. with the "nursery humidity floor" rule; the true reach of a condition combination (will it fire Wednesday at 2 a.m.?) only unfolds in the real time stream. Humans evaluate rules by mental simulation, and mental-simulation accuracy falls off steeply with condition count — past two boolean conditions, untrained expectations are basically unreliable.

Meanwhile many household actions are irreversible or inconveniently so: the notification already sent, the socket already cutting a running dryer cycle, the family member woken at 3 a.m. Trial-and-error learning is priced out by physical consequences — software can do canary releases; "just run it and see" in home automation is paid for on the spot by the people who live there.

Rehearsal's mechanism is therefore separating execution from effect: the same condition evaluation, the same rule engine, but the output points at a display layer instead of the device layer. Replay-based rehearsal ("would have fired 4 times last month, at these times") gives the reach; simulation-based rehearsal injects one condition true and watches the whole chain decide. Both run outside the real time stream — the essence of staging.

Studying it

End-user programming research has long named the absence of testing and debugging support as a core adoption barrier: Ko and colleagues' analysis of learning barriers in end-user programming identifies the difficulty of answering "why didn't it do what I thought it would" among the deepest; field studies of home automation repeatedly find users' rule creation suppressed by "afraid of what might happen", restricting themselves to low-consequence rules. Together these point to rehearsal capacity as a ceiling on how far orchestration systems get used.

To evaluate rehearsal itself: have users create and activate rules with and without rehearsal, comparing pre-activation revision rates, first-week incident rates, and confidence self-ratings; retrospective studies track how "activate-and-regret" (deleted or disabled within 48 hours) shifts when rehearsal is introduced.

One methodological caution: rehearsal's ecological validity rests on representative history — a newly installed sensor has none, replay covers neither seasons nor schedule shifts. Treat "insufficient rehearsal data" as its own condition in design and evaluation.

Where it stops holding

• Rehearsal covers only what the engine can see. It simulates condition evaluation and the rule chain, not the physical execution's real outcomes (a jammed blind, a dead bulb) — execution-layer faults do not exist at the rehearsal layer; a passed rehearsal is not end-to-end reliability.
• Simulated triggers miss time-dependent defects. Timezone offsets, day boundaries, daylight-saving switches only surface in the real time stream; replay detects them only if the history happens to contain boundary events.
• Rehearsal has a usage barrier; building it is not using it. Forcing rehearsal on every rule deters creation of low-consequence ones; the right shape is consequence-graded — mandatory for locks, notifications, purchases; optional for dimming.
• Rehearsal results are fooled by "looks right". What users confirm in rehearsal is often the rule they imagine rather than the rule as built (isomorphic to the confirmation problem in natural-language rule creation); whether rehearsal works at all depends on how concretely it shows — the next card's subject.

Applying it

• Offer two rehearsals for every rule, after save and before activation: replay ("fired 4 times last month: three sunsets, one manual light-on") and single simulation ("suppose the door opens now and it's dark — would run: porch light 100%, notify mum").
• Embed the rehearsal entry next to the activation control, not in a menu — it belongs to the activation decision, not to settings.
• Grade mandatory rehearsal by consequence: rules with irreversible or spilling actions (locks, notifying others, spending) cannot be activated un-rehearsed.
• Make rehearsal results persist: keep "would have fired 4 times last month" beside the rule and compare with actual firings after activation — divergence is the earliest signal of semantic drift.
• How to check: track rehearsal usage and the share of rules modified after rehearsing — the modification rate is a direct reading of rehearsal's value (users catching their own mistakes); then watch the 48-hour regret rate move.

Related

• Same group: Z5.07.2 Rehearsals must show the specific actions that would fire · Z5.07.3 Without a test environment, side effects surface only in live runs · Z5.07.4 Rehearsals must cover boundary conditions, not just the happy path
• Nearby: Z5.03 Traceable causality · Z5.04.3 Natural-language descriptions must be verifiable
• Search terms: dry-run · staging · end-user debugging · smart home rules