Push by relevance rather than event

What it is

Relevance-based notification does not push on every event. It first asks whether a change affects someone's responsibility, subscription, dependency, current task, or stated interest, and only then decides whether to interrupt, at what intensity, and when. That an event occurred is an objective fact; whether it is worth interrupting a particular person is a separate judgment, and that second judgment is what a relevance system exists to make.

Why it happens

Event streams are naturally organized around producers — who changed what, and when — while recipients care about what they need to do next. These two organizations do not line up, and relevance filtering is essentially a bridge between them: it uses object relationships, role, deadline, and learned preference as filter conditions to decide which side of the bridge an event lands on.

The second-order mechanism is that this bridge inevitably runs into the precision–recall tradeoff: the more aggressive the filter, the more relevant events it silently drops (false negatives); the more conservative, the more irrelevant events leak through (false positives), which defeats the point of filtering. This tradeoff cannot be solved once with a fixed parameter, because the optimal point differs by user and by event class — for high-stakes responsibility changes, tolerating more false positives beats risking a single false negative; for routine activity, the opposite holds. What makes this worse is the invisibility of a dropped event: when notification volume is too high, users at least know they are being bothered and can go check; when over-filtering causes a miss, the miss is silent — the user has no idea an event they should have seen never appeared, until the consequence surfaces and gets traced back. That is harder to detect and harder to correct than noise, and it is the largest hidden risk in any relevance system.

Studying it

• Paradigm: run event-by-event, per-subscription, and role/dependency-relevance routing rules in parallel over the same real event stream, with a human-labeled ground truth of "should this person have been notified" for each event, then compute each rule's precision, recall, and actual user action rate.
• Variables: relevance feature set, push timing, precision, recall, action rate, mute rate, and count of misses discovered after the fact.
• Methodological caution: clicks cannot serve as a proxy for relevance — clicking amplifies existing attentional bias (more prominent items get clicked more), so genuine relevance labels need to come from recipient role and the actual task consequence of an event, not interaction data itself.

Where it stops holding

Relevance models depend on historical relationship data, which new members, rare risk types, and cross-team boundary information typically lack; the model tends to read "never seen before" as "not relevant," which is exactly where false-negative risk peaks. Over-personalization carries a deeper cost too: users lose the chance for discovery — information outside their existing subscription scope that would have been valuable never appears in the filtered view at all. That is not a bug; it is the necessary side effect of optimizing for relevance. For this reason, critical alert categories cannot be handed entirely to a predictive model — a safety channel that bypasses relevance filtering and reaches people directly must remain.

Applying it

• Let users see why each alert arrived and adjust it manually by object, role, time window, and intensity, turning an opaque filter into an explainable, correctable one.
• Route high-certainty responsibility changes (a task assigned to me, an approval I must sign) directly and immediately; digest routine activity — split by certainty, not by guessed relevance.
• Keep a searchable full activity stream as a backstop for filter errors, and give safety-critical events an independent, unfiltered alert channel.
• Verification: periodically sample "filtered-out" events for human review and compute the miss rate by role, checking whether a specific role (new members, cross-team collaborators) is being systematically dropped; use this miss rate together with satisfaction, not complaints about volume alone, to drive rule iteration.

Related

• Same group: V6.05.1 Notification volume grows fastest in collaboration tools · V6.05.2 All-user notifications train people to ignore
• Nearby: V4.03 Notification and subscription granularity · V2.03 Change awareness
• Search terms: notification relevance · attention management · alert prioritization · precision recall