Skip to main content
HUMΛN
HUMΛN
←Community
HumanOS
HumanOS

ML risk scoring that reflects real stakes: embeddings and audit history

HUMΛN Team··15 min·ML + Compliance

A keyword hit on “patient” fires the high-stakes path. Good—except the utterance was about a patient waitlist for a SaaS trial, not a hospital discharge note. Regex matched. The world model did not.

HumanOS risk scoring starts with context, not string coincidence—because the failure mode is rarely “wrong keyword.” It is wrong stake.

Context first, regex second

Embeddings place the utterance in a semantic neighborhood before rules and models fire. Hospital discharge and SaaS onboarding may share a token; they do not share a harm profile. That is why risk that only greps is a demo heuristic wearing a compliance badge.

Scroll-stopper: A risk score that cannot tell “patient” in medicine from “patient” in marketing is not cautious—it is loud.

Multi-dimensional risk

A single float from a toy classifier is not an operational risk system. Dimensions like stake, harm_potential, and velocity belong in the same decision record as rules outputs. The HumanOS layer combines:

  • Rules — explicit, auditable, versioned.
  • ML scaffold — org-flagged, calibration-aware, explainable components.

Roll out gradually: flags, backfill jobs, admin endpoints—no silent flips on Friday. So that operators can refuse a model that changes production risk on partial data.

KNN from audit history

Past governed decisions are not only logs—they are training signal for similarity: tasks like this one escalated before; here is the neighborhood in embedding space. That is how risk learns from what actually happened—without inventing scores from marketing copy.

Footgun: previous rows without embeddings until backfill completes—gate behavior so you do not change production risk on partial data. Cold-start falls back to rules-only.

Explainability

Compliance needs why, not only what. Surface:

  • fired_rules — deterministic triggers with codes.
  • Model components — bounded narrative suitable for audit (“nearest neighbors were escalation-heavy”), not a black box “trust us.”

ML does not override Fourth Law: when uncertain, escalate to a human. Scores inform; they do not silence the escalation path.

Checklist: ML risk rollout

  • Domain classifier evaluated on held-out org tasks—not only demo prompts.
  • Embeddings backfill job idempotent; progress visible to admins.
  • Fallback to rules-only when ML disabled or cold-start.
  • Fourth Law path preserved—ML does not override human escalation when uncertain.

Go deeper

— Part of