TeaGuard is a trust & safety backend for anonymous review platforms — the kind of product where anyone can post about anyone, with no verified identity to hold them accountable. It analyzes submitted text across three independent risk dimensions — privacy risk, defamation risk, and AI-generated writing — and returns confidence-scored, human-readable labels instead of a single opaque "flagged" boolean.

Those labels drive a compose-time prompt, not a filter: in the common case the author sees a heads-up before the post goes live and decides for themselves whether to revise it. That intervention point is the product; the pipeline behind it exists to make the prompt specific enough to be worth reading.

The system is deliberately built as a fusion of signals, not one model doing everything: an LLM classifier (Groq / Llama-3.3-70B) handles anything requiring semantic judgment, a regex-based rule engine catches literal patterns fast and cheaply, and a custom stylometric heuristic scores writing uniformity as a weak AI-authorship signal. Scores combine through weighted, max-based aggregation — not averaging — so one severe signal can't be diluted by two unrelated low ones.

A live build runs at /projects/teaguard — real requests hit the actual Flask backend and its full detection pipeline. First request may take ~20s if the free-tier backend is cold.

Anonymous review platforms have a structural problem: the same anonymity that makes honest reviews possible also makes privacy leaks and defamation cheap. A moderation system for this context can't just be a binary spam filter — it has to make graded, explainable calls that a human moderator can act on, appeal, and audit.

The central question was:

Can a lightweight moderation system make defensible, explainable risk calls without either over-blocking legitimate speech or under-catching real harm?

The compose-time prompt isn't an invention here — it's an implementation of a documented intervention, Preliminary Flagging Before Posting, catalogued by the Prosocial Design Network.

What matters about that literature is that its evidence base is behavioral, not classifier-side. OpenWeb (2020) found roughly half of prompted users revised or abandoned the comment, alongside a 12.5% increase in civil comments platform-wide. A Twitter field experiment (Katsaros et al., 2021) found prompted users posted 6% fewer offensive Tweets than an unprompted control, with reduced recidivism afterward and fewer offensive replies to the prompted posts themselves.

The pattern works because most flagged posts are written in the heat of the moment by people acting in good faith — they lack awareness, not intent. One design constraint follows directly from that: the author must always be able to post anyway. A prompt that can't be dismissed stops being a prompt and becomes a filter, which is a different intervention with a different (and weaker) evidence base.

Wording as a design decision

The reference pattern's own mockup reads "Your comment is likely to be hurtful to others." I deliberately didn't copy it.

The same source records that in 3% of cases, users responded to the prompt by adding more slurs and profanity than they had originally intended. Second-person copy that adjudicates intent — your comment, hurtful — is the most likely thing to produce that reaction, because it asks the author to accept a verdict about themselves before they can act on it.

So TeaGuard's prompts describe the text, not the author. Instead of "this is harmful," the privacy prompt reads "This could identify someone — it mentions details like a name, employer, or neighborhood." The classifier's job is to point at a passage. Deciding what it means is left to the person who wrote it, which is also the only honest division of labor given how often the classifier is wrong.

1. A review is submitted with its text and creator ID.
2. Three signals run independently: the LLM classifier reasons about privacy/defamation/AI-authorship in context; the rule engine pattern-matches phone numbers, addresses, workplaces, and explicit accusation keywords; the stylometric heuristic scores sentence-length uniformity.
3. Scores are fused per risk category with category-specific weights (e.g., rules are weighted higher on privacy, the LLM higher on defamation, since each signal is strongest where the other is blind).
4. The fused score determines a risk label and a suggested action: allow, prompt the author (a compose-time heads-up they can act on or dismiss — the author can always post anyway), or hold for human review.
5. Every submission and appeal is written to an immutable audit log.

The most consequential early decision was how to combine three scores into one. Averaging is the obvious default, but it has a specific failure mode here: a text with one severe, specific signal (a phone number, an explicit accusation) and two neutral signals would average down into "medium risk" — exactly the case where under-reacting is costly. Aggregating by max instead means the system's floor is set by its most confident signal, not diluted by its least relevant ones.

The stylometric heuristic originally scored raw sentence-length variance as its AI-authorship signal. Testing against short adversarial samples surfaced a real bug: raw variance scales with sentence length, so short AI-generated text was scoring as more human-like than genuine human writing, inverting the signal the heuristic was supposed to provide. Switching to coefficient of variation (variance normalized by mean sentence length) fixed the inversion.

A second calibration finding, documented rather than silently patched over: formal, non-native English tends to read as "AI-like" to any style-based heuristic, because both share low sentence-length variety.

This is a local instance of a caveat the pattern literature states plainly — any system that scores text for harm is human-built and "will naturally carry the perspectives and biases of its creators." Treating it that way changed the response. Rather than trying to tune the false-positive pattern out of existence, the system routes uncertain cases to human review instead of auto-flagging, and the user-facing copy for this signal never asserts that the text is machine-written. A cheap heuristic's blind spot should lower its authority, not be hidden behind confident wording.

Two decisions were about what happens after a review is scored, not during scoring:

• Anti-gaming appeals. An appeal endpoint that automatically re-classifies content would create an obvious loop: flag → appeal → get cleared by the same logic that flagged it. The appeal endpoint intentionally never re-runs the scoring pipeline — it only updates status and writes to the audit log, keeping a human decision in the loop.
• Rate limiting. Flask-Limiter caps submissions at 10/min and 100/day. Live load testing confirmed the system allows the first 10 rapid submissions and correctly rejects the 11th and 12th with HTTP 429.
• Offline fallback. If the LLM API is unreachable, the system falls back to the rule-based and stylometric signals alone rather than failing the request outright.

• Backend — Flask, SQLite
• LLM signal — Groq API, Llama-3.3-70B
• Rule engine — Python re, hand-built pattern library (phone/email/address/workplace/school + accusation keywords)
• Stylometric signal — pure-Python sentence-length coefficient of variation
• Reliability — Flask-Limiter (rate limiting), offline heuristic fallback
• Storage — SQLite (submissions, labels, audit log)

• The rule engine is blind to indirect identification — e.g., "the only bartender at the rooftop bar on 57th Street" identifies someone without tripping a single regex pattern.
• The stylometric signal is a weak heuristic, not a classifier — it currently informs escalation to human review rather than acting alone.
• No labeled evaluation set yet; calibration has been driven by targeted adversarial test cases rather than aggregate precision/recall metrics.
• The success metric isn't precision/recall. The pattern literature measures behavior change — revision rate, abandonment rate, post-anyway rate — because a perfectly accurate prompt nobody acts on has changed nothing. TeaGuard doesn't yet instrument any of those, which is the largest gap between this build and the studies it's modeled on.
• Backlash is an unmeasured cost. The reference studies observed a small share of users escalating their language after being prompted. Copy choices here are meant to reduce that, but the effect is unmeasured — this build has no way to detect an author who edited a post to make it worse.

The most useful decisions in this project weren't about the LLM at all — they were about what not to trust to a single score. Max-based fusion, an appeal path that can't clear itself, and rate limiting validated against live load are all guardrails around the parts of the system that are the easiest to over-trust once they're working. The interesting engineering here was less "get the model to classify well" and more "design the system so a wrong or gamed classification doesn't become the last word."

The copy turned out to be part of that same problem. A prompt that tells the author what their words mean is a system asserting authority it hasn't earned — and the evidence says a share of people will push back by writing something worse. Pointing at the passage and leaving the judgment to the author isn't softer language; it's the same guardrail, applied to the one surface the user actually reads.