Documentation - Advanced
Advanced policy configuration & ModBot
Take your policies from functional to intelligent - write better LLM labels, deploy ModBot for automated decisions, and manage complex multi-policy architectures.
At scale, the quality of your label descriptions and internal guidelines directly determines how much you can automate - and how much your team spends on manual review.
Who this is for
This guide is for teams that have a working Checkstep configuration - policies defined, rules set, content flowing through the moderation queue - and want to go further. You're past the "getting it running" phase and into "making it smarter."
If you haven't configured your basic policies and rules yet, start with the Building Your Content Policies guide first. The concepts here build directly on that foundation.
Writing better LLM labels
If you're using a prompt-based (LLM) strategy - and most accounts are - the quality of your labels is the single biggest lever you have for detection accuracy. This is where policy knowledge meets prompt engineering.
The mental shift: behaviors, not policies
The most common mistake when writing LLM labels is copying your policy text into the label description. Your policy says "financial spam is not allowed." But the LLM needs to know what financial spam looks like - the actual behaviors it should detect in a post.
"Financial spam - content related to financial spam or scams on the platform"
"False promises of guaranteed investment returns, deceptive loan offers, unsolicited cryptocurrency promotions, fake giveaways requiring payment"
The strong label describes observable behaviors. The LLM can match these against actual content. The weak label is circular - it tells the model to look for spam by describing spam.
Go micro, not macro
Think about what the harmful content actually looks like at the individual post level. If your policy prohibits "political discussion," what does that mean in practice? It might mean debates about election outcomes, commentary on specific political figures, partisan attacks, or discussion of policy legislation. Each of these is a more useful detection target than the umbrella term.
The car analogy
Writing good labels is like explaining a car's quirks to someone borrowing it - "it's weird when it shifts from 3rd to 4th gear" is more useful than "it drives a little funny." The more specific and behavioral your descriptions, the better the LLM can identify what you're looking for.
Labels you can experiment with safely
Remember: labels from your strategies are applied to all incoming content whether or not you have a rule for them. This means you can add new labels to your LLM strategy and observe what they catch beforecreating rules that take action. Use this to iterate on your label descriptions - refine the wording, check what's being tagged, and only create a rule once you're confident the label is catching the right content.
ModBot: AI-assisted moderation decisions
ModBot is a secondary AI review layer. Where your strategies and rules handle top-of-funnel scanning (labeling content and routing it), ModBot reads your policies like a moderator and makes nuanced decisions on individual pieces of content.
Where ModBot fits in the pipeline
ModBot sits after the top-of-funnel scanning. Content that your rules send for review - either to a human moderator queue or to a ModBot queue - can be reviewed by ModBot before (or instead of) human review. This is the "more sophisticated AI bot" option for content in the review confidence range.
Why not run ModBot on everything?
You couldrun ModBot on 100% of your content - and it would work. But it's a high-token-cost operation. The strategy of most platforms is to use fast, cheap top-of-funnel scanning to identify the 1–2% of content that's potentially problematic, and then spend the heavier ModBot resources only on that subset. Think of it as triage: the strategies are your intake nurses, ModBot is the specialist.
How ModBot makes decisions
ModBot reads your policy description and internal guidelines, examines the flagged content, and produces a decision with written rationale. Understanding this process is key to getting better decisions out of it.
Reads your policy
ModBot pulls in the full policy description and internal guidelines for the relevant policy. Everything you've written in those fields is context for the decision.
Examines the content
The flagged content - including any surrounding context like conversation history, user metadata, or related messages - is analyzed against the policy.
Identifies the relevant policy clause
ModBot pinpoints the specific part of your policy that applies (or doesn't apply). It will quote from your policy text in its rationale.
Makes a decision with rationale
The bot decides to enforce or not enforce, and writes an explanation citing the content, the applicable policy clause, and the reasoning. This rationale is visible in the content timeline.
Example: alcohol policy
Two posts, two decisions
Post A shows bottles of alcohol with the caption "Let me know if you need me to hook you up." Post B shows a similar image with "Can't wait for the weekend." Both are flagged by image recognition for alcohol-related content.
ModBot reads the illegal activities policy - which prohibits attempts to buy, sell, or trade alcohol - and decides differently. Post A's combination of the image and the transactional language ("hook you up") strongly suggests an offer to supply alcohol. Post B is a casual social post with no transactional intent. ModBot removes Post A and approves Post B, citing the specific policy clause in each rationale.
This is the kind of nuanced, context-aware decision that simple threshold-based rules can't make. The quality of that decision depends entirely on how well your policy and internal guidelines are written.
Tuning policies for ModBot
When ModBot makes an incorrect decision, the fix is almost always in your policy text - not in the AI configuration. This is the key difference between tuning your top-of-funnel strategies (which is about thresholds and labels) and tuning ModBot (which is about policy clarity).
The feedback loop
The process for improving ModBot's decisions follows a consistent pattern:
Spot an incorrect decision
Either through QA review, user appeals, or monitoring the content timeline. Note what ModBot decided and what it should have decided.
Read the rationale
ModBot tells you why it made the decision. Look at which policy clause it cited. Is the policy ambiguous on this point? Does the policy actually cover this scenario?
Update the policy or guidelines
Clarify the ambiguity, add the missing scenario, or tighten the language. If the policy says "attempts to buy, sell, or trade" but you also want to prohibit casual promotion of alcohol, add that to the policy text.
Monitor the change
After updating, watch subsequent decisions on similar content. ModBot should now handle the scenario correctly. If not, refine further.
Writing guidelines that ModBot reads well
Because ModBot is reading your internal guidelines as an LLM, certain writing patterns produce better results:
Be explicit about exceptions
If self-referential slurs are acceptable on your platform, state that clearly. If academic discussion of extremist content is permitted, define what qualifies as "academic." ModBot won't infer exceptions - it needs them written down.
Include concrete examples
Pair each guideline with examples of content that should and shouldn't be actioned. ModBot uses these as reference points when evaluating similar content. The more examples you provide, the better calibrated its decisions become.
Describe behaviors, not just categories
Just as with LLM labels, ModBot performs better when guidelines describe what harmful content looks like rather than what category it falls into. "Posts offering to supply controlled substances, including coded language like 'DM me for details' alongside product imagery" is more useful than "drug-related content."
Multi-policy architectures
As your platform grows - or if you serve multiple communities with different standards - you'll face decisions about how to structure your policies. This is where Checkstep's flexibility creates both power and complexity.
When one policy set isn't enough
Consider a platform like a community hosting service, where each community has different moderation standards. A music-only community might block anything that isn't music discussion. A general community might be more permissive. Both exist on the same platform and share core safety policies (no CSAM, no credible threats) but diverge on content focus.
Two approaches
| Approach | Trade-offs |
|---|---|
| Single policy set | Simpler to manage. Use content type tags and rule scoping to differentiate behavior per community. Works well when communities share most policies and only differ on a few rules. Can strain when communities have fundamentally different standards. |
| Multiple policy sets | Each community (or community type) gets its own policy configuration. Maximum flexibility, but you're now managing three, four, five sets of policies instead of one. Core policies (CSAM, violence, spam) are duplicated across sets. |
Most platforms start with a single policy set and use rule scoping to handle differences. When the customization needed per community becomes so significant that rule scoping creates more complexity than it saves, it's time to split into multiple policy sets.
Using ModBot to bridge the gap
One powerful pattern: keep your rules and top-of-funnel scanning consistent across communities, but write community-specific internal guidelines. ModBot reads the guidelines and makes context-aware decisions - so the same flagged content can receive different treatment based on which community it came from, without duplicating your rule configuration.
The full mental model
Putting it all together - how every layer of Checkstep's policy system connects, from 10 million pieces of content down to a single moderation decision.
Content volume
All user-generated content flowing through your platform - millions of pieces per day. 100% of this is scanned by your active strategies.
Strategies label (fast, cheap)
Pre-trained models, LLM classifiers, keyword lists - your top-of-funnel scanning applies labels to everything. This is a high-speed, lower-cost operation.
Rules evaluate (instant)
Content with labels is checked against your rules. High-confidence hits are auto-enforced. Low-confidence content is trusted. The middle band is flagged for review. Roughly 1–2% of content enters the review pipeline.
ModBot or human reviews (slower, expensive)
The flagged 1–2% gets the full treatment - ModBot reads the policy and writes a rationale, or a human moderator reviews the content with full context. This is where the expensive, high-quality decision making happens.
Action + compliance
Enforcement actions trigger compliance workflows - user notifications, transparency portal entries, appeal options - all handled automatically by Checkstep.
The elegance of this model is cost efficiency. You run cheap, fast scanning on everything and reserve expensive, nuanced review for the small percentage that needs it. Your policies, guidelines, and rules are the instructions that make every layer work correctly.
What's next
You've reached the far end of Checkstep's policy configuration. From here, optimization is an ongoing practice - not a one-time setup.
Ongoing optimization
Plan regular reviews of your Reporting Dashboard - monthly at minimum. Look at which policies are triggering most, how ModBot's accuracy is tracking, and whether moderator throughput is sustainable. Use the quality assurance feature (secondary review of a percentage of moderator decisions) to monitor consistency.
Policy evolution
Content moderation is never finished. New types of harmful content emerge, regulations change, your community evolves. Build a regular cadence for reviewing and updating your policies - both the public descriptions and the internal guidelines that drive ModBot's decisions.
Get help when you need it
The initial calibration of thresholds, ModBot configuration, and multi-policy architecture is something most customers do with Checkstep's team. If you're hitting complexity that this guide doesn't cover, reach out to your account manager - advanced configuration is a collaborative process.