MintSense
Understanding Confidence Scores
Every message classified by MintSense includes a confidence score — a number between 0 and 1 that indicates how certain the AI is about its classification. This guide explains what confidence scores mean and how to use them.
What Is a Confidence Score?
A confidence score represents MintSense's certainty about a message's intent. Higher scores mean more certainty.
- 0.9 - 1.0: Very high confidence — clear buyer intent, safe to auto-reply
- 0.7 - 0.89: High confidence — likely buyer intent, review suggested reply
- 0.5 - 0.69: Medium confidence — ambiguous, human review recommended
- Below 0.5: Low confidence — unclear intent, needs human judgment
How Confidence Scores Are Used
ReplyMint uses confidence scores to decide which messages to auto-reply to:
- Messages with high confidence (0.85+) and clear buyer intent can trigger auto-reply if enabled
- Messages with medium confidence (0.6-0.84) surface to your inbox for manual review
- Messages with low confidence or negative sentiment are never auto-replied to
Examples
High Confidence (0.95)
Comment: "How much is this? Do you ship to Canada?"
Classification: Buyer Intent – Pricing + Shipping
Why high confidence? Direct pricing question + shipping question = clear buyer intent
Medium Confidence (0.68)
Comment: "Love this! Might need to grab one"
Classification: Interest – Potential Buyer
Why medium confidence? Shows interest but not asking a specific question. Could be casual engagement or genuine buying consideration.
Low Confidence (0.42)
Comment: "Fire 🔥"
Classification: General Engagement
Why low confidence? Generic praise without context. Could mean anything — not enough signal to classify with certainty.
Adjusting Your Auto-Reply Threshold
In MintSense AI → Automation, you can set a minimum confidence threshold for auto-reply.
- 0.85 (recommended): Only auto-reply to very clear buyer intent
- 0.75: More aggressive — may auto-reply to some ambiguous messages
- 0.95: Conservative — only the clearest buyer signals get auto-reply
Start at 0.85 and adjust based on how many false positives you see. If MintSense is auto-replying to messages it shouldn't, raise the threshold. If it's missing too many buyers, lower it slightly.
Handling False Positives
A false positive is when MintSense classifies a message as buyer intent but it's not. For example:
Comment: "The price is way too high 😒"
Misclassification: Buyer Intent – Pricing (wrong — this is a complaint)
If you see false positives, check your auto-reply threshold. Most false positives have medium confidence (0.6-0.8). Raising the threshold to 0.85+ fixes this.
When to Ignore Confidence Scores
Confidence scores don't account for:
- Negative sentiment: Even high-confidence negative comments should not be auto-replied to
- Spam: Spam is filtered separately from confidence scoring
- Unsafe content: Hate, threats, and personal info are flagged separately
MintSense never auto-replies to messages flagged as negative, spam, or unsafe — regardless of confidence score.