1 files changed, 153 insertions, 0 deletions
diff --git a/TEST_MODERATOR_README.md b/TEST_MODERATOR_README.md
new file mode 100644
index 0000000..c2eb94c
--- /dev/null
+++ b/TEST_MODERATOR_README.md
@@ -0,0 +1,153 @@
+# Test Moderator CLI
+
+A command-line tool for testing the intelligent roleplay moderator against text files before deploying it to production.
+
+## Features
+
+- **File Testing**: Test individual text files or entire directories
+- **Mock Submissions**: Creates realistic mock Reddit submissions for testing
+- **AI Evaluation**: Uses the same GPT-5-nano evaluation logic as the production bot
+- **Detailed Results**: Shows AI reasoning, confidence scores, and proposed actions
+- **Mod Mail Preview**: Shows what mod mail messages would be sent to users
+
+## Usage
+
+### Prerequisites
+
+1. **OpenAI API Key**: You need a valid OpenAI API key with access to GPT-5-nano
+2. **Python Dependencies**: Install the required packages:
+   ```bash
+   pip install openai
+   ```
+
+### Basic Usage
+
+```bash
+# Test a single file
+python test_moderator.py --file sample_post.txt
+
+# Test all .txt files in a directory
+python test_moderator.py --directory test_posts/
+
+# Test with custom author name
+python test_moderator.py --file post.txt --author myuser
+
+# Use API key from command line
+python test_moderator.py --file post.txt --api-key sk-your-key-here
+```
+
+### Environment Variables
+
+The tool automatically loads configuration from a `.env` file in the project root. Create a `.env` file with:
+
+```bash
+# OpenAI API Credentials
+OPENAI_API_KEY=sk-your-key-here
+```
+
+Alternatively, you can set environment variables directly:
+
+```bash
+export OPENAI_API_KEY="sk-your-key-here"
+```
+
+### Command Line Options
+
+- `--file, -f`: Test a single text file
+- `--directory, -d`: Test all .txt files in a directory
+- `--author, -a`: Author name for mock submissions (default: testuser)
+- `--api-key, -k`: OpenAI API key (or set OPENAI_API_KEY env var)
+- `--verbose, -v`: Show detailed output
+
+## Sample Test Files
+
+The `test_posts/` directory contains sample files demonstrating different types of content:
+
+- **`artwork_showcase.txt`**: Art-focused post that should be re-flaired
+- **`high_quality_roleplay.txt`**: Well-written roleplay that should be allowed
+- **`low_effort_post.txt`**: Minimal content that should be removed
+- **`basic_roleplay.txt`**: Simple roleplay that might be borderline
+- **`poetic_roleplay.txt`**: Creative, high-quality roleplay content
+
+## Understanding Results
+
+### AI Evaluation Output
+
+The tool shows:
+
+- **Should be Art**: Whether the post should be re-flaired as Art
+- **Is Low Effort**: Whether the post should be removed for low effort
+- **Confidence**: AI confidence score (0.0 to 1.0)
+- **Reasoning**: Detailed explanation of the AI's decision
+
+### Actions
+
+- **🎨 Change flair to Art**: Post will be re-flaired and user notified
+- **🗑️ Remove post (low effort)**: Post will be removed and user notified
+- **✅ Allow post**: Post passes all checks and will be allowed
+
+### Mod Mail Preview
+
+The tool shows exactly what mod mail messages would be sent to users, including:
+
+- Subject lines
+- Message content
+- Reasoning explanations
+
+## Example Output
+
+```
+1. test_posts/artwork_showcase.txt
+------------------------------------------------------------
+Title: Artwork Showcase
+Word Count: 45
+Has Media: True
+Media Type: text
+Content Preview: Check out this amazing artwork I drew of Tokai Teio! She's so beautiful and I spent hours getting every detail perfect...
+
+🤖 AI Evaluation:
+  Should be Art: True
+  Is Low Effort: False
+  Confidence: 0.95
+  Reasoning: This post is primarily showcasing artwork with detailed description of the art process, making it more suitable for the Art flair than Roleplay.
+
+📋 Actions:
+  🎨 Change flair to Art
+
+📧 Mod Mail (Art Flair Change):
+  Subject: Your post flair has been changed to Art
+  Message: Your roleplay post has been automatically re-flaired as 'Art' because it appears to be primarily showcasing artwork or visual content rather than roleplay.
+  Reasoning: This post is primarily showcasing artwork with detailed description of the art process, making it more suitable for the Art flair than Roleplay.
+```
+
+## Setup
+
+1. **Install Dependencies**: `pip install openai python-dotenv`
+2. **Create .env File**: Add `OPENAI_API_KEY=sk-your-key-here` to your `.env` file
+3. **Run Tests**: `python test_moderator.py --directory test_posts/`
+
+## Tips for Testing
+
+1. **Test Edge Cases**: Include borderline content to see how the AI handles ambiguous cases
+2. **Check Confidence Scores**: Low confidence scores indicate the AI is uncertain
+3. **Review Reasoning**: The AI's explanations help understand its decision-making process
+4. **Test Different Lengths**: Try very short posts, very long posts, and everything in between
+5. **Include Media References**: Test posts that mention images, videos, or other media
+
+## Troubleshooting
+
+### Common Issues
+
+- **API Key Error**: Make sure your OpenAI API key is valid and has access to GPT-5-nano
+- **File Not Found**: Check that file paths are correct and files exist
+- **JSON Parse Error**: The AI sometimes returns malformed JSON; the tool handles this gracefully
+- **Rate Limiting**: If you hit OpenAI rate limits, wait a moment and try again
+
+### Getting Help
+
+If you encounter issues:
+
+1. Check that all dependencies are installed
+2. Verify your OpenAI API key is working
+3. Ensure test files are valid text files
+4. Check the error messages for specific guidance