Email Flow Overview¶

Understanding how email flows through Mailborder is essential for troubleshooting and optimization. This document explains the complete email processing pipeline.

High-Level Email Flow¶

┌─────────────┐
│   Internet  │
│   (SMTP)    │
└──────┬──────┘
       │
       ▼
┌─────────────────────────────────────┐
│         Postfix MTA                 │
│  (Port 25 - SMTP Reception)         │
└──────┬───────────────┬──────────────┘
       │               │
       │               ▼
       │        ┌────────────────┐
       │        │   mb-milter    │
       │        │  (Policy Hook) │
       │        └────────┬───────┘
       │                 │
       ▼                 ▼
┌──────────────────────────────────────┐
│    Postfix Content Filter Queue      │
│   /var/spool/mailborder/incoming/    │
└──────┬───────────────────────────────┘
       │
       ▼
┌──────────────────────────────────────┐
│         mb-virtuoso                  │
│    (Queue Manager / Orchestrator)    │
└──────┬───────────────────────────────┘
       │
       ▼
┌──────────────────────────────────────┐
│          mb-filter                   │
│  (Spam & Virus Scanner)              │
│                                      │
│  ├─ Rspamd (spam scoring)            │
│  ├─ ClamAV (virus scanning)          │
│  ├─ RBL checks                       │
│  └─ SPF/DKIM/DMARC validation        │
└──────┬───────────────────────────────┘
       │
       ▼
    Decision
       │
       ├─ PASS ────────────┐
       ├─ SPAM ───────────►│ Quarantine
       ├─ VIRUS ──────────►│ Quarantine/Reject
       └─ POLICY BLOCK ───►│ Reject
                           │
                           ▼
                    ┌──────────────┐
                    │  Relay Host  │
                    │ (Your Server)│
                    └──────────────┘
                           │
                           ▼
                    ┌──────────────┐
                    │ User Mailbox │
                    └──────────────┘

Detailed Processing Stages¶

Stage 1: SMTP Reception (Postfix)¶

Component: Postfix MTA

Process: 1. Listening on port 25 (SMTP) 2. Client connects from Internet 3. SMTP greeting: 220 mailborder.example.com ESMTP 4. Client sends HELO/EHLO 5. Client sends MAIL FROM:<sender@domain.com> 6. Client sends RCPT TO:<recipient@yourdomain.com>

At this stage: - Basic validation (syntax checking) - DNS checks (sender domain exists) - Recipient validation (domain accepted by Mailborder)

Possible Outcomes: - Accept: Proceed to next stage - Temp Fail (421): Greylisting, queue full, temporary error - Reject (550): Invalid recipient, policy block, blacklist

Stage 2: Milter Policy Hook (mb-milter)¶

Component: mb-milter service

Purpose: Early policy enforcement before accepting message

Checks Performed: - Sender/recipient validation - Rate limiting - Connection-level policies - Early RBL checks - GeoIP filtering

Actions Available: - Accept: Continue processing - Reject: Block at SMTP level (no bounce) - Tempfail: Temporary rejection (sender retries) - Discard: Accept but silently discard - Quarantine: Accept but hold for review

Advantage of Milter: - Reject before accepting entire message - Saves bandwidth and processing time - Legitimate SMTP rejection (sender's server handles bounce)

Stage 3: Content Filter Queue¶

Component: Postfix queue in /var/spool/mailborder/incoming/

Process: 1. Message accepted and written to disk 2. Queue file created with unique ID 3. Message waiting for scanner pickup

Queue Structure:

/var/spool/mailborder/
├── incoming/       # New messages
├── active/         # Currently processing
├── deferred/       # Temporary failures
├── hold/           # Admin hold
└── corrupt/        # Damaged messages

Queue Management: - Maximum queue size enforced - Messages expire after 5 days (configurable) - Retry logic for temporary failures

Stage 4: Queue Processing (mb-virtuoso)¶

Component: mb-virtuoso daemon

Purpose: Orchestrate email scanning and delivery

Process: 1. Poll /var/spool/mailborder/incoming/ for new messages 2. Pick up message files (FIFO order by default) 3. Send message to mb-filter via Unix socket 4. Receive verdict from mb-filter 5. Take action based on verdict 6. Log transaction to database

Performance: - Process multiple messages in parallel - Number of workers: 2× CPU cores - Unix socket communication (fast, local)

Actions Based on Verdict: - Pass: Deliver to relay host - Spam: Move to quarantine storage - Virus: Quarantine or reject (configurable) - Policy Block: Reject with error message

Stage 5: Scanning (mb-filter)¶

Component: mb-filter daemon

Purpose: Comprehensive content analysis

Scan Steps:

1. Rspamd Analysis - Bayesian classification - Statistical analysis - Pattern matching - URL reputation - Header analysis - MIME structure validation

2. ClamAV Virus Scan - Signature-based detection - Heuristic analysis - Archive scanning (ZIP, RAR, 7z, etc.) - Document macro detection - Executable analysis

3. RBL Queries - Check sender IP against real-time blacklists - Query multiple RBLs in parallel - Weighted scoring based on RBL reputation

4. Sender Authentication - SPF: Validate sender IP authorized - DKIM: Verify cryptographic signature - DMARC: Check policy compliance

5. Content Filtering - Subject line analysis - Body content keywords - Attachment type checking - File name analysis

6. Policy Rules - Custom policy evaluation - Whitelist/blacklist checks - Domain-specific rules - User-specific rules

Output:

{
  "verdict": "spam",
  "spam_score": 8.5,
  "virus_detected": false,
  "spf_result": "pass",
  "dkim_result": "pass",
  "dmarc_result": "pass",
  "rbl_hits": ["zen.spamhaus.org", "bl.spamcop.net"],
  "action": "quarantine"
}

Stage 6: Action Execution¶

Based on verdict, mb-virtuoso takes action:

Pass (Deliver) 1. Connect to relay host (your mail server) 2. Deliver via SMTP 3. Log successful delivery 4. Delete from queue

Quarantine 1. Move message to /var/spool/mailborder/quarantine/ 2. Create database record with metadata 3. Generate quarantine notification (if enabled) 4. Delete from queue

Reject 1. Generate bounce message to sender 2. Log rejection reason 3. Delete from queue

Temporary Failure 1. Move to /var/spool/mailborder/deferred/ 2. Retry after delay (exponential backoff) 3. Maximum retries: 5 4. Abandon after 24 hours

Stage 7: Delivery to Relay Host¶

Component: Postfix SMTP client

Process: 1. DNS lookup for relay host 2. Connect on port 25 (or configured port) 3. SMTP transaction 4. Deliver message 5. Confirm delivery

Relay Host Configuration:

Example: mail.example.com or 192.168.1.10

Authentication (if required): - Username/password - TLS encryption recommended

Delivery Confirmation: - 250 OK - Success - 4xx - Temporary failure (retry) - 5xx - Permanent failure (bounce)

Email Processing Metrics¶

Timing¶

Typical processing times:

Stage	Time
SMTP Reception	0.1-0.5 seconds
Milter Policy	0.1-0.3 seconds
Queue Pickup	0.1-1.0 seconds
Spam Scanning	0.5-2.0 seconds
Virus Scanning	0.3-1.5 seconds
RBL Queries	0.1-0.5 seconds
Delivery to Relay	0.2-1.0 seconds
Total	1.3-6.8 seconds

Factors Affecting Speed: - Message size - Number of attachments - Archive complexity - System load - Network latency to RBLs - Relay host response time

Throughput¶

Capacity depends on hardware:

Hardware	Messages/Hour	Messages/Day
2 CPU, 4 GB RAM	~2,000	~48,000
4 CPU, 8 GB RAM	~5,000	~120,000
8 CPU, 16 GB RAM	~12,000	~288,000
16 CPU, 32 GB RAM	~25,000	~600,000

Message States¶

Messages can be in several states:

New - In /var/spool/mailborder/incoming/ - Waiting for scanner pickup

Active - Currently being scanned - In /var/spool/mailborder/active/

Deferred - Temporary delivery failure - Will retry automatically - In /var/spool/mailborder/deferred/

Quarantined - Held for review - In /var/spool/mailborder/quarantine/

Delivered - Successfully delivered to relay host - Removed from queue - Logged in database

Bounced - Permanent delivery failure - Bounce message sent to sender - Logged in database

Expired - Exceeded maximum queue time (default 5 days) - Bounce sent to sender - Removed from queue

Monitoring Email Flow¶

Real-Time Monitoring¶

Via Web Interface: - Dashboard → Email Statistics - Shows live message counts and rates

Via Command Line:

# View queue
sudo mailq

# Count messages in queue
sudo mailq | grep -c "^[A-F0-9]"

# Watch real-time processing
sudo tail -f /var/log/mailborder/filter.log

# Service status
sudo mb-status

Log Files¶

Email Processing Log:

/var/log/mailborder/virtuoso.log

Scanning Results:

/var/log/mailborder/filter.log

SMTP Transactions:

/var/log/mailborder/postfix.log

Policy Decisions:

/var/log/mailborder/milter.log

Troubleshooting Email Flow¶

Email Not Arriving¶

Check: 1. Is MX record pointing to Mailborder?

dig example.com MX

Is Postfix listening?
```
netstat -tlnp | grep :25
```
Is firewall allowing port 25?
```
sudo iptables -L -n | grep 25
```

Any errors in logs?

sudo tail -f /var/log/mailborder/postfix.log

Email Stuck in Queue¶

Check: 1. View queue:

sudo mailq

Is relay host reachable?
```
telnet mail.example.com 25
```
Check virtuoso logs:
```
sudo journalctl -u mb-virtuoso -n 50
```
Flush queue manually:
```
sudo postqueue -f
```

Slow Processing¶

Check: 1. System resources:

top
free -h
df -h

Service status:
```
sudo mb-status
```

Scanning times in logs:

sudo grep "scan_time" /var/log/mailborder/filter.log | tail -20

Queue size:
```
sudo mailq | grep -c "^[A-F0-9]"
```

All Email Quarantined¶

Check: 1. Spam thresholds: - Web Interface → Email Security → Spam Filtering

Recent configuration changes:
```
sudo mb-audit --type config --last 24h
```
Rspamd status:
```
sudo systemctl status rspamd
```
Test with known-good email:
Send from trusted source
Check spam score in logs

Best Practices¶

Email Flow Optimization¶

Tune Spam Thresholds
Monitor false positive rate
Adjust based on user feedback
Start conservative, tighten gradually
Enable Greylisting
Reduces spam significantly
May delay first-time senders
Whitelist important senders
Use RBLs Wisely
Enable reputable RBLs only
Avoid overly aggressive RBLs
Monitor RBL false positives
Optimize Queue Processing
Increase concurrent scans if CPU available
Monitor queue depth
Alert on queue backup
Monitor Relay Host
Ensure relay host can handle volume
Check relay host queue
Monitor delivery times

Performance Monitoring¶

Set up alerts for: - Queue depth > 1000 - Average scan time > 5 seconds - Failed delivery rate > 5% - Service downtime

Advanced Topics¶

Custom Email Routing¶

Route email based on criteria: - Recipient domain - Sender domain - Message size - Content type

See Policy Enforcement

Priority Processing¶

Prioritize certain emails: - Executive mailboxes - Critical domains - Time-sensitive content

See Advanced Policies

Email Archiving¶

Archive all email for compliance: - BCC to archive server - Journal all transactions - Long-term storage

See Compliance

Next Steps¶

Spam Detection - Understand spam scoring
Virus Scanning - Antivirus configuration
Policy Enforcement - Custom rules
Quarantine Management - Handle held email