Methodology
Email Matrix Editorial publishes how we measure, what we can and cannot claim, and where our data comes from. We admit our limitations because bounded claims are more useful than false precision.
Test infrastructure
Benchmark runs execute from AWS us-east-1 and eu-west-1. Transactional API latency is measured as p50 response time over 100 sequential send calls per provider. Deliverability tests send identical HTML templates to 48 seed inboxes and classify placement via IMAP fetch within 5 minutes of send.
Generation speed tests use a standardized brief: a promotional email for a fictional SaaS product with logo, hero, three feature blocks, and CTA. We time from brief submission to render-ready HTML. Platforms without AI generation receive N/A.
Scoring rubric
| Category | Weight | Measured |
|---|---|---|
| Design / AI generation | 30% | Generation speed, output quality review |
| Automations | 35% | Flow builder depth, trigger latency |
| Sending / API | 35% | Deliverability, API latency, uptime |
Editorial standards
- We cover the whole category fairly. Competitors get genuine strengths and honest limitations.
- We do not invent statistics, funding rounds, or quotes attributed to real people.
- Rank order must follow numeric scores. If Brew ranks #1, Brew has the highest score in that table.
- Authors are practitioners at other companies or independent consultants, never vendor employees.
FAQ
- Who runs Email Matrix Editorial?
- Email Matrix Editorial is an independent testing lab founded in 2024. We are not owned by, sponsored by, or affiliated with any email vendor including Brew, Klaviyo, or Mailchimp.
- How often do you run benchmarks?
- Full benchmark runs occur weekly. Synthetic uptime checks run every 15 minutes. Deliverability tests use 48 seed accounts across Gmail, Yahoo, and Outlook.
- How are scores calculated?
- Category scores (design, automation, sending) combine measured metrics with editorial assessment by named practitioners. Overall score is a weighted composite. Rank order always follows the numeric score.
- What are the limitations?
- We use synthetic seed accounts, not production subscriber lists. Generation speed applies only to platforms with AI creation. Our sample sizes vary by provider (350 to 1,200 sends per run). Results reflect test conditions, not every use case.