Measured on real traffic.
A WAF that blocks everything catches every attack. What matters is how many attacks it stops while letting real users through. Here is what we measured, how, and the caveats.
The WAF Comparison Project
open-appsec's WAF Comparison Project publishes two datasets: 1,040,242 legitimate requests recorded while browsing 692 real sites, and 73,924 attack payloads. Balanced accuracy is the average of the share of attacks blocked and the share of legitimate requests allowed.
| LightMoon | Attacks blocked | Legitimate blocked | Balanced accuracy |
|---|---|---|---|
balanced + CRS rules | 94.3% | 1.14% | 96.6% |
balanced | 86.6% | 0.32% | 93.1% |
Published results for other products
For context, the project's 2026 report measured the products below. These are their numbers and their setup; we did not re-run them.
Balanced accuracy. Other products: open-appsec WAF Comparison Project 2026 report. LightMoon: our run of the same datasets, September 2026.
Read these numbers carefully
- The report ran its Python client over the network against each product. We ran the same requests in process, encoded the way that client encodes them, and counted a 403 as a block, as the project does.
- We used the legitimate set to find and fix false positives. To check that the fixes generalise, we tuned on half the sites and measured on the other half: on sites we never tuned with, LightMoon with CRS blocked 1.5% of legitimate requests, against 0.77% on the sites we tuned with.
- open-appsec's engine is made by the authors of the benchmark.
- The benchmark measures signature detection only. Bot checks, rate limits, bans, response inspection and sink guards are outside it.
GoTestWAF
GoTestWAF v0.5.8 sends 675 attack requests and 141 legitimate ones. We ran it against LightMoon and against ModSecurity with CRS 4.29, both in front of the same origin on one machine, with rate limits and bans off on both.
| Attacks blocked | Legitimate blocked | Score | |
|---|---|---|---|
LightMoon balanced + CRS | 50.5% | 3 of 141 | 76.2% |
LightMoon strict + CRS | 56.2% | 17 of 141 | 78.7% |
LightMoon balanced | 41.6% | 0 of 141 | 56.8% |
| ModSecurity + CRS, paranoia 1 | 48.7% | 13 of 141 | 63.4% |
| ModSecurity + CRS, paranoia 2 | 56.9% | 87 of 141 | 59.4% |
Reproduce it
Both harnesses are in the repository. The legitimate dataset is a 1.2 GB download.
npm run build
CRS=1 node bench/waf-comparison/run.mjs # WAF Comparison Project
bench/gotestwaf/run.sh # GoTestWAF, needs DockerPerformance
Per-request cost on one core of a desktop machine (Node 24):
| Request | balanced | strict |
|---|---|---|
| Browser page view, 14 headers, 3 cookies | 33 µs | 46 µs |
| API POST with a 1 KB JSON body | 120–140 µs | 170 µs |
| Static asset | 40 µs | 39 µs |
curl | 9–14 µs | 6–10 µs |
Rules only run when they could match: each rule's regex is analysed for the literals any match must contain, and one Aho-Corasick pass per value decides which rules to run.