Engineering

What always-on tracing costs, and why sampling does not help

We measured our own PHP extension on a WordPress page. The request-level layer is free. The function observer costs about 4% — and it costs the same whether it profiles 3% of requests or none of them.

21 September 2026 · Adam Buhl

PHPRay records every PHP request in production. The first question anybody sensible asks is what that costs, and the honest answer has to come with a method, not a marketing number. So we measured it, and one of the results was uncomfortable enough that we changed the default because of it.

The setup

WordPress 7.1, default theme, 31 published posts, 89 KB of HTML, rendering in about 48 ms warm. PHP 8.3.33 as mod_php under Apache with opcache on, MariaDB 11 in a separate container. Everything on a 16-core laptop on mains power, with the web container pinned to CPUs 4–7, the database to 2–3 and the load generator to 12–15.

The client was ab -n 300 -c 1 — sequential, so this is latency and not throughput. 120 requests of warm-up discarded after every restart. Variants interleaved over five rounds, because measuring one variant to completion and then the next attributes thermal drift and background load to whichever ran later.

The numbers

Median across five rounds of the per-round median request time.

Per-round p50 values, so you can see the spread rather than trust a single median:

The first two distributions overlap almost completely. The third is separated from both. So: the always-on layer — one record per request with wall time, CPU, peak memory, SQL as fingerprints, outbound HTTP, errors and N+1 detection — is not what costs. The function-level attribution is the whole cost.

The part that surprised us

Our default samples 3% of requests for per-function profiling. We assumed, for months, that a 3% sample rate meant roughly 3% of the profiling cost. A follow-up run with three rounds added a variant where the observer is registered but sampling is set to zero, so no request is ever profiled:

Lowering the sample rate saves nothing. Once any extension registers a zend_observer handler, the engine moves every user function call onto the observed path for the life of that process.

Our handlers return immediately for requests we are not profiling. That is not where the time goes. The dispatch has already been paid for by the time our code gets a say. The registration callback already declines core and vendor code and only observes files it can attribute to a plugin or theme — the obvious optimisation was in place before we measured, and it is not what remains.

So the trade-off is binary, not tunable. phpray.profile_functions is either on, and every request on that worker pays about 4% on this workload, or off, and the per-plugin breakdown is unavailable in that process. If you are building anything on zend_observer, the decision that matters is whether the observer is registered at all in that process — not how often it fires.

The second half, which cost a customer 31 minutes

The day before we ran these measurements, we installed the extension server-wide for one PHP version on a production shared host, with phpray.enabled=0. We assumed that made it inert. It does not.

phpray.enabled is PHP_INI_PERDIR, which is read per request, long after module startup. MINIT had already installed the observer and the SQL, cURL and file hooks in every worker of that PHP version — including accounts that had never asked for it. One customer's shop was down for 31 minutes.

The fix is a switch at the right level: phpray.master_switch is PHP_INI_SYSTEM, and it is the first thing MINIT checks. With 0, module startup returns before installing anything, so a host always has one line that makes the extension inert without uninstalling it. Two .phpt tests cover both directions, and the installer no longer arms a whole PHP version: you enable it per account.

If your extension can be dangerous, the switch that makes it harmless has to be readable before any per-directory configuration exists. That is a small API detail with a large blast radius, and we learned it the expensive way.

What we will not claim

• That the overhead is zero. With the default configuration it is not.
• That these numbers transfer to your application. This is one WordPress page on one machine. A request dominated by database or external HTTP time will show a smaller relative cost; a tight PHP loop will show more.
• Anything about throughput. The client was sequential on purpose.

The raw per-round numbers, the harness and the method are in the repository: docs/BENCHMARK-WORDPRESS-2026-09-20.md. If you measure something different on your workload, we would genuinely like the data.

What we are doing about the 4%

Two directions, in this order. First, decide per worker, not per request: a host could arm the observer in a small share of its PHP workers and leave the rest untouched, so a fleet pays a few percent on a few percent of traffic instead of a few percent on all of it. Second, approximate the same answer without the observer — attribution of time to a plugin can be derived from the SQL, HTTP and file hooks we already install, which cost nothing measurable. Coarser, but it answers the same customer question.

Until one of them lands, the honest guidance is the one we ship: leave phpray.profile_functions=0 on latency-sensitive fleets and arm it when you are investigating.