Product analytics for coding agents: a practical architecture

Product analytics for coding agents works when the agent receives a narrow measurement contract instead of unrestricted database access. The contract should name meaningful metrics, preserve the reason each metric exists, expose a small set of typed queries, and return enough evidence for a person to review the answer. As of July 2026, Poolstatis implements this shape with four primitives, six Query DSL branches, and MCP tools that operate on registered metric keys.

The architecture matters because a coding agent has a different failure mode from a dashboard user. A person can notice that a chart title looks odd, ask who defined the event, or compare the result with institutional knowledge. An agent tends to accept whatever schema it can reach. If the analytics surface contains thousands of raw event names, loosely typed properties, and arbitrary SQL, the agent can produce a syntactically valid answer that has no stable business meaning.

The safer alternative is not to remove autonomy. It is to put meaning in the interface the agent uses.

What changes when the agent is the primary analytics user?

Agent-first product analytics moves semantic decisions ahead of query execution. A traditional analytics flow often starts with a person opening a dashboard, selecting events, adjusting filters, and interpreting the chart. An agent-first flow starts with a product question, resolves that question to registered metrics and funnels, runs a typed query, checks data quality, and returns a decision packet.

This shift changes the product surface in four ways:

1. Definitions become API data. A metric needs a stable key, human-readable name, source, type, lifecycle status, and purpose. A funnel needs ordered metric keys, a conversion window, and a goal.
2. Queries become bounded operations. The agent chooses trend, funnel, entities, retention, lifecycle, or stickiness. It does not assemble arbitrary joins.
3. Data quality travels with the answer. The response should expose whether evidence is registered, what time window was used, and what identity or property assumptions affect the result.
4. Human review stays explicit. An agent may propose a metric, but activation changes the shared measurement contract and should remain reviewable.

The result is an analytics system designed around product decisions rather than around chart construction. A chart can still be produced by a client, but chart configuration is not the source of truth.

What should the analytics contract contain?

An analytics contract for coding agents should contain immutable facts, mutable state, semantic metrics, and goal-driven funnels. Poolstatis names these four primitives Event, Entity, Metric, and Funnel. Each primitive solves a separate measurement problem, so the agent does not need to infer whether a value represents history, current state, meaning, or journey structure.

Primitive	What it represents	Example	Why the agent needs it
Event	An immutable fact at a specific time	signup.completed	Preserves what happened without rewriting history
Entity	Mutable state for a user, account, or other object	account.plan = "team"	Keeps current segmentation state out of event duplication
Metric	A registered definition with a required purpose	activated_account	Tells the agent what to count and which decision the count supports
Funnel	Ordered metric steps with a required goal	signup_to_first_answer	Gives conversion analysis an explicit product journey

The distinction between an event and a metric is especially important. An event is evidence. A metric is an interpretation of evidence for a recurring decision. One event can support more than one metric, and a metric definition can change while old events remain immutable.

A narrow query layer sits on top of the contract. Poolstatis uses one discriminated Query DSL endpoint, where the kind field selects a supported operation. Trend and funnel queries reference registry metric keys rather than raw event names. This rule prevents a coding agent from silently turning an implementation detail into a business KPI.

The contract should also expose read-only context before mutation. A good first tool returns the registered metrics, funnels, entity types, observed events, and registered coverage for the project. The agent can inspect the existing vocabulary before proposing anything new. This reduces duplicate metrics such as signup, sign_up, and user_registered that count the same behavior with slightly different names.

Why not give the agent raw SQL?

Raw SQL optimizes for query freedom, while an agent-facing analytics surface must optimize for reproducible meaning. SQL is useful for internal engineering and debugging. It is a poor default product contract for autonomous analytics because the query text does not guarantee that the selected tables, joins, identity grain, or time semantics match the product question.

Consider the question: “Did the onboarding rewrite improve activation?” A plausible SQL query can still be wrong in several ways:

• it may count events rather than unique actors;
• it may use signup.started instead of the agreed activation entry metric;
• it may join mutable plan data as if it were historical state;
• it may compare incomplete current-week data with a complete prior week;
• it may include events that arrived but were never covered by an active metric;
• it may use a different conversion window from the shared onboarding funnel.

A typed analytics query cannot eliminate every interpretation error, but it can make the important choices explicit and reusable. The metric key resolves to a stored source and purpose. The funnel resolves to fixed steps and a fixed window. Date parsing follows one contract. The response can include the metric definition and query window alongside the values.

The storage layer should remain replaceable behind that interface. Poolstatis routes event reads and writes through an EventStore seam. The public Query DSL stays deliberately narrow because every method must be implementable on a second event store, not only through database-specific SQL. That constraint is useful for agents too: a question keeps the same meaning when the storage implementation changes.

Give the agent freedom to choose a supported analysis, not freedom to redefine the measurement model inside every query.

How does the build-measure-decide loop work?

The build-measure-decide loop should connect a shipped change to registered evidence and finish with a reviewable next action. The following sequence keeps the agent productive without letting it skip identity, instrumentation, or data-quality gates.

1. Start with the decision

The agent states the product question before adding events. “Measure onboarding” is too broad. “Decide whether removing the invite step increases the share of new accounts that reach their first typed query within 24 hours” names an action, an outcome, an actor grain, and a time boundary.

That statement guides the metric purpose and funnel goal. It also tells the team which result would change the product.

2. Inspect the current measurement model

The agent reads project schema, registered metrics, funnels, property definitions, and recently observed event names. Existing definitions should be reused when they already represent the intended outcome. A new metric should not be created only because the event name differs.

This inspection step also catches drift. An observed event can be accepted by ingest but marked registered=false when no active metric covers it. The event remains available for diagnosis, while the quality signal tells the agent not to treat it as trusted KPI evidence.

3. Propose the smallest instrumentation change

The agent adds only the event or property needed to answer the decision. Immutable occurrences belong in events. Current plan, role, lifecycle stage, and similar state belong on entities. Low-cardinality properties that change how the decision is interpreted should receive their own definitions and trust review.

For a new metric, the agent writes a purpose that names the decision. In Poolstatis, a newly registered metric starts as proposed. A human can inspect its name, source, type, category, tags, and purpose before activation.

4. Verify accepted, registered evidence

Successful network delivery is not enough. The agent checks that the server accepted the event, the intended active metric covers it, identity is stable, and the timestamp belongs to the expected window. Duplicate retry behavior should be idempotent so a lost HTTP response does not inflate the count.

This gate separates “the SDK sent something” from “the product has decision-ready evidence.”

5. Run a typed query

The agent selects the smallest supported query that matches the question:

• use a trend for change over time;
• use a funnel for ordered conversion;
• use retention for return behavior by cohort;
• use lifecycle for new, returning, resurrecting, and dormant states;
• use stickiness for frequency distribution;
• use entities for current state and segmentation.

The agent passes registry keys and explicit date windows. It does not guess raw event names from autocomplete.

6. Return a decision packet

The final answer includes the observed movement, sample size or counts, comparison window, metric purpose, data-quality state, and a reproducible query description. If evidence is incomplete, the agent says what is missing and stops short of a causal claim.

The next action should match the evidence. A conversion increase after a release is an association unless the assignment came from a valid experiment. The agent may recommend keeping the change, investigating a segment, or collecting more data, but it should not upgrade correlation into causation.

What evidence should an agent return?

An agent analytics answer should be small enough to read in a coding session and complete enough to audit later. A useful answer is not a dump of query rows. It is a structured packet that connects the question, definition, data, and decision.

A practical packet contains:

Question: Did the shorter setup improve first value?
Metric: first_typed_query
Purpose: Decide whether setup changes help verified accounts reach a real product answer.
Window: 2026-07-16 through 2026-07-29
Comparison: previous 14 complete days
Result: 38.4% → 47.1% (+8.7 percentage points)
Evidence: 184 eligible accounts, 100% registered coverage
Limit: observational comparison, not randomized assignment
Next move: keep the shorter setup and inspect invite-step drop-off

Each line carries a different part of the reasoning. The purpose prevents the metric name from being read out of context. The window makes recency and completeness visible. Registered coverage warns about instrumentation drift. The limit prevents a stronger conclusion than the method supports.

For automated consumers, the same packet should be available as typed JSON. For humans, it should be rendered as concise prose or a decision card. The representation can change, but the underlying fields should not.

When does agent-first analytics fit?

Agent-first analytics fits teams that already let coding agents change product code and want the same session to inspect measurement, verify instrumentation, and answer a bounded product question. It is most useful when metric definitions are scattered across dashboards, event names drift between releases, or engineering repeatedly translates product questions into one-off SQL.

Use this architecture when:

• coding agents can reach a reviewed MCP or API surface;
• the team can name decision owners and metric purposes;
• product identity is stable enough for unique counts and funnels;
• instrumentation changes can ship with tests and verification;
• humans want an audit trail without operating every query manually.

A dashboard-first platform may be the better primary interface when most analysis is exploratory, analysts need broad ad hoc joins, or business users depend on a large library of visual reports. The two models can coexist. A semantic registry and typed agent surface can sit beside a warehouse and BI environment, as long as both sides agree on metric meaning.

Do not use autonomous analytics to hide missing foundations. If the product has no stable actor identity, no accepted-event verification, or no owner for metric definitions, an agent will reproduce those gaps faster. Fix identity and semantics before increasing query autonomy.

Implementation checklist

Use this checklist before calling an agent-facing analytics integration complete:

• Every recurring product metric has a stable key and a decision-oriented purpose.
• Every funnel has ordered registered metric keys, a goal, and an explicit conversion window.
• Events remain immutable, while mutable user or account state lives on entities.
• New definitions start in a reviewable state rather than becoming trusted automatically.
• Ingest accepts diagnostic evidence but marks uncovered events as unregistered.
• The agent can inspect existing schema and observed events before creating definitions.
• Query operations are typed and reference semantic keys instead of raw SQL.
• Identity grain, date window, counts, and registered coverage travel with the answer.
• Observational results are not described as causal experiment results.
• The final response names a next action or states why the evidence is insufficient.

Poolstatis exposes this contract through its MCP reference, Query DSL, and instrumentation standard. The implementation is source-available, so a team can inspect the exact schemas and storage seam rather than trusting a marketing description.