How to Be a Good Collaborator to a Statistician and a Data Engineer
statistical collaboration, statistical consulting, data engineering, reproducibility, analytic variability, estimand, data quality
Most failed projects are not failures due to technical matters, but failures of a good collaboration. They fail earlier, in the part nobody writes up: a question that was never pinned down, a column whose meaning changed in 2023, a “quick pull” that quietly became the number in the board deck. The statistician gets blamed for the estimate and the data engineer gets blamed for the pipeline, but the decisive errors were usually committed by well-meaning people who thought they were being helpful by not bothering anyone.
The influential statistician and geneticist, Ronald A. Fisher put it elegantly, in his 1938 presidential address to the Indian Statistical Congress:
consulting the statistician after the experiment is finished is often merely asking him to perform a post mortem examination — he can perhaps say what the experiment died of.1.
This is a piece about the habits that keep the patient alive. It is deliberately two-sided. Collaborators owe analysts specific things; analysts owe collaborators specific things back, and “the data wasn’t clean” is not a professional deliverable.
Why This Is Harder Than It Looks
The intuition that a well-specified question has one obvious analysis is wrong, and it is wrong by a measurable amount.
In the best-known demonstration, 29 teams comprising 61 analysts were given the same dataset and the same question — whether soccer referees give more red cards to dark-skin-toned players. The 29 analyses used 21 unique combinations of covariates. Estimated effects ranged from an odds ratio of 0.89 to 2.93, median 1.31; 20 teams (69%) reported a statistically significant positive effect and 9 (31%) did not. Neither analysts’ prior beliefs nor their expertise nor peer ratings of analysis quality explained the spread2.
The same pattern shows up in neuroimaging. Seventy teams analysed one fMRI dataset against nine pre-specified hypotheses. No two teams chose identical workflows. The resulting variation in hypothesis-test outcomes was substantial even between teams whose intermediate statistical maps were highly correlated3.
The lesson is not that analysis is arbitrary. It is that the analysis is underdetermined by the question, and the gap gets filled by defensible-looking judgment calls — the same garden of forking paths that makes untested model assumptions so consequential. Every one of those calls is a place where your context — what the variable actually measures, which exclusions are clinically or commercially sensible, what decision the number feeds — either arrives in time or doesn’t.
The variation in results across competent analysts is large enough that your contribution as a collaborator — supplying context that narrows the defensible choice set — is not a courtesy. It is part of the estimate.
What You Owe the Statistician
In Research Settings
Bring them in at design, not at submission. The single highest-leverage change is temporal. Kass and colleagues put “statistical analysis is more than a set of computations” and “plan ahead, really ahead” near the top of their ten rules for effective statistical practice, precisely because analysis choices are constrained by design choices that have already been locked in by the time most people call4. A statistician consulted at design can change the sample size. A statistician consulted at submission can only change the adjectives.
State the estimand before the method. Not “can you run a regression,” but: what quantity, in what population, under what contrast, with what handling of intercurrent events. If you cannot say it in a sentence, that is the finding, and it is worth knowing before you collect data.
Disclose everything you already looked at. Selective reporting is not a moral failing you have to confess; it is a fact about the data-generating process that changes what the numbers mean. An analyst who learns on submission day that you already ran eleven subgroup comparisons has been handed a different problem than the one they thought they were solving.
Settle authorship and credit at the start. The ASA’s Ethical Guidelines for Statistical Practice — eight principles across 72 elements, approved in 2022 — put responsibilities to stakeholders and integrity of data and methods on the same footing5. Statisticians are expected to refuse to produce misleading analyses. That obligation is much easier to honour when they are a named collaborator than when they are an unacknowledged service provider being paid by the hour.
Accept that the honest answer may be “this design can’t answer that.” The most valuable thing a statistician does is sometimes to tell you your study is not powered to detect what you care about. Treating this as obstruction rather than information is how you end up with an underpowered study and a strained relationship.
In Industry and Analytics Settings
Bring the decision, not the query. “Pull last quarter’s same-store sales by region” is a specification. “We’re deciding whether to close three underperforming stores and I need to know whether the recent dip is real” is a problem. The second lets the analyst tell you the first request would have misled you — because, say, the comp-store definition excludes the exact stores you’re evaluating.
Define the metric once, in writing, and stop redefining it in meetings. Most “the numbers don’t match” escalations are two correct calculations of two different things. A metric definition should name its grain, its filters, its date logic, its denominator, and its owner.
Say what happens if the number is wrong. An estimate for a monthly newsletter and an estimate feeding a pricing change deserve different levels of scrutiny, and analysts cannot calibrate effort they cannot see. Tell them the blast radius.
Give real deadlines and real slack. “EOD” for a request that requires joining three source systems is not a deadline; it is a request for an unvalidated guess with your name on it.
What You Owe the Data Engineer
Data engineers are the ones who find out that your business logic is inconsistent, usually at 2 a.m. Treat their constraints as physics, not preferences.
Send the reproduction, not the vibe. “The dashboard looks off” costs an engineer hours. “Row for store 412, 2026-07-14, shows fuel gallons of 0 but the POS extract shows 8,431 — here’s the query” costs minutes.
A defect report has four parts, and “something is wrong” is none of them. Telling an engineer or statistician that a number is wrong hands them a search problem with no search criteria. A usable report specifies:
- What is wrong — the exact rows, cells, or figures, with the query or filter that reproduces them.
- Why it is wrong — the business or domain fact that contradicts it: “these stores were acquired in March, so they can’t have January comp sales.”
- What correct looks like — the expected value or behaviour, with the independent source it comes from (the POS extract, the vendor invoice, the trial protocol).
- How to build the correct logic — the rule in implementable terms: “exclude stores from the comp base until they have 13 full months of history, keyed on
acquisition_date, notopen_date.”
The fourth part is the one collaborators most often skip, because it feels like doing the engineer’s job. It is the opposite: you are the only person who knows the rule. The engineer can translate a rule into SQL in an afternoon; they cannot conjure the rule itself, and if forced to guess, you have just delegated your business logic to the person with the least context for it — which is how the definition-drift problem below gets minted in the first place.
Announce upstream changes before they ship. A renamed field, a new store format, a changed promotion code, a vendor migration — every one of these breaks something downstream. The cost of a two-line heads-up is roughly zero. The cost of discovering it via a silently wrong report is not.
Understand that “just add a column” is rarely just adding a column. It may mean a backfill, a grain change, a contract change with every downstream consumer, and a reconciliation against numbers people have already quoted externally.
Stop building shadow pipelines. The spreadsheet you maintain because the official table was inconvenient will eventually be cited in a meeting as a source of truth, and someone will have to reconcile it. If the official table is wrong, file that as a defect rather than routing around it.
Treat data quality as a shared problem, not a service-desk ticket. In dbt Labs’ 2025 practitioner survey (459 respondents, collected October–December 2024 — a self-selected industry sample, not a probability sample), poor data quality was the most commonly cited challenge, named by more than 56% of respondents6. Data quality originates in operational systems and business processes, which means it is mostly not within the data team’s control, which means it will not be fixed by escalating to the data team.
What Statisticians and Data Engineers Owe Back
The obligations run both directions, and this is where the genre usually goes soft.
Learn the domain well enough to be wrong intelligently. Vance and Smith’s ASCCR framework — Attitude, Structure, Content, Communication, Relationship — is drawn from directing statistical collaboration labs across more than 900 direct projects and supervision of 4,000 more, and it puts Attitude and Relationship on equal footing with Content7. The framework exists because technical correctness delivered incomprehensibly has an effect size of zero.
Report uncertainty as uncertainty. Give the interval and the assumptions it rests on. Treating a P-value as a binary verdict rather than as continuous evidence throws away most of what the analysis produced — the case for S-values and the full compatibility/consonance function rather than a single dichotomized interval.
Say what would change your mind. Before running the analysis, state which results would be surprising and what you would do about them. This is the analyst-side equivalent of pre-registration and it is free.
Make it reproducible or don’t ship it. Given the demonstrated spread across analysts2, 3, an analysis whose choices aren’t recorded is not a result — it’s an anecdote with decimal places. Code in version control, the query saved next to the number, the exclusions written down.
Push back early and in writing. An analyst who accepts an impossible framing in the meeting and privately resents it for six weeks has chosen the worst available option.
Explain the constraint, not just the refusal. “That’s a three-week change because the fact table’s grain is transaction-line and you’re asking for basket-level” is a sentence a business partner can act on. “That’s not possible” is not.
Everything Runs on Assumptions — Ledger Them, Don’t Launder Them
Every model is a bundle of assumptions: linearity, exchangeability, missingness mechanisms, the proposition that the measurement instrument measures the thing. But so is everything else in the collaboration. You assume the chair holds, the laptop boots, the warehouse job ran last night, the engineer understood the ticket, the stakeholder meant what the ticket says. There is no assumption-free work — there is only work whose assumptions are written down and work whose assumptions are discovered later, at a worse time, by someone angrier.
The failure mode is not making assumptions. It is laundering them — letting them pass silently into the result so the final number looks cleaner than the process that produced it. The analyst-variability results above2, 3 are precisely this: dozens of defensible assumption sets, each invisible in its own write-up, visible only when laid side by side.
So the practice, for both sides of the table, is a two-column discipline. For each assumption, state:
- The assumption, in one falsifiable sentence.
- What is different if it’s wrong — which conclusions survive, which die, and what you’d check first.
A short worked example:
| Assumption | If it’s wrong |
|---|---|
| The nightly load completed before the snapshot | Yesterday’s figures are partial; totals are biased downward; check the job log before anything else |
acquisition_date marks when a store enters the comp base |
Comp growth mixes organic and acquired stores; the trend is uninterpretable; re-key the base |
| Missingness in loyalty ID is unrelated to basket size | Basket-level estimates are biased toward small transactions; bound the effect or model the missingness |
| The stakeholder’s “sales” means net of returns | Every reconciliation against finance fails by the return rate; re-confirm the definition |
Two things happen when you do this. First, most of the second column turns out to be checkable in minutes — the ledger converts vague dread into a short verification queue. Second, disagreements relocate to where they belong: not “your number is wrong” but “I don’t accept row three,” which is a conversation two professionals can actually finish. An assumption stated with its failure consequence is a hypothesis; an assumption left implicit is a landmine with your name on the deed.
An Intake Template
Put this at the top of every request. If it cannot be filled in, the work isn’t ready to start.
| Field | Why it matters |
|---|---|
| Decision | What action changes based on the answer |
| Question | One sentence, in domain language |
| Estimand / metric | Quantity, population, contrast, grain, denominator |
| Population & exclusions | Who’s in, who’s out, and why |
| Time window | Including how “current” is defined |
| Known data issues | Migrations, backfills, definition changes |
| Prior looks | What has already been run or explored |
| Blast radius | External? Regulatory? Internal only? |
| Deadline & slack | Real date, and what happens if it slips |
| Owner | Who signs off on the definition |
| Assumption ledger | Each assumption, and what changes if it’s wrong |
Two Checklists
Before You Ask
Before You Deliver
The Failure Modes Worth Naming
The post-mortem request. Design is locked, data are collected, and the ask is to find something. See Fisher, above.
The definition drift. Two teams, two correct numbers, one metric name. Costs more meeting-hours annually than any modelling error.
The heroic pull. An engineer bypasses the pipeline under deadline pressure; the one-off becomes load-bearing; nobody remembers the filters six months later.
The silent constraint. The analyst knows the design cannot support the claim, says so once, softly, in a meeting, and is not heard. Write it down. Writing it down is the whole technique.
None of this is sophisticated. That is rather the point: the returns to being a good collaborator come almost entirely from unglamorous, cheap, early habits, and almost not at all from anyone being cleverer.
Bring the decision, define the metric, disclose the prior looks, warn about upstream changes — and, on the other side, explain the constraint, report the uncertainty, and make it reproducible.
Comments