NYC comparator methods lab: demo dashboard

panel = fetch("/statistics/nyc-comparator/dashboard/panel.json").then((r) => { if (!r.ok) throw new Error("Could not load demonstration panel"); return r.json(); })
meta = fetch("/statistics/nyc-comparator/dashboard/meta.json").then((r) => { if (!r.ok) throw new Error("Could not load demonstration metadata"); return r.json(); })
rows = { if (panel.synthetic !== meta.demo) throw new Error("Panel and metadata disagree about synthetic data"); return panel.rows; }
P = buildPanel(rows, meta.start, meta.end)
fake = ["2019", "2020", "2021", "2022", "2023", "2024", "2025"].map((y) => `${y}-01-01`)
specs = specGrid(6)
primarySpec = specs.find((s) => s.id === "12-mo mean, equal")
primary = analyze(P, {focus: meta.focus, peers: P.cities, event: meta.event, spec: primarySpec, fakeEvents: fake})
Z = 1.959963984540054
models = [
  {key: "inTime", label: "Focus city's own history (in-time placebos)", short: "Own history", dash: null},
  {key: "poisson", label: "Focal count-only component (not a valid CI)", short: "Counts only", dash: "4,3"},
  {key: "crossCity", label: "Cross-city placebo spread", short: "Peer spread", dash: "1,3"},
  {key: "inTimeOmit", label: "Own history omitting 2020–2023 events (sensitivity)", short: "Omit 2020–23", dash: "8,3,2,3"}
]
teal = "#555555"
coral = "#222222"
grey = "#9aa5aa"
pct = (x, base) => 100 * x / base
fp = (x) => `${x >= 0 ? "+" : "−"}${Math.abs(x).toFixed(0)}%`
f2 = (x) => `${x >= 0 ? "+" : "−"}${Math.abs(x).toFixed(2)}`

function strip(res, focus = meta.focus) {
  const d = models.map((m) => {
    const se = res.ses[m.key];
    return {model: width < 640 ? m.short : m.label, est: pct(res.est, res.baseRate),
            lo: pct(res.est - Z * se, res.baseRate), hi: pct(res.est + Z * se, res.baseRate),
            mde: pct(mde(se), res.baseRate)};
  });
  const lim = Math.max(100, ...d.map((r) => Math.max(Math.abs(r.lo), Math.abs(r.hi)))) * 1.05;
  return Plot.plot({
    height: 230, marginTop: 22, marginLeft: width < 640 ? 90 : 310, width: Math.min(width, 900),
    ariaLabel: `Normal reference bands for the ${focus}–peer gap change under four illustrative noise summaries`,
    x: {domain: [-lim, lim], label: `Change in ${focus}–peer gap, % of focal baseline rate`, grid: true},
    y: {label: null, domain: d.map((r) => r.model)},
    marks: [
      Plot.ruleX([0], {stroke: "#1f2a30", strokeDasharray: "3,3"}),
      Plot.ruleY(d, {y: "model", x1: "lo", x2: "hi", stroke: teal, strokeWidth: 6, strokeOpacity: 0.55}),
      Plot.dot(d, {y: "model", x: "est", fill: coral, r: 5}),
      Plot.tip(d, Plot.pointerY({y: "model", x: "est",
        title: (r) => `${r.model}\nNormal reference band: ${fp(r.lo)} to ${fp(r.hi)}\nApproximate 80%-power magnitude: ±${Math.abs(r.mde).toFixed(0)}%`}))
    ]
  });
}
function pathPlot(res, focus) {
  const d = eventPath(res.g, meta.event).filter((r) => r.rel >= -48);
  return Plot.plot({
    height: 280, width: Math.min(width, 560),
    x: {label: "Months relative to Jan 2026"},
    y: {label: `${focus} minus peers (per 100k/month)`, grid: true},
    marks: [
      Plot.ruleY([0], {stroke: grey}),
      Plot.ruleX([-0.5], {stroke: "#1f2a30", strokeDasharray: "3,3"}),
      Plot.line(d, {x: "rel", y: "effect", stroke: teal}),
      Plot.dot(d.filter((r) => r.rel >= 0), {x: "rel", y: "effect", fill: coral, r: 3}),
      Plot.tip(d, Plot.pointerX({x: "rel", y: "effect", title: (r) => `${r.month.slice(0, 7)}: ${f2(r.effect)}`}))
    ]
  });
}
function permPlot(res, focus) {
  return Plot.plot({
    height: Math.max(300, 16 * res.isp.length), marginLeft: 130, width: Math.min(width, 520),
    x: {label: "Post-period gap ÷ pre-period noise (RMSPE ratio)", grid: true},
    y: {label: null},
    marks: [
      Plot.barX(res.isp, {y: "city", x: "ratio", fill: (d) => d.city === focus ? coral : grey,
        sort: {y: "x", reverse: true}}),
      Plot.tip(res.isp, Plot.pointerY({y: "city", x: "ratio", title: (d) => `${d.city}: ${d.ratio.toFixed(2)}`}))
    ]
  });
}

Synthetic-data demonstration. These results cannot establish an effect of Zohran Mamdani’s administration. Read the report and limitations · Less Likely Statistics. Supplied material: Kremu. No registration record was supplied.

Data status

meta.demo
  ? html`<p class="demo-banner"><strong>Demo data.</strong> Every number on this dashboard comes from a synthetic panel and can't be cited. We simulated these results for teaching, and they make no claims about observed NYC outcomes. Seed ${meta.seed}; NYC's added post-event effect is zero. Common shocks multiply different baseline rates, so additive parallel trends need not hold. ${panel.note}</p>`
  : html`<p class="data-note">Source: ${meta.provenance.source_url}. Retrieved ${meta.provenance.retrieved}. Geography: ${meta.provenance.geography}. Revision note: ${meta.provenance.revision_note}. Built ${meta.built}.</p>`

What changes are compatible with six months of data?

{
  const se = primary.ses.inTime, b = primary.baseRate;
  const lo = pct(primary.est - Z * se, b), hi = pct(primary.est + Z * se, b);
  const cc = primary.ses.crossCity;
  return html`
    <p class="lede">Under the own-history normal approximation, the synthetic NYC–peer gap change has a reference band from
      <span class="num">${fp(lo)}</span> to <span class="num">${fp(hi)}</span> of NYC's baseline rate.
      The approximate change needed for 80% power is <span class="num">±${pct(mde(se), b).toFixed(0)}%</span> of that rate.</p>
    <p class="sub">Each bar applies ±1.96 times an illustrative noise summary; demonstrated 95% coverage is not established. These summaries measure different sources of variation. The count-only component excludes peer counts and dependence. The cross-city diagnostic uses spread across peer cities,
      which can be sensitive to differences in city size. This diagnostic is not the site's full interval calculation: ${fp(pct(primary.est - Z * cc, b))} to ${fp(pct(primary.est + Z * cc, b))}.
      This panel is fixed to the primary specification (12-month mean baseline, all ${P.cities.length - 1} peers, equal weights) and doesn't respond to any control.
      Percentages scale a change in the NYC–peer gap by NYC's baseline rate; they are not NYC's own percentage rate change.
      The own-history band uses only ${primary.itp.filter((d) => Number.isFinite(d.estimate)).length} overlapping placebo windows. The report also gives an illustrative <em>t</em> reference; neither reference is calibrated for these dependent draws. The fourth row omits 2020–2023 events, leaving three draws; it is a sensitivity choice, not a better-calibrated model. The MDE is conditional normal planning arithmetic, not measured design power.</p>
    ${strip(primary)}`;
}

NYC minus peers, relative to Dec 2025

pathPlot(primary, meta.focus)

Readout

{
  const b = primary.baseRate, sl = preSlope(primary.g, meta.event, 24), lag = trendLag(12, 6);
  const row = (k, v, why) => html`<tr><td>${k}</td><td class="v">${v}</td><td class="why">${why}</td></tr>`;
  return html`<table class="readout">${[
    row("Estimate", `${f2(primary.est)} per 100k/mo (${fp(pct(primary.est, b))})`, "Mean post gap minus 12-month baseline"),
    row("Peer rank", `${primary.rank} of ${primary.rankTotal}`, `Upper-tail share = ${primary.permRatioP.toFixed(2)}; finite-ratio floor is 1/${primary.rankTotal}`),
    row("Versus NYC's past Januaries", `p = ${primary.inTimeP.toFixed(2)}`, "Fake event dates 2019–2025, including pandemic years"),
    row("Drift that erases it", `${(primary.est / lag).toFixed(4)} per mo`, `Pre-period slope ${sl.slope.toFixed(4)} (SE ${sl.se.toFixed(4)})`)
  ]}</table>`;
}

Compatibility curves: how well each hypothesized change fits the data

{
  const b = primary.baseRate;
  const grid = d3.range(-150, 150.5, 1);
  const d = models.flatMap((m) => sValues(primary.est, primary.ses[m.key], grid.map((g) => g / 100 * b))
    .map((r, i) => ({model: m.label, h: grid[i], p: r.p})));
  return Plot.plot({
    height: 260, width: Math.min(width, 900), ariaLabel: "Conditional normal compatibility curves; line styles identify four noise summaries",
    color: {legend: true, domain: models.map((m) => m.label), range: [teal, "#1f2a30", grey, "#777777"]},
    x: {label: "Hypothesized gap change, % of NYC's baseline rate", grid: true},
    y: {label: "p-value", domain: [0, 1]},
    marks: [Plot.ruleX([0], {strokeDasharray: "3,3"}), ...models.map((m) => Plot.line(d.filter((r) => r.model === m.label), {x: "h", y: "p", stroke: "model", strokeDasharray: m.dash})),
            Plot.tip(d, Plot.pointerX({x: "h", y: "p", stroke: "model",
              title: (r) => `${fp(r.h)}: p = ${r.p.toFixed(2)}, S = ${(-Math.log2(r.p)).toFixed(1)} bits`}))]
  });
}

Surprise, in coin flips

{
  const b = primary.baseRate;
  const rows = sValues(primary.est, primary.ses.inTime, [-0.5, -0.25, 0, 0.25, 0.5].map((x) => x * b));
  return html`<table class="readout">${rows.map((r) =>
    html`<tr><td>${fp(pct(r.hypothesis, b))} change</td><td class="v">${r.s_bits.toFixed(1)} bits</td><td class="why">p = ${r.p.toFixed(2)}</td></tr>`)}</table>
    <p class="sub">An S-value of <em>s</em> bits is as surprising as <em>s</em> heads in a row from a fair coin, under the stated normal model. It is not a probability that the hypothesis is true.</p>`;
}

All ten defensible specifications at once

{
  const d = specs.map((s) => {
    const r = analyze(P, {focus: meta.focus, peers: P.cities, event: meta.event, spec: s, fakeEvents: fake});
    const se = r.ses[specModel];
    return {id: s.id, est: pct(r.est, r.baseRate), lo: pct(r.est - Z * se, r.baseRate),
            hi: pct(r.est + Z * se, r.baseRate), primary: s.id === primarySpec.id};
  });
  return html`<p class="sub">Here are all ten at once, so nobody has to trust the one we picked (or hunt for one they like).
    The emphasized row is the fixed primary specification. These percentages describe changes in the NYC–peer gap, scaled by NYC's baseline rate. They can fall below −100% because peer rates can also change. "t-1" is the site's own baseline.</p>
    ${Plot.plot({
      height: 360, marginLeft: 180, width: Math.min(width, 900),
      x: {label: "NYC–peer gap change, % of NYC's baseline rate", grid: true},
      y: {label: null, domain: d.slice().sort((a, b) => a.est - b.est).map((r) => r.id)},
      marks: [
        Plot.ruleX([0], {strokeDasharray: "3,3"}),
        Plot.ruleY(d, {y: "id", x1: "lo", x2: "hi", stroke: (r) => r.primary ? teal : grey, strokeWidth: (r) => r.primary ? 6 : 3}),
        Plot.dot(d, {y: "id", x: "est", fill: (r) => r.primary ? coral : "#1f2a30", r: (r) => r.primary ? 5.5 : 4}),
        Plot.tip(d, Plot.pointerY({y: "id", x: "est", title: (r) => `${r.id}\n${fp(r.est)} (normal reference: ${fp(r.lo)} to ${fp(r.hi)})`}))
      ]
    })}`;
}

Is NYC unusual among its peers?

html`<p class="sub">Every city takes a turn as the focus, with the same January 2026 date. NYC ranks
  ${primary.rank} of ${primary.rankTotal} finite ratios (upper-tail share = ${primary.permRatioP.toFixed(2)}). We use this ranking as a descriptive comparison.</p>
  ${permPlot(primary, meta.focus)}`

Is January 2026 unusual in NYC’s own past?

{
  const b = primary.baseRate;
  const d = [...primary.itp.map((r) => ({year: r.fake_event.slice(0, 4), est: pct(r.estimate, b), kind: "Placebo"})),
             {year: "2026", est: pct(primary.est, b), kind: "Actual"}];
  return html`<p class="sub">The same estimator at fake January start dates. The 2020–2023 event windows include simulated pandemic-era shocks.
      p = ${primary.inTimeP.toFixed(2)}.</p>
    ${Plot.plot({
      height: 300, width: Math.min(width, 480),
      x: {label: null, type: "point"}, y: {label: "% of NYC's baseline", grid: true},
      marks: [Plot.ruleY([0], {stroke: grey}),
              Plot.dot(d, {x: "year", y: "est", r: 6, fill: (r) => r.kind === "Actual" ? coral : grey}),
              Plot.tip(d, Plot.pointerX({x: "year", y: "est", title: (r) => `${r.year}: ${fp(r.est)}`}))]
    })}`;
}

How much hidden drift would change the reading?

{
  const sl = preSlope(primary.g, meta.event, 24), lag = trendLag(12, 6), b = primary.baseRate;
  const span = 3 * Math.max(Math.abs(sl.slope), sl.se);
  const d = driftAdjust(primary.est, primary.ses.inTime, d3.range(-span, span + span / 100, span / 60), lag)
    .map((r) => ({...r, e: pct(r.estimate, b), l: pct(r.lo, b), h: pct(r.hi, b)}));
  return html`<p class="sub">If NYC and its peers were already drifting apart before January, then part of the gap would be that drift.
      The shaded band uses a naive OLS slope interval that ignores serial correlation. This illustrative linear scenario does not implement HonestDiD inference.</p>
    ${Plot.plot({
      height: 300, width: Math.min(width, 900),
      x: {label: "Assumed drift, per 100k per month", grid: true},
      y: {label: "Adjusted estimate, % of baseline", grid: true},
      marks: [
        Plot.rect([0], {x1: () => sl.slope - Z * sl.se, x2: () => sl.slope + Z * sl.se, fill: teal, fillOpacity: 0.08}),
        Plot.areaY(d, {x: "delta", y1: "l", y2: "h", fill: teal, fillOpacity: 0.2}),
        Plot.line(d, {x: "delta", y: "e", stroke: "#1f2a30"}),
        Plot.ruleY([0], {strokeDasharray: "3,3"})
      ]
    })}`;
}

You’re outside the fixed primary analysis

html`<p class="explore-banner">Settings chosen after seeing the data are exploratory. Trying combinations until one looks unusual is the forking-paths problem, which is the reason this dashboard exists. Anything found here should be reported together with the specification-curve page.</p>`

Your configuration

xRes === null
  ? html`<p class="demo-banner">Select at least 5 peers other than ${xFocus} to estimate anything.</p>`
  : (() => {
      const se = xRes.ses[xModel], b = xRes.baseRate;
      return html`<p class="lede">${xFocus}: <span class="num">${fp(pct(xRes.est, b))}</span>
          (normal reference: ${fp(pct(xRes.est - Z * se, b))} to ${fp(pct(xRes.est + Z * se, b))}).
          Approximate 80%-power change: ±${pct(mde(se), b).toFixed(0)}% of the focal baseline rate. Peer rank ${xRes.rank} of ${xRes.rankTotal} finite ratios.</p>
        ${strip(xRes, xFocus)}`;
    })()

Path

xRes === null ? html`` : pathPlot(xRes, xFocus)

Peer ranking

xRes === null ? html`` : permPlot(xRes, xFocus)

viewof xFocus = Inputs.select(P.cities, {value: meta.focus, label: "Focus city"})
viewof xSpec = Inputs.select(specs, {format: (s) => s.id, value: primarySpec, label: "Specification"})
viewof xModel = Inputs.radio(new Map(models.map((m) => [m.label, m.key])), {value: "inTime", label: "Noise model"})
viewof xPeers = Inputs.checkbox(P.cities, {value: P.cities, label: "Peers (focus is excluded automatically)"})

What this dashboard claims

We don’t claim that the 2026 administration changed homicide in either direction. We illustrate how the calculation responds to sampling variation, simulated trends and analytical choices. The original site’s homicide methodology (checked October 1, 2026) discloses its uncertainty components and limited pretrend power.

Fixed primary specification. Outcome: monthly homicide rate per 100k. Event: January 2026. Post-period: January–June 2026. Baseline: mean of the 12 months before the event. Peers: all cities with a complete series, equally weighted. Primary noise model: NYC’s own in-time placebos.

What the percentages mean. The estimate is the post-period focal–peer gap minus its baseline gap. Dividing this contrast by the focal city’s 12-month baseline rate expresses it as a percentage of that rate. It is not the focal city’s own percentage change and has no universal −100% lower bound.

Noise summaries and sensitivity. With a single treated city, the variance model drives the result. Different city sizes can distort cross-city inference through heteroskedasticity¹. This demo does not establish the direction of distortion for the real site. The independent-Poisson focal count component excludes peer noise and dependence. NYC’s historical spread has seven overlapping draws; omitting 2020–2023 events leaves three. The displayed normal bands and planning MDEs have no demonstrated coverage or power calibration.

Peer ranking. Every city is compared with all the other cities, so the original focal city remains in a placebo’s peer pool. Both RMSPE windows use the chosen pre-baseline mean as their center; for the year-over-year specification, the ratio is still this gap diagnostic rather than a year-over-year residual statistic. The ratio rank is an upper-tail descriptive share. A real focal treatment effect could enter placebo comparator pools.

Verification. The supplied parity tests compare the JavaScript estimators with the R reference implementation to an absolute tolerance of 1e-9. Thus, agreement tells us that the two implementations match, and it says nothing about whether the inferential model is valid.

References. See the report for sources and limitations. Placebo ranks are descriptive diagnostics unless exchangeability of cities or event dates is justified; the normal intervals are model-dependent approximations, not validated causal uncertainty bands.

1. Ferman B, Pinto C. (2019). “Inference in differences-in-differences with few treated groups and heteroskedasticity.” The Review of Economics and Statistics. 101:452–467. doi: 10.1162/rest_a_00759.

Linked from (6)