Small Link Sets and the Limits of Statistics
Most link profiles that people analyse are small. Eleven referring domains, forty, two hundred. At those sizes, percentages are unstable, averages are dominated by single rows, and month-over-month changes are indistinguishable from noise. The tools present the same statistics regardless of n, which is where the trouble starts.
The honest move with a small set is to stop computing and start listing. A list of forty domains, read, is more informative than any statistic computed over forty domains.
Why percentages break at small n
A single row is a large fraction of a small set. With 20 referring domains, one link is 5% — so “exact-match anchors rose from 5% to 10%” describes one new link. The percentage makes it sound like a trend.
Worse, the denominator itself moves. Index churn adds and removes rows for reasons unrelated to your site, per lost-link reports and index churn. At n=20, two churn events can swing every percentage in your report by ten points with nothing having happened.
The practical threshold isn’t a number anyone can give you honestly, but the shape of the problem is clear: below roughly a hundred referring domains, report counts; above it, percentages start to be worth computing, with the denominator stated. Treat that as a rule of thumb, not a finding.
Why averages break even faster
Authority scores are compressed at the top, so an average over them is already suspect — the argument in Domain Rating versus Domain Authority. At small n it’s worse: one high-scoring domain can lift a mean by fifteen points.
Hypothetically, twelve referring domains scoring 8, 11, 14, 15, 17, 19, 21, 22, 24, 26, 31, and 88. Mean 24.7. Median 20. The mean’s most informative digit is entirely produced by one row, and the correct description is “eleven modest domains and one strong one,” which no summary statistic conveys. (Illustrative figures, not measurements.)
At n=12, just show the twelve numbers. It fits on a line.
What “no signal” looks like, and why it’s the usual answer
Two failures of statistical honesty are common in link reporting, and they’re opposite.
Reading noise as signal. A profile goes from 41 to 44 referring domains and the report calls it 7% growth. Three domains, one of which may be a scraper. No conclusion is available.
Treating absence of change as failure. A quarter with two new referring domains from a deliberate programme is not obviously a bad quarter; at that volume the variance is larger than the effect anyone could measure. Whether the programme works is not answerable from two data points, in either direction.
The sentence that covers both: “at this volume, month-to-month variation is not distinguishable from normal index churn; the meaningful unit here is the individual link.”
What you can legitimately say with a small set
Quite a lot, as long as it’s about specific links rather than distributions.
Enumerate. “We have 41 referring domains. Here they are, grouped into five categories, with a note on why each link exists where we know.” That’s a real analysis and it’s not statistical.
Characterise composition qualitatively. “Of 41, twelve are directory or profile listings, nine are from partners or suppliers, six are content scrapers, eleven are editorial citations from trade sites, three we can’t explain.” Extremely useful, no inference required.
Name the specific gaps. “None of the four trade publications our competitors are cited by mention us.” Checkable, actionable, not a percentage. Method in what a link gap analysis can and can’t tell you.
Report verified events. New citations, verified removals, with URLs. At small n these are the whole story, and they’re first-party facts rather than index statistics.
State the position relative to competitors in counts. “41 referring domains against 180, 240, and 95 for the three sites outranking us” is a legitimate comparison and needs no statistics at all.
The tests you cannot run
Worth saying explicitly, because tools sometimes imply otherwise.
Before/after on a link. One link, one site, no control. Not a design that supports inference — see how to tell whether a link did anything.
Correlating your own link growth with your own traffic. Two time series from one unit, both trending, both influenced by everything else you did. This is the classic spurious correlation setup, and the sample size is one site.
Significance testing on rank movements. Positions are ordinal, volatile, personalised, and not independent across keywords. Applying a t-test to them produces a p-value with no defensible interpretation.
Extrapolating from your profile to a benchmark. Your 41 domains are not a sample of anything; they’re the population of your links.
How to write the small-n caveat once
Keep a standing sentence and reuse it. Something like: “This profile has 41 referring domains. At that size, percentage and average figures move substantially on single links and are not reported; findings below are stated as counts and specific links.”
That one line pre-empts the request for a percentage, explains why it’s absent, and signals that the omission is deliberate rather than an oversight. It fits the broader habits in building a link report you can defend.
The part that’s genuinely uncomfortable
Small-n honesty makes reports look less impressive. Percentages, averages, and month-over-month deltas read as rigour; counts and lists read as thin. The incentive runs the wrong way, and it’s the main reason unstable statistics appear in link reports at all.
The compensating argument is practical rather than moral. A report built on numbers that move on churn will contradict itself within two quarters, and someone will notice. A report built on enumerated links and stated counts won’t, because every claim in it is checkable and stays true. That’s a slower kind of credibility and it’s the only kind that compounds.