Sampling, Export Caps, and Missing Rows

Every backlink export you have ever run was truncated. Row caps, plan limits, per-domain sampling, and default filters all remove rows before the file reaches you, usually without a warning in the file itself. If you then compute percentages over that file, the percentages describe the sample.

The fix isn’t a better tool. It’s knowing which truncation applied and reporting the denominator you actually had.

The five places rows disappear

Plan and export caps. Interfaces cap rows per view and exports cap rows per file, and the cap depends on your subscription. The cap is usually documented; it is almost never visible in the CSV.

Default filters you didn’t set. Many link views default to “one link per domain,” “live links only,” or “exclude subdomains.” Each is a defensible default and each changes your totals substantially. “One link per referring domain” in particular converts a link count into a domain count silently.

Sort-then-cut ordering. When a cap applies, you get the top N by whatever the default sort is — usually a vendor authority score. That’s a biased sample, not a random one. The rows you lose are systematically the low-authority tail, which is exactly where scrapers, junk, and anomalies live.

Per-source aggregation. A domain with 90,000 links to you may be represented by a handful of example rows plus a count. Your file then has 6 rows for something that contributes 90,000 to the headline number.

Index coverage. Before any of the above, the vendor’s crawl never saw some links at all. That’s a different problem — what a link index actually contains — but it compounds with all of these.

Why the bias direction matters

If truncation were random, a sample would still support percentages with wider error bars. It isn’t random. Caps interact with sorts, so truncation removes the tail.

Consequences worth spelling out:

• Toxicity and spam percentages computed on a capped export understate the junk, because the junk sorts last. Whatever a toxicity score tells you, it tells you about the top slice.
• Anchor distributions computed on a capped export overstate branded and editorial anchors for the same reason.
• Country, TLD, and language breakdowns skew toward the high-authority sites, which are not distributed like the tail.
• “New links this week” views are the exception — they’re usually sorted by date, so their truncation removes older rows instead. Different bias, same principle.

How to find your actual denominator

Four checks, two minutes.

1. Compare the export’s row count to the interface’s total. If the file has exactly 10,000 rows and the interface says 214,000 links, you have a cap, not a dataset.
2. Look for a round number. 1,000, 10,000, 100,000, 25 pages of 200. Round row counts are caps. Real profiles do not end on round numbers.
3. Re-run with the sort inverted. Sort ascending by authority and export again. If the two files barely overlap, you’ve measured the size of what you’re missing.
4. Read the filter state, not the filter panel’s default. Note “live links only,” “one per domain,” subdomain handling, and any date window, and write them down next to the file.

What to do when the cap binds

You have four options and they’re all legitimate if labelled.

Segment and export in slices. Split by date range, by target page, or by TLD, and export each slice under the cap. Then reassemble. Tedious, and it produces a genuinely fuller dataset.

Work at the referring-domain level. Domain-level exports are far smaller than link-level ones and are the better unit for most analysis anyway, per referring domains versus total backlinks.

Use the API with pagination if you have access. APIs usually expose the same caps per request but allow you to page past them, subject to units or credits.

Accept the sample and say so. “Analysis over the top 10,000 links by vendor authority score, of approximately 214,000 reported; low-authority tail not included.” That sentence costs nothing and prevents someone rebuilding your analysis and getting different numbers.

The number that goes in the report

Report both: the vendor’s stated total, and the size of the set you analysed.

A hypothetical example: “Vendor reports 214,000 links from 2,140 referring domains. Analysis below is over all 2,140 referring domains (no cap binding at domain level) and over a 10,000-row link sample, top-sorted by authority. Percentages are of the sample.” (Illustrative figures.)

That framing has a useful property: it tells the reader exactly which of your numbers are population statistics and which are sample statistics. Almost no link report does this, and it’s the cheapest credibility available.

The pathology to watch for

The specific failure this causes is a percentage that moves because a cap moved. Upgrade a plan, and the export cap rises; the tail arrives; the “toxic link percentage” jumps. Nothing happened to the site. The measurement instrument changed.

That’s the same class of problem as a vendor recalibrating a score — a change in the observer, reported as a change in the observed. It’s covered generally in when a metric moves but nothing changed, and export caps are its most avoidable instance, because unlike a vendor’s formula, you control this one.

What nobody outside the vendors knows

How per-domain example rows are selected. Whether the sample within an aggregated source is representative or just the first ones crawled. How caps interact with “live only” filters. Whether two vendors’ caps remove comparable fractions of comparable profiles.

Given all that, the durable discipline is small: never publish a percentage without the denominator next to it, and never let the denominator be a cap you didn’t mention.