Link Signals

Lost-Link Reports and Index Churn

A “lost links” report tells you that a link the vendor’s index previously recorded is no longer being recorded. That is a statement about the index, not about the web. Most large lost-link spikes are recrawl events, template changes, or index maintenance — not links being taken down.

Separating the two is mostly a matter of knowing the handful of mechanisms that produce a “lost” row without anyone removing anything.

The page was recrawled and had changed. The link may have moved into a paginated archive, been pushed off a “recent posts” module, or been removed from a template. Nobody edited an article; the link’s host page simply renders differently now.

The page is gone but the link isn’t the reason. A site redesign, a CMS migration, an expired domain, a paywall, or a robots.txt change all remove the vendor’s ability to see the link. The link’s absence from the index says nothing about editorial intent.

The crawler couldn’t fetch it this time. Timeouts, rate limiting, Cloudflare-style challenges, and geo-blocks all look like absence. Some vendors will drop a link after repeated failures and re-add it later. That produces a lost row followed weeks later by a “new” row for the same link.

The URL now resolves differently. If the linking page moved and the vendor’s index tracks the old URL, you may see one lost row and one new row for what is, editorially, the same citation.

The vendor changed its rules. Dedup policy, link-lifetime policy, and what counts as a distinct link are vendor decisions that get revised. A change there can retire thousands of rows across every customer’s account simultaneously — the class of event covered in when a metric moves but nothing changed.

The symmetric error, and it’s just as common.

Indexes discover fast and forget slowly. If a link is removed today, the vendor won’t know until it recrawls that page, and recrawl intervals for low-priority pages can be long. So your live link count includes links that no longer exist, in unknown proportion.

This is the reason a lost-link report is a lagging indicator in both directions: it over-reports removals that were crawl artefacts, and under-reports removals that haven’t been recrawled. Neither error is random, and neither has a published magnitude.

Triage: which lost rows deserve attention

Sort by whether losing the link would matter, not by when it appeared.

Tier one — verify by hand. Links you or someone on your team specifically earned or placed, links from domains you’d describe as significant, and links to pages that matter commercially. Open the URL. Look for the link. If the page is fine and the link is gone, it’s a real removal.

Tier two — check for a pattern. Multiple lost rows from the same domain in the same window usually means a template or platform change on their side, not a series of decisions about you.

Tier three — ignore. Single links from domains you’ve never heard of, especially in a large batch on the same date. This is the background hum of the index: scraped pages rotating, aggregators expiring, low-value directories dying. Investigating it costs hours and finds nothing.

A rough proportion, hypothetically: of 300 lost rows in a month, perhaps a handful sit in tier one. The report’s default sort will not put them first. (Illustrative.)

The verification step people skip

Open the source URL and look. That’s it — the whole verification protocol, and it settles nearly every case.

Four outcomes and their readings:

  • Page loads, link present. The index is wrong or lagging. Nothing happened.
  • Page loads, link absent. Real removal. Worth noting, possibly worth an email.
  • Page 404s or the domain is gone. Real loss, not a decision about you.
  • Page won’t load for you either. Ambiguous. Likely a fetch problem the crawler also hit; check again in a week before recording anything.

Recording that outcome next to each tier-one row turns a noisy report into three verified facts, which is a far better deliverable than a count.

Report verified removals separately from index-level churn. Two lines: “3 verified removals from domains we’d previously earned links from” and “212 rows dropped from the vendor index this month, unverified — mostly low-value and consistent with normal churn.”

Never report net link change as a performance figure. “Net +40 referring domains” mixes real acquisition, real loss, crawl discovery of old links, and index maintenance into one number, and the mixture ratio is unknown.

Prefer referring-domain-level loss. A domain dropping out entirely is a meaningful event. Losing 3 of 900 links from a domain that still links to you is not.

Give the report a stated window and vendor. Churn rates differ substantially between indexes, so a comparison across tools or across time with different tools is not a comparison.

What this implies for monitoring

If you run alerts on lost links, alert on the tier-one set only — a watchlist of specific URLs or domains — not on the aggregate. Aggregate lost-link alerts will fire on index maintenance and train everyone to ignore the channel. There’s a general version of that problem in alert thresholds for link metrics.

And when someone asks whether you’re losing links: the honest answer is that you’re losing index rows at a rate you can’t fully attribute, and here are the specific verified removals, which is the part that’s actually actionable.

What nobody outside the vendors knows

How long a link persists in an index after removal. How recrawl priority is assigned. What fraction of any lost-link report is genuine. Whether two vendors’ churn rates differ because of crawl frequency or retention policy — the underlying question of what a link index actually contains.

The durable takeaway is smaller than it sounds and more useful: a lost-link report is a to-do list of things to check, and it becomes evidence only after you’ve checked them.

toolsdata-qualitymetrics