Signs it is time to revisit crawl budget

Teams tend to reach for this after something has already gone wrong. A few signals that crawl budget is due some attention.

Technical fixes remove obstacles; content earns the position. Both are needed and they are not interchangeable. Most teams find the first pass takes an afternoon and the maintenance takes minutes a month.

The signals

• Nobody can say when it was last reviewed
• The answer depends on who you ask
• Large sites can waste crawling on pages nobody should see
• Faceted navigation generates infinite URLs if unchecked

How to approach it

Clean internal linking is most of the fix. None of that requires a large budget, only a decision and someone to own it. Assume whoever inherits this will have half your context and none of your patience.

How to tell if yours is fine

Search work compounds slowly, which is why it gets abandoned about two months before it would have paid off. Three things worth confirming about crawl budget before you move on:

• Someone can say what the current setup is without going to look
• Large sites can waste crawling on pages nobody should see — and you know whether that is true here
• There is a way to tell whether the last change to this helped

None of this needs a rewrite. Most of it is a morning's work once someone decides to do it.