What to ask your agency about crawl budget
Every audit we run turns up some version of this. If you are briefing an agency or a freelancer on crawl budget, these questions are worth asking early.
Technical fixes remove obstacles; content earns the position. Both are needed and they are not interchangeable. Write the reasoning down alongside the decision, because the reasoning is what changes first.
Questions worth asking
- Who will actually do this work, and have they done it before?
- How will we know afterwards whether it worked?
- What happens if it needs changing in a year?
- What are you assuming that we have not confirmed?
What a good answer sounds like
Large sites can waste crawling on pages nobody should see. Getting it slightly wrong is survivable. Ignoring it entirely is not. Doing this properly once is usually cheaper than doing it approximately three times.
Clean internal linking is most of the fix. It is worth being explicit about, because assumptions differ quietly. Check it against what you would want a competitor's site to get wrong.
What this looks like day to day
Search work compounds slowly, which is why it gets abandoned about two months before it would have paid off. Three things worth confirming about crawl budget before you move on:
- Someone can say what the current setup is without going to look
- Large sites can waste crawling on pages nobody should see — and you know whether that is true here
- There is a way to tell whether the last change to this helped
The point is not perfection, it is knowing which of these you have consciously chosen to skip.