Jun 28, 2022
Any seasoned SEO professional will tell you how important website crawling is to maintaining strong technical SEO. But here are the better questions: How often should Banner Design you perform website crawls? And how often do they actually perform them ? In this article, we will discuss what SEO publications suggest as “best practice” web crawling cadence and the actual rate of performance of SEO professionals. Next, I'll demonstrate the benefits of an Banner Design accelerated web crawl cadence by sharing a case study from FOX. What is a website crawl, anyway? Using specialized tools like Screaming Frog or DeepCrawl, you can peek “under the hood” of a website – much Banner Design like a mechanic does when inspecting cars. But instead of inspecting the mechanical parts of a car, you inspect the optimizable elements of a website, including the quality of its metadata, XML sitemaps, response codes, and more. When something isn't working as expected in SEO, it's up to you to diagnose the problem and find the solution to fix it. How often should you perform technical Banner Design website crawls for SEO? Example of the Screaming Frog SEO Spider UI. What are industry experts saying about website crawl cadence “best practices”? Industry publications seem to agree that 'mini' technical audits should be conducted on a monthly basis and 'in-depth' technical audits should Banner Design be conducted on a quarterly or semi-annual basis. Advertising Continue reading below However, few discussions specifically address the “optimal frequency” to perform website crawls: "Part of your ongoing SEO strategy includes regular audits so you can find and fix issues quickly (we recommend quarterly)." - Erika Varagouli, SEMrush “It is recommended to Banner Design perform an automated scan once a month. This will often be enough to raise major issues, such as any errors on the page in the form of broken links, page titles and metadata, or duplicate content. - Institute of Digital Marketing “I perform an SEO audit for my clients in the first month, monthly (mini audit) and Banner Design quarterly (in-depth audit).” - Anna Crowe, Journal of Search Engines A website crawl and a technical audit are not the same thing (thanks for the clear separation of the two, Barry Adams).