The first “pillar” of SEO, the good technical construction of the website, is fundamental if we want it to be correctly taken into account by search engines.
The first brick of the Botify suite, Botify Analytics, is a crawler that crawls your entire site as crawlers do. During the crawl, Botify Analytics collects hundreds of metrics on each of the pages it discovers.
Once analyzed, these millions of data items are the subject of a comprehensive report that reveals a wealth of information, allowing identifying blocking points and optimization projects. The report is built around all the priorities of an SEO manager: SEO compliance, internal linking, content quality, performance etc.
All this information is available with a global vision, a vision by segment (you can “divide” your site by page templates, and business logic: product categories, brands, price ranges, etc.) and at the URL link.
Botify Analytics is the only log analyzer dedicated to SEO
If Botify Analytics allows you to simulate the theoretical behaviour of crawlers by crawling your site, Botify Log Analyzer allows you to analyze the real behaviour.
Botify Log Analyzer analyzes HTTP server logs and uses the only reliable way to study robot behaviour comprehensively. On each page browsed by a robot, it requests the server to record this call in a log file. By the way, the server will record precious information: date and time of the call requested URL, return code, user-agent, referer, etc.
Therefore, log files will make it possible to count very reliably the volume of pages crawled by robots and “human” visits from search engines.
With this information, Botify Log Analyzer will allow you to monitor every day:
- Crawl volume by page type
- The crawl volume according to the HTTP code (301 redirects well crawled by Google? Validation of production release? Generation of server errors? Etc ...
- The appearance of new pages and their crawl by search engines
- More advanced scenarios: crawl ratio? After how many days will a non-crawled URL “go out of the index” by Google? Etc...