The Core

Why We Are Here => Web Development => Topic started by: Rumbas on May 18, 2017, 08:39:13 AM

Title: Bulk index checking?
Post by: Rumbas on May 18, 2017, 08:39:13 AM
I often need to check a current sitemap against actual indexed pages on that domain. Eg. I need to ensure that all pages from the (xml) sitemap have crawled and indexed.
Could be done manually, but I got too many to check.

Any ideas, tools or services which - safely - can check a couple of thousand URLs?
Title: Re: Bulk index checking?
Post by: Adam C on May 18, 2017, 10:39:53 AM
http://urlprofiler.com/ + proxies
Title: Re: Bulk index checking?
Post by: Rumbas on May 18, 2017, 12:49:34 PM
Thanks Adam.

Getting the same answers everywhere; eg. use proxies and a tool. Either Scrapebox or URLprofiler.com

Fired up: http://indexchecking.com/ and it did the job. As least my spots checks showed validity from the tool.
Title: Re: Bulk index checking?
Post by: Drastic on May 18, 2017, 01:16:35 PM
Scrapebox will do it but with unreliable output, or it was last time I used it. Glad you found that, noted.