Doing an audit for a big ecommerce client and have all the numbers in:
Site: domain.com shows: 345.000 URL's
Total indexed from Search Console: 595.000 URL's
The XML sitemap has 440.000 URLs, only 364.000 indexed though.
What would you do? What advice? Got a couple of points, but would love your take initially :)
crawl the site and look for duplicates (md5 of the page source) and then create a master real list.
From that I would look to shrink it further, prioritising what I want to rank, rather than what exists.
On all the pages I don't care about I would make all the internal links nofollow and/or noindex and/or link via a single URL that redirects and is itself blocked with noindex / robots.txt.
With the pages that are left I'd move that linkjuice around internally to really show some power on the pages that matter.
Rinse, repeat, keep on going and refining.
Is it Magento? If so check the date on the sitemap.
Also the search console can take time to catch up with changes. I had that.
Have they changed site design recently, so urls?
Quote from: Rumbas on February 03, 2016, 12:49:51 PM
Site: domain.com shows: 345.000 URL's
Total indexed from Search Console: 595.000 URL's
The XML sitemap has 440.000 URLs, only 364.000 indexed though.
If what you're looking for is simply a number of URLs indexed by Google, go for the total indexed according to Search Console Index Status.
site: is and always has been
dodgyThe XML sitemap represents a list of URLs that your client has defined for indexation, but many others may exist (or Google thinks exist but could be robots.txt blocked) that Google will keep in the index.
Index Status could be flaky too, but I have less reason to distrust it than the other measures noted as a representation of indexed URLs from a domain.