Author Topic: Google pushing new robots.txt standard  (Read 88 times)

littleman

  • Administrator
  • Hero Member
  • *****
  • Posts: 4458
    • View Profile

ergophobe

  • Inner Core
  • Hero Member
  • *
  • Posts: 4834
    • View Profile
Re: Google pushing new robots.txt standard
« Reply #1 on: July 04, 2019, 12:00:28 AM »
>>Google follows at least five redirect hops.

I know John Mueller has said that in Webmaster office hours, but I don't think I've seen that in help docs before.

>>Google currently enforces a size limit of 500 kibibytes (KiB), and ignores content after that limit.

That's a pretty massive robots.txt. I wonder how many are out there that go beyond that.