The Core

Why We Are Here => Web Development => Topic started by: Brad on July 26, 2018, 07:48:18 PM

Title: MIT have a bot?
Post by: Brad on July 26, 2018, 07:48:18 PM
I've looked at the bot threads at WmW but didn't find much.

I have two, of what I take are bots, tag teaming Wordpress.

One claims to be from Cambridge Mass., MIT

The other just US.

Both are checking ever tag, category.  And some posts and pages.  They might be working together.

Anyone heard of MIT having a bot?
Title: Re: MIT have a bot?
Post by: ergophobe on July 26, 2018, 08:45:39 PM
They certainly have a lot of research on crawling the web, most of it from back in the 20th century.
site:mit.edu web crawler

But more recently....
http://news.mit.edu/2016/artificial-intelligence-system-surfs-web-improve-performance-1110