Thread
:
Performance Boost, phase 1
View Single Post
Coda
Developer
#
25
A crawler is a program that loads a website and follows all of the links. Search engines use these in order to know what content you have so that people can search for it. Sometimes it's called a spider. Baidu's crawler, unfortunately, is excessively aggressive. Hopefully taking out the traffic it generates will speed the site up a little more.
And yes, we might have a couple legitimate users in China. I know we have a few users in Asia, I just don't know where.
Posted 02-16-2012, 06:37 PM