Gnomit's Spider
Our spider collects the keywords, titles and descriptions of webpages from the web.
Our spider will only collect the homepage of your website and will not traverse
any deeper, thus it is very unlikely to cause any excess traffic.
Our spider implements the
robots.txt protocol with the User-agent "Gnomit".
Our spider will also honor the meta tag "robots". By adding a line like
<meta name="robots" content="noindex" />
to your websites' header, the content of your homepage will be no further processed
and will not be included in our database. If you still want to allow other robots
to index your site, you may use the meta name
"robots-gnomit".
In your webservers' logfile you can identify Gnomit's Spider with a User-Agent
like:
Mozilla/5.0 (compatible; en-US; +http://www.gnomit.com/info/spider) Gnomit/1.1",