Gnomit's Spider

Our spider collects the keywords, titles and descriptions of webpages from the web.

Our spider will only collect the homepage of your website and will not traverse any deeper, thus it is very unlikely to cause any excess traffic.

Our spider implements the robots.txt protocol with the User-agent "Gnomit". Our spider will also honor the meta tag "robots". By adding a line like

  <meta name="robots" content="noindex" />

to your websites' header, the content of your homepage will be no further processed and will not be included in our database. If you still want to allow other robots to index your site, you may use the meta name "robots-gnomit".

In your webservers' logfile you can identify Gnomit's Spider with a User-Agent like:

  Mozilla/5.0 (compatible; en-US; +http://www.gnomit.com/info/spider) Gnomit/1.1",