Plukkie crawler/robot

Project:
Indexing worldwide for new to release worldwide search engine.

Indexing The Netherlands and Belgium to update Dutch/Belgium search engine.

Indexing Luxembourg, Germany, French, United Kingdom and Ireland for a new to release Luxembourg/Germany/French/United Kingdom/Ireland search engine.

If you don't want search engine users can search some or all pages then create or add to a robots.txt file placed in the root web directory from your website a rule for example disallow all pages:

User-agent: plukkie
Disallow: /

It can take some days before crawler plukkie take over the changes in the robots.txt file.

If you think crawler plukkie has a bug in filtering or visiting URLs or reading robots.txt files please email info (at) ykoon (dot) nl so we can investigate the reported bug and find a solution.

Thanks the crawler team.

May 2009