RobotPack
by SOHO-iT inc

  RobotPack is a robot-exclusion manager tool to inform robots and other automated search engine tools to not index parts of your web site. It provides FTP access to your website so you can easily select the files and directories which shouldn't be indexed by search engines. When done, it will create the robots.txt file for you and upload it to your server.

RobotPack also comes with the Open Robots Directory (ORDY), which allows you to update the Robots database and share it freely with everyone. This database is updated frequently by our editors over the Internet.

Related software:
robots.txt, robotpack, ordy, open robots directory, webcrawler, robots exclusion standard, bots, spider, ...