is a visual editor for Web robot exclusion files that lets you create the ROBOTS.TXT files required to instruct search engines which parts of a Web site are not to be indexed and made searchable by the general public.
RoboGen allows you to log onto your FTP server and select the documents and directories which are not to be made searchable, as well as sort the FTP trees.
It features a database of over 180 known user-agents and ten major search engines, enabling you to edit the database and add additional user-agents.
You can also add, remove, and modify existing agent rules, and manage information for multiple servers.
RoboGen SE runs on
Windows 95/98/Me/NT/2000/Windows 10/11
and is available under the
Time-limited Trial
license
— the installer is 730 KB.
We’ve catalogued it under
Miscellaneous.
✓
Verified clean. Every RoboGen SE build on SoftLookup is scanned for viruses, spyware, adware, trojans and backdoors. We re-test on every update.
Help fellow users decide. Share your experience with RoboGen SE.