Crawler

TasteRexBot

TasteRexBot reads public restaurant menus so diners can find and rate individual dishes. It identifies itself with the user agent TasteRexBot/1.0 (+https://tasterex.com/bot).

How it behaves

• Reads public pages only. It never signs in, submits forms or gets around paywalls.
• Follows robots.txt.
• Makes one request at a time per site, with a pause between requests.
• Looks for menu pages only, and publishes dish names only. See where menu data comes from.

Blocking it

Add this to your site's robots.txt:

User-agent: TasteRexBot
Disallow: /

Your restaurant stays listed so diners can still rate it, and you can manage your menu yourself byclaiming your restaurant. Questions or problems:restaurants@tasterex.com.