# robots.txt for Affrah - أفراح # https://www.robotstxt.org/ User-agent: * Allow: / # Sitemap - full URL required by spec Sitemap: https://affrah.com/sitemap.xml # AI site summary: https://affrah.com/llms.txt # AI crawlers - allow indexing for better discoverability User-agent: GPTBot Disallow: /api/chat Disallow: /ai-chat.xhtml Allow: / User-agent: ChatGPT-User Disallow: /api/chat Disallow: /ai-chat.xhtml Allow: / User-agent: Google-Extended Disallow: /api/chat Disallow: /ai-chat.xhtml Allow: / User-agent: PerplexityBot Disallow: /api/chat Disallow: /ai-chat.xhtml Allow: / User-agent: Claude-Web Disallow: /api/chat Disallow: /ai-chat.xhtml Allow: / User-agent: anthropic-ai Disallow: /api/chat Disallow: /ai-chat.xhtml Allow: / # Block admin and private areas Disallow: /admin/ Disallow: /login Disallow: /register Disallow: /forgot-password Disallow: /companyLogin Disallow: /companyRegister # AI chat - prevent indexing and bot access (interactive, not for crawlers) Disallow: /api/chat Disallow: /ai-chat.xhtml