# https://www.robotstxt.org/robotstxt.html User-agent: * Allow: / # Public API resources (docs for humans, LLMs, and OG images) Allow: /api/og Allow: /api/llm.txt Allow: /llms.txt Allow: /api/openapi.json # Disallow authentication and private pages Disallow: /login Disallow: /es/login Disallow: /onboarding/ Disallow: /es/onboarding/ Disallow: /scheduler/ Disallow: /settings/ Disallow: /admin/ Disallow: /api/ Disallow: /offline # Sitemap Sitemap: https://socialrobot.io/sitemap.xml