# robotstxt.org User-agent: * Sitemap: https://getyarn.io/sitemap.xml User-agent: * Disallow: /api/ User-agent: GumGum-Bot Disallow: / User-agent: SemrushBot Disallow: / User-agent: SemrushBot-SA Disallow: / User-agent: SemrushBot-BA Disallow: / User-agent: SemrushBot-SI Disallow: / User-agent: SemrushBot-SWA Disallow: / User-agent: SemrushBot-CT Disallow: / User-agent: SemrushBot-BM Disallow: / User-agent: AhrefsBot Disallow:/ User-agent: grapeshot Disallow: / User-agent: DotBot Disallow: / User-agent: MJ12bot Disallow: /