# robots.txt # # Robots.txt is used to allow crawlers to index the website. User-agent: * Disallow: