Send the following on WhatsApp
Continue to ChatQuery : Right way to allow all access to website in robots.txt? user-agent: * disallow: Is this the right way to allow all access to all content of a website to crawlers in a robots.txt? https://vmapp.org/query/102878/Right-Way-To-Allow-All-Access-To-Website-In-Robots-Txt-User-Agent-Disallow-Is-This-The-Right-Way-To-Allow-All-Access-To-All-Content-Of-A-Website-To-Crawlers-In-A-Robots-Txt/