How to set up a robots.txt to disallow OpenAI's GPTBot from crawling your website for LLM Models Edwin Akrong - 08 Aug 2023