Google posts draft to formalize Robots Exclusion Protocol Specification
Google open sources its robots.txt parser as part of this announcement. Please visit Search Engine Land for the full article. Search Engine Land: News & Info About SEO, PPC, SEM, Search Engines & Search Marketing https://searchengineland.com/google-posts-draft-to-formalize-robots-exclusion-protocol-specification-318998