When search engine robots crawl a website, they will typically first access a site's robots.txt file. The robots.txt file tells bots and other crawlers what is and is not allowed to be crawled on your site.
This is used mainly to avoid overloading your site with requests; it is not a mechanism for keeping a web page out of Google (or other search engines).
IMPORTANT: The robot.txt file is configured specifically to the site owners preferences. For this reason, we do not provide this file by default.
To create a robot.txt file you can search for a generator of your choice or choose from one of the suggested resources below:
- https://www.seoptimer.com/robots-txt-generator
- https://smallseotools.com/robots-txt-generator/
- https://www.sureoak.com/seo-tools/robots-txt-generator
Once you have generated the file send it to vcgsupport@vcgcorporate.com. Please allow up to 2 business days for us to update the website.
If you have any questions or need help generating your sitemap please reach out to our support staff at 800-558-3054 or email us at vcgsupport@vcgcorporate.com.