How to Find Shopify Robots.txt File & Submit Your Sitemap
Your Shopify robots.txt file is located in the root directory of your Shopify store’s primary domain name. It will look like domain name slash robots.txt as in the following example:
examplestore.com/robots.txt
Submitting Your Sitemap
If you think a specific page, file, or content of your Shopify store is blocked by robots.txt and you want to edit it so that Google and other search engines can index it, think again.
You DO NOT need to edit your robots.txt file to ensure Google indexes your website content.
You can submit your sitemap to Google to let it know all about your website’s content, files, and updates. Sitemaps are generated automatically, and they are located in the root directory of your Shopify store’s domains. I.e., katies-apperal.com/sitemap.xml.
A Shopify sitemap file contains links to all your products, pages, blog posts, product images, and collections. Search engines use this file to index your site so that your store’s pages appear in search results.
- For Basic Shopify Plan: Only your store’s primary domain has a generated sitemap file, and it is discoverable by search engines.
- For Advanced Shopify or Shopify Plus Plan: If you use international domains, sitemap files are generated for all of your domains.
Mind that it might take some time for search engines to crawl and index your web pages. Google doesn’t guarantee how long the process will take. You can use Google Search Console to request indexing for each page manually or submit your store’s sitemap.
Here’s how to submit your Shopify sitemap to Search Console:
Step 1. Go to your Search Console account. Click ≡ and select Add Property.
Step 2. From the Select Property Type box, select URL Prefix. Then, enter the Shopify domain you want to add as a property. (Include the https:// part.)
Step 3. Click to Continue. Select HTML Tag on the Verify Ownership window.
Step 4. Highlight the full HTML tag and copy it to your clipboard.
<meta name="google-site-verification" content="IV7BPLESttSpBdxSWN1s4zlr4HIcuHkGQYmE3wLG59w" />
Step 5. Go to Shopify Admin > Online Store > Themes.
Step 6. Find the theme you wish to edit and click Actions, then Edit Code.
Step 7. See the Layout section, find and click theme.liquid.
Step 8. Paste the tag you copied on a blank line directly below the opening <head> tag.
Step 9. Save and return to Search Console to click Verify.
Notice that the verification process doesn’t happen immediately. If you receive an error message, wait 15 minutes to click Verify again.
How to Edit Shopify Robots.txt File
If you are asking, “Can I edit my Shopify Robots.txt file?” the answer is no. You cannot edit a robots.txt file if you are using a hosting service such as Shopify. Shopify automatically handles the robots.txt files for you, and rest assured they are doing that correctly.
For example, Shopify automatically blocks your shopping cart page from showing up in search engines. This helps your store’s SEO since it allows finding the product pages more easily in the search results.
In short, this is to make sure your product pages rank higher than your checkout page.
How to Hide a Shopify Page From Search Engines?
Although you cannot edit your Shopify robots.txt files, you can hide pages that are not included in your robots.txt. It’s possible by customizing the <head> section of your store’s theme.liquid layout file.
1. Open your Shopify admin page and go to Online Store > Themes.
2. Find the theme you want to edit. Then, click Actions > Edit Code.
3. Click theme.liquid.
4. To exclude the search template, paste the code below in the <head> section:
{% if template contains 'search' %}<meta name="robots" content="noindex">{% endif %}
5. If you want to exclude a specific page , paste the code below in the <head> section:
{% if handle contains 'page-handle-you-want-to-exclude' %}<meta name="robots" content="noindex">{% endif %}
- Don’t forget to replace page-handle-you-want-to-exclude with the correct page handle.
6. Save changes.
Source: Shopify
What is a Robots.txt File Used For?
The primary purpose of a robots.txt file is to manage crawler traffic to your site and often to keep a page off Google, depending on the page type.
Web page: You can manage traffic to a web page using robots.txt files, but you can’t hide your web pages from Google search results. It can still appear in search results.
Media file: A robots.txt file can help you manage traffic and prevent image, video, and audio files from appearing in Google search results.
Resource file: Robots.txt files can block resource files such as unimportant script, image, or style files and help you manage traffic to those files.
How to keep a Shopify page out of Google?
If you want to keep a web page out of Google, you should use noindex directives or password protect your page.
Know The Limitations of a Robots.txt File
- A page with robots.txt file can still be indexed by Google if linked by other sites.
- Different crawlers index syntax differently, and some may not understand certain instructions.
- It’s up to the crawler to obey your robots.txt instructions or not. Not all search engines support robots.txt directives.
Note: In all cases, it’s important to test your Shopify store’s robots.txt files. You can use Google’s Robots.txt Tester in that respect.
Helpful articles you might like:
.png)
