Shopify robots.txt: how to edit it safely
Shopify lets you edit robots.txt.liquid directly. What the default file already blocks, how to add your own rules, and how to test a change safely.
Shopify lets you edit robots.txt.liquid, a theme file that generates the store's robots.txt with your custom rules layered onto sensible platform defaults. Most stores never need to touch it — the default already blocks cart, checkout, account and search pages — but adding rules for a filtered or faceted URL pattern is a common, safe reason to.
Robots.txt is one small piece of the wider Shopify SEO strategy: it controls what gets crawled, not what ranks, and getting it wrong can undo work done everywhere else on the site.
On this page
What does Shopify's default robots.txt already block?
Cart, checkout, account, internal search results and a handful of administrative paths, out of the box, with no configuration needed [1]. These exclusions exist because none of those pages are meant to be search results — a checkout page has no content to rank, and an internal search results page usually just duplicates content that lives elsewhere on the store.
Confirm this by visiting yourstore.com/robots.txt directly. Reading the live file is the only way to know what is actually being served, rather than assuming the platform default matches what you remember reading about it.
How do you add a custom rule to robots.txt.liquid?
By editing the theme file directly, in your theme's code editor, adding lines within the sections the file provides for custom rules rather than rewriting Shopify's defaults [1]. This is meaningfully different from a self-hosted platform, where robots.txt is a plain static file you upload with full control over every line — Shopify's version is generated, with your custom additions layered on top.
Open robots.txt.liquid in the theme code editor
It sits with the theme's other Liquid files, usually under templates or the theme root depending on the theme's structure.
Add your rule inside the designated custom section
Follow the pattern Shopify's own documentation shows rather than guessing at syntax — a malformed rule can be silently ignored.
Preview the theme before publishing
Shopify's theme preview lets you check the change without it going live on the published store.
Fetch the live robots.txt URL after publishing
Confirm the rule reads exactly as intended. This step catches the mistakes preview alone sometimes misses.
What is the most common Shopify robots.txt mistake?
A leftover broad disallow rule, usually added during a theme migration or a staging-to-live move, that ends up blocking far more than intended — sometimes the entire store. It is rare, but it is the single most damaging mistake possible here, since a page nobody can crawl cannot rank regardless of anything else being right about it.
| Good use | Wrong tool for the job |
|---|---|
| Blocking internal search result URLs from crawling | Removing an already-indexed page from search results |
| Excluding a staging or duplicate subdomain | Hiding sensitive data (robots.txt is public) |
| Managing crawl budget on a very large catalog | Fixing a ranking problem on an indexable page |
Does blocking a URL in robots.txt remove it from Google's index?
Not reliably, and this is a common misunderstanding [2]. Robots.txt controls crawling — whether a crawler is allowed to fetch a page — not indexing directly. A URL that was already indexed before being disallowed can remain indexed, often showing with no snippet, because Google can no longer crawl the page to confirm it should be removed.
To actually remove an indexed page, use a noindex meta tag or header, which requires the page to stay crawlable so the directive itself can be read. Using robots.txt for this job typically produces the opposite of the intended result: a worse-looking result that stays indexed rather than a page that is cleanly gone [3].
Should you disallow app-generated or filter URLs?
Often yes, but check what they actually are first. Shopify apps commonly generate their own URL patterns — a wishlist app, a quick-view modal, a currency switcher — some of which produce crawlable, indexable pages that add nothing but crawl waste. A faceted filter combination on a large collection (size, color and price all combined) can multiply into thousands of near-duplicate URLs the same way.
Before writing a disallow rule for a filter pattern, we check whether the pattern is actually being crawled at meaningful volume in Search Console. Blocking something nobody was crawling anyway is wasted effort; not blocking something that is eating real crawl budget is a missed fix.
Confirm the pattern in Search Console's crawl stats or URL inspection tool before writing a rule for it. A rule based on a guess rather than actual crawl data risks either missing the real problem or blocking a pattern that was never an issue in the first place.
Do you need to update robots.txt after a theme change?
Check it every time, even when the change seems unrelated to crawling. A theme update can alter which file governs robots.txt.liquid, occasionally reset custom additions to a default state, or shift a URL pattern the file was written to reference. None of this shows up as an error in the Shopify admin — it only shows up as a rule that silently stopped doing what it used to do.
Build this into your regular update process: after any theme update, fetch the live robots.txt and diff it against what you expect, the same way you would check a canonical tag or a schema block. It takes two minutes and it is the difference between catching a regression immediately and discovering it months later in a traffic report.
When should a Shopify store add rules beyond the default?
When a specific URL pattern is being crawled wastefully and cannot be fixed a better way — a faceted filter combination generating thousands of near-duplicate URLs, or a legacy path from an old app that still resolves but serves nothing useful. Robots.txt is the right tool once you have confirmed canonical tags and redirects are not sufficient on their own.
It is the wrong first move for ordinary duplicate content, like collection-scoped product URLs — those are usually better solved with a correct canonical tag, which lets the preferred version stay indexed while the duplicate is consolidated into it rather than simply hidden from crawling.
How does robots.txt interact with international markets?
Carefully, and it is worth checking specifically if you run Shopify Markets. A rule intended for one market's subfolder can accidentally match a broader path pattern than intended, silently blocking a market you meant to keep crawlable. This is exactly the kind of mistake that survives for months because nothing in the Shopify admin flags it.
The same applies to blog content shared across markets: a rule meant to exclude a staging version of your Shopify blog should be scoped precisely enough that it cannot accidentally catch a live market's blog subfolder too. When in doubt, fetch the live robots.txt for every market subfolder individually rather than assuming one check covers all of them.
How does robots.txt fit alongside a sitemap?
They do different jobs, and both belong in the same file for a reason: robots.txt tells crawlers what not to fetch, while a Sitemap: line inside it points crawlers toward the URLs you do want found. Shopify generates a sitemap automatically, and referencing it from robots.txt is standard practice that most themes already handle without any manual step [3].
Check that the sitemap reference in your live robots.txt actually resolves to a working sitemap URL, particularly after a domain change or a Markets rollout that introduces new subfolders. A broken or stale sitemap reference is a smaller problem than a bad disallow rule, but it is an easy thing to verify while you are already checking the file for everything else in this guide.
Who should own robots.txt changes on an ongoing basis?
One person or one clearly defined process, not whoever happens to be in the theme code that week. Because a robots.txt mistake can be both severe and invisible until traffic drops, it should not be an incidental edit made while someone is in the file for an unrelated reason. Treat it the way you would treat a production database change: reviewed before it ships, verified after it does.
For a small store, this can simply mean one person checks the live file monthly as part of a broader technical review. For a larger catalog with frequent theme updates, it is worth adding a robots.txt fetch to whatever automated monitoring already watches for site outages, so a bad change is caught in minutes rather than in next month's traffic report.
Questions people also ask
Can you edit robots.txt directly on Shopify?
Yes, through a theme file called robots.txt.liquid, which you can add rules to or override sections of without replacing the whole file. This is different from platforms where robots.txt is a static file you upload.
What does Shopify's default robots.txt already block?
Cart, checkout, account and internal search result pages, along with a handful of other administrative paths, are excluded by default. Most stores do not need to touch these rules at all.
How do you test a Shopify robots.txt change before it goes live?
Preview the theme with the change applied and fetch the robots.txt URL directly in a browser or with curl to confirm the rule reads the way you intended, before publishing the theme live.
Can a robots.txt mistake get a Shopify store deindexed?
A broad disallow rule accidentally applied to the whole store, most often during a theme migration, is the realistic worst case. It is rare, but it is the single most damaging robots.txt mistake we have seen.
Does blocking a page in robots.txt remove it from Google's index?
Not reliably. Robots.txt blocks crawling, not indexing — a URL already indexed can still appear with no description if it is later disallowed. A noindex tag, which requires the page to be crawlable, is the correct tool for removing an indexed page.
Sources
- Customize robots.txt — Shopify Developer Documentation (checked 22 September 2026)
- Introduction to robots.txt — Google Search Central (checked 22 September 2026)
- How to write and submit a robots.txt file — Google Search Central (checked 22 September 2026)
Get your Shopify robots.txt reviewed before you edit it
Book a free 30-minute consultation with Ibrahem and Jeremy. We look at your site while we talk, and you leave with a prioritised plan.
Who wrote this
Ibrahem HelalCo-founder
Leads strategy and delivery.
Works on: SEO, GEO
Reviewed before publishing by Jeremy Zava, Co-founder.
- Shopify SEO strategy: the full planA Shopify SEO strategy in order: fix duplicate URLs and speed, build collection pages, add schema and content, then earn links and measure revenue.
- Shopify blog SEO: a working setupHow to structure a Shopify blog so posts support collections instead of sitting alone, plus the template and metafield settings that actually affect search.
- Shopify Markets SEO: domains and hreflangShopify Markets can serve country storefronts by subfolder, subdomain or domain. Here is which setup search engines handle best, and how to get hreflang right.
- Shopify schema markup: what to fix firstShopify's Online Store 2.0 themes already output basic Product JSON-LD. Here is what they leave out, and how to find and fix the gaps without a developer.