Magento 2 Technical SEO Checklist for Developers

Magento 2 gives you most of the technical SEO controls you need, but the defaults are not tuned for a large catalog, and a few settings interact in ways that create duplicate URLs or thin pages by the thousand. This checklist is what I go through on a Magento 2.4 store before anyone starts writing meta descriptions. It is about crawlability, indexation and URL hygiene, the parts a developer controls.

1. URLs and rewrites

Start with the settings under Stores > Configuration > Catalog > Catalog > Search Engine Optimization and Stores > Configuration > General > Web.

• Use Web Server Rewrites (web/seo/use_rewrites) must be Yes, so URLs do not contain index.php.
• Auto redirect to Base URL (web/url/redirect_to_base) should send a 301, so the bare domain and the www version do not both answer.
• Use Categories Path for Product URLs (catalog/seo/product_use_categories). When it is on, a product in three categories gets three URLs. Unless you have a strong reason, keep it off and let products live at one short URL.
• Generate category/product URL Rewrites (catalog/seo/generate_category_product_rewrites). With category paths off, turning this off as well stops Magento from writing rewrite rows for every product and category combination. On large catalogs this keeps url_rewrite from growing into millions of rows.
• Create Permanent Redirect for URLs if URL Key Changed (catalog/seo/save_rewrites_history) should stay on, so a renamed product redirects instead of returning 404.

You can set these from the CLI, which makes them part of your deployment notes:

bin/magento config:set web/seo/use_rewrites 1
bin/magento config:set catalog/seo/product_use_categories 0
bin/magento config:set catalog/seo/save_rewrites_history 1
bin/magento config:set catalog/seo/product_canonical_tag 1
bin/magento config:set catalog/seo/category_canonical_tag 1

Changing URL settings on a live store has side effects on existing rewrites. If the table is already bloated or full of redirect chains, clean it first. The process is in URL rewrites cleanup.

2. Canonical tags

Enable Use Canonical Link Meta Tag For Products and For Categories. The product canonical points to the short product URL, which matters if category paths are on anywhere. The category canonical points to the category URL without query parameters, which covers filter and sort parameters from layered navigation.

One known side effect: the category canonical also drops the page parameter, so page 2 of a category declares page 1 as canonical. Search engines can then treat deeper pages as duplicates and crawl products linked only from them less often. If your category pages are long, consider a customization that keeps p in the canonical while still dropping filters and sort order. Test it carefully, because it changes the head of every category page.

3. Layered navigation and faceted URLs

Layered navigation is the biggest source of crawl waste on Magento stores. Every filter combination is a URL like /shoes.html?color=49&size=167&product_list_order=price, and crawlers will follow them all if the links are plain anchors.

The canonical tag tells search engines which URL to index, but it does not stop crawling. On a big catalog you also want to:

• Disallow sort, limit and mode parameters in robots.txt, since they never produce useful landing pages.
• Decide which filters deserve indexable landing pages, such as brand or main material, and build those as real category pages or SEO landing pages instead of relying on query strings.
• Keep the remaining filter links from being crawled, with robots.txt rules on their parameters.

If you build custom filters, decide at that point how their URLs are formed and whether they should be crawlable.

4. robots.txt and meta robots

Magento generates robots.txt from configuration. Go to Content > Design > Configuration, edit the scope and open Search Engine Robots. There you set the default meta robots value and the custom robots.txt instructions. A starting point for most stores:

User-agent: *
Disallow: /checkout/
Disallow: /customer/
Disallow: /wishlist/
Disallow: /catalogsearch/
Disallow: /sendfriend/
Disallow: /review/product/list/
Disallow: /*?*product_list_order=
Disallow: /*?*product_list_limit=
Disallow: /*?*product_list_mode=
Disallow: /*?*SID=

Sitemap: https://www.example.com/sitemap.xml

The most important check on staging and development environments is the opposite: default robots should be NOINDEX,NOFOLLOW there and INDEX,FOLLOW on production. A staging database copied to production, or the reverse, is a classic way to deindex a live store. After every database refresh, check the value.

Search result pages should not be indexed. Blocking /catalogsearch/ in robots.txt stops crawling, and a noindex meta tag covers pages that were already found. You can set it per layout handle in your theme:

<?xml version="1.0"?>
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
    <head>
        <meta name="robots" content="NOINDEX,FOLLOW"/>
    </head>
</page>

Save it as catalogsearch_result_index.xml in your theme's Magento_CatalogSearch/layout folder. Keep in mind that a page blocked in robots.txt cannot have its meta robots read, so for already indexed search pages, let them be crawled until they drop out, then add the disallow rule.

5. XML sitemap

Configure the sitemap under Stores > Configuration > Catalog > XML Sitemap and create the file in Marketing > SEO & Search > Site Map. Points to check:

• Generation is enabled and runs on cron, so the file is not months old.
• The sitemap includes only URLs that return 200 and are canonical. Disabled products and products not visible individually should not be in it.
• Product image entries are useful for image search but make files much larger. Turn them off if generation gets slow.
• Each store view that is a separate site gets its own sitemap, and robots.txt for that domain points to it.

6. Status codes

Crawl a sample of URLs and check the responses:

• Disabled products should return 404. If you want to keep their link equity, add a 301 to the closest category with a custom URL rewrite.
• Deleted categories should not leave orphan rewrites that point to nowhere.
• Redirect chains, where A goes to B and B to C, should be collapsed. They pile up with every URL key change.
• The 404 page should return a real 404 status, not 200. The default CMS no-route page does this correctly, but custom themes sometimes break it.

Out of stock products are a business decision. If Display Out of Stock Products is off, those products disappear from listings but their pages still load. That is usually fine for temporary stockouts. For discontinued products, redirect them.

7. HTTPS and mixed content

All base URLs should be HTTPS, including the unsecure base URL. Check CMS blocks and product descriptions for hard coded http:// image links, which cause mixed content warnings. A simple SQL search on cms_block, cms_page and the text attribute tables finds them.

8. Performance as a ranking input

Core Web Vitals are part of page experience signals, and Magento storefronts with Luma often fail on LCP and INP on mobile. Full page cache with Varnish, image optimization and a lighter frontend make the biggest difference. The detailed checklist is in Magento 2 Core Web Vitals.

9. Structured data and international SEO

Luma outputs some schema.org microdata on product pages, but it is incomplete for rich results. Most stores should replace it with JSON-LD, as explained in structured data for Magento 2. If you run several languages or countries, Magento does not output hreflang tags at all. See hreflang for multilanguage Magento stores.

10. Things to recheck after every deployment

• Default meta robots per store view.
• robots.txt content and the sitemap reference.
• Canonical tags on a product, a category and a filtered category.
• One disabled product URL returning 404.
• No new blocks marked uncacheable on product or category layouts, which would hurt speed.

FAQ

Should I use category paths in Magento product URLs?

Usually no. Category paths give each product several URLs depending on the category, which relies on canonical tags to fix duplication. Short product URLs are simpler and keep the rewrite table smaller.

Does Magento add canonical tags by default?

Magento supports canonical tags for products and categories, controlled by two settings in Catalog Search Engine Optimization. Check that both are enabled on every store view.

Where do I edit robots.txt in Magento 2?

In Content, Design, Configuration. Edit the website or store view scope and open the Search Engine Robots section. The custom instructions there are served as robots.txt.

Should layered navigation URLs be indexed?

Generally not. Keep filter and sort URLs out of the index and build dedicated landing pages for the few filter values that match real search demand.

I am available for freelance Magento and Adobe Commerce work, including technical SEO audits. If you need help with this, get in touch.