Skip to main content

Pagination vs. Incremental Page Loading For Search Engine Crawlability

By June 12, 2024Technical SEO

In this episode of SEO With Chris, I’m talking about pagination vs. incremental page loading. Specifically, how incremental page loading (load more and infinite scroll) pose crawlability issues that pagination does not.

In order for a website’s pages to show up in search results search engines have to be able to find them and that’s where pagination and incremental page loading come into play.

(This content is available in video form, or in a slightly modified text form below the video).

What Is Pagination?

When an e-commerce page, blog page or resources section has too many products/blog posts/articles to show them all on the first page, the page will usually display 10 or 20 or 30 and then you’ll click over to page two in order to see the next batch.

Here’s an example of what pagination looks like. The hiking shirts category page from REI, by default, shows 30 items per page. To see more, you can scroll down to the bottom, and click to page two to see 30 more shirts, and so on and so forth until you reach page 8. Each time you click to another page, the URL will change to /page=2, /page=3, etc.

screenshot of traditional pagination from REI

Historically all websites used this method of making website content available. It’s very easy to implement, it’s easy for humans to use and it’s really good for search engines as well. Search engines have no problem crawling traditional forms of pagination.

Incremental Page Loading

As technology has marched on, we saw the introduction of incremental page loading, which comes in two forms: Load more buttons and infinite scroll.

Load More Buttons

The first type of incremental page loading is the load more or the show more button. In this configuration, you’ll see a default number of products/articles, and a button that you have to click in order to see more. The button usually says “Show More” or “Load More”, but it can be labeled anything. Once you click the button, more articles are dynamically loaded with JavaScript.

Here’s an example from They have a tag page for the Philadelphia Phillies, which displays 20 articles above a “Show More” button. Clicking “Show More” triggers JavaScript to dynamically load another 20 articles. Those articles are not present in the HTML until the Show More button is clicked. This is problematic for search engines, and I’ll show you why in a minute.

show more incremental page loading example from NBC Philadephia

At the time of this publishing, the next article after the “Show More” button is clicked is about a man who met his kidney donor at a Phillies game. But when we inspect the page, and search for it, that link doesn’t appear…until after you click “Show More.” Without additional configuration, search engines cannot find that link.

Infinite Scroll

The second type of incremental page loading is called infinite scroll. Infinite scroll is similar to a load more button, except that you don’t have to click a button, and more content is loaded automatically as you scroll. This configuration also uses JavaScript to dynamically load content which is not present in the HTML prior to scrolling.

Infinite scroll is challenging to show in a screenshot, so check out this example to see what I mean:

Problems For Search Engines

Both methods of incremental page loading cause issues for search engines, and prevent them from comprehensively crawling the site. Here’s what Google says about crawling load more and infinite scroll:

“Once you’ve selected the most appropriate UX strategy for your site and SEO, make sure the Google crawler can find all of your content.

For example, you can implement pagination using links to new pages on your ecommerce site, or using JavaScript to update the current page. Load more and infinite scroll are generally implemented using JavaScript. When crawling a site to find pages to index, Google only follows page links marked up in HTML with <a href> tags. The Google crawler doesn’t follow buttons (unless marked up with <a href>) and doesn’t trigger JavaScript to update the current page contents.”

These two technologies that I just discussed are problematic because Google clearly states that they will not execute the JavaScript needed in order to crawl them.

Finding Crawlability Issues From Incremental Page Loading

We know that pagination is crawlable. Load more, show more, etc. are not crawlable. Infinite scroll is also not crawlable. Here’s how to check for them…

One thing is you can crawl the site with Screaming Frog, Sitebulb or another crawler and look for pages with URLs that have ?page=2, ?page=3, etc. in the URL. That would be an indicator that pagination is present and you may not have an issue. If you crawl the site and don’t find those pages, it may be an indicator that they have a load more or an infinite scroll setup.

If it’s a small site and you know it has e-commerce pages or blog/resources pages, you can go look at those manually and see if incremental loading is present.

Another thing you can do is crawl the site normally and then also crawl the XML sitemap. If the URL counts are off between the sitemap crawl and the regular crawl then that might be an indicator that you have pages that are present in the site map but can’t get discovered because of incremental page loading. (You might also find that you have island pages, which could be a separate issue).

Fixing Crawlability Issues From Incremental Page Loading

There are a few ways to effectively fix crawlability issues created by incremental page loading.

Change To Traditional Pagination

One option is you can implement a traditional style pagination setup. It’s time tested, we know it works, it’s pretty lightweight and it’s crawlable. I’m not a UX person so I can’t speak to the user experience impact of traditional pagination vs. incremental loading, but I do know that traditional pagination is crawlable.

Hidden Pagination

The other option is to implement hidden pagination on pages with an incremental scroll configuration. It’s just a version of pagination that search engines can see but visitors can’t because it’s hidden in the HTML.

Remember the NBC Philadelphia page from earlier? Here’s the hidden pagination they’re using on their incremental loading pages:

screenshot of hidden pagination nbc philadelphia

Hiding Content With CSS 

The other option is loading all of the content in the HTML and then using CSS to hide it underneath a load more/show more button. In that situation, clicking the button simply unhides the content – you’re not dynamically loading it like you would with JavaScript. Search engines can crawl it because they don’t need to click the button – it’s right there in the HTML the whole time.


Whereas traditional pagination is crawlable, incremental loading by default is not, and to fix it you need to implement one of several different configurations. Questions? Comment here or find me on Twitter/X.


Need SEO help? Contact me!

    Chris Berkley

    Chris is a digital marketing consultant specializing in SEO and Analytics across industries including healthcare, education, finance and others.

    Leave a Reply