Use splats in SSG redirects instead of adding a redirect for every exact prerendered path

[lindsaylevine]

right now there are more redirects than necessary being added to _redirects for dynamic SSG'd pages (for ex, a user like moop-moop may have thousands of paths prerendered via getStaticPaths and each of these is getting its own redirect). we should use a splat instead.

#26 (comment)

cc @moop-moop

[calavera]

a user like moop-moop may have thousands of paths prerendered via getStaticPaths and each of these is getting its own redirec

can you elaborate why we currently do this? I'm trying to understand the problem to see if we can come up with a better solution.

[FinnWoelm]

Hi @calavera, nice to meet you! 😊

We create two redirects for every SSG page to support Next.js preview mode. The redirects look like this:

/my-static-page  /.netlify/functions/next_my_static_page  200!  Cookie=__prerender_bypass,__next_preview_data
/_next/data/%BUILD_ID%/my-static-page.json  /.netlify/functions/next_my_static_page  200!  Cookie=__prerender_bypass,__next_preview_data

In other words, if the special preview-mode cookies are present, we ignore the local, statically-generated-and-prerendered file and instead hit the Netlify function. We need two redirects: One for the actual route (/my-static-page) and one for the "data route". For a large site, this could lead to several thousand rewrites in the _redirects file.

We thought we could tackle this with splats. For example, if a Next.js page with dynamic routing [id].js generates 100 static, prerendered pages (/1, /2, etc...), then we can have just a single rewrite for each the page (/:id) and the data route (/_next/data/%BUILD_ID%/:id.json).

But because the redirects for preview mode are force-redirects (so that we ignore the pre-rendered HTML and JSON files), the splat rewrite/redirect would cause any pages or assets that match /pages/:id to be incorrectly rendered.

For example, a separate Next.js /static HTML page would no longer be rendered in preview mode. Neither would it render correctly an asset like site.xml or my-asset.png in the /public folder, since all those redirects would be force-rewritten to the Netlify function for /[id].js. Our current viewpoint is that using splat redirects for this is not a possibility.

We don't really know how to best address this. One idea we have been daydreaming about would be to have two _redirects files — one without preview mode redirects and one with all redirects, including for preview mode.

By default, Netlify would use the _redirect file without preview mode redirects. This file has only a few redirects and should provide a fast experience for all public users (who are never in preview mode).

When a user is in preview mode and makes a HTTP request, Netlify would detect the presence of those cookies (__prerender_bypass and/or __next_preview_data) and then use the much bigger _redirects file, that includes all the preview mode redirects. These HTTP requests may now be a bit slower due to all the extra redirects, but this only affects users who are in preview mode, who are probably internal users/site editors, where performance is probably less of a concern.

This "daydream" would require Netlify to implement such custom logic specifically for Next.js... 😊

We would be thrilled to hear any ideas and suggestions!

[lindsaylevine]

@FinnWoelm thank you for the thorough breakdown!!! i happened to respond to david (@calavera) in our slack but what you said is still super helpful context, some of which i didn't include. let me break down what david brainstormed on slack:

1. Take all the routes that you want to support in preview mode and generate a json document with them, don't add them to the redirects file
2. Add that json document to the zip file that we use to package the function
3. When AWS unpacks that function, it will put the json file in the same directory
4. Read that file in the function and return a 404 if the incoming request is not in the json file

he said "I don't think that's much better [than what we currently have] but it might be quicker and less visible to our users"

other thoughts quoted from david:

you can load the file outside the lambda handler, and it will be loaded only once for as many requests the function serves, and it's granted to be atomic because it would change only when the deploy is different, and if you want to go the extra evil mile, generate the json payload inside the function's file 🙊
the problem with all this, of course, is that we'd deploy more functions than we could/should because the SHA will change when the routes change
but we don't really enforce the number of times you can deploy a function

[FinnWoelm]

Hi @calavera,

@lindsaylevine told me briefly about your chat. Thanks so much for looking into this and helping us figure out a solution! 😊

I'm not sure if I'm correctly understanding your proposal/brainstorm. I can't quite imagine how it could work. The issue is that preview mode requires forced rewrites for all of these paths, because there are pre-rendered HTML files that are shadowed by default. Any splat redirect/rewrite will break (force rewrite) requests for static html files and static assets that match the rewrite mask, but that are not actually supposed to be served by that particular Netlify Function.

It is valid to have static files/assets, such as /video.mp4 and /my-page.html, as well as hundreds of pre-rendered pages (e.g. /1, /2, /3, etc...) at the same level of URL depth. A rewrite for /:id /.netlify/functions/... 200! would correctly enable preview mode for all our pre-rendered pages, but it would also rewrite our requests for our static files/assets to the Netlify Function. That's of course not correct.

It's a tricky issue. Perhaps I'm misunderstanding your proposal/brainstorm! @lindsaylevine suggested that we could have a chat about it some time (next year)! 😊

[lindsaylevine]

closing because it seems this isn't really a real issue at the moment outside of moop-moop bringing it up, and we can re-address when edge handlers are out.