Skip to content

feat: refresh hooks api implementation #1950

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 48 commits into from
Mar 22, 2023
Merged
Show file tree
Hide file tree
Changes from 41 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
55d6022
feat: subclass NextServer to override request handler
orinokai Feb 23, 2023
c3d5607
feat: add revalidate API route
orinokai Feb 23, 2023
3030689
feat: subclass NextServer to override request handler
orinokai Feb 23, 2023
699ad3a
feat: add revalidate API route
orinokai Feb 23, 2023
8033d60
Merge branch 'rs/refresh-hooks-poc' of github.com:netlify/netlify-plu…
orinokai Mar 1, 2023
3c805d4
feat: add refresh hooks api implementation
orinokai Mar 1, 2023
796e458
chore: increase revalidate ttl for testing
orinokai Mar 1, 2023
bf3f8d9
chore: add time to page for testing
orinokai Mar 1, 2023
2a2da7e
feat: add error reporting for caught exceptions
orinokai Mar 1, 2023
cae0a91
fix: use req.url verbatim
orinokai Mar 1, 2023
b04cb8e
chore: test throw in revalidate handler
orinokai Mar 1, 2023
de649e9
chore: fix lint issue
orinokai Mar 1, 2023
98407ea
fix: update revalidate error handling
orinokai Mar 1, 2023
f691362
fix: error handling in revalidate api function
orinokai Mar 1, 2023
443beca
chore: refactor revalidate handling for easier testing
orinokai Mar 6, 2023
e043e9d
test: add tests for refresh hooks
orinokai Mar 6, 2023
95f7c94
test: add e2e test for refresh hooks revalidate
orinokai Mar 6, 2023
238596e
feat: revalidate data routes
orinokai Mar 6, 2023
cf0fc72
fix: export named NetlifyNextServer for module interop
orinokai Mar 6, 2023
0f7d1db
fix: revalidate error messaging in demo
orinokai Mar 6, 2023
8cc7abe
fix: remove revalidate data routes for milestone 1
orinokai Mar 6, 2023
fef98a1
fix: amend test for revalidate success/failure
orinokai Mar 6, 2023
bf97d73
feat: add additional paths for routes
orinokai Mar 10, 2023
fef1c59
chore: refactor tests
orinokai Mar 10, 2023
99ed48d
chore: add tests for handlerUtils
orinokai Mar 10, 2023
3ec181f
chore: remove eslint directive
orinokai Mar 10, 2023
6b4a2ce
test: remove rsc trailing slash
orinokai Mar 10, 2023
021f825
Revert "test: remove rsc trailing slash"
orinokai Mar 10, 2023
a40925f
feat: source and transform data routes from prerenderManifest
orinokai Mar 14, 2023
6aa81df
feat: update revalidate page with more test scenarios
orinokai Mar 14, 2023
8f60f43
fix: revalidate api endpoint path order
orinokai Mar 14, 2023
9fc4123
chore: update revalidate test paths
orinokai Mar 17, 2023
0e61a26
fix: cache and normalize manifest routes
orinokai Mar 20, 2023
a8ea2bd
chore: update tests for caching and localizing routes
orinokai Mar 20, 2023
36acbdb
chore: fix eslint complaint
orinokai Mar 20, 2023
d53a11d
chore: update cypress tests
orinokai Mar 21, 2023
4e8daa5
fix: consolidated static route matching for revalidate with i18n
orinokai Mar 21, 2023
8bcdb1c
chore: improve comments on revalidate api route
orinokai Mar 21, 2023
ce62c92
chore: update snapshots and cypress tests
orinokai Mar 21, 2023
3da9e46
Merge branch 'main' into rs/refresh-hooks-api
orinokai Mar 21, 2023
228ed12
Merge branch 'main' into rs/refresh-hooks-api
orinokai Mar 21, 2023
f85d2f5
chore: reset with-revalidate TTL after testing
orinokai Mar 22, 2023
a197783
chore: add comment about paths to revalidate endpoint
orinokai Mar 22, 2023
5da076d
Merge branch 'main' into rs/refresh-hooks-api
orinokai Mar 22, 2023
d029976
chore: update cypress test to match old ttl for revalidate api
orinokai Mar 22, 2023
6ef27a5
Merge branch 'rs/refresh-hooks-api' of github.com:netlify/netlify-plu…
orinokai Mar 22, 2023
1445c26
chore: move tests to resolve edge clash
orinokai Mar 22, 2023
fd32f38
chore: add comments to functions generation
orinokai Mar 22, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion cypress/integration/default/dynamic-routes.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ describe('Static Routing', () => {
it('renders correct page via ODB on a static route', () => {
cy.request({ url: '/getStaticProps/with-revalidate/', headers: { 'x-nf-debug-logging': '1' } }).then((res) => {
expect(res.status).to.eq(200)
expect(res.headers).to.have.property('x-nf-render-mode', 'odb ttl=60')
expect(res.headers).to.have.property('x-nf-render-mode', 'odb ttl=300')
expect(res.body).to.contain('Dancing with the Stars')
})
})
Expand Down
69 changes: 69 additions & 0 deletions cypress/integration/default/revalidate.spec.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
describe('On-demand revalidation', () => {
it('revalidates static ISR route with default locale', () => {
cy.request({ url: '/api/revalidate/?select=0' }).then((res) => {
expect(res.status).to.eq(200)
expect(res.body).to.have.property('message', 'success')
})
})
it('revalidates static ISR route with non-default locale', () => {
cy.request({ url: '/api/revalidate/?select=1' }).then((res) => {
expect(res.status).to.eq(200)
expect(res.body).to.have.property('message', 'success')
})
})
it('revalidates root static ISR route with default locale', () => {
cy.request({ url: '/api/revalidate/?select=2' }).then((res) => {
expect(res.status).to.eq(200)
expect(res.body).to.have.property('message', 'success')
})
})
it('revalidates root static ISR route with non-default locale', () => {
cy.request({ url: '/api/revalidate/?select=3' }).then((res) => {
expect(res.status).to.eq(200)
expect(res.body).to.have.property('message', 'success')
})
})
it('revalidates dynamic prerendered ISR route with default locale', () => {
cy.request({ url: '/api/revalidate/?select=4' }).then((res) => {
expect(res.status).to.eq(200)
expect(res.body).to.have.property('message', 'success')
})
})
it('fails to revalidate dynamic non-prerendered ISR route with fallback false', () => {
cy.request({ url: '/api/revalidate/?select=5', failOnStatusCode: false }).then((res) => {
expect(res.status).to.eq(500)
expect(res.body).to.have.property('message')
expect(res.body.message).to.include('Invalid response 404')
})
})
it('revalidates dynamic non-prerendered ISR route with fallback blocking', () => {
cy.request({ url: '/api/revalidate/?select=6' }).then((res) => {
expect(res.status).to.eq(200)
expect(res.body).to.have.property('message', 'success')
})
})
it('revalidates dynamic non-prerendered ISR route with fallback blocking and non-default locale', () => {
cy.request({ url: '/api/revalidate/?select=7' }).then((res) => {
expect(res.status).to.eq(200)
expect(res.body).to.have.property('message', 'success')
})
})
it('revalidates dynamic prerendered appDir route', () => {
cy.request({ url: '/api/revalidate/?select=8' }).then((res) => {
expect(res.status).to.eq(200)
expect(res.body).to.have.property('message', 'success')
})
})
it('fails to revalidate dynamic non-prerendered appDir route', () => {
cy.request({ url: '/api/revalidate/?select=9' }).then((res) => {
expect(res.status).to.eq(200)
expect(res.body).to.have.property('message', 'success')
})
})
it('revalidates dynamic prerendered appDir route with catch-all params', () => {
cy.request({ url: '/api/revalidate/?select=10' }).then((res) => {
expect(res.status).to.eq(200)
expect(res.body).to.have.property('message', 'success')
})
})
})
25 changes: 25 additions & 0 deletions demos/default/pages/api/revalidate.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
export default async function handler(req, res) {
const query = req.query
const select = Number(query.select) || 0

const paths = [
'/getStaticProps/with-revalidate/', // valid path
'/fr/getStaticProps/with-revalidate/', // valid path (with locale)
'/', // valid path (index)
'/fr/', // valid path (index with locale)
'/getStaticProps/withRevalidate/2/', // valid path (with dynamic route)
'/getStaticProps/withRevalidate/3/', // invalid path (fallback false with dynamic route)
'/getStaticProps/withRevalidate/withFallbackBlocking/3/', // valid path (fallback blocking with dynamic route)
'/fr/getStaticProps/withRevalidate/withFallbackBlocking/3/', // valid path (fallback blocking with dynamic route and locale)
'/blog/nick/', // valid path (with prerendered appDir dynamic route)
'/blog/greg/', // invalid path (with non-prerendered appDir dynamic route)
'/blog/rob/hello/', // valid path (with appDir dynamic route catch-all)
]

try {
await res.revalidate(paths[select])
return res.json({ code: 200, message: 'success' })
} catch (err) {
return res.status(500).send({ code: 500, message: err.message })
}
}
7 changes: 4 additions & 3 deletions demos/default/pages/getStaticProps/with-revalidate.js
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
import Link from 'next/link'

const Show = ({ show }) => (
const Show = ({ show, time }) => (
<div>
<p>This page uses getStaticProps() to pre-fetch a TV show.</p>
<p>This page uses getStaticProps() to pre-fetch a TV show at {time}</p>

<hr />

Expand All @@ -22,9 +22,10 @@ export async function getStaticProps(context) {
return {
props: {
show: data,
time: new Date().toISOString(),
},
// ODB handler will use the minimum TTL=60s
revalidate: 1,
revalidate: 300,
}
}

Expand Down
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@
"testMatch": [
"**/test/**/*.spec.js",
"**/test/**/*.spec.ts",
"**/*.test.ts",
"!**/test/e2e/**",
"!**/test/fixtures/**",
"!**/test/sample/**",
Expand Down
8 changes: 8 additions & 0 deletions packages/runtime/src/helpers/functions.ts
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,10 @@ export const generateFunctions = async (
await ensureDir(join(functionsDir, functionName))
await writeFile(join(functionsDir, functionName, `${functionName}.js`), apiHandlerSource)
await copyFile(bridgeFile, join(functionsDir, functionName, 'bridge.js'))
await copyFile(
join(__dirname, '..', '..', 'lib', 'templates', 'server.js'),
join(functionsDir, functionName, 'server.js'),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we get some comments on why we're copying this file? (This whole functions.ts file could use those tbh hah, but I won't put that on you)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep, i totally agree! i don't know enough about some of the functions in there to comment it all, but i've added comments to generateFunctions() in fd32f38

)
await copyFile(
join(__dirname, '..', '..', 'lib', 'templates', 'handlerUtils.js'),
join(functionsDir, functionName, 'handlerUtils.js'),
Expand All @@ -67,6 +71,10 @@ export const generateFunctions = async (
await ensureDir(join(functionsDir, functionName))
await writeFile(join(functionsDir, functionName, `${functionName}.js`), handlerSource)
await copyFile(bridgeFile, join(functionsDir, functionName, 'bridge.js'))
await copyFile(
join(__dirname, '..', '..', 'lib', 'templates', 'server.js'),
join(functionsDir, functionName, 'server.js'),
)
await copyFile(
join(__dirname, '..', '..', 'lib', 'templates', 'handlerUtils.js'),
join(functionsDir, functionName, 'handlerUtils.js'),
Expand Down
36 changes: 21 additions & 15 deletions packages/runtime/src/templates/getHandler.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,7 @@ import { outdent as javascript } from 'outdent'

import type { NextConfig } from '../helpers/config'

import type { NextServerType } from './handlerUtils'

/* eslint-disable @typescript-eslint/no-var-requires */

const { promises } = require('fs')
const { Server } = require('http')
const path = require('path')
Expand All @@ -22,17 +19,17 @@ const {
getMaxAge,
getMultiValueHeaders,
getPrefetchResponse,
getNextServer,
normalizePath,
} = require('./handlerUtils')
const { NetlifyNextServer } = require('./server')
/* eslint-enable @typescript-eslint/no-var-requires */

type Mutable<T> = {
-readonly [K in keyof T]: T[K]
}

// We return a function and then call `toString()` on it to serialise it as the launcher function
// eslint-disable-next-line max-params
// eslint-disable-next-line max-params, max-lines-per-function
const makeHandler = (conf: NextConfig, app, pageRoot, staticManifest: Array<[string, string]> = [], mode = 'ssr') => {
// Change working directory into the site root, unless using Nx, which moves the
// dist directory and handles this itself
Expand Down Expand Up @@ -68,22 +65,30 @@ const makeHandler = (conf: NextConfig, app, pageRoot, staticManifest: Array<[str
// We memoize this because it can be shared between requests, but don't instantiate it until
// the first request because we need the host and port.
let bridge: NodeBridge
const getBridge = (event: HandlerEvent): NodeBridge => {
const getBridge = (event: HandlerEvent, context: HandlerContext): NodeBridge => {
const {
clientContext: { custom: customContext },
} = context

if (bridge) {
return bridge
}
const url = new URL(event.rawUrl)
const port = Number.parseInt(url.port) || 80
base = url.origin

const NextServer: NextServerType = getNextServer()
const nextServer = new NextServer({
conf,
dir,
customServer: false,
hostname: url.hostname,
port,
})
const nextServer = new NetlifyNextServer(
{
conf,
dir,
customServer: false,
hostname: url.hostname,
port,
},
{
revalidateToken: customContext.odb_refresh_hooks,
},
)
const requestHandler = nextServer.getRequestHandler()
const server = new Server(async (req, res) => {
try {
Expand Down Expand Up @@ -119,7 +124,7 @@ const makeHandler = (conf: NextConfig, app, pageRoot, staticManifest: Array<[str
process.env._NETLIFY_GRAPH_TOKEN = graphToken
}

const { headers, ...result } = await getBridge(event).launcher(event, context)
const { headers, ...result } = await getBridge(event, context).launcher(event, context)

// Convert all headers to multiValueHeaders

Expand Down Expand Up @@ -180,6 +185,7 @@ export const getHandler = ({ isODB = false, publishDir = '../../../.next', appDi
// We copy the file here rather than requiring from the node module
const { Bridge } = require("./bridge");
const { augmentFsModule, getMaxAge, getMultiValueHeaders, getPrefetchResponse, getNextServer, normalizePath } = require('./handlerUtils')
const { NetlifyNextServer } = require('./server')

${isODB ? `const { builder } = require("@netlify/functions")` : ''}
const { config } = require("${publishDir}/required-server-files.json")
Expand Down
82 changes: 82 additions & 0 deletions packages/runtime/src/templates/handlerUtils.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
import { normalizeRoute, unlocalizeRoute, localizeRoute, localizeDataRoute } from './handlerUtils'

describe('normalizeRoute', () => {
it('removes a trailing slash from a route', () => {
expect(normalizeRoute('/foo/')).toEqual('/foo')
})
it('ignores a string without a trailing slash', () => {
expect(normalizeRoute('/foo')).toEqual('/foo')
})
it('does not remove a lone slash', () => {
expect(normalizeRoute('/')).toEqual('/')
})
})

describe('unlocalizeRoute', () => {
it('removes the locale prefix from an i18n route', () => {
expect(
unlocalizeRoute('/fr/foo', {
defaultLocale: 'en',
locales: ['en', 'fr', 'de'],
}),
).toEqual('/foo')
})
it('removes the locale prefix from a root i18n route', () => {
expect(
unlocalizeRoute('/de', {
defaultLocale: 'en',
locales: ['en', 'fr', 'de'],
}),
).toEqual('/')
})
it('does not modify a default locale route', () => {
expect(
unlocalizeRoute('/foo', {
defaultLocale: 'en',
locales: ['en', 'fr', 'de'],
}),
).toEqual('/foo')
})
})

describe('localizeRoute', () => {
it('adds the locale prefix to an i18n route', () => {
expect(
localizeRoute('/foo', {
defaultLocale: 'en',
locales: ['en', 'fr', 'de'],
}),
).toEqual('/en/foo')
})
it('adds the locale prefix to a root i18n route', () => {
expect(
localizeRoute('/', {
defaultLocale: 'en',
locales: ['en', 'fr', 'de'],
}),
).toEqual('/en')
})
it('does not modify a prefixed i18n route', () => {
expect(
localizeRoute('/en/foo', {
defaultLocale: 'en',
locales: ['en', 'fr', 'de'],
}),
).toEqual('/en/foo')
})
})

describe('localizeDataRoute', () => {
it('adds the locale prefix to a data route', () => {
expect(localizeDataRoute('/_next/data/build/foo.json', '/en/foo')).toEqual('/_next/data/build/en/foo.json')
})
it('removes the index suffix from a root route', () => {
expect(localizeDataRoute('/_next/data/build/index.json', '/en')).toEqual('/_next/data/build/en.json')
})
it('does not add the locale prefix if it already exists in the data route', () => {
expect(localizeDataRoute('/_next/data/build/en/foo.json', '/en/foo')).toEqual('/_next/data/build/en/foo.json')
})
it('does not modify an RSC data route', () => {
expect(localizeDataRoute('/foo.rsc', '/foo')).toEqual('/foo.rsc')
})
})
Loading