update openapi-typescript-fetch bench

[ajaishankar]

Changes

Update bench to use openapi-typescript correctly.

Create and reuse the fetch method instead of instantiating it on every invocation.

This should speed up the benchmark slightly.

How to Review

This change moves the method creation outside of the bench

Checklist

• Unit tests updated
• docs/ updated (if necessary)
• pnpm run update:examples run (only applicable for openapi-typescript)

[changeset-bot]

⚠️ No Changeset found

Latest commit: 5f4d299

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[ajaishankar]

@drwpow it's great that lots of progress has been made on a (very elegant) fetch implementation and it is now part of openapi-typescript! 👍

The fetch was completely missing two years back when I first published openapi-typescript-fetch.

Just wanted to update the bench with correct usage...

[drwpow]

Thank you for all the inspiration, and pushing it forward as well!

To be honest, the benchmarks are a bit flaky (all benchmarks are to some degree, but these even moreso). I’ve been procrastinating fixing the absolute numbers which are pretty unreliable even on the same machine (the relative benchmarks seem accurate, but absolute ops/s fluctuate way more wildly than they should).

Either way, thanks for this change. Will update the benchmarks soon with this improved implementation, and hopefully figure out a way to make the runs more consistent.

[ajaishankar]

I was surprised why there would be so much of a difference in the benchmark.

Turns out my target was ES2015 and Typescript was transpiling async/await into some generators.

Once the target was set to ESNext that overhead went away and I'm getting same/better ops/s as openapi-fetch (on my machine 😉)

Again, not sure how relevant these benchmarks are.

API wise openapi-fetch is so much cleaner, thanks for this and also openapi-typescript! 👍

[drwpow]

Yeah as I said the absolute numbers are kinda garbage from my findings, and I need to replace it with something more reliable. But the relative numbers (e.g. 1.38x faster than axios) tend to be fairly consistent.

Also as someone that is always learning about benchmarks, you’re benchmarking in V8 (Node, Chrome), which rewards certain particular operations and punishes others. JavaScriptCore (Safari) and SpiderMonkey (Firefox) will have different benchmarks on the same code. But in general, things like RegExs, functional programming (.reduce() vs for … of), and classic Big O Notation (i.e. “am I iterating over this array once, or 5 times?”) will all affect the runtime. For a fetch library, IMHO it is very important to sweat every little drop of performance, because these are the core of so many applications, and get executed billions of times. For most code we write this is bikeshedding and splitting hairs, but for fetching, I think it matters.

[ajaishankar]

Got it!

Time permitting, I'll try to add the following to the benchmark that exercises the following:

1. Path construction from request payload args
2. Query string serialization
3. Exception handling
4. Middleware handling (loop vs recursion in openapi-typescript-fetch)

In my case I'd to make a clone of the payload because of a design choice (all params are mashed into a single argument and deconstructed) whereas in openapi-fetch this overhead may not be there.

Will also publish a esnext version of the library for comparison.