Skip to content

Commit 164bfe8

Browse files
RaisinTenMylesBorins
RaisinTen
authored and
MylesBorins
committed
src: add initial support for single executable applications
Compile a JavaScript file into a single executable application: ```console $ echo 'console.log(`Hello, ${process.argv[2]}!`);' > hello.js $ cp $(command -v node) hello $ npx postject hello NODE_JS_CODE hello.js \ --sentinel-fuse NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2 $ npx postject hello NODE_JS_CODE hello.js \ --sentinel-fuse NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2 \ --macho-segment-name NODE_JS $ ./hello world Hello, world! ``` Signed-off-by: Darshan Sen <[email protected]> PR-URL: #45038 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Michael Dawson <[email protected]> Reviewed-By: Joyee Cheung <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Colin Ihrig <[email protected]>
1 parent 2239e24 commit 164bfe8

14 files changed

+615
-1
lines changed

‎configure.py

+10
Original file line numberDiff line numberDiff line change
@@ -146,6 +146,12 @@
146146
default=None,
147147
help='use on deprecated SunOS systems that do not support ifaddrs.h')
148148

149+
parser.add_argument('--disable-single-executable-application',
150+
action='store_true',
151+
dest='disable_single_executable_application',
152+
default=None,
153+
help='Disable Single Executable Application support.')
154+
149155
parser.add_argument("--fully-static",
150156
action="store_true",
151157
dest="fully_static",
@@ -1357,6 +1363,10 @@ def configure_node(o):
13571363
if options.no_ifaddrs:
13581364
o['defines'] += ['SUNOS_NO_IFADDRS']
13591365

1366+
o['variables']['single_executable_application'] = b(not options.disable_single_executable_application)
1367+
if options.disable_single_executable_application:
1368+
o['defines'] += ['DISABLE_SINGLE_EXECUTABLE_APPLICATION']
1369+
13601370
o['variables']['node_with_ltcg'] = b(options.with_ltcg)
13611371
if flavor != 'win' and options.with_ltcg:
13621372
raise Exception('Link Time Code Generation is only supported on Windows.')

‎doc/api/index.md

+1
Original file line numberDiff line numberDiff line change
@@ -52,6 +52,7 @@
5252
* [Readline](readline.md)
5353
* [REPL](repl.md)
5454
* [Report](report.md)
55+
* [Single executable applications](single-executable-applications.md)
5556
* [Stream](stream.md)
5657
* [String decoder](string_decoder.md)
5758
* [Test runner](test.md)

‎doc/api/single-executable-applications.md

+140
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
# Single executable applications
2+
3+
<!--introduced_in=REPLACEME-->
4+
5+
> Stability: 1 - Experimental: This feature is being designed and will change.
6+
7+
<!-- source_link=lib/internal/main/single_executable_application.js -->
8+
9+
This feature allows the distribution of a Node.js application conveniently to a
10+
system that does not have Node.js installed.
11+
12+
Node.js supports the creation of [single executable applications][] by allowing
13+
the injection of a JavaScript file into the `node` binary. During start up, the
14+
program checks if anything has been injected. If the script is found, it
15+
executes its contents. Otherwise Node.js operates as it normally does.
16+
17+
The single executable application feature only supports running a single
18+
embedded [CommonJS][] file.
19+
20+
A bundled JavaScript file can be turned into a single executable application
21+
with any tool which can inject resources into the `node` binary.
22+
23+
Here are the steps for creating a single executable application using one such
24+
tool, [postject][]:
25+
26+
1. Create a JavaScript file:
27+
```console
28+
$ echo 'console.log(`Hello, ${process.argv[2]}!`);' > hello.js
29+
```
30+
31+
2. Create a copy of the `node` executable and name it according to your needs:
32+
```console
33+
$ cp $(command -v node) hello
34+
```
35+
36+
3. Inject the JavaScript file into the copied binary by running `postject` with
37+
the following options:
38+
39+
* `hello` - The name of the copy of the `node` executable created in step 2.
40+
* `NODE_JS_CODE` - The name of the resource / note / section in the binary
41+
where the contents of the JavaScript file will be stored.
42+
* `hello.js` - The name of the JavaScript file created in step 1.
43+
* `--sentinel-fuse NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2` - The
44+
[fuse][] used by the Node.js project to detect if a file has been injected.
45+
* `--macho-segment-name NODE_JS` (only needed on macOS) - The name of the
46+
segment in the binary where the contents of the JavaScript file will be
47+
stored.
48+
49+
To summarize, here is the required command for each platform:
50+
51+
* On systems other than macOS:
52+
```console
53+
$ npx postject hello NODE_JS_CODE hello.js \
54+
--sentinel-fuse NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2
55+
```
56+
57+
* On macOS:
58+
```console
59+
$ npx postject hello NODE_JS_CODE hello.js \
60+
--sentinel-fuse NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2 \
61+
--macho-segment-name NODE_JS
62+
```
63+
64+
4. Run the binary:
65+
```console
66+
$ ./hello world
67+
Hello, world!
68+
```
69+
70+
## Notes
71+
72+
### `require(id)` in the injected module is not file based
73+
74+
`require()` in the injected module is not the same as the [`require()`][]
75+
available to modules that are not injected. It also does not have any of the
76+
properties that non-injected [`require()`][] has except [`require.main`][]. It
77+
can only be used to load built-in modules. Attempting to load a module that can
78+
only be found in the file system will throw an error.
79+
80+
Instead of relying on a file based `require()`, users can bundle their
81+
application into a standalone JavaScript file to inject into the executable.
82+
This also ensures a more deterministic dependency graph.
83+
84+
However, if a file based `require()` is still needed, that can also be achieved:
85+
86+
```js
87+
const { createRequire } = require('node:module');
88+
require = createRequire(__filename);
89+
```
90+
91+
### `__filename` and `module.filename` in the injected module
92+
93+
The values of `__filename` and `module.filename` in the injected module are
94+
equal to [`process.execPath`][].
95+
96+
### `__dirname` in the injected module
97+
98+
The value of `__dirname` in the injected module is equal to the directory name
99+
of [`process.execPath`][].
100+
101+
### Single executable application creation process
102+
103+
A tool aiming to create a single executable Node.js application must
104+
inject the contents of a JavaScript file into:
105+
106+
* a resource named `NODE_JS_CODE` if the `node` binary is a [PE][] file
107+
* a section named `NODE_JS_CODE` in the `NODE_JS` segment if the `node` binary
108+
is a [Mach-O][] file
109+
* a note named `NODE_JS_CODE` if the `node` binary is an [ELF][] file
110+
111+
Search the binary for the
112+
`NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2:0` [fuse][] string and flip the
113+
last character to `1` to indicate that a resource has been injected.
114+
115+
### Platform support
116+
117+
Single-executable support is tested regularly on CI only on the following
118+
platforms:
119+
120+
* Windows
121+
* macOS
122+
* Linux (AMD64 only)
123+
124+
This is due to a lack of better tools to generate single-executables that can be
125+
used to test this feature on other platforms.
126+
127+
Suggestions for other resource injection tools/workflows are welcomed. Please
128+
start a discussion at <https://github.com/nodejs/single-executable/discussions>
129+
to help us document them.
130+
131+
[CommonJS]: modules.md#modules-commonjs-modules
132+
[ELF]: https://en.wikipedia.org/wiki/Executable_and_Linkable_Format
133+
[Mach-O]: https://en.wikipedia.org/wiki/Mach-O
134+
[PE]: https://en.wikipedia.org/wiki/Portable_Executable
135+
[`process.execPath`]: process.md#processexecpath
136+
[`require()`]: modules.md#requireid
137+
[`require.main`]: modules.md#accessing-the-main-module
138+
[fuse]: https://www.electronjs.org/docs/latest/tutorial/fuses
139+
[postject]: https://github.com/nodejs/postject
140+
[single executable applications]: https://github.com/nodejs/single-executable

‎doc/contributing/maintaining-single-executable-application-support.md

+81
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# Maintaining Single Executable Applications support
2+
3+
Support for [single executable applications][] is one of the key technical
4+
priorities identified for the success of Node.js.
5+
6+
## High level strategy
7+
8+
From the [Next-10 discussions][] there are 2 approaches the project believes are
9+
important to support:
10+
11+
### Compile with Node.js into executable
12+
13+
This is the approach followed by [boxednode][].
14+
15+
No additional code within the Node.js project is needed to support the
16+
option of compiling a bundled application along with Node.js into a single
17+
executable application.
18+
19+
### Bundle into existing Node.js executable
20+
21+
This is the approach followed by [pkg][].
22+
23+
The project does not plan to provide the complete solution but instead the key
24+
elements which are required in the Node.js executable in order to enable
25+
bundling with the pre-built Node.js binaries. This includes:
26+
27+
* Looking for a segment within the executable that holds bundled code.
28+
* Running the bundled code when such a segment is found.
29+
30+
It is left up to external tools/solutions to:
31+
32+
* Bundle code into a single script.
33+
* Generate a command line with appropriate options.
34+
* Add a segment to an existing Node.js executable which contains
35+
the command line and appropriate headers.
36+
* Re-generate or removing signatures on the resulting executable
37+
* Provide a virtual file system, and hooking it in if needed to
38+
support native modules or reading file contents.
39+
40+
However, the project also maintains a separate tool, [postject][], for injecting
41+
arbitrary read-only resources into the binary such as those needed for bundling
42+
the application into the runtime.
43+
44+
## Planning
45+
46+
Planning for this feature takes place in the [single-executable repository][].
47+
48+
## Upcoming features
49+
50+
Currently, only running a single embedded CommonJS file is supported but support
51+
for the following features are in the list of work we'd like to get to:
52+
53+
* Running an embedded ESM file.
54+
* Running an archive of multiple files.
55+
* Embedding [Node.js CLI options][] into the binary.
56+
* [XCOFF][] executable format.
57+
* Run tests on Linux architectures/distributions other than AMD64 Ubuntu.
58+
59+
## Disabling single executable application support
60+
61+
To disable single executable application support, build Node.js with the
62+
`--disable-single-executable-application` configuration option.
63+
64+
## Implementation
65+
66+
When built with single executable application support, the Node.js process uses
67+
[`postject-api.h`][] to check if the `NODE_JS_CODE` section exists in the
68+
binary. If it is found, it passes the buffer to
69+
[`single_executable_application.js`][], which executes the contents of the
70+
embedded script.
71+
72+
[Next-10 discussions]: https://github.com/nodejs/next-10/blob/main/meetings/summit-nov-2021.md#single-executable-applications
73+
[Node.js CLI options]: https://nodejs.org/api/cli.html
74+
[XCOFF]: https://www.ibm.com/docs/en/aix/7.2?topic=formats-xcoff-object-file-format
75+
[`postject-api.h`]: https://github.com/nodejs/node/blob/71951a0e86da9253d7c422fa2520ee9143e557fa/test/fixtures/postject-copy/node_modules/postject/dist/postject-api.h
76+
[`single_executable_application.js`]: https://github.com/nodejs/node/blob/main/lib/internal/main/single_executable_application.js
77+
[boxednode]: https://github.com/mongodb-js/boxednode
78+
[pkg]: https://github.com/vercel/pkg
79+
[postject]: https://github.com/nodejs/postject
80+
[single executable applications]: https://github.com/nodejs/node/blob/main/doc/contributing/technical-priorities.md#single-executable-applications
81+
[single-executable repository]: https://github.com/nodejs/single-executable

‎lib/internal/main/single_executable_application.js

+55
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
'use strict';
2+
const {
3+
prepareMainThreadExecution,
4+
markBootstrapComplete,
5+
} = require('internal/process/pre_execution');
6+
const { getSingleExecutableCode } = internalBinding('sea');
7+
const { emitExperimentalWarning } = require('internal/util');
8+
const { Module, wrapSafe } = require('internal/modules/cjs/loader');
9+
const { codes: { ERR_UNKNOWN_BUILTIN_MODULE } } = require('internal/errors');
10+
11+
prepareMainThreadExecution(false, true);
12+
markBootstrapComplete();
13+
14+
emitExperimentalWarning('Single executable application');
15+
16+
// This is roughly the same as:
17+
//
18+
// const mod = new Module(filename);
19+
// mod._compile(contents, filename);
20+
//
21+
// but the code has been duplicated because currently there is no way to set the
22+
// value of require.main to module.
23+
//
24+
// TODO(RaisinTen): Find a way to deduplicate this.
25+
26+
const filename = process.execPath;
27+
const contents = getSingleExecutableCode();
28+
const compiledWrapper = wrapSafe(filename, contents);
29+
30+
const customModule = new Module(filename, null);
31+
customModule.filename = filename;
32+
customModule.paths = Module._nodeModulePaths(customModule.path);
33+
34+
const customExports = customModule.exports;
35+
36+
function customRequire(path) {
37+
if (!Module.isBuiltin(path)) {
38+
throw new ERR_UNKNOWN_BUILTIN_MODULE(path);
39+
}
40+
41+
return require(path);
42+
}
43+
44+
customRequire.main = customModule;
45+
46+
const customFilename = customModule.filename;
47+
48+
const customDirname = customModule.path;
49+
50+
compiledWrapper(
51+
customExports,
52+
customRequire,
53+
customModule,
54+
customFilename,
55+
customDirname);

‎node.gyp

+6-1
Original file line numberDiff line numberDiff line change
@@ -151,7 +151,8 @@
151151

152152
'include_dirs': [
153153
'src',
154-
'deps/v8/include'
154+
'deps/v8/include',
155+
'deps/postject'
155156
],
156157

157158
'sources': [
@@ -449,6 +450,7 @@
449450

450451
'include_dirs': [
451452
'src',
453+
'deps/postject',
452454
'<(SHARED_INTERMEDIATE_DIR)' # for node_natives.h
453455
],
454456
'dependencies': [
@@ -523,6 +525,7 @@
523525
'src/node_report.cc',
524526
'src/node_report_module.cc',
525527
'src/node_report_utils.cc',
528+
'src/node_sea.cc',
526529
'src/node_serdes.cc',
527530
'src/node_shadow_realm.cc',
528531
'src/node_snapshotable.cc',
@@ -633,6 +636,7 @@
633636
'src/node_report.h',
634637
'src/node_revert.h',
635638
'src/node_root_certs.h',
639+
'src/node_sea.h',
636640
'src/node_shadow_realm.h',
637641
'src/node_snapshotable.h',
638642
'src/node_snapshot_builder.h',
@@ -675,6 +679,7 @@
675679
'src/util-inl.h',
676680
# Dependency headers
677681
'deps/v8/include/v8.h',
682+
'deps/postject/postject-api.h'
678683
# javascript files to make for an even more pleasant IDE experience
679684
'<@(library_files)',
680685
'<@(deps_files)',

‎src/node.cc

+17
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,7 @@
3939
#include "node_realm-inl.h"
4040
#include "node_report.h"
4141
#include "node_revert.h"
42+
#include "node_sea.h"
4243
#include "node_snapshot_builder.h"
4344
#include "node_v8_platform-inl.h"
4445
#include "node_version.h"
@@ -122,6 +123,7 @@
122123
#include <cstring>
123124

124125
#include <string>
126+
#include <tuple>
125127
#include <vector>
126128

127129
namespace node {
@@ -310,6 +312,18 @@ MaybeLocal<Value> StartExecution(Environment* env, StartExecutionCallback cb) {
310312
first_argv = env->argv()[1];
311313
}
312314

315+
#ifndef DISABLE_SINGLE_EXECUTABLE_APPLICATION
316+
if (sea::IsSingleExecutable()) {
317+
// TODO(addaleax): Find a way to reuse:
318+
//
319+
// LoadEnvironment(Environment*, const char*)
320+
//
321+
// instead and not add yet another main entry point here because this
322+
// already duplicates existing code.
323+
return StartExecution(env, "internal/main/single_executable_application");
324+
}
325+
#endif
326+
313327
if (first_argv == "inspect") {
314328
return StartExecution(env, "internal/main/inspect");
315329
}
@@ -1244,6 +1258,9 @@ static ExitCode StartInternal(int argc, char** argv) {
12441258
}
12451259

12461260
int Start(int argc, char** argv) {
1261+
#ifndef DISABLE_SINGLE_EXECUTABLE_APPLICATION
1262+
std::tie(argc, argv) = sea::FixupArgsForSEA(argc, argv);
1263+
#endif
12471264
return static_cast<int>(StartInternal(argc, argv));
12481265
}
12491266

0 commit comments

Comments
 (0)