Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Update xsnap/xsnap-worker argument processing and usage output #52

Open
wants to merge 5 commits into
base: Agoric
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
# agoric
# agoric

Repository for producing snapshot-supporting XS runtimes. See [xsnap](./xsnap).
22 changes: 11 additions & 11 deletions xsnap/documentation/xsnap-worker.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
# xsnap-worker

`sources/xsnap-worker.c` contains a variant of `xsnap` which accepts execution commands over a file descriptor. It is designed to function as a "JS coprocessor", driven by a parent process (which can be written in any language). The parent does a fork+exec of `xsnap-worker`, then writes netstring-formatted commands to the child. The child executes those commands (evaluating JS or delivering the command to a handler function), possibly emitting one or more requests to the parent during execution, then finally finishes the command and writing a status to the parent (including metering information).
`sources/xsnap-worker.c` contains a variant of `xsnap` which accepts execution commands over a file descriptor. It is designed to function as a "JS coprocessor", driven by a parent process (which can be written in any language). The parent does a fork+exec of `xsnap-worker`, then writes [netstring](https://en.wikipedia.org/wiki/Netstring)-formatted commands to the child. The child executes those commands (evaluating JS or delivering the command to a handler function), possibly emitting one or more requests to the parent during execution, then finally finishes the command and writing a status to the parent (including metering information).

By default, the process starts from an empty JS environment (for now, you must install SES/`lockdown` yourself, but in the future it is likely to start from an empty SES environment). If the child is started with a `-r SNAPSHOTFILENAME` argument, it will start from a previously-written JS engine snapshot instead.
By default, the process starts from an empty JS environment (in the future it may start from an empty [hardened JavaScript](https://github.com/endojs/endo/blob/master/packages/ses/README.md) environment, but for now you must [`lockdown()`](https://github.com/endojs/endo/blob/master/packages/ses/README.md#lockdown) yourself). If the process is started with a `-r $SNAPSHOT` option, it will instead start from a previously-written heap snapshot file.

The launch arguments are:
Supported options are:

* `-h`: print this help message
* `-i <interval>`: set the metering check interval: larger intervals are more efficient but are likely to exceed the execution budget by more computrons
* `-l <limit>`: limit each delivery to `<limit>` computrons
* `-p`: print the current meter count before every `print()`
* `-r <snapshot filename>`: launch from a JS snapshot file, instead of an empty environment
* `-s SIZE`: set `parserBufferSize`, in kiB (1024 bytes)
* `-v`: print the `xsnap` version and exit with rc 0
* `-n`: print the agoric-upgrade version and exit with rc 0
* All `argv` strings that do not start with a hyphen are ignored. This allows the parent to include dummy no-op arguments to e.g. label the worker process with a vat ID and name, so admins can use `ps` to distinguish between workers being run for different purposes.
* `-v`: print xsnap and XS version information
* `-i <interval>`: set the metering check interval (defaults to 1). Larger intervals are more efficient but are likely to exceed the execution budget by more computrons.
* `-l <limit>`: limit each delivery to `<limit>` computrons (defaults to no limit)
- `-p`: prefix `print` output with the current meter count in square brackets
* `-r <snapshot>`: read the heap snapshot file to create the XS machine (rather than initializing to an empty environment)
* `-s <size>`: set parser buffer size, in kiB (defaults to 8192 = 8_388_608 bytes)

All arguments that do not start with a hyphen are ignored, so they may be used to label the worker process with identifying information visible to e.g. `ps` (such as a vat ID and name).

Once started, the process listens on file descriptor 3, and will write to file descriptor 4. The process will perform a blocking read on fd3 until a complete netstring is received. The first character of the body of this netstring indicates what command to execute, with the remainder of the body as the command's payload. The commands are:

Expand Down
55 changes: 25 additions & 30 deletions xsnap/readme.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,10 @@
# xsnap
Revised: January 4, 2022

Warning: These notes are preliminary. Omissions and errors are likely. If you encounter problems, please ask for assistance.

## About

`xsnap` is a custom XS runtime that read and write XS snapshots. See [XS snapshots](./documentation/XS Snapshots.md) for details and the C programming interface. `xsnap` can create a machine from scratch or from a snapshot.
`xsnap` is a custom [Moddable XS](https://github.com/Moddable-OpenSource/moddable/blob/public/documentation/xs/XS%20Differences.md) runtime that can dump and load from [heap snapshots](./documentation/XS%20Snapshots.md).

`xsnap` also uses the metering version of XS. There are options to constraint how much computation a machine can do. See [XS metering](./documentation/XS Metering.md)
`xsnap` also uses the metering version of XS. There are options to constraint how much computation a machine can do. See [XS metering](./documentation/XS%20Metering.md).

## Additions

Expand Down Expand Up @@ -40,23 +37,23 @@ Besides ECMAScript built-ins and intrinsics, `xsnap` provides additional feature

## Build

### Linux
### Linux

cd ./makefiles/lin
make

The debug version is built in `$MODDABLE/build/bin/lin/debug`
The release version is built in `$MODDABLE/build/bin/lin/release `

### macOS
### macOS

cd ./xs/makefiles/mac
make

The debug version is built in `$MODDABLE/build/bin/mac/debug`
The release version is built in `$MODDABLE/build/bin/mac/release `

### Windows
### Windows

cd .\xs\makefiles\win
build
Expand All @@ -67,27 +64,27 @@ The release version is built in `$MODDABLE/build/bin/win/release `
## Usage

xsnap [-h] [-v]
[-d <snapshot>] [-r <snapshot>] [-w <snapshot>]
[-i <interval>] [-l <limit>] [-p]
[-e] [-m] [-s] strings...
[-d] [-i <interval>] [-l <limit>] [-q] [-r <snapshot>] [-w <snapshot>]
[-i <interval>] [-l <limit>] [-q]
[-e] [-m] [-s] <string>...

- `-h`: print this help message
- `-v`: print XS version
- `-d <snapshot>`: dump snapshot to stderr
- `-r <snapshot>`: read snapshot to create the XS machine
- `-w <snapshot>`: write snapshot of the XS machine at exit
- `-i <interval>`: metering interval (defaults to 1)
- `-l <limit>`: metering limit (defaults to none)
- `-p`: prefix `print` output with metering index
- `-e`: eval `strings`
- `-m`: `strings` are paths to modules
- `-s`: `strings` are paths to scripts

Without `-e`, `-m`, `-s`, if the extension is `.mjs`, strings are paths to modules, else strings are paths to scripts.
- `-d <snapshot>`: read heap snapshot file and dump a textual representation to stderr
- `-i <interval>`: metering interval (defaults to 1)
- `-l <limit>`: metering limit (defaults to no limit)
- `-q`: prefix `print` output with the current meter count in square brackets
- `-r <snapshot>`: read heap snapshot file to create the XS machine
- `-w <snapshot>`: write heap snapshot file of the XS machine at exit
- `-e`: evaluate each `<string>` (in its own unique scope)
- `-m`: interpret each `<string>` as the path for a module to load
- `-s`: interpret each `<string>` as the path for a script to load

Without `-e`, `-m`, `-s`, each string with suffix ".mjs" is interpreted as the path for a module to load and each other string is interpreted as the path for a script to load.

## Examples

Add the debug or release directory here above to your path.
Add the debug or release directory here above to your path.

### helloworld

Expand Down Expand Up @@ -139,7 +136,7 @@ Just to test how a `Proxy` instance, its target and its handler survive the snap

### modules

Use the `-m` option for modules
Use the `-m` option for modules

cd ./examples/modules
xsnap -m before.js -w snapshot.xsm
Expand All @@ -149,7 +146,7 @@ Modules imported before writing the snapshot are available after reading the sna

### metering

Use the `-l` option to limit the number of byte codes that can be executed.
Use the `-l` option to limit the number of byte codes that can be executed.

cd ./examples/metering
xsnap test.js -l 10000
Expand All @@ -174,18 +171,16 @@ There is a performance gain but a precision lost.

### metering-built-ins

Use the `-p` option to prefix `print` output with the metering index.
Use the `-q` option to prefix `print` output with the current meter count in square brackets.

cd ./examples/metering-built-ins
xsnap test.js -p
xsnap test.js -q

The tests builds, sorts and reverses an array of 100 random numbers. Observe the metering index around `sort` and `reverse`.
The tests builds, sorts and reverses an array of 100 random numbers. Observe the increasing metering count.

...
[3935] 99 0.4153946155753893
[3957] sort
[11266] reverse
[16260] 0 0.000007826369259425611
...


37 changes: 22 additions & 15 deletions xsnap/sources/xsnap-worker.c
Original file line number Diff line number Diff line change
Expand Up @@ -290,8 +290,15 @@ int main(int argc, char* argv[])
continue;
if (!strcmp(argv[argi], "-h")) {
xsPrintUsage();
return 0;
} else if (!strcmp(argv[argi], "-i")) {
return E_SUCCESS;
}
else if (!strcmp(argv[argi], "-v")) {
char version[C_PATH_MAX];
xsVersion(version, sizeof(version));
printf("xsnap %s (XS %s)\n", XSNAP_VERSION, version);
return E_SUCCESS;
}
else if (!strcmp(argv[argi], "-i")) {
argi++;
if (argi < argc)
interval = atoi(argv[argi]);
Expand Down Expand Up @@ -334,16 +341,11 @@ int main(int argc, char* argv[])
return E_BAD_USAGE;
}
}
else if (!strcmp(argv[argi], "-v")) {
char version[16];
xsVersion(version, sizeof(version));
printf("xsnap %s (XS %s)\n", XSNAP_VERSION, version);
return E_SUCCESS;
}
else if (!strcmp(argv[argi], "-n")) {
printf("deprecated\n");
return E_SUCCESS;
} else {
}
else {
xsPrintUsage();
return E_BAD_USAGE;
}
Expand Down Expand Up @@ -735,13 +737,18 @@ void xsBuildAgent(xsMachine* machine)

void xsPrintUsage()
{
printf("xsnap [-h] [-i <interval>] [-l <limit>] [-s <size>] [-m] [-r <snapshot>] [-s] [-v]\n");
printf("xsnap [-h] [-v]\n");
printf(" [-i <interval>] [-l <limit>] [-p] [-r <snapshot>] [-s <size>]\n");
printf(" <ignored-operand>...\n");
printf("\t-h: print this help message\n");
printf("\t-i <interval>: metering interval (default to 1)\n");
printf("\t-l <limit>: metering limit (default to none)\n");
printf("\t-s <size>: parser buffer size, in kB (default to 8192)\n");
printf("\t-r <snapshot>: read snapshot to create the XS machine\n");
printf("\t-v: print XS version\n");
printf("\t-v: print xsnap and XS version information\n");
printf("\t-i <interval>: metering interval (defaults to 1)\n");
printf("\t-l <limit>: metering limit (defaults to no limit)\n");
printf("\t-p: prefix `print` output with the current meter count in square brackets\n");
printf("\t-r <snapshot>: read heap snapshot file to create the XS machine\n");
printf("\t-s <size>: parser buffer size, in kiB (defaults to 8192 = 8_388_608 bytes)\n");
printf("All arguments that do not start with a hyphen are ignored, so they may be used\n");
printf("to label the worker process with identifying information visible to e.g. `ps`.\n");
}

void xs_clearTimer(xsMachine* the)
Expand Down
42 changes: 24 additions & 18 deletions xsnap/sources/xsnap.c
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,16 @@ int main(int argc, char* argv[])
for (argi = 1; argi < argc; argi++) {
if (argv[argi][0] != '-')
continue;
if (!strcmp(argv[argi], "-d")) {
if (!strcmp(argv[argi], "-h")) {
xsPrintUsage();
return 0;
}
else if (!strcmp(argv[argi], "-v")) {
xsVersion(path, sizeof(path));
printf("XS %s\n", path);
return 0;
}
else if (!strcmp(argv[argi], "-d")) {
argi++;
if (argi < argc)
argd = argi;
Expand All @@ -158,8 +167,6 @@ int main(int argc, char* argv[])
}
else if (!strcmp(argv[argi], "-e"))
option = 1;
else if (!strcmp(argv[argi], "-h"))
xsPrintUsage();
else if (!strcmp(argv[argi], "-i")) {
argi++;
if (argi < argc)
Expand Down Expand Up @@ -203,10 +210,6 @@ int main(int argc, char* argv[])
option = 3;
else if (!strcmp(argv[argi], "-t"))
option = 4;
else if (!strcmp(argv[argi], "-v")) {
xsVersion(path, sizeof(path));
printf("XS %s\n", path);
}
else if (!strcmp(argv[argi], "-w")) {
argi++;
if (argi < argc)
Expand Down Expand Up @@ -391,20 +394,23 @@ void xsBuildAgent(xsMachine* machine)

void xsPrintUsage()
{
printf("xsnap [-h] [-e] [i <interval] [l <limit] [-m] [-r <snapshot>] [-s] [-v] [-w <snapshot>] strings...\n");
printf("\t-d <snapshot>: dump snapshot to stderr\n");
printf("\t-e: eval strings\n");
printf("xsnap [-h] [-v]\n");
printf(" [-d] [-i <interval>] [-l <limit>] [-q] [-r <snapshot>] [-w <snapshot>]\n");
printf(" [-e] [-m] [-s] <string>...\n");
printf("\t-h: print this help message\n");
printf("\t-i <interval>: metering interval (default to 1)\n");
printf("\t-l <limit>: metering limit (default to none)\n");
printf("\t-m: strings are paths to modules\n");
printf("\t-r <snapshot>: read snapshot to create the XS machine\n");
printf("\t-s: strings are paths to scripts\n");
printf("\t-v: print XS version\n");
printf("\t-w <snapshot>: write snapshot of the XS machine at exit\n");
printf("\t-d <snapshot>: read heap snapshot file and dump a textual representation to stderr\n");
printf("\t-i <interval>: metering interval (defaults to 1)\n");
printf("\t-l <limit>: metering limit (defaults to no limit)\n");
printf("\t-q: prefix `print` output with the current meter count in square brackets\n");
printf("\t-r <snapshot>: read heap snapshot file to create the XS machine\n");
printf("\t-w <snapshot>: write heap snapshot file of the XS machine at exit\n");
printf("\t-e: evaluate each string (in its own unique scope)\n");
printf("\t-m: interpret each string as the path for a module to load\n");
printf("\t-s: interpret each string as the path for a script to load\n");
printf("without -e, -m, -s:\n");
printf("\tif the extension is .mjs, strings are paths to modules\n");
printf("\telse strings are paths to scripts\n");
printf("\teach string with suffix \".mjs\" is interpreted as the path for a module to load\n");
printf("\tand each other string is interpreted as the path for a script to load\n");
}

static int gxStep = 0;
Expand Down
7 changes: 2 additions & 5 deletions xsnap/xsbug-node/readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ The xsnap pipes are used to communicate with xsnap as usual. The script reads th

This module defines the xsbug `Machine` class.

The xsbug protocol uses XML so the module needs a SAX parser. A few have been tried but [Saxophone](https://github.com/matteodelabre/saxophone) was the only one that worked well.
The xsbug protocol uses XML so the module needs a SAX parser. A few have been tried but [Saxophone](https://github.com/matteodelabre/saxophone) was the only one that worked well.

The xsbug `Machine` class defines a bunch of commands and internal events. These are all the commands and events that **xsbug** uses.

Expand Down Expand Up @@ -133,7 +133,4 @@ Then instruments can be sampled at will.
xsSampleInstrumentation(machine, xsnapInstrumentCount, xsnapInstrumentValues);
#endif

Every time instruments are sampled, the xsbug `Machine` object triggers the `'instruments'` event.



Every time instruments are sampled, the xsbug `Machine` object triggers the `'instruments'` event.