From 7e68d06879893b0226e7e201520cc0d9ca1be29d Mon Sep 17 00:00:00 2001 From: Richard Gibson Date: Mon, 8 Jul 2024 17:08:57 -0400 Subject: [PATCH 1/5] chore: Fix Markdown links and trailing whitespace --- xsnap/documentation/xsnap-worker.md | 4 ++-- xsnap/readme.md | 20 +++++++++----------- xsnap/xsbug-node/readme.md | 7 ++----- 3 files changed, 13 insertions(+), 18 deletions(-) diff --git a/xsnap/documentation/xsnap-worker.md b/xsnap/documentation/xsnap-worker.md index 4ddbb1d..8a87345 100644 --- a/xsnap/documentation/xsnap-worker.md +++ b/xsnap/documentation/xsnap-worker.md @@ -1,8 +1,8 @@ # xsnap-worker -`sources/xsnap-worker.c` contains a variant of `xsnap` which accepts execution commands over a file descriptor. It is designed to function as a "JS coprocessor", driven by a parent process (which can be written in any language). The parent does a fork+exec of `xsnap-worker`, then writes netstring-formatted commands to the child. The child executes those commands (evaluating JS or delivering the command to a handler function), possibly emitting one or more requests to the parent during execution, then finally finishes the command and writing a status to the parent (including metering information). +`sources/xsnap-worker.c` contains a variant of `xsnap` which accepts execution commands over a file descriptor. It is designed to function as a "JS coprocessor", driven by a parent process (which can be written in any language). The parent does a fork+exec of `xsnap-worker`, then writes [netstring](https://en.wikipedia.org/wiki/Netstring)-formatted commands to the child. The child executes those commands (evaluating JS or delivering the command to a handler function), possibly emitting one or more requests to the parent during execution, then finally finishes the command and writing a status to the parent (including metering information). -By default, the process starts from an empty JS environment (for now, you must install SES/`lockdown` yourself, but in the future it is likely to start from an empty SES environment). If the child is started with a `-r SNAPSHOTFILENAME` argument, it will start from a previously-written JS engine snapshot instead. +By default, the process starts from an empty JS environment (in the future it may start from an empty [hardened JavaScript](https://github.com/endojs/endo/blob/master/packages/ses/README.md) environment, but for now you must [`lockdown()`](https://github.com/endojs/endo/blob/master/packages/ses/README.md#lockdown) yourself). If the child is started with a `-r SNAPSHOTFILENAME` argument, it will start from a previously-written JS engine snapshot instead. The launch arguments are: diff --git a/xsnap/readme.md b/xsnap/readme.md index ef88a62..2726f29 100644 --- a/xsnap/readme.md +++ b/xsnap/readme.md @@ -5,9 +5,9 @@ Warning: These notes are preliminary. Omissions and errors are likely. If you en ## About -`xsnap` is a custom XS runtime that read and write XS snapshots. See [XS snapshots](./documentation/XS Snapshots.md) for details and the C programming interface. `xsnap` can create a machine from scratch or from a snapshot. +`xsnap` is a custom [Moddable XS](https://github.com/Moddable-OpenSource/moddable/blob/public/documentation/xs/XS%20Differences.md) runtime that can dump and load from [heap snapshots](./documentation/XS%20Snapshots.md). -`xsnap` also uses the metering version of XS. There are options to constraint how much computation a machine can do. See [XS metering](./documentation/XS Metering.md) +`xsnap` also uses the metering version of XS. There are options to constraint how much computation a machine can do. See [XS metering](./documentation/XS%20Metering.md). ## Additions @@ -40,7 +40,7 @@ Besides ECMAScript built-ins and intrinsics, `xsnap` provides additional feature ## Build -### Linux +### Linux cd ./makefiles/lin make @@ -48,7 +48,7 @@ Besides ECMAScript built-ins and intrinsics, `xsnap` provides additional feature The debug version is built in `$MODDABLE/build/bin/lin/debug` The release version is built in `$MODDABLE/build/bin/lin/release ` -### macOS +### macOS cd ./xs/makefiles/mac make @@ -56,7 +56,7 @@ The release version is built in `$MODDABLE/build/bin/lin/release ` The debug version is built in `$MODDABLE/build/bin/mac/debug` The release version is built in `$MODDABLE/build/bin/mac/release ` -### Windows +### Windows cd .\xs\makefiles\win build @@ -87,7 +87,7 @@ Without `-e`, `-m`, `-s`, if the extension is `.mjs`, strings are paths to modul ## Examples -Add the debug or release directory here above to your path. +Add the debug or release directory here above to your path. ### helloworld @@ -139,7 +139,7 @@ Just to test how a `Proxy` instance, its target and its handler survive the snap ### modules -Use the `-m` option for modules +Use the `-m` option for modules cd ./examples/modules xsnap -m before.js -w snapshot.xsm @@ -149,7 +149,7 @@ Modules imported before writing the snapshot are available after reading the sna ### metering -Use the `-l` option to limit the number of byte codes that can be executed. +Use the `-l` option to limit the number of byte codes that can be executed. cd ./examples/metering xsnap test.js -l 10000 @@ -174,7 +174,7 @@ There is a performance gain but a precision lost. ### metering-built-ins -Use the `-p` option to prefix `print` output with the metering index. +Use the `-p` option to prefix `print` output with the metering index. cd ./examples/metering-built-ins xsnap test.js -p @@ -187,5 +187,3 @@ The tests builds, sorts and reverses an array of 100 random numbers. Observe the [11266] reverse [16260] 0 0.000007826369259425611 ... - - diff --git a/xsnap/xsbug-node/readme.md b/xsnap/xsbug-node/readme.md index d96ab7c..7f484d5 100644 --- a/xsnap/xsbug-node/readme.md +++ b/xsnap/xsbug-node/readme.md @@ -60,7 +60,7 @@ The xsnap pipes are used to communicate with xsnap as usual. The script reads th This module defines the xsbug `Machine` class. -The xsbug protocol uses XML so the module needs a SAX parser. A few have been tried but [Saxophone](https://github.com/matteodelabre/saxophone) was the only one that worked well. +The xsbug protocol uses XML so the module needs a SAX parser. A few have been tried but [Saxophone](https://github.com/matteodelabre/saxophone) was the only one that worked well. The xsbug `Machine` class defines a bunch of commands and internal events. These are all the commands and events that **xsbug** uses. @@ -133,7 +133,4 @@ Then instruments can be sampled at will. xsSampleInstrumentation(machine, xsnapInstrumentCount, xsnapInstrumentValues); #endif -Every time instruments are sampled, the xsbug `Machine` object triggers the `'instruments'` event. - - - +Every time instruments are sampled, the xsbug `Machine` object triggers the `'instruments'` event. From 85c440718120c18edcd2bb82faa05ffc1b024628 Mon Sep 17 00:00:00 2001 From: Richard Gibson Date: Mon, 8 Jul 2024 21:46:31 -0400 Subject: [PATCH 2/5] chore: Add link to top-level README --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index a1a424a..25b4307 100644 --- a/README.md +++ b/README.md @@ -1 +1,3 @@ -# agoric \ No newline at end of file +# agoric + +Repository for producing snapshot-supporting XS runtimes. See [xsnap](./xsnap). From 25c101e65b5601d5bf29f4768e04a28325fc8371 Mon Sep 17 00:00:00 2001 From: Richard Gibson Date: Mon, 8 Jul 2024 17:30:26 -0400 Subject: [PATCH 3/5] fix: Check for -h/-v args first (and exit upon encountering either) --- xsnap/sources/xsnap-worker.c | 20 +++++++++++--------- xsnap/sources/xsnap.c | 17 ++++++++++------- 2 files changed, 21 insertions(+), 16 deletions(-) diff --git a/xsnap/sources/xsnap-worker.c b/xsnap/sources/xsnap-worker.c index b52c34a..e051bb0 100644 --- a/xsnap/sources/xsnap-worker.c +++ b/xsnap/sources/xsnap-worker.c @@ -290,8 +290,15 @@ int main(int argc, char* argv[]) continue; if (!strcmp(argv[argi], "-h")) { xsPrintUsage(); - return 0; - } else if (!strcmp(argv[argi], "-i")) { + return E_SUCCESS; + } + else if (!strcmp(argv[argi], "-v")) { + char version[C_PATH_MAX]; + xsVersion(version, sizeof(version)); + printf("xsnap %s (XS %s)\n", XSNAP_VERSION, version); + return E_SUCCESS; + } + else if (!strcmp(argv[argi], "-i")) { argi++; if (argi < argc) interval = atoi(argv[argi]); @@ -334,16 +341,11 @@ int main(int argc, char* argv[]) return E_BAD_USAGE; } } - else if (!strcmp(argv[argi], "-v")) { - char version[16]; - xsVersion(version, sizeof(version)); - printf("xsnap %s (XS %s)\n", XSNAP_VERSION, version); - return E_SUCCESS; - } else if (!strcmp(argv[argi], "-n")) { printf("deprecated\n"); return E_SUCCESS; - } else { + } + else { xsPrintUsage(); return E_BAD_USAGE; } diff --git a/xsnap/sources/xsnap.c b/xsnap/sources/xsnap.c index 6429155..b6c8db5 100644 --- a/xsnap/sources/xsnap.c +++ b/xsnap/sources/xsnap.c @@ -146,7 +146,16 @@ int main(int argc, char* argv[]) for (argi = 1; argi < argc; argi++) { if (argv[argi][0] != '-') continue; - if (!strcmp(argv[argi], "-d")) { + if (!strcmp(argv[argi], "-h")) { + xsPrintUsage(); + return 0; + } + else if (!strcmp(argv[argi], "-v")) { + xsVersion(path, sizeof(path)); + printf("XS %s\n", path); + return 0; + } + else if (!strcmp(argv[argi], "-d")) { argi++; if (argi < argc) argd = argi; @@ -158,8 +167,6 @@ int main(int argc, char* argv[]) } else if (!strcmp(argv[argi], "-e")) option = 1; - else if (!strcmp(argv[argi], "-h")) - xsPrintUsage(); else if (!strcmp(argv[argi], "-i")) { argi++; if (argi < argc) @@ -203,10 +210,6 @@ int main(int argc, char* argv[]) option = 3; else if (!strcmp(argv[argi], "-t")) option = 4; - else if (!strcmp(argv[argi], "-v")) { - xsVersion(path, sizeof(path)); - printf("XS %s\n", path); - } else if (!strcmp(argv[argi], "-w")) { argi++; if (argi < argc) From f89b3f86c08968397f25ecd6047411e43d88de93 Mon Sep 17 00:00:00 2001 From: Richard Gibson Date: Mon, 8 Jul 2024 17:36:19 -0400 Subject: [PATCH 4/5] chore: Output correct `xsnap`/`xsnap-worker` usage ...and order options alphabetically with select exceptions --- xsnap/sources/xsnap-worker.c | 17 +++++++++++------ xsnap/sources/xsnap.c | 25 ++++++++++++++----------- 2 files changed, 25 insertions(+), 17 deletions(-) diff --git a/xsnap/sources/xsnap-worker.c b/xsnap/sources/xsnap-worker.c index e051bb0..0dafbb3 100644 --- a/xsnap/sources/xsnap-worker.c +++ b/xsnap/sources/xsnap-worker.c @@ -737,13 +737,18 @@ void xsBuildAgent(xsMachine* machine) void xsPrintUsage() { - printf("xsnap [-h] [-i ] [-l ] [-s ] [-m] [-r ] [-s] [-v]\n"); + printf("xsnap [-h] [-v]\n"); + printf(" [-i ] [-l ] [-p] [-r ] [-s ]\n"); + printf(" ...\n"); printf("\t-h: print this help message\n"); - printf("\t-i : metering interval (default to 1)\n"); - printf("\t-l : metering limit (default to none)\n"); - printf("\t-s : parser buffer size, in kB (default to 8192)\n"); - printf("\t-r : read snapshot to create the XS machine\n"); - printf("\t-v: print XS version\n"); + printf("\t-v: print xsnap and XS version information\n"); + printf("\t-i : metering interval (defaults to 1)\n"); + printf("\t-l : metering limit (defaults to no limit)\n"); + printf("\t-p: prefix `print` output with the current meter count in square brackets\n"); + printf("\t-r : read heap snapshot file to create the XS machine\n"); + printf("\t-s : parser buffer size, in kiB (defaults to 8192 = 8_388_608 bytes)\n"); + printf("All arguments that do not start with a hyphen are ignored, so they may be used\n"); + printf("to label the worker process with identifying information visible to e.g. `ps`.\n"); } void xs_clearTimer(xsMachine* the) diff --git a/xsnap/sources/xsnap.c b/xsnap/sources/xsnap.c index b6c8db5..17f1e2d 100644 --- a/xsnap/sources/xsnap.c +++ b/xsnap/sources/xsnap.c @@ -394,20 +394,23 @@ void xsBuildAgent(xsMachine* machine) void xsPrintUsage() { - printf("xsnap [-h] [-e] [i ] [-s] [-v] [-w ] strings...\n"); - printf("\t-d : dump snapshot to stderr\n"); - printf("\t-e: eval strings\n"); + printf("xsnap [-h] [-v]\n"); + printf(" [-d] [-i ] [-l ] [-q] [-r ] [-w ]\n"); + printf(" [-e] [-m] [-s] ...\n"); printf("\t-h: print this help message\n"); - printf("\t-i : metering interval (default to 1)\n"); - printf("\t-l : metering limit (default to none)\n"); - printf("\t-m: strings are paths to modules\n"); - printf("\t-r : read snapshot to create the XS machine\n"); - printf("\t-s: strings are paths to scripts\n"); printf("\t-v: print XS version\n"); - printf("\t-w : write snapshot of the XS machine at exit\n"); + printf("\t-d : read heap snapshot file and dump a textual representation to stderr\n"); + printf("\t-i : metering interval (defaults to 1)\n"); + printf("\t-l : metering limit (defaults to no limit)\n"); + printf("\t-q: prefix `print` output with the current meter count in square brackets\n"); + printf("\t-r : read heap snapshot file to create the XS machine\n"); + printf("\t-w : write heap snapshot file of the XS machine at exit\n"); + printf("\t-e: evaluate each string (in its own unique scope)\n"); + printf("\t-m: interpret each string as the path for a module to load\n"); + printf("\t-s: interpret each string as the path for a script to load\n"); printf("without -e, -m, -s:\n"); - printf("\tif the extension is .mjs, strings are paths to modules\n"); - printf("\telse strings are paths to scripts\n"); + printf("\teach string with suffix \".mjs\" is interpreted as the path for a module to load\n"); + printf("\tand each other string is interpreted as the path for a script to load\n"); } static int gxStep = 0; From 87b170f9a8aed9095da7e17b9397bad7b4e16bb9 Mon Sep 17 00:00:00 2001 From: Richard Gibson Date: Mon, 8 Jul 2024 17:41:37 -0400 Subject: [PATCH 5/5] docs: Update READMEs to describe correct `xsnap`/`xsnap-worker` usage --- xsnap/documentation/xsnap-worker.md | 20 ++++++++-------- xsnap/readme.md | 37 +++++++++++++---------------- 2 files changed, 27 insertions(+), 30 deletions(-) diff --git a/xsnap/documentation/xsnap-worker.md b/xsnap/documentation/xsnap-worker.md index 8a87345..4f1d259 100644 --- a/xsnap/documentation/xsnap-worker.md +++ b/xsnap/documentation/xsnap-worker.md @@ -2,19 +2,19 @@ `sources/xsnap-worker.c` contains a variant of `xsnap` which accepts execution commands over a file descriptor. It is designed to function as a "JS coprocessor", driven by a parent process (which can be written in any language). The parent does a fork+exec of `xsnap-worker`, then writes [netstring](https://en.wikipedia.org/wiki/Netstring)-formatted commands to the child. The child executes those commands (evaluating JS or delivering the command to a handler function), possibly emitting one or more requests to the parent during execution, then finally finishes the command and writing a status to the parent (including metering information). -By default, the process starts from an empty JS environment (in the future it may start from an empty [hardened JavaScript](https://github.com/endojs/endo/blob/master/packages/ses/README.md) environment, but for now you must [`lockdown()`](https://github.com/endojs/endo/blob/master/packages/ses/README.md#lockdown) yourself). If the child is started with a `-r SNAPSHOTFILENAME` argument, it will start from a previously-written JS engine snapshot instead. +By default, the process starts from an empty JS environment (in the future it may start from an empty [hardened JavaScript](https://github.com/endojs/endo/blob/master/packages/ses/README.md) environment, but for now you must [`lockdown()`](https://github.com/endojs/endo/blob/master/packages/ses/README.md#lockdown) yourself). If the process is started with a `-r $SNAPSHOT` option, it will instead start from a previously-written heap snapshot file. -The launch arguments are: +Supported options are: * `-h`: print this help message -* `-i `: set the metering check interval: larger intervals are more efficient but are likely to exceed the execution budget by more computrons -* `-l `: limit each delivery to `` computrons -* `-p`: print the current meter count before every `print()` -* `-r `: launch from a JS snapshot file, instead of an empty environment -* `-s SIZE`: set `parserBufferSize`, in kiB (1024 bytes) -* `-v`: print the `xsnap` version and exit with rc 0 -* `-n`: print the agoric-upgrade version and exit with rc 0 -* All `argv` strings that do not start with a hyphen are ignored. This allows the parent to include dummy no-op arguments to e.g. label the worker process with a vat ID and name, so admins can use `ps` to distinguish between workers being run for different purposes. +* `-v`: print xsnap and XS version information +* `-i `: set the metering check interval (defaults to 1). Larger intervals are more efficient but are likely to exceed the execution budget by more computrons. +* `-l `: limit each delivery to `` computrons (defaults to no limit) +- `-p`: prefix `print` output with the current meter count in square brackets +* `-r `: read the heap snapshot file to create the XS machine (rather than initializing to an empty environment) +* `-s `: set parser buffer size, in kiB (defaults to 8192 = 8_388_608 bytes) + +All arguments that do not start with a hyphen are ignored, so they may be used to label the worker process with identifying information visible to e.g. `ps` (such as a vat ID and name). Once started, the process listens on file descriptor 3, and will write to file descriptor 4. The process will perform a blocking read on fd3 until a complete netstring is received. The first character of the body of this netstring indicates what command to execute, with the remainder of the body as the command's payload. The commands are: diff --git a/xsnap/readme.md b/xsnap/readme.md index 2726f29..c9d8b9a 100644 --- a/xsnap/readme.md +++ b/xsnap/readme.md @@ -1,7 +1,4 @@ # xsnap -Revised: January 4, 2022 - -Warning: These notes are preliminary. Omissions and errors are likely. If you encounter problems, please ask for assistance. ## About @@ -67,23 +64,23 @@ The release version is built in `$MODDABLE/build/bin/win/release ` ## Usage xsnap [-h] [-v] - [-d ] [-r ] [-w ] - [-i ] [-l ] [-p] - [-e] [-m] [-s] strings... + [-d] [-i ] [-l ] [-q] [-r ] [-w ] + [-i ] [-l ] [-q] + [-e] [-m] [-s] ... - `-h`: print this help message - `-v`: print XS version -- `-d `: dump snapshot to stderr -- `-r `: read snapshot to create the XS machine -- `-w `: write snapshot of the XS machine at exit -- `-i `: metering interval (defaults to 1) -- `-l `: metering limit (defaults to none) -- `-p`: prefix `print` output with metering index -- `-e`: eval `strings` -- `-m`: `strings` are paths to modules -- `-s`: `strings` are paths to scripts - -Without `-e`, `-m`, `-s`, if the extension is `.mjs`, strings are paths to modules, else strings are paths to scripts. +- `-d `: read heap snapshot file and dump a textual representation to stderr +- `-i `: metering interval (defaults to 1) +- `-l `: metering limit (defaults to no limit) +- `-q`: prefix `print` output with the current meter count in square brackets +- `-r `: read heap snapshot file to create the XS machine +- `-w `: write heap snapshot file of the XS machine at exit +- `-e`: evaluate each `` (in its own unique scope) +- `-m`: interpret each `` as the path for a module to load +- `-s`: interpret each `` as the path for a script to load + +Without `-e`, `-m`, `-s`, each string with suffix ".mjs" is interpreted as the path for a module to load and each other string is interpreted as the path for a script to load. ## Examples @@ -174,12 +171,12 @@ There is a performance gain but a precision lost. ### metering-built-ins -Use the `-p` option to prefix `print` output with the metering index. +Use the `-q` option to prefix `print` output with the current meter count in square brackets. cd ./examples/metering-built-ins - xsnap test.js -p + xsnap test.js -q -The tests builds, sorts and reverses an array of 100 random numbers. Observe the metering index around `sort` and `reverse`. +The tests builds, sorts and reverses an array of 100 random numbers. Observe the increasing metering count. ... [3935] 99 0.4153946155753893