ICU support in V8

[lukehutch]

Can you please enable ICU support in Javet V8?

Removing the original issue text -- see the update below

[newk5]

I believe this is already supported, check https://www.caoccao.com/Javet/tutorial/advanced/internationalization_i18n.html

[lukehutch]

Thanks for the docs link. I followed the docs, and for this code

const numberFormat = new Intl.NumberFormat('en-US');
console.log(numberFormat.format(1234567.89));  // "1,234,567.89"

I get:

TypeError: Internal error. Icu error.

Here is my pom.xml:

        <dependency>
            <groupId>com.caoccao.javet</groupId>
            <artifactId>javet</artifactId>
            <version>4.1.1</version>
        </dependency>
        <dependency>
            <groupId>com.caoccao.javet</groupId>
            <artifactId>javet-v8-linux-x86_64</artifactId>
            <version>4.1.1</version>
        </dependency>
        <dependency>
            <groupId>com.caoccao.javet</groupId>
            <artifactId>javet-v8-macos-arm64</artifactId>
            <version>4.1.1</version>
        </dependency>
        <dependency>
            <groupId>com.caoccao.javet</groupId>
            <artifactId>swc4j</artifactId>
            <version>1.3.0</version>
        </dependency>
        <dependency>
            <groupId>com.caoccao.javet</groupId>
            <artifactId>javet-v8-macos-arm64-i18n</artifactId>
            <version>4.1.1</version>
        </dependency>
        <dependency>
            <groupId>com.caoccao.javet</groupId>
            <artifactId>javet-v8-linux-x86_64-i18n</artifactId>
            <version>4.1.1</version>
        </dependency>

Here is my init code:

    static {
        try (var icuStream = V8Executor.class.getResourceAsStream("/icudt76l.dat")) {
            if (icuStream != null) {
                var tempFile = File.createTempFile("icudt76l-", ".dat");
                tempFile.deleteOnExit();
                try (var fos = new FileOutputStream(tempFile)) {
                    icuStream.transferTo(fos);
                }
                // Set up V8 to use the ICU data
                V8RuntimeOptions.V8_FLAGS.setIcuDataFile(tempFile.getAbsolutePath());
            } else {
                throw new RuntimeException("ICU data file not found in resources");
            }
        } catch (IOException e) {
            throw new RuntimeException("Failed to extract ICU data to temp file", e);
        }
    }

Here is how I create the runtime:

        v8Runtime = V8Host.getV8I18nInstance().createV8Runtime();

[lukehutch]

@caoccao do you have any idea what might be causing this exception?

[caoccao]

Where do you get that icu file?

[lukehutch]

From here: https://github.com/nodejs/node/tree/main/deps/icu-small/source/data/in

[caoccao]

Please find a better one in V8.

[lukehutch]

@caoccao The v8 github repo does not contain any .dat files from ICU. I don't know where to get them from. The ICU github repo contains many individual .dat files, not a single one compiled for release.

Can you please update the instructions in your documentation about where to get a good ICU .dat file from?

[caoccao]

Usually you can find it from the latest actions, as the doc indicates. However these days have been special. Please check the announcements in the discord channel to get better idea of what's going on. There's no need to update the doc because most probably next month things will get back to normal. And it's not that hard to find a usable one.

[lukehutch]

I have looked at the Actions page of Javet, but I don't see how to get the ICU dat files from there. As far as I can see this is only build logs.

I have also looked at announcements in the Discord channel... I don't see anything there currently. And I can watch this in the future, sure. But I need to find this ICU file quickly.

If you're not going to update the documentation, can you please provide me with a link to the dat file that I need? The documentation really doesn't make this process easy to figure out.

[lukehutch]

@caoccao I also tried building the .dat file using https://github.com/apotocki/icu4c-iosx/tree/master , and that generated file gives the same error: TypeError: Internal error. Icu error. -- note that this exception is thrown inside JS. It's possible this error is coming from the ICU data missing a section, but I don't know.

By the way, if I comment out the line V8RuntimeOptions.V8_FLAGS.setIcuDataFile(tempFile.getAbsolutePath());, then V8Host.getV8I18nInstance().createV8Runtime() crashes the JRE with a SIGSEGV (the JNI code does not check if the value is null) -- not a good experience (Javet should throw an exception if the location of the ICU data has not been specified).

Are the ICU data files also arch-specific?

Why not ship the V8 .dat files in the ICU-enabled versions of the Javet jars? That would make this whole process much easier...

[caoccao]

There are boundaries between an SDK and an SDK consumer. All what you said belong to the SDK consumer's responsibility. Please do null check by yourself. Please find a suitable ICU files by yourself. An SDK shouldn't enforce its users to pay the tax that they don't want to pay, e.g. to ship an ICU file in the JAR.

[caoccao]

I've just run some actions. You may download the ICU files there.

[lukehutch]

I downloaded your ICU file from Actions (icudtl.dat), and this file does in fact work. Thank you.

However, since this is the only working source I have been able to find to get the .dat file from, and since these files aren't available on Actions after a certain amount of time (there were no files available before from builds 3+ months old), there is still a problem here.

There are boundaries between an SDK and an SDK consumer. All what you said belong to the SDK consumer's responsibility. Please do null check by yourself. Please find a suitable ICU files by yourself. An SDK shouldn't enforce its users to pay the tax that they don't want to pay, e.g. to ship an ICU file in the JAR.

The .dat file is absolutely required for Javet to work when i18n is enabled, and as we have seen, it is both very difficult to even find the right file, and very difficult to find one that actually works. You need to provide a working .dat file to your users, in an easy-to-source way.

If you don't want to ship the .dat file in the ICU-specific Javet Jars, then please at least provide one more Javet jar that has the .dat file in it, so that the .dat file can be pulled in as an optional Maven dependency.

[caoccao]

No. There are many versions of ICU files for various purpose. The choice is left to the users.