Unicode 16.0

[Carbon225]

Validation tests pass.

This test needs to be updated (we need to find a character that is not assigned to a script, 771 is now Hebrew):

    # unknown script for character
    c = chr(771)
    e: CurableSequence = ens_process(f'bitcoin.bitcin.bi̇tcin.bitсin{c}').error
    assert e.general_info == 'Contains visually confusing characters from multiple scripts (Latin plus other scripts)'
    assert (
        e.sequence_info
        == 'This character is disallowed because it is visually confusing with another character from the Latin script'
    )

There is no pyunormalize for unicode 16.0.

UnicodeWarning: Unicode version mismatch: pyunormalize is using 15.1.0, but the ENS Normalization spec is for 16.0.0 (2024-09-10T20:47:54.200Z)

[djstrong]

@Carbon225 There is pyunormalize for unicode 16.0

[Carbon225]

I wrote this code:

for cp in tqdm(range(0, 0x10FFFF)):
    c = chr(cp)
    e = ens_process(f'bitcoin.bitcin.bi̇tcin.bitсin{c}').error
    try:
        assert e.general_info == 'Contains visually confusing characters from multiple scripts (Latin plus other scripts)'
        assert (
            e.sequence_info
            == 'This character is disallowed because it is visually confusing with another character from the Latin script'
        )
        print(f'{cp} - OK')
        break
    except AssertionError:
        pass

It should find a new example for this test.
But it did not find any characters that cause this message.

I will make this test xfail, most likely all codepoints are now included in ens-normalize-js.

[Carbon225]

I changed this in the readme:

Based on JavaScript implementation version 1.9.0

Original implementation based on JavaScript implementation version 1.9.0

I did not want people thinking we are using some old reference implementation.

[lightwalker-eth]

@Carbon225 @djstrong Looks good!

I shared 3 comments with feedback. Suggest we action the comment I shared on the README.md file in this PR. For the other comments, appreciate your advice and good to action those separately in a different PR.

[lightwalker-eth]

Suggested change

	* Original implementation based on [JavaScript implementation version 1.9.0](https://github.com/adraffy/ens-normalize.js/tree/4873fbe6393e970e186ab57860cc59cbbb1fa162).
	* Supports updates to the ENS Normalize reference implementation for Unicode 16.0.

Spent some time thinking about this line. I think it's best we just fully remove the old idea / bullet point here. I agree it made since when we first released this library, but we already are making reference to Raffy's JS implementation up above and no benefit for anyone if we go into this detail here.

Alternatively, I think it's now useful to make a note here to identify which Unicode version we've updated this library to support.

[Carbon225]

@lightwalker-eth I like the idea of mentioning unicode support. But what do you mean by "supports updates"?
Is it supposed to mean "Up-to-date with ENS Normalize reference implementation as of Unicode version 16.0"?
I'm happy to go with your version if you think it sounds ok.

[lightwalker-eth]

@Carbon225 Good catch. My suggestion could have been more clear. I like the way you proposed writing it:

"Up-to-date with ENS Normalize reference implementation as of Unicode version 16.0."