CLDR-15595 Reserve strings "undefined", "undefine", and "auto" in LDML

[sffc]

CLDR-15595

• This PR completes the ticket.

This change describes all of LDML, so I wasn't sure exactly where to put it, so I made a spot for it in the top-level tr35.md.

[srl295]

Text LGTM but I think it's in the wrong place, put it under #Valid_Attribute_Values

[srl295]

Also There Oughtta Be A Test™, under TestDtdData (for the DTD) and perhaps something else to verify that it doesn't show up in actual data.

[srl295]

@sffc ping?

[sffc]

This seems like a tricky test to write. I need to figure out how to check the full set of identifiers.

[srl295]

This seems like a tricky test to write. I need to figure out how to check the full set of identifiers.

see the DtdData class

[macchiati]

That class can be used to iterate through all of the elements and all of their attributes, and check if any value, like " undefined" is valid.

…

On Fri, Oct 6, 2023, 17:24 Steven R. Loomis ***@***.***> wrote: This seems like a tricky test to write. I need to figure out how to check the full set of identifiers. see the DtdData class — Reply to this email directly, view it on GitHub <#3278 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACJLEMC4X7D5UCPK5WZVJLDX6APDZAVCNFSM6AAAAAA5AQLDQCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJQHA4TIMRVGY> . You are receiving this because your review was requested.Message ID: ***@***.***>

[macchiati]

I made a quick program to do a test for that, but we would need to enhance the @match values further before it could be a useful test. (#3328)

So that should not stand in the way of this PR.

[macchiati]

I think "specifies many enumerations" is too vague to be normative.

One way to more completely specify it would be to restrict it to attribute values. That is, however, too broad, because attributes can be representing other information. Or maybe focus on the terms that can be in bcp47 subtags?

One way to make progress in v44 would be to explicitly list those enumerations that cannot contain these terms, and then in v45 work on a more general solution.

[srl295]

I made a quick program to do a test for that, but we would need to enhance the @match values further before it could be a useful test. (#3328)

So that should not stand in the way of this PR.

So you have confirmed that the DTD doesn't currently use these terms?

Do we need to also check the data that there’s not some data element, such as a calendar or unit with <type name="undefine" … ?

I agree that if we're confident we're not already in violation of the requirement, this could proceed into the spec.

( i still think it should be located under #Valid_Attribute_Values though )

[macchiati]

So you have confirmed that the DTD doesn't currently use these terms?

No, I've confirmed something quite different: that we have *no* guarantees in the DTD that those terms *cannot* be used. (That is, there are `@MATCH` values that permit them.) It would be possible to make a test that no current (and past?) CLDR data had values in {define, defined, auto}. One slight complication would be that the multivalue attributes would need to split the values to check.

…

On Mon, Oct 9, 2023 at 4:17 PM Steven R. Loomis ***@***.***> wrote: I made a quick program to do a test for that, but we would need to enhance the @match <https://github.com/match> values further before it could be a useful test. (#3328 <#3328>) So that should not stand in the way of this PR. So you have confirmed that the DTD doesn't currently use these terms? Do we need to also check the data that there’s not some data element, such as a calendar or unit with <type name="undefine" … ? I agree that if we're confident we're not already in violation of the requirement, this could proceed into the spec. ( i still think it should be located under #Valid_Attribute_Values though ) — Reply to this email directly, view it on GitHub <#3278 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACJLEMHLDZ2HLQNVGOYCCLTX6QBQZAVCNFSM6AAAAAA5AQLDQCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONJTGEYDEMBWGU> . You are receiving this because your review was requested.Message ID: ***@***.***>

[sffc]

Could @macchiati or @srl295 suggest a next step on this PR?

[srl295]

Could @macchiati or @srl295 suggest a next step on this PR?

Recapping existing suggestions:

1. I suggested that it should be moved to be a subsection of ### Valid Attribute Values

2. In the absence of a general statement and enforcing test, @macchiati suggested that it would be better to make an explicit list of enumerations where the given list is not allowed.

   Perhaps something like this:

> Certain attributes have values which are reserved for use by implementations and will not be added as explicit entries.
>
> The reserved values are as follows:
> - undefined … ( explanation etc )
> - … 
> 
> The attributes where these will not be added are as follows:
>
> - `day type=`
> - `unit type=`
> - …

3. Validate that those values are not currently added.
4. Follow-up issue to automate the above, and simplify the spec (perhaps by an annotation or other metadata).

[sffc]

In the absence of a general statement and enforcing test, @macchiati suggested that it would be better to make an explicit list of enumerations where the given list is not allowed.

It seems infeasible to produce and maintain this list.

I feel like a hand-wavy solution is okay for now, until we can write the general test?

[srl295]

In the absence of a general statement and enforcing test, @macchiati suggested that it would be better to make an explicit list of enumerations where the given list is not allowed.

It seems infeasible to produce and maintain this list.

Is it? As I noted, there's the DtdData class which has full access to annotation metadata. And there's also an XML Schema form of the data.

There's even grep 'ATTLIST.*|' common/dtd/ldml.dtd | fgrep -v draft which produces a bunch of things with DTD enumerations. You could filter that to find items of interest.

edit also, adding a general test would obviate future maintenance.

I feel like a hand-wavy solution is okay for now, until we can write the general test?

A hand-wavy solution isn't appropriate for a spec. In a sense it's already hand wavy, you can implement to what's spec'ed now, and you have brought this up to CLDR-TC. So, you would likely not see the auto calendar added for example.