param --subject should support wildcard matching for subject

[mrvudoan]

Proposed change

Ex.
nats stream ls --subject=*.tokenX

Use case

on a nats-server used day by day, the subject is constructed by day rule but all available categories of subjects may NOT appear for everyday.
Ex:
available subjects,
YYYY-MM-DD.C1
YYYY-MM-DD.C2
YYYY-MM-DD.C3

for day 1: stream STR-2025-02-10
2025-02-10.C1
2025-02-10.C2

for day 2: stream STR-2025-02-11
2025-02-11.C1
2025-02-11.C3

for day 3: stream STR-2025-02-12
2025-02-12.C2
2025-02-12.C3

then this command
nats stream ls --subject=*.C2
should list streams:
STR-2025-02-10
STR-2025-02-12

Contribution

No response

[ripienaar]

This already works:

[rip@s1-lon]% nats s info ORDERS_10 --json|jq .config.subjects
[
  "js.in.orders_10"
]
~
[rip@s1-lon]% nats s ls --subject '*.in.orders_10'
╭──────────────────────────────────────────────────────────────────────────────────╮
│                          Streams matching *.in.orders_10                         │
├───────────┬─────────────┬─────────────────────┬──────────┬────────┬──────────────┤
│ Name      │ Description │ Created             │ Messages │ Size   │ Last Message │
├───────────┼─────────────┼─────────────────────┼──────────┼────────┼──────────────┤
│ ORDERS_10 │             │ 2024-06-04 10:25:56 │ 1,000    │ 10 MiB │ 1.01s        │
╰───────────┴─────────────┴─────────────────────┴──────────┴────────┴──────────────╯

Maybe you need to quote your patterns

[mrvudoan]

This does not work in my case.

I'm using the nats v0.1.1 (also tried with the 0.1.6) on Windows 10.
Here are some of my tries but none of them works.

nats stream ls --subject=.C2
nats stream ls --subject '.C2'

=> the expected result are streams:
STR-2025-02-10
STR-2025-02-12

nats stream ls --subject=.C4
nats stream ls --subject='.C4'
nats stream ls --subject '*.C4'

=> expected for no stream as the result.

but for all of these tries, all streams are listed.

if i used
nats stream ls --subject=2025-02-12.C3
then only the stream STR-2025-02-12 is returned (but this is not useful).

Thanks for reading.

[ripienaar]

show stream info for a stream that would match *.C4?

[mrvudoan]

well, I just want to have the result as same as from 'nats stream ls' but now use the --subject param supported with wildcard (*, >).

[ripienaar]

I understand what you want, I am trying to see why it is not working for you, hence asking for more information.

[ripienaar]

Would also help to see what your command does when run with --trace

[mrvudoan]

well, this is our internal server and by the company policy I can not send out any information like that.
The above example is for describing the issue, not the real data.
Just for brief, this server has about 735 streams (I used nats stream ls > list-stream.txt, then count the line in a text editor because I found no nats-cli command to count the number of streams on a server).
Moreover, when I use the --trace flag, it writes out a lot of info spreading on many screen pages. I tried to redirect into a file but these tracing line does not go into except the result of the nats stream ls).

[ripienaar]

Without seeing information of what is happening or what configuration you have, I am afraid I can't help you.

This works with both nats-0.1.6 and nats-0.1.4 even nats-0.0.35

[ripienaar]

The thing to look for in --trace is this:

$ nats s ls --subject '*.in.orders_10' --trace
10:41:57 >>> $JS.API.STREAM.LIST
{"offset":0,"subject":"*.in.orders_10"}

Note it sends a request with the subject listed there and correctly shown.

If that is working correctly and you do not see results, well then you probably dont actually have a stream that match. Or your nats server is ancient.

[mrvudoan]

In my case, the first lines of trace look like as following:

**> nats -s 192.168.20.52:4222 stream ls --subject ".MV" --trace > d:\trace.txt
17:24:20 >>> $JS.API.STREAM.LIST
{"offset":0,"subject":".MV"}

17:24:20 <<< $JS.API.STREAM.LIST
{"type":"io.nats.jetstream.api.v1.stream_list_response","total":735,"offset":

I will have my server admin to get the nats server version.

[mrvudoan]

Well, my nats-server version is v2.10.7.

[ripienaar]

Here the search string is “.MV” which is not valid, it only matches using nats patterns like *.MV which means 1 token before MV.

Token s are separated by dot.

[mrvudoan]

well, I don't know why the '*' is hidden by this github comment editor. Sorry for not taking a preview before commit my comment.

[ripienaar]

Ah ok. You should surround quotes text with 3 backticks and it won’t interpret as formatting.

Well then given that query. What subject are you trying to match in the stream?

[mrvudoan]

As mentioned in my first comment, I used an example to describe my issue.
Here in my real case, I want to use "*.MV" as the searching subject pattern to list all streams containing subjects look like
Day1.MV
Day2.MV
...

But this does not work as expected.
The command
nats stream ls --subject "*.MV"
always lists all streams on the nats-server, even the streams with no message at all (so wierd).

Because my application has the subject in format of "DayN.MV", and for some days, there is no message for this MV subject at all. I want to list out only streams having messages of that MV-suffix subject.

[ripienaar]

The match is against the configuration of the stream not the contents of the stream.

[mrvudoan]

Sorry, I don't get what you mean.

Which stream configuration should I take a review?
I thought this matching is against the subjects of a stream, i.e., a nats server may have multiple streams. In turn, each stream may have different subjects. And then with the command nats stream ls --subject=SUBJECT , I expect to have a list of streams containing that SUBJECT.

[ripienaar]

Sorry, I don't get what you mean.

Which stream configuration should I take a review? I thought this matching is against the subjects of a stream, i.e., a nats server may have multiple streams. In turn, each stream may have different subjects. And then with the command nats stream ls --subject=SUBJECT , I expect to have a list of streams containing that SUBJECT.

Do “nats stream info” see the subjects there in the configuration? That’s what it’s matching on. It will even match empty streams.

[ripienaar]

If you want to filter out empty streams - s ls cant do it but there's a much more capable nats s find that can do like nats s find --subject 'js.*.orders_0' --empty and have various other filters also, but thats only for empty streams of course.

I dont think we have a tool/api that can match on the subjects of messages actually stored in the streams, its aimed at finding streams that could potentially store a certain subject.

[mrvudoan]

I used nats stream subjects <stream-name> to list all subjects in a given stream and repeat for other streams. Then I can know which streams contain which subjects.
Using nats stream ls --subject=SUBJECT is just my expectation to have a list of them.
BTW, nats stream info returns some info not looking useful as nats stream subjects, e.g.

Subjects: => because all subjects in my stream start with DayN token, so this line contains only one value looks like DayN.>

Number of Subjects: xxx

So, I should develop a tool myself to scan all possible streams and match the search SUBJECT (wildcard supported) against their subjects.

Thank you a ton for so patient with me.

[ripienaar]

No worries, yeah getting the stream subjects using the info api and matching on that is probably your best bet - assuming its not just a case of empty or not streams.