Search depth for headings

[joshuataylor]

Hi,

Is it possible to set a search depth for headings, so headings from 1-5 are covered?

this

should

be

searchable

too

Or have this settable as a config, to not break backwards compatibility?

[fabriciomurta]

There's no means to do this right now with Retype, unfortunately. Search is currently a client-side database that is meant to be the smallest possible while still bringing in useful results. But there's not really a configuration on "how deep in headings" this database should be populated with. There's actually no means to set how "comprehensive" the search database goes.

We do think on ways to expand search capabilities and your feedback is more than welcome, thank you!

[geoffreymcgill]

Is it possible to set a search depth for headings, so headings from 1-5 are covered?

Nice suggestion. Yes, I think we can make this work.

We will keep this thread updated with our progress.

Here's another search related discussion, see #59.

[joshuataylor]

Awesome, let me know if you need a tester for this :).

Yeah I saw that, but wasn't sure if I should bump it or create a new one. IMHO, full content search is a different topic (and shouldn't be OOTB, as yes that causes search index size to explode).

[geoffreymcgill]

The enhancement of all heading levels to the search index is in the queue and should make it into the Retype v1.11 release.

At the moment, we do not have a release date for v1.11 but we will be sure to keep this thread updated with our progress.

[DannyBen]

In case opinions are still collected, here is mine:

I a bit confused about the various discussions on the search subject.

I understand why it currently only searches in filenames and H1 headers. It is low-cost, and for me, about 70% effective. A good compromise from engineering standpoint.

While allowing the user to opt in for a deeper heading search will probably not hurt, I feel that for most users it will be not here nor there - the "there" being full text search.

As I see it, full text search option should be higher in priority than deeper header searches - since it will make the deep header search unnecessary.

The mdbook documentation builder, which was my weapon of choice in the pre-retype era, is doing it quite effortlessly, from the user standpoint. It does not feel heavy or an afterthought. My vote is to add something like this to Retype, but keep the elegance that retype brings. In case there is a fear that some users of retype will not like the larger search index, they can opt out to the "shallow search" behavior.

[geoffreymcgill]

Hi @DannyBen. I appreciate the feedback. Adding the heading levels H4, H5, and H6 to the search index is a trivial enhancement for us to make. Little effort is involved and it certainly wouldn't be taking away any engineering effort from a larger full-text search index project.

With adding the three extra heading levels, the increase in size of the client-side search index would also be trivial. Relatively few authors use an H4 (or smaller). I have occasionally used an H4 for a small bold Example or Sample heading above a code sample, but now prefer using the Code blocks component configured with a title.

I have seen instances where the author uses H4 but likely should have been using an H3. In that scenario, the improved search indexing of all headings will certainly help.

We have this noted as a topic to discuss tomorrow at a team meeting.

We have also been discussing the full-text search index recently.

[geoffreymcgill]

The enhancement required changing just one value in the code:

heading.Level is (>= 1 and <= 3)) // Before
heading.Level is (>= 1 and <= 6)) // After

I still want to run this change against several large sample projects to get a feel for how these extra levels of headings affect the experience and usefulness of the search results. So far, things feel good.

[geoffreymcgill]

As of v1.11.0, the Retype search now indexes all heading elements from H1 to H6. Hope this helps.

[geoffreymcgill]

Search has undergone a huge improvement in the upcoming Retype v2.0. A full text index is now the default, with options to configure a partial or basic search index mode.

All Retype v1.x releases use the equivalent of the new basic mode.

[DannyBen]

Cool. The days of the 1.x releases, when there were more frequent releases with a couple of features each, were much more enjoyable from user perspective. Seems like many features are piled onto the 2.0 release, so each feature is prolonging the release of the other features...

[geoffreymcgill]

Yes, agreed. Big bang releases are not going to happen again. We will be doing much smaller releases in the future. The v2.0 build was way more involved than anticipated and its just the way it worked out.

[sboklund]

@geoffreymcgill Do you have an ETA for v2.0? :)

[geoffreymcgill]

@sboklund We should have a 2.0.0-preview release available today.

[geoffreymcgill]

Retype v2.0.0-preview is now available. See #287.