Index Erlang/OTP apps into typesense

[josevalim]

We can probably ask them to notify us on new tags/releases if necessary.

[josevalim]

@garazdawi do you have a preference of the version granularity you want us to index into search? Do you want every MAJOR.MINOR? Or also MAJOR.MINOR.PATCH?

[garazdawi]

We can probably ask them to notify us on new tags/releases if necessary.

Yes, no problem.

do you have a preference of the version granularity you want us to index into search? Do you want every MAJOR.MINOR? Or also MAJOR.MINOR.PATCH?

The docs we publish are only for the latest patch release on each branch + the master branch, at the moment OTP-28.0-rc0, OTP-27.2, OTP-26.2.5.6, OTP-25.3.2.16, etc.

So I want the indexing for each patch release, but I don't care about the results for anything but the latest patch on each branch. We may want to have indexing for each patch in the future, but I doubt it.

If you are crawling erlang.org for the data, what you want to crawl is currently:

• erlang.org/doc/ ## This is 27
• erlang.org/docs/28

and then when 28 is released it will be:

• erlang.org/docs/27
• erlang.org/doc/ ## This is 28
• erlang.org/docs/29

erlang.org/docs/27 also exists now for consistency, but it should not be used.

With algolia we currently use sitemap.xml to describe which files to crawl. We also insert this metadata into the docs to help with crawling:

<meta name="major-vsn" content="27">
<link rel="canonical" href="https://www.erlang.org/doc/system/installation_guide.html" />