2.0.7

Implement point in time API for opensearch 2.x. Works slightly different than Elastic's _pit API.

Unfortunately, this does not fix search_after for opensearch and there are still some errors related to _shard_doc not being a mapped field; which means we can't sort on that like we are supposed to in Elastic. This will need some further workarounds. Pull requests for this are welcome; my bandwidth to deal with opensearch specific bugs is limited.

What's Changed

• Build now uses Opensearch 2.8.0
• Support creating PITs when using Opensearch 2.x by @hanswesterbeek in #79
• Searchafter add fixme for sort on _shard_doc for opensearch by @jillesvangurp in #81

Full Changelog: 2.0.6...2.0.7

Closable client

• client now implements closable
• needed so the new sniffing node selector can close its client and not leak lots of Java threads

2.0.5

What's Changed

• add new experimental SniffingNodeSelector with affinity for nodes by @jillesvangurp in #76. This enables a smarter strategy for selecting nodes that periodically updates its list of available nodes as well as tries to use the same node based on either the thread name or a specified AffinityId co-routine scope. I neither is available it falls back to using randomly selected ndes. The round robin strategy is still used by default and you should not use the SniffingNodeSelector if your cluster is behind a load balancer or running in docker.

Compatibility breaking changes:

• RestClient.nextNode is now a suspend function

Full Changelog: 2.0.4...2.0.5

2.0.4

What's Changed

• Fix OS1/OS2 typo by @hanswesterbeek in #71
• Kotlin 1.8.21 and other library updates
• _pit API seems no longer supported in the latest Opensearch 2.7 release. I've restricted searchAfter to not allow OS1/OS2 anymore. If somebody needs this, please create a PR to add support for the new APIs in Opensearch. Or you can use scrolling searches

New Contributors

• @hanswesterbeek made their first contribution in #71

Full Changelog: 2.0.3...2.0.4

2.0.3 - filter & filters aggregations and other changes

A couple of feature and change requests came in that I took care of. Thanks @jelle-blaauw & @legzo for filing issues for this.

• #67 - implement filter and filters aggregation
• #68 - implement missing configuration properties for date histogram aggregation
• #70 - port is now nullable in the node configuration

Full Changelog: 2.0.2...2.0.3

2.0.2 search_after opt in for sort

Two minor fixes:

• search after now has a new parameter that allows you to override the error it will throw if you attempt to add a sort to the query. Added this because it's very easy to break search_after this way. So, opting in signals that you know what you are doing.

• and,or,not functions now have a variant that accepts a varargs argument.

Full Changelog: 2.0.1...2.0.2

Support dense vector and knn queries

This release adds support for doing knn searches along with a nice example based on openai generated embeddings.

What's Changed

• knn/vector search by @jillesvangurp in #66

Full Changelog: 2.0.0...2.0.1

2.0.0 Release

After 13 release candidates, lots of documentation updates and many small feature additions, improvements, etc. I decided that enough is enough. It's ready. Many thanks to the wonderful people that have contributed code in the past few months.

You can read about all the wonderful features in the manual. I'm not going to repeat all that here.

The short version of it is more features than ever, richer DSLs for everything, and it works on any platform that Kotlin can compile to. A lot of work went into creating this library. And I've tried my best to make this the nicest way to implement search features using Kotlin and provide a best in class developer experience (DX).

Why now?

We recently completed the process of migrating all our es-kotlin-client code to kt-search at FORMATION. This also unblocked upgrading our cluster to version 8. And since it always was my plan to not release 2.0 until we had done that, we can now finally do that.

Also, it's April 1st. So, why not? ;-). Now is a good day to do this.

Migrating from es-kotlin-client

If you have used es-kotlin-client before, kt-search 2.0.0 the successor to that. The short version of it is that migrating should be fairly easy. The query DSL is mostly backward compatible so your queries should be easy to port over. Completing the migration only took us a few days.

Changes

There are no major new changes since RC-13 a few days ago. Just one change related to #64 to allow you to get to the highlight object in search hits. Full highlight support will come in a future release. For now, you can just use the DSL to set this up manually.

What's next?

Kt-search is not done. My plan is to keep adding features, expanding DSL support, and improving things for the foreseeable future. I have a few specific things in mind that I might start working on soonish:

• Highlight support was requested #64 and seems like a nice feature to do next.
• We are currently working on using geospatial aggregations and I plan to add DSL support for that as we go.
• I'm interested in exploring vector search and will likely add support for that
• I've recently helped a client with Elastic Fleet and I was shocked at how hard it was to automate things around that. Clearly adding some kt-search support for that would be very helpful.

Of course none of this should stop you from using kt-search right now. It's usable. It's easy to extend it. But do let me know if you are missing some feature. Or better still, open an issue and create a pull request.

2.0.0-RC-13

This release adds support for geo queries. We use geoshape queries a lot at FORMATION, so adding this was kind of the obvious move.

In the process of implementing this, I found few more things that seemed nice to address:

• implement geo queries by @jillesvangurp in #65 geo_grid, geo_boundary, geo_shape, and geo_distance queries are now supported
• move VariantRestriction annotation to searchdsl module
• allow using raw json as a string literal in JsonDsl - useful for embedding raw geojson geometries in shape queries
• make WaitFor the default for bulk operations - I keep forgetting to add this and it seems like a sane default.
• Elasticsearch 8 is now used by default for the tests. Version 7 is still tested for in the matrix tests.

Also, we have completed the migration away from the old es-kotlin-client at FORMATION and are now using kt-search exclusively. Implementing that revealed no serious issues other than the race conditions with the CIO engine that we addressed in RC-10.

Note, ignore the 11 and 12 tags. I made a mess while tagging the wrong thing and moving to RC-13 seemed like the most obvious way to move forward.

There are still some features I might want to squeeze in for 2.0.0 but they are pretty minor and I'm running out of excuses to not get this over with soonish.

Full Changelog: 2.0.0-RC-10...2.0.0-RC-13

Switch ktor client engine to Java (on jvm)

We experienced some threading issues with the CIO engine. Switching to the Java client seems to fix this. So, we're making this the default.

You can still switch back to the CIO client by using ktorClientWithCIOEngine or by overriding the client parameter with your own implementation. If you do, make sure to take care of basic authentication (if using elastic cloud) and json handling like our default implementations do.

The issues only surface under intense load that we have in the FORMATION Spring Boot integration tests.