7.1.0

7.1.0 - 2023-05-05

Features

Some bigger organizations deploying Quepid want to nest it under another domain, like https://tools.bigcorp.com/quepid instead of the more typical https://quepid.bigcorp.com. @worleydl open a PR to support this eleven months ago, and now we finally have it over the finish line! Just specify the RAILS_RELATIVE_URL_ROOT for when you want to nest Quepid under another domain. #500 by @worleydl.

Improvements

• Custom scorers are sorted by the order from oldest to newest in the modal picker UI. Instead, lets sort them alphabetically. #717 by @epugh fixes #695.

• teamSvc unit test was failing a lot, and now has been dealt with... By ignoring the checks. :-(.

• When creating a new Book of judgements, seed the Scorer with the one from the Case you were using. #716 by @epugh fixes #705.

Bugs

• Using templates in OpenSearch (and Elasticsearch) clashes with how we display (or hide) the fieldSpec field. #706 by @mkr fixes #699.

• The "Find Missing Docs" UI doesn't actual work with OpenSearch. #707 by @mkr fixes #700. Nicer help text as well.

• Generated link to individual OpenSearch (and Elasticsearch) document changed, and didn't render properly. o19s/splainer-search#117 by @mkr fixes #701.

• Swapping from one Scorer to another Scorer would lose the labels in the popup window. #717 by @epugh fixes #704 and #696 by @epugh. It may also fix #613 by @atarora ;-).

7.0.0 Ready for Human Rating! And the Future.

7.0.0 - 2023-04-24

Are you ready to launch a Human Rating Program? Quepid is now finally able to support you and your fellow human judges in gathering explicit ratings on your search results.

Since the dawn of Quepid, we've suffered from an ahem sub optimal rating interface for gathering human judgements.

The rating interface failed most best practices for human rating, including suffering from position bias by listing documents in the order the search engine returned them, only supporting a single rating per query/doc pair, requiring lots of mouse movement and clicking.
All that, and the UI is a combination of features required for a Relevancy Engineer with those of a Human Judge.
It's enough to make you weep.
Just to make life harder, the rating interface requires a live connection to your search engine, which often required additional technical hoops to be jumped by your decidedly untechnical subject matter experts.

However that is no longer the case!

This rating interface features:

• You can have up to three independent ratings for every query/doc pair, opening the door to interesting measurements of rating quality.
• A static dataset for rating that is populated from your live search engine. Now your set of query/doc pairs won't change over time unless you want them to.
• Query/doc pair are randomly sampled, but with a bias in favour of higher positioned results over lower positioned so you can get to meaningful numbers quicker.
• Ability to mark a query/doc pair as "I can't Rate This Document" so we can find edge/confusing cases.
• You control what shows up on the card using the same display configuration as in your typical Quepid case.
• Supports thumbnails and hero images in the card.
• There are Scoring Guidelines for your Raters to understand what the scoring scale means.

To make the life of a Relevancy Engineer better, you can now import your queries and ratings from a Book of Judgments.
Indeed you can roundtrip from a Case with query/doc pairs to a Book of judgements, get them rated, and then import then back into your Case.

This work was inspired by the great work that the folks at the Wikimedia Foundation did on Discernatron, their human rating tool.

Quepid is now the big 7.0! There have been 98 PR's since release 6.14.0. We are now running on Rails 7, which is great for opening the door to future improvements and keeping us from accumulating tech debt. This also means we took the opportunity to bump the versions of Redis and MySQL we use, as well as Node, Ruby, and all the other libraries. We're now on a modern infrastructure everywhere except for the Relevance Engineer's UI which is still on AngularJS.

Do back up your data before doing this upgrade! Normally I'm pretty cavalier about updates in Quepid-land, but this changes a lot of things, including the MySQL version from 5.6 to 8.... So please back up your data, and for good measure, use those export options to export your precious ratings that you've collected.

Below are details on some selected PR's.

Features

• As mentioned above, the human rater interface is a big feature. There are too many PR's and Issues to call them out individually. However, I do want to thank everyone who contributed to that really important feature. You know who you are!

• Allow a Case to be marked Public to facilitate sharing analytics. Public cases don't require you to log in for certain screens. #595.

• Jupyter notebook for calculating Jaccard Similarity between Snapshots. #586 by @atarora.

• Add Reciprocal Rank as a default Scorer. #525 by @david-fisher.

Improvements

• Rails 7 Upgrade! This would be enough to move us from Quepid 6 to Quepid 7 by itself. Turned out to be pretty painless upgrade. Most of the work was in #627.

• We enabled RenovateBot to provide automatic PR's for dependencies. In the past three months we had a flood of dependency updates, which improves our security profile and helps us deal with tech debt by ensuring we aren't falling behind.

• Refactor to remove manualMaxScore and manualMaxScoreValue from custom scorers as not used. #609.

6.14.0

6.14.0 - 2022-11-22

Fixes inspired by folks working with the Chorus project.

Improvements

• Rework the selection of settings for a search engine so that we are smarter about if you pick a TMDB demo server, use great defaults, but if you are picking your own search engine, then we make fewer assumptions about query structure, which reduces the chance the first query will blow up! This was first identified by @macohen as part of querqy/chorus#129. Tracked as #580 by @epugh and fixed in #582,

Bugs

• Command line tool thor import:ratings didn't handle a csv file with a header row. Now filters off the header row. Thanks @wrigleyDan for spotting this problem. #581 by @epugh and fixed in #583.

Full Changelog: v6.13.0...v6.14.0

Megan's Birthday Edition

6.13.0 - 2022-10-28

It's my sister Megan's birthday 🎂 today, so I figured I would give her a release of Quepid. Happy birthday Megan!

There is so much to celebrate in this release, however I have to call out adding support for OpenSearch, enabling API keys to work with Elasticsearch, and the first release supporting Jupyter notebooks runing in Quepid!

Features

• Quepid now supports OpenSearch! #319 was opened 18 months ago by @flaxsearch, and contributed to by @DmitryKey. Huge thanks to @mkr for stepping up and adding the support to both splainer-search and Quepid in #559.

• Quepid now supports Elastic Cloud and Elasticsearch auth with API Keys! Thanks to the work by @aditya-kanekar in #563 and then extended by @worleydl in #566. @aditya-kanekar also wrote up the docs on https://github.com/o19s/quepid/wiki/Troubleshooting-Elastic-Cloud-and-Quepid for how to set up Elastic Cloud!

• Jupyter Notebooks In Quepid 🎊. Everyone wants to customize their analytics and have different visualizations, so let's make that easier. Quepid now integrates Jupyterlite, a in browser version of Jupyter. So you can write your notebooks using the Python you know and love, and not have to worry about installing dependencies, as Quepid ships all the typical ones. This is an area that I expect a lot of improvement and change as we get to know how to ship sample analytics as Jupyter notebooks. #544 by @epugh.

• Our first sample notebook let's you compare the scores of two snapshots using a histogram. Really drives home the "before and after" story of Relevance Tuning. To support this notebook, we needed to preserve the score and if all the docs are rated in the snapshot datastructure. #550 by @epugh.

• Snapshots now include the total number of results for a query, useful for analytics. Also expose the Quepid API for looking up snapshots in the snapshots modal UI. #553 by @epugh fixes #539 by @renekrie.

• Curious what version of Quepid you are running? So are the rest of us! Display the Quepid version number in the page footer, #570 by @jzonthemtn is fixed by #576 by @epugh.

Improvements

• We've moved away from the "master" terminology to "main" for the default code branch, and updated links for that.

• Moved to Ruby 3! Ruby 3.1.2 on Bullseye is apparently twice as fast as Ruby 2. These two changes let Quepid run on Apple Silicon. It also lets us simplify our Dockerfile setup for Chromium, which we use for testing our frontend application. Lastly, we ripped out Webpacker (Webpack). Webpacker was added for JS toolingduring our migration to Rails 6, however we never actually used it in our development tooling, and is no longer preferred as part of Rails 7. #558 by @epugh and @mkr.

• Integrate updating of database schema annotations into build processes. This leverages the annotations gem, which we've had for years, but wasn't documented and therefore wasn't being manually run. #555 by @epugh.

• When we first moved to Rails from python, we had various database table constraints that were not enforced. Over the years we've started enforcing them, but never cleaned up the old data, till now! #552 by @epugh. Mostly of interest to folks with a deployment of Quepid back to 2016 ;-).

• During the Rails 6 migration we found out Redis wasn't critical to running Quepid, and attempted to make it optional. However that turned out to be confusing, and future features will require Redis. So let's put it back in our default production docker compose setup. #557 by first time contributor @OkkeKlein. Thanks!

Bugs

• Number of results to show is not preserved when cloning a case. #578 by @epugh fixes #565 submitted by @MassimilianoUngheretti-TomTom.

• Occasionally the Case snapshot isn't successful and there is no snapshot status information displayed to the user. #569 by @atarora fixes #568, also by @atarora.

• The Snapshot Comparison Jupyter notebook doesn't show the number of results found. #571 fixes #572, both by @atarora.

• If your TC_URL isn't defined, then that causes Quepid to blow up. #573 by @jzonthemtn is fixed by #575 by @epugh.

• docker-compose.prod.yml is out of date and needs Redis and Nginx. #557 by @OkkeKlein fixes #554.

• Changing Search Engines doesn't reset the api_method, so you could have ES attmepting to use JSONP, which is Solr specific. #560 opened by @epugh fixed by #559 by @mkr and @epugh.

New Contributors

• @aditya-kanekar made their first contribution in #563
• @OkkeKlein made their first contribution in #557
• @mkr made their first contribution in #559

Full Changelog: v6.12.1...v6.13.0

6.12.1

6.12.1 - 2022-08-11

Bugs

• "Show Only Rated" feature in Quepid busted for ES. Highlighting on _id after a terms match in ES causes a index out of bounds exception. Also, the API method in explainOther needed to have the right case, might make a constants file at some point. o19s/splainer-search#109 by @worleydl fixed the issue in Splainer. Bumped in Quepid via #547 by @epugh.

6.12.0 - 2022-08-09

including 6.12.0 in this release notes as Eric forgot to tag it.

Features

• Search box (to find queries). #506 and #512 by @epugh fixes #493 by @theaaronposc. Thanks Aaron!

• You can now generate a new Case through the thor command line tools, useful when setting up lots of cases or as part of a script. #533 by @epugh.

• Surface how many unrated documents you have for a query as an alert bubble. #532 by @epugh.

• The Frog Pond Report tells you about the state of your ratings. Understand the distribution of queries that need ratings and decide if you need to "hop to it" ;-). #526 by @epugh.

• Create missing queries when importing ratings. #509 by @epugh.

Improvements

• Gray out Queries with no ratings. Previously they showed up as Red with a 0.0 score, the same as if everything had bad ratings! #516 by @worleydl and suggested by @renekrie.

• Keycloak Version 17 changed the default url to not have the /auth in the namespaces. #536 by @epugh fixes #528. We now test Quepid with Keycloak 18.

• Average Precision Scorer AP@10 wasn't implemented following the most common definition of AP. #535 by @david-fisher fixes this! Thanks David!

• Taking a Snapshot is intensive process that makes a unique query to the search engine. #513 by @epugh provides nicer error message.

• Remove some more unused AngularJS code. #529 by @epugh.

• 'Information need' import doesn't works for an empty case. #508 by @epugh fixes #507 by @atarora.

• Bump to latest Rails to pick up CVE fixes, Ruby 2.7.6 recommended by Heroku, Bundler 2.3.18, plus general updates to gem dependencies.

Bugs

• The detailed export of the case doesn't exports the 0 documents matching queries. #504 by @atarora fixes #501.

• Move the logic for the default ES and Solr server definitions from server AND client side to just client side. #505 by @epugh fixes #503.

• Limited encoding renders snapshots unusable. #510 by @epugh fixes #499 by @KennyLindahl. Thanks Kenny for the analysis!

• Fixed missing query fields from cloning a case. #520 by @epugh

6.11.0

6.11.0 - 2022-04-23

Features

• Documenting the explicit Information Need of the user is a core part of running human judgements, and historically people used the Notes field for this. Now we have a Information Need attribute for every Query, and you can import/export that data via CSV, as well as edit it in the UI. Mouse over the query to see exactly what the user is trying to do! #480 by @epugh.

• The details of a specific query exposed by the Explain Query modal is in a Human readable format, but isn't in a JSON compliant format. Now we sort the keys alphabetically, which helps the user dig through the details, and have a Copy as JSON button so you can use other tools to look at the JSON information. #497 by
  @epugh fixes #492.

• Delete All Queries for a Case was requested by @negyesi in #475. At the case level added a Delete button that brings up a modal that lets you delete the case, archive the case, or delete all the queries. #486 by @epugh.

• Frustrated that your custom API that looks like Solr to Quepid has to deal with JSONP callbacks? Now you don't, as we have introduced the ability to swap from the default JSONP to GET api methods when you define your Solr end point. This is going to simplify the work that folks have to do. #495 by @epugh. For more information refer to the docs at https://github.com/o19s/quepid/blob/master/docs/endpoints_solr.md#solr-endpoints-structure.

• Do you have a relative image url in your search engine that you want to show via thumb: or image:? You can now provide some JSON in the field specification to provide that data: id,title:title,{"name": "image_url", "type":"image", "prefix": "https://www.example.org/images"}, description. This feature is experimental as we learn more about how to have richer field specifications, and may change! #487 by @epugh.

Improvements

• The link to view a single Solr document specifies XML, which I realize is a very OK boomer approach these days! o19s/splainer-search#107 by @begomeister fixes #494 by swapping to JSON format instead.

• github.com/teleborder/FileSaver.js repo disappeared, so migrate to an actual published node module file-saver instead. #498 by @epugh.

• We had a mish-mash of query and query_text as headers in our CSV files, so we've standardized on query to make export/import simpler. #489 by @epugh.

• We discovered that Redis was ONLY used in Quepid to support sending events to Google Analytics! If your QUEPID_GA isn't specified, then Redis is no longer required. Thanks to @rbednarzCBI for updating the production docker settings to remove Redis in #488.

Bugs

• The various CSV upload and export features didn't really handle , in the text properly, so we gave that a good going over. #482 by @atarora fixed by #483 and #485 by @epugh

• Tracking the history of your tries, we had a limit in the database that prevented you from having lots of tries. #478 by @KennyLindahl and #470 by @korlowski. Fixed by #479 by @epugh. Thanks @KennyLindahl for the additional testing work you did to help me fix this!

• Fixed the ability to peek at the next page of result when you have less then 10 results left. Thanks @anegyesi for submitting #473 and fixed in #496.

• Noticed the case level graph isn't useful? PR #491 by @worleydl, Save scores less often, fixes stretching of graph, fixes this.

• @thePanz opened a ticket a year ago (#330) about ratings getting progressively slower in the UI, and a huge amount of updates to the backend. Thanks to @worleydl and #490 this should be fixed.

6.10.1

6.10.1 - 2022-02-28

Bugs

• Updated Splainer-Search to https://github.com/o19s/splainer-search/releases/tag/v2.13.0 to enhance snapshots with ES, and make our parsing of ES.

• When Quepid switches between HTTP and HTTPS protocols, the flash message "Reloaded on HTTP protocol for Quepid app to match search engine URL" causes some CSS layout issues which led to the "Run my Queries" button to slide off the page! Darn you CSS. First spotted by @nathancday, while giving a demo of course! Fixed by #463 by @epugh.

• When reloading from HTTP to HTTPS Elasticsearch during the case wizard, we lose the default query params for ES, we end up
  with the default Solr query params! #457 by @epugh and @wrigleyDan. Fixed by #464 by @epugh.

• Build was failing for Apple Silicon M1 users! #430 by @llermaly and fixed by @wrigleyDan via #460.

v6.10.0

6.10.0 - 2022-01-03

We've taken a rag and polished up the Find and Rate Missing Documents modal box in this release. We've also completely
redone how we attempted to handle http and https connections from Quepid to http and https search engines that was first
introduced into 6.9.0 based on what we've learned in the real world.

Improvements

• Reworked the handing of http and https connections to Quepid when connecting to http and https search engines. Added in Development mode a nginx based proxy to make http://localhost/ and https://localhost/ work, which is also how you would deploy Quepid in a Production set up. Still works fine on http://localhost:3000 if you only use http based search engine connection. Big thanks to @jzonthemtn for #451 which fixes #444 and #438.

• The Find and Rate Missing Documents query interface assumes you use the Lucene query syntax. Turns out we have a highlighter built in, so enable that for Lucene syntax. #453 by @epugh.

• Writing your own scorer? The modal popup window is rather cramped, so let's give the editor room to breathe by making them larger! #452 by @epugh.

• In the early days of Quepid, looking up a single Solr doc would trigger a facet.field on every field you display in Quepid. For some Solr's, this can turn a quick lookup for a single document to a 30 second or more ordeal because of unexpected faceting! Thanks to @jeffryedvm for opening #442. Fixed by #456 by @epugh.

Bugs

• On the Find and Rate Missing Documents screen the ability to show just the rated documents had an issue that you had to click the button twice, making you think it was broken. #454 and #423 by @epugh are fixed in #455 by @epugh.

v6.9.1

6.9.1 - 2021-10-27

We forgot one improvement to splainer-search, so we are rolling another release!

Improvements

• Splainer-search 2.11.0 handles nulls the same as empty arrays ({ }) in Solr output. Nulls might happen when a Proxy is used. #440 by @epugh fixes issue discovered by @mkr in o19s/splainer-search#97.

• Node 15 is not a LTS version, which leads to a warning message! #437 by @epugh bumps this.

v6.9.0

6.9.0 - 2021-10-27 - Spooky HTTPS Support!

Features

• Chrome 94 forces you to be on https URL to access HTTPS secured search engines. Quepid now checks to see whether it is running on the same protocol, HTTPS or HTTP that the search engine is running on, and if not, prompts you to reload the app on the same protocol as the search engine. This required updating the messenging in the Case Wizard and the Tune Relevance screens. We also simplified the logic in the home_controller.rb around bootstrapping a Case and Try. Thanks to @tboeghk for pairing with @epugh on how to solve this problem. #431 by @epugh fixes #426 by @DmitryKey and #432 by @bbirrell.

Improvements

• Remove remnants of "SecureApp" angular1 app that was refactored a while ago into regular Rails app for login screens. #434 by @epugh fixes #433.

• Logic to enable Google Analytics tracking was crufty and caused classloader issues in development mode. #419 by @epugh fixes #418.