From 0f44589bf56bf88cc5ee7bc4ce07e296de4ab5bf Mon Sep 17 00:00:00 2001 From: "Carrie Warner (Mattermost)" <74422101+cwarnermm@users.noreply.github.com> Date: Mon, 28 Apr 2025 17:10:32 -0400 Subject: [PATCH 1/5] OpenSearch as option to Elasticsearch --- source/about/editions-and-offerings.rst | 2 +- source/about/mattermost-v9-changelog.md | 8 +- source/comply/data-retention-policy.rst | 2 +- source/conf.py | 2 + source/configure/bleve-search.rst | 2 +- .../environment-configuration-settings.rst | 48 +-- source/configure/optimize-your-workspace.rst | 2 +- source/deploy/backup-disaster-recovery.rst | 10 +- .../deploy/software-hardware-requirements.rst | 4 +- source/scale/additional-ha-considerations.rst | 2 +- source/scale/elasticsearch-setup.rst | 121 ++++++++ source/scale/elasticsearch.rst | 273 ------------------ source/scale/enterprise-search.rst | 189 ++++++++++++ source/scale/opensearch-setup.rst | 61 ++++ source/scale/scaling-for-enterprise.rst | 10 +- source/upgrade/admin-onboarding-tasks.rst | 4 +- source/upgrade/important-upgrade-notes.rst | 12 +- 17 files changed, 418 insertions(+), 334 deletions(-) create mode 100644 source/scale/elasticsearch-setup.rst delete mode 100644 source/scale/elasticsearch.rst create mode 100644 source/scale/enterprise-search.rst create mode 100644 source/scale/opensearch-setup.rst diff --git a/source/about/editions-and-offerings.rst b/source/about/editions-and-offerings.rst index a14c222c6bf..854e30af89d 100644 --- a/source/about/editions-and-offerings.rst +++ b/source/about/editions-and-offerings.rst @@ -58,7 +58,7 @@ Mattermost Enterprise is an enterprise-grade collaboration system that supports This offering includes all the features of `Mattermost Professional <#mattermost-professional>`__, plus: -- :doc:`Enterprise-scale search with dedicated indexing and usage resourcing via cluster support `. +- :doc:`Enterprise-scale search with dedicated indexing and usage resourcing via cluster support `. - :doc:`Sychronization of access controls, channels, and teams with AD/LDAP Groups `. - :doc:`eDiscovery and compliance export automation `. - :doc:`Enterprise mobile device management with custom EMM support via AppConfig `. diff --git a/source/about/mattermost-v9-changelog.md b/source/about/mattermost-v9-changelog.md index 2a9749433c3..65cf11dc12c 100644 --- a/source/about/mattermost-v9-changelog.md +++ b/source/about/mattermost-v9-changelog.md @@ -80,7 +80,7 @@ ### Important Upgrade Notes - - Added support for Elasticsearch v8. Also added Beta support for [Opensearch v1.x and v2.x](https://opensearch.org/). A new config setting ``ElasticsearchSettings.Backend`` has been added to differentiate between Elasticsearch and Opensearch. The default value is `elasticsearch`, which breaks support for AWS Elasticsearch v7.10.x since the official v8 client only works from Elasticsearch v7.11 and higher versions. See the important note below for details. + - Added support for Elasticsearch v8. Also added Beta support for [OpenSearch v1.x and v2.x](https://opensearch.org/). A new config setting ``ElasticsearchSettings.Backend`` has been added to differentiate between Elasticsearch and OpenSearch. The default value is `elasticsearch`, which breaks support for AWS Elasticsearch v7.10.x since the official v8 client only works from Elasticsearch v7.11 and higher versions. See the important note below for details. - Mattermost supports Elasticsearch v7.17+. However, we recommend upgrading your Elasticsearch v7 instance to v8.x. See the [Elasticsearch upgrade](https://www.elastic.co/guide/en/elasticsearch/reference/current/setup-upgrade.html) documentation for details. - When using Elasticsearch v8, ensure you set ``action.destructive_requires_name`` to ``false`` in ``elasticsearch.yml`` to allow for wildcard operations to work. @@ -88,8 +88,8 @@ **AWS Elasticsearch** If you're using AWS Elasticsearch, you must: -1. Upgrade to AWS Opensearch. Refer to the [Opensearch upgrade](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/version-migration.html) documentation for details. -2. Disable "compatibility mode" in Opensearch. +1. Upgrade to AWS OpenSearch. Refer to the [OpenSearch upgrade](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/version-migration.html) documentation for details. +2. Disable "compatibility mode" in OpenSearch. 3. Upgrade Mattermost server. 4. Update the Mattermost `ElasticsearchSettings.Backend` configuration setting value from `elasticsearch` to `opensearch` manually or using [mmctl](https://docs.mattermost.com/manage/mmctl-command-line-tool.html#mmctl-config-set). This value cannot be changed using the System Console. See the Mattermost [Elasticsearch backend type](https://docs.mattermost.com/configure/environment-configuration-settings.html#elastic-backendtype) documentation for additional details. 5. Restart the Mattermost server. @@ -160,7 +160,7 @@ New setting options were added to ``config.json``. Below is a list of the additi #### Changes to the Enterprise plan: - Under ``ElasticsearchSettings`` in ``config.json``: - - Added ``Backend`` to differentiate between Elasticsearch and Opensearch. The default value is Elasticsearch. + - Added ``Backend`` to differentiate between Elasticsearch and OpenSearch. The default value is ``elasticsearch``. ### API Changes - Added new API endpoints to manage remote clusters. diff --git a/source/comply/data-retention-policy.rst b/source/comply/data-retention-policy.rst index 1f104b03c5c..e7eca7acbda 100644 --- a/source/comply/data-retention-policy.rst +++ b/source/comply/data-retention-policy.rst @@ -47,7 +47,7 @@ You can also run the deletion job manually at any time by selecting **Run Deleti .. note:: - If using data retention and :doc:`ElasticSearch `, ensure the :ref:`ElasticSearch aggregate search indexes ` setting is set to a value that is greater than your data retention policy in days. + If using data retention with :doc:`ElasticSearch `, ensure the :ref:`ElasticSearch aggregate search indexes ` setting is set to a value that is greater than your data retention policy in days. Frequently Asked Questions (FAQs) --------------------------------- diff --git a/source/conf.py b/source/conf.py index 11dda588b5c..49da0867265 100644 --- a/source/conf.py +++ b/source/conf.py @@ -3792,6 +3792,8 @@ def setup(_: Sphinx): "https://docs.mattermost.com/scale/scale-to-80000-users.html", "scale/scale-to-88000-users.html": "https://docs.mattermost.com/scale/scale-to-90000-users.html", +"scale/elasticsearch.html": + "https://docs.mattermost.com/scale/enterprise-search.html", # Upgrade redirects "upgrade/upgrading-mattermost-server.html#prepare-to-upgrade-to-the-latest-version": diff --git a/source/configure/bleve-search.rst b/source/configure/bleve-search.rst index 366b0514982..271f448acdb 100644 --- a/source/configure/bleve-search.rst +++ b/source/configure/bleve-search.rst @@ -42,4 +42,4 @@ The following conditions are applied when using Bleve search: How does search work with Bleve disabled? ------------------------------------------- -Mattermost performs full text searches against the database unless you have an :ref:`Enterprise license ` and :doc:`Elasticsearch ` configured. \ No newline at end of file +Mattermost performs full text searches against the database unless you have an :ref:`Enterprise license ` and :doc:`enterprise search ` configured. \ No newline at end of file diff --git a/source/configure/environment-configuration-settings.rst b/source/configure/environment-configuration-settings.rst index 850b1b9b36e..a801f400f70 100644 --- a/source/configure/environment-configuration-settings.rst +++ b/source/configure/environment-configuration-settings.rst @@ -943,10 +943,9 @@ Disable database search ~~~~~~~~~~~~~~~~~~~~~~~ +---------------------------------------------------------------+------------------------------------------------------------------------------+ -| When other search engines are configured, such as | - System Config path: **Environment > Database** | -| :doc:`Elasticsearch `, | - ``config.json`` setting: ``".SqlSettings.DisableDatabaseSearch: false",`` | -| the database can be disabled to perform searches. | - Environment variable: ``MM_SQLSETTINGS_DISABLEDATABASESEARCH`` | -| | | +| When `enterprise-scale search `, | - System Config path: **Environment > Database** | +| database search can be disabled from performing searches. | - ``config.json`` setting: ``".SqlSettings.DisableDatabaseSearch: false",`` | +| | - Environment variable: ``MM_SQLSETTINGS_DISABLEDATABASESEARCH`` | | - **true**: Disables the use of the database to perform | | | searches. If another search engine isn't configured, | | | setting this value to ``true`` will result in empty search | | @@ -956,20 +955,20 @@ Disable database search Search behavior in Mattermost depends on which search engines are enabled: -- When :doc:`Elasticsearch ` is enabled, Mattermost will try to use it first. +- When :doc:`Elasticsearch ` or :doc:`AWS OpenSearch ` is enabled, Mattermost will try to use it first. - If Elasticsearch fails or is disabled, Mattermost will attempt to use :doc:`Bleve `, if enabled. If this occurs, you will see the warning ``Encountered error on SearchPostsInTeamForUser``. -- If both Elasticsearch and Bleve fail or are disabled, Mattermost tries to search the database directly, if this is enabled. +- If these fail or are disabled, Mattermost tries to search the database directly, if this is enabled. - If all of the above methods fail or are disabled, the search results will be empty. .. note:: Disabling this configuration setting in larger deployments may improve server performance in the following areas: - - Reduced Database Load: When database search is enabled, every search query executed by users needs to interact with the database, leading to additional load on the database server. By disabling database search, you can avoid these queries, thereby reducing the database load. - - Improved Response Time: Database searches can be time-consuming, especially with large datasets. Disabling database search can result in faster response times because the system no longer spends time fetching and processing search results from the database. - - Offloading Search to Indexing Services: Disabling database search often means that searches are offloaded to specialized indexing services like Elasticsearch, which are optimized for search operations. These services can provide faster and more efficient search capabilities compared to traditional database searches. - - Lower Resource Consumption: Running search queries directly against the database can be resource-intensive (using CPU and memory). With database search disabled, these resources can be allocated to other critical functions, improving overall system performance. - - Enhanced Scalability: As the number of users and data volume grow, database search can become less efficient. Specialized search services are designed to scale more effectively, enhancing overall system scalability and performance. + - **Reduced Database Load**: When database search is enabled, every search query executed by users needs to interact with the database, leading to additional load on the database server. By disabling database search, you can avoid these queries, thereby reducing the database load. + - **Improved Response Time**: Database searches can be time-consuming, especially with large datasets. Disabling database search can result in faster response times because the system no longer spends time fetching and processing search results from the database. + - **Offloading Search to Indexing Services**: Disabling database search often means that searches are offloaded to specialized indexing services like Elasticsearch, which are optimized for search operations. These services can provide faster and more efficient search capabilities compared to traditional database searches. + - **Lower Resource Consumption**: Running search queries directly against the database can be resource-intensive (using CPU and memory). With database search disabled, these resources can be allocated to other critical functions, improving overall system performance. + - **Enhanced Scalability**: As the number of users and data volume grow, database search can become less efficient. Specialized search services are designed to scale more effectively, enhancing overall system scalability and performance. - However, the ability to perform database searches in Mattermost is a critical feature for many users, particularly when other search engines aren't enabled. Disabling this feature will result in users seeing an error if they attempt to use the Mattermost Search box. It’s important to balance performance improvements with the needs of your organization and users. Applied schema migrations @@ -1219,7 +1218,7 @@ Elasticsearch .. include:: ../_static/badges/ent-selfhosted.rst :start-after: :nosearch: -Elasticsearch provides enterprise-scale deployments with optimized search performance and prevents performance degradation and timeouts. Learn more about :doc:`Elasticsearch ` in our product documentation. +Elasticsearch provides enterprise-scale deployments with optimized search performance and prevents performance degradation and timeouts. Learn more about :doc:`Elasticsearch ` in our product documentation. You can configure the Elasticsearch environment in which Mattermost is deployed in **System Console > Environment > Elasticsearch**. You can also edit the ``config.json`` file as described in the following tables. Changes to configuration settings in this section require a server restart before taking effect. @@ -1256,7 +1255,7 @@ Enable Elasticsearch indexing :systemconsole: Environment > Elasticsearch :configjson: .Elasticsearchsettings.Backend :environment: MM_ELASTICSEARCHSETTINGS_BACKEND - :description: Set the type of search backend as either Elasticsearch or Opensearch. + :description: Set the type of search backend as either Elasticsearch or OpenSearch. Backend type ~~~~~~~~~~~~~ @@ -1265,28 +1264,11 @@ Backend type | The type of search backend. | - System Config path: **Environment > Elasticsearch** | | | - ``config.json`` setting: ``".Elasticsearchsettings.Backend: elasticsearch",`` | | - ``elasticsearch`` - (**Default**) | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_BACKEND`` | -| - ``opensearch`` - Required for AWS Opensearch | | +| - ``opensearch`` - Required for AWS OpenSearch | | | customers. | | +----------------------------------------------------+-----------------------------------------------------------------------------------+ -.. important:: - - Mattermost v9.11 introduces support for `Elasticsearch v8 `__ and beta support for `Opensearch v1.x and v2.x `_. - - - Mattermost supports Elasticsearch v7.17+. However, we recommend upgrading your Elasticsearch v7 instance to v8.x. See the `Elasticsearch upgrade `_ documentation for details. - - Customers using Elasticsearch v8 must set ``action.destructive_requires_name`` to ``false`` in ``elasticsearch.yml`` to enable wildcard operations. - - **AWS Elasticsearch Customers** - - The official AWS Elasticsearch v8 client only works from Elasticsearch v7.11 and later. This is a breaking change for customers using AWS Elasticsearch v7.10.x. If you're using AWS Elasticsearch, you must upgrade to `AWS Opensearch `_. See the `AWS Amazon Opensearch upgrade `_ documentation for details. - - If you're using AWS Elasticsearch, you must: - - 1. Upgrade to AWS Opensearch for future compatibility. Refer to the `Opensearch upgrade `_ documentation for details. - 2. Disable "compatibility mode" in Opensearch. - 3. Upgrade the Mattermost server. - 4. Change the default ``ElasticsearchSettings.Backend`` configuration value from ``elasticsearch`` to ``opensearch`` using :ref:`mmctl config set `, or by editing the ``config.json`` file manually. This value cannot be changed using the System Console. See the Mattermost :ref:`Elasticsearch backend type ` documentation for additional details. - 5. Restart the Mattermost server. +Learn more about :ref:`enterprise search version support `. .. config:setting:: server-connection-address :displayname: Server connection address (Elasticsearch) @@ -1710,7 +1692,7 @@ Aggregate search indexes .. note:: - If you’re using :doc:`data retention ` and :doc:`Elasticsearch `, configure this with a value greater than your data retention policy. + If you’re using :doc:`data retention ` and :doc:`Elasticsearch `, configure this with a value greater than your data retention policy. .. config:setting:: post-aggregator-start-time :displayname: Post aggregator start time (Elasticsearch) diff --git a/source/configure/optimize-your-workspace.rst b/source/configure/optimize-your-workspace.rst index b382d684412..20639e36188 100644 --- a/source/configure/optimize-your-workspace.rst +++ b/source/configure/optimize-your-workspace.rst @@ -45,7 +45,7 @@ The following optimization areas can alert you to workspace suggestions, warning | Workspace access | Is the Mattermost workspace may not be accessible to users? | If your web server settings don't pass a live URL test, your workspace may not be accessible to others. | | | | See the :ref:`Web server configuration settings ` product documentation to learn more: | +-----------------------+----------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| Search performance | As your user base grows, is search getting slower? | See the :doc:`Elasticsearch ` product documentation to learn more. | +| Search performance | As your user base grows, is search getting slower? | See the :doc:`Enterprise search ` product documentation to learn more. | +-----------------------+----------------------------------------------------------------------------------------------------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Data privacy | Do you need more control and insights into your data? | See the product documentation to learn more: | | | | | diff --git a/source/deploy/backup-disaster-recovery.rst b/source/deploy/backup-disaster-recovery.rst index 90b2cd29923..c6190687fc2 100644 --- a/source/deploy/backup-disaster-recovery.rst +++ b/source/deploy/backup-disaster-recovery.rst @@ -59,9 +59,9 @@ Set up in one data center As a first step, set up Mattermost in a single data center. At a very basic high level, this would be something like below: .. image:: ../images/dr1.png - :alt: An architecture diagram showing a single proxy that's forwarding traffic to 2 nodes, a database with single writer + n readers, and an S3 bucket and ES/OS using AWS Opensearch service. + :alt: An architecture diagram showing a single proxy that's forwarding traffic to 2 nodes, a database with single writer + n readers, and an S3 bucket and ES/OS using AWS OpenSearch service. -The diagram above has a single proxy, forwarding traffic to 2 nodes. There's also a database with single writer + n readers and an S3 bucket and ES/OS using AWS Opensearch service. +The diagram above has a single proxy, forwarding traffic to 2 nodes. There's also a database with single writer + n readers and an S3 bucket and ES/OS using AWS OpenSearch service. At this stage, we are ignoring other details like LDAP/SAML, SMTP etc. @@ -121,7 +121,7 @@ Now you have bi-directional replication working between these S3 replica and sou Replicate ES/OS storage ^^^^^^^^^^^^^^^^^^^^^^^ -1. To replicate ES/OS storage, set up CCR (cross-cluster replication) for AWS Opensearch with the following requirements: +1. To replicate ES/OS storage, set up CCR (cross-cluster replication) for AWS OpenSearch with the following requirements: - Elasticsearch 7.10 or OpenSearch 2.x - Fine-grained access control enabled @@ -159,7 +159,7 @@ Replicate ES/OS storage To recap: -- Use Opensearch 2.x. +- Use OpenSearch 2.x. - Enable fine-grained access control. - Create the master user, and note the server credentials. - Set the IAM policy as above. @@ -314,7 +314,7 @@ Failover RDS to secondary To perform the failover, go to the RDS global cluster, and under **Actions**, select **Switchover or Failover global database**, and then select **switchover** to switch over without any data loss (which will take more time to complete). Alternatively, you can choose **failover** for a quicker failover at the expense of data-loss. If the entire region is unavailable anyways, then **failover** is no worse than **switchover**. -After this is done, the app nodes which were stuck trying to connect should move forward and everything should be functional. You can read/write, upload images and everything should be replicated. Everything except Opensearch data. +After this is done, the app nodes which were stuck trying to connect should move forward and everything should be functional. You can read/write, upload images and everything should be replicated. Everything except OpenSearch data. Failover ES/OS to secondary ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/source/deploy/software-hardware-requirements.rst b/source/deploy/software-hardware-requirements.rst index a3b00c2caa6..73338dad165 100644 --- a/source/deploy/software-hardware-requirements.rst +++ b/source/deploy/software-hardware-requirements.rst @@ -149,7 +149,7 @@ Search limitations on PostgreSQL: - One of them is: ``The length of a tsvector (lexemes + positions) must be less than 1 megabyte``, which means that, based on the file content, even files with content less than 1 MB won't be searchable if they hit the ``tsvector`` limit of 1 MB. -- If any of the above is an issue, you can either enable the :doc:`Elasticsearch feature ` or install MySQL instead. +- If any of the above is an issue, you can :doc:`set up and enable enterprise search `. Search limitations on MySQL: @@ -222,5 +222,5 @@ For Enterprise Edition deployments with a multi-server setup, we highly recommen - Prometheus to track system health of your Mattermost deployment, through :doc:`performance monitoring feature ` available in Mattermost Enterprise. - Grafana to visualize the system health metrics collected by Prometheus with the :doc:`performance monitoring feature `. Grafana 5.0.0 and later is recommended. -- Elasticsearch to support highly efficient database searches in a cluster environment. Elasticsearch v7.17+ is supported, and Elasticsearch v8.x or Opensource is recommended from Mattermost v9.11. :doc:`Learn more here `. +- Elasticsearch to support highly efficient database searches in a cluster environment. Elasticsearch v7.17+ is supported, and Elasticsearch v8.x or AWS OpenSearch is recommended from Mattermost v9.11. :doc:`Learn more `. - MinIO or AWS S3. Mattermost is compatible with object storage systems which implement the S3 API. Other S3-compatible systems may work, but are not officially supported. Learn more about file storage configuration options :ref:`in our documentation `. diff --git a/source/scale/additional-ha-considerations.rst b/source/scale/additional-ha-considerations.rst index 0a2d7ec80c4..c39c6674bb6 100644 --- a/source/scale/additional-ha-considerations.rst +++ b/source/scale/additional-ha-considerations.rst @@ -3,6 +3,6 @@ .. This page intentionally not accessible via the LHS navigation pane because it's included in other pages -`Elasticsearch `__ provides enterprise-scale deployments with optimized search performance and prevents performance degradation and timeouts. Elasticsearch allows you to search large volumes of data quickly, in near real-time, by creating and managing an index of post data. Mattermost’s implementation uses `Elasticsearch `_ as a distributed, RESTful search engine supporting highly efficient database searches in a :doc:`cluster environment `. Visit the :doc:`Mattermost Elasticsearch product documentation ` for deployment and configuration details. +`Elasticsearch `__ provides enterprise-scale deployments with optimized search performance and prevents performance degradation and timeouts. Elasticsearch allows you to search large volumes of data quickly, in near real-time, by creating and managing an index of post data. Mattermost’s implementation uses `Elasticsearch `_ as a distributed, RESTful search engine supporting highly efficient database searches in a :doc:`cluster environment `. Visit the :doc:`Mattermost Elasticsearch product documentation ` for deployment and configuration details. Performance monitoring support enables a Mattermost server to track system health for large Enterprise deployments through integrations with `Prometheus `__ and `Grafana `__. These integrations support data collection from several Mattermost servers, which is particularly useful if you’re running Mattermost :doc:`in high availability mode `. Once you’re tracking system health, you can :doc:`set up performance alerts ` on your Grafana dashboard. Visit the :doc:`Mattermost Performance Monitoring product documentation ` for installation details. \ No newline at end of file diff --git a/source/scale/elasticsearch-setup.rst b/source/scale/elasticsearch-setup.rst new file mode 100644 index 00000000000..ac69d2e50d1 --- /dev/null +++ b/source/scale/elasticsearch-setup.rst @@ -0,0 +1,121 @@ +Elasticsearch server setup +=========================== + +.. include:: ../_static/badges/ent-cloud-selfhosted.rst + :start-after: :nosearch: + +Elasticsearch allows you to search large volumes of data quickly, in near real-time, by creating and managing an index of post data. The indexing process can be managed from the System Console after setting up and connecting an Elasticsearch server. The post index is stored on the Elasticsearch server and updated constantly after new posts are made. In order to index existing posts, a bulk index of the entire post database must be generated. + +Deploying Elasticsearch includes the following two steps: `setting up Elasticsearch <#set-up-elasticsearch>`__, and `configuring Mattermost <#configure-mattermost>`_. + +Set up Elasticsearch +--------------------- + +We highly recommend that you set up Elasticsearch server on a dedicated machine separate from the Mattermost Server. + +1. Download and install the latest release of `Elasticsearch v8 `_, or `Elasticsearch v7.17+ `_. See the Elasticsearch documentation for installation details. + +2. Set up Elasticsearch with ``systemd`` by running the following commands: + + .. code-block:: sh + + sudo /bin/systemctl daemon-reload + sudo /bin/systemctl enable elasticsearch.service + sudo systemctl start elasticsearch.service + +3. Confirm Elasticsearch is working on the server by running the following command: + + .. code-block:: sh + + curl localhost:9200 + +4. Get your network interface name by running the following command: + + .. code-block:: sh + + ip addr + +5. Edit the Elasticsearch configuration file in ``vi`` by running the following command: + + .. code-block:: sh + + vi /etc/elasticsearch/elasticsearch.yml + +6. In this file, replace the ``network.host`` value of ``_eth0_`` with your network interface name, and save your changes. + +7. Restart Elasticsearch by running the following commands: + + .. code-block:: sh + + sudo systemctl stop elasticsearch + sudo systemctl start elasticsearch + +8. Confirm the ports are listenings by running the following command: + + .. code-block:: sh + + netstat -plnt + + You should see the following ports, including the ones listening on ports 9200 and 9300. Confirm these are listening on your server's IP address. + +9. Create an Elasticsearch directory and give it the proper permissions. + +10. Install the `icu-analyzer plugin `__ to the ``/usr/share/elasticsearch/plugins`` directory by running the following command: + + .. code-block:: sh + + sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install analysis-icu + +11. Test the connection from Mattermost to Elasticsearch by running the following command: + + .. code-block:: sh + + curl 172.31.80.220:9200 + +Configure Mattermost +--------------------- + +Follow these steps to configure Mattermost to use your Elasticsearch server and to generate the post index: + +.. tip:: + + Advanced Elasticsearch configuration settings for large deployments can be configured outside the System Console in the ``config.json`` file. See the :ref:`Elasticsearch configuration settings ` documentation to learn more. + +1. Go to **System Console > Environment > Elasticsearch**. +2. Set **Enable Elasticsearch Indexing** to ``true`` to enable the other the settings on the page. Once the configuration is saved, new posts made to the database are automatically indexed on the Elasticsearch server. +3. Set the Elasticsearch server connection details: + + a. Enter **Server Connection Address** for the Elasticsearch server you set up earlier. + b. (Optional) Enter **Server Username** used to access the Elasticsearch server. For AWS Elasticsearch, leave this field blank. + c. (Optional) Enter **Server Password** associated with the username. For AWS Elasticsearch, leave this field blank. + d. Set **Enable Cluster Sniffing** (Optional). Sniffing finds and connects to all data nodes in your cluster automatically. For AWS Elasticsearch, this field should be set to ``false``. + +.. tip:: + + From Mattermost v7.8, optional CA and client certificate configuration settings are available for use with basic auth credentials or to replace them. See the :ref:`Elasticsearch configuration settings ` documentation for details. + +4. Select **Test Connection** and then select **Save**. If the server connection is unsuccessful you won't be able to save the configuration or enable searching with Elasticsearch. + +5. Select **Build Index** to build the post index of existing posts. This process can take up to a few hours depending on the size of the post database and number of messages. The progress percentage can be seen as the index is created. To avoid downtime, set **Enable Elasticsearch for search queries** to ``false`` so that database search is available during the indexing process. + + .. important:: + + Complete bulk indexing before enabling Elasticsearch in the next step. Otherwise, search results will be incomplete. + +6. Enable Elasticsearch by setting **Enable Elasticsearch for search queries** to ``true``, and setting **Enable Elasticsearch for autocomplete** to ``true``. + +7. Save your configuration updates and restart the Mattermost server. + +.. note:: + + - Additional advanced Elasticsearch settings for large deployments can be configured outside the System Console in the ``config.json`` file. Read the :ref:`Elasticsearch configuration settings ` documentation to learn more. + - If your deployment has a large number of posts (typically in excess of one million but not strictly defined), the reindexing progress percentage may stay at 99% for a long time. The size of the data to be indexed is estimated, and on large databases, estimations can become inaccurate. While progress estimates may be inaccurate, and the progress percentage may appear stuck at near completion, indexing will continue behind the scenes until complete. + - Search results for files shared before upgrading to Mattermost Server v5.35 may be incomplete until an extraction command is run using the :ref:`mmctl `. After running this command, the search index must be rebuilt. Go to **System Console > Environment > Elasticsearch > Bulk Indexing**, then select **Index Now** to rebuild the search index to include older file contents. + - For high post volume deployments, we strongly encourage you to read and properly configure the Mattermost :ref:`LiveIndexingBatchSize ` configuration setting. + +Limitations +------------ + +1. Elasticsearch uses a standard selection of "stop words" to keep search results relevant. Results for the following words will not be returned: "a", "an", "and", "are", "as", "at", "be", "but", "by", "for", "if", "in", "into", "is", "it", "no", "not", "of", "on", "or", "such", "that", "the", "their", "then", "there", "these", "they", "this", "to", "was", "will", and "with". +2. Searching stop words in quotes returns more results than just the searched terms (`ticket `__). +3. Search results are limited to a user's team and channel membership. This is enforced by the Mattermost server. The entities are indexed in Elasticsearch in a way that allows Mattermost to filter them when querying, so the Mattermost server narrows down the results on every Elasticsearch request applying those filters. \ No newline at end of file diff --git a/source/scale/elasticsearch.rst b/source/scale/elasticsearch.rst deleted file mode 100644 index 8c390f98506..00000000000 --- a/source/scale/elasticsearch.rst +++ /dev/null @@ -1,273 +0,0 @@ -Elasticsearch -============= - -.. include:: ../_static/badges/ent-cloud-selfhosted.rst - :start-after: :nosearch: - -Elasticsearch provides enterprise-scale deployments with optimized search performance, dedicated indexing and usage resourcing via cluster support without performance degradation and timeouts, resulting in faster, more predicable search results. Mattermost's implementation uses `Elasticsearch `_ as a distributed, RESTful search engine supporting highly efficient database searches in a :doc:`cluster environment `. - -.. important:: - - - The default Mattermost database search starts to show performance degradation at around 2 million posts, on a server with 32 GB RAM and 4 CPUs. If you anticipate your Mattermost server reaching more than 2.5 million posts, we recommend enabling Elasticsearch for optimum search performance **before** reaching 3 million posts. - - For deployments with over 5 million posts, Elasticsearch is required to avoid significant performance issues (such as timeouts) with search and at-mentions. - - We highly recommend that you `set up Elasticsearch <#set-up-an-elasticsearch-server>`__ on a different machine than the Mattermost Server. - -Deployment guide ----------------- - -Elasticsearch allows you to search large volumes of data quickly, in near real-time, by creating and managing an index of post data. The indexing process can be managed from the System Console after setting up and connecting an Elasticsearch server. The post index is stored on the Elasticsearch server and updated constantly after new posts are made. In order to index existing posts, a bulk index of the entire post database must be generated. - -Deploying Elasticsearch includes the following two steps: `setting up the Elasticsearch server <#set-up-an-elasticsearch-server>`_, and `configuring Elasticsearch in Mattermost <#configure-elasticsearch-in-mattermost>`_. - -Set up an Elasticsearch server -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -1. Download and install the latest release of `Elasticsearch v8 `_, or `Elasticsearch v7.17+ `_. See the Elasticsearch documentation for installation details. - -.. important:: - - - Mattermost v9.11 adds support for `Elasticsearch v8 `__ as well as beta support for `Opensearch v1.x and v2.x `_. - - Mattermost supports Elasticsearch v7.17+. However, we recommend upgrading your Elasticsearch v7 instance to v8.x. See the `Elasticsearch upgrade `_ documentation for details. - - When using Elasticsearch v8, ensure you set ``action.destructive_requires_name`` to ``false`` in ``elasticsearch.yml`` to allow for wildcard operations to work. - - **AWS Elasticsearch** - - If you're using AWS Elasticsearch, you must: - - 1. Upgrade to AWS Opensearch. Refer to the `Opensearch upgrade `_ documentation for details. - 2. Disable "compatibility mode" in Opensearch. - 3. Upgrade Mattermost server. - 4. Update the Mattermost ``ElasticsearchSettings.Backend`` configuration setting value from ``elasticsearch`` to ```opensearch``` manually or using :ref:`mmctl `. This value cannot be changed using the System Console. See the Mattermost :ref:`Elasticsearch backend type ` documentation for additional details. - 5. Restart the Mattermost server. - -2. Set up Elasticsearch with ``systemd`` by running the following commands: - - .. code-block:: sh - - sudo /bin/systemctl daemon-reload - sudo /bin/systemctl enable elasticsearch.service - sudo systemctl start elasticsearch.service - -3. Confirm Elasticsearch is working on the server by running the following command: - - .. code-block:: sh - - curl localhost:9200 - -4. Get your network interface name by running the following command: - - .. code-block:: sh - - ip addr - -5. Edit the Elasticsearch configuration file in ``vi`` by running the following command: - - .. code-block:: sh - - vi /etc/elasticsearch/elasticsearch.yml - -6. In this file, replace the ``network.host`` value of ``_eth0_`` with your network interface name, and save your changes. - -7. Restart Elasticsearch by running the following commands: - - .. code-block:: sh - - sudo systemctl stop elasticsearch - sudo systemctl start elasticsearch - -8. Confirm the ports are listenings by running the following command: - - .. code-block:: sh - - netstat -plnt - - You should see the following ports, including the ones listening on ports 9200 and 9300. Confirm these are listening on your server's IP address. - -9. Create an Elasticsearch directory and give it the proper permissions. - -10. Install the `icu-analyzer plugin `__ to the ``/usr/share/elasticsearch/plugins`` directory by running the following command: - - .. code-block:: sh - - sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install analysis-icu - -11. Test the connection from Mattermost to Elasticsearch by running the following command: - - .. code-block:: sh - - curl 172.31.80.220:9200 - -Configure Elasticsearch in Mattermost -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Follow these steps to configure Mattermost to use your Elasticsearch server, and to generate the post index. - -.. tip:: - - Advanced Elasticsearch configuration settings for large deployments can be configured outside the System Console in the ``config.json`` file. See the :ref:`Elasticsearch configuration settings ` documentation to learn more. - -1. Go to **System Console > Environment > Elasticsearch**. -2. Set **Enable Elasticsearch Indexing** to ``true`` to enable the other the settings on the page. Once the configuration is saved, new posts made to the database are automatically indexed on the Elasticsearch server. -3. Set the Elasticsearch server connection details: - - a. Enter **Server Connection Address** for the Elasticsearch server you set up earlier. - b. (Optional) Enter **Server Username** used to access the Elasticsearch server. For AWS Elasticsearch, leave this field blank. - c. (Optional) Enter **Server Password** associated with the username. For AWS Elasticsearch, leave this field blank. - d. Set **Enable Cluster Sniffing** (Optional). Sniffing finds and connects to all data nodes in your cluster automatically. For AWS Elasticsearch, this field should be set to ``false``. - -.. tip:: - - From Mattermost v7.8, optional CA and client certificate configuration settings are available for use with basic auth credentials or to replace them. See the :ref:`Elasticsearch configuration settings ` documentation for details. - -4. Select **Test Connection** and then select **Save**. If the server connection is unsuccessful you won't be able to save the configuration or enable searching with Elasticsearch. - -5. Select **Build Index** to build the post index of existing posts. This process can take up to a few hours depending on the size of the post database and number of messages. The progress percentage can be seen as the index is created. To avoid downtime, set **Enable Elasticsearch for search queries** to ``false`` so that database search is available during the indexing process. - - .. important:: - - Complete bulk indexing before enabling Elasticsearch in the next step. Otherwise, search results will be incomplete. - -6. Enable Elasticsearch by setting **Enable Elasticsearch for search queries** to ``true``, and setting **Enable Elasticsearch for autocomplete** to ``true``. - -7. Save your configuration updates and restart the Mattermost server. - -.. note:: - - - Additional advanced Elasticsearch settings for large deployments can be configured outside the System Console in the ``config.json`` file. Read the :ref:`Elasticsearch configuration settings ` documentation to learn more. - - If your deployment has a large number of posts (typically in excess of one million but not strictly defined), the reindexing progress percentage may stay at 99% for a long time. The size of the data to be indexed is estimated, and on large databases, estimations can become inaccurate. While progress estimates may be inaccurate, and the progress percentage may appear stuck at near completion, indexing will continue behind the scenes until complete. - - Search results for files shared before upgrading to Mattermost Server v5.35 may be incomplete until an extraction command is run using the :ref:`mmctl `. After running this command, the search index must be rebuilt. Go to **System Console > Environment > Elasticsearch > Bulk Indexing**, then select **Index Now** to rebuild the search index to include older file contents. - - For high post volume deployments, we strongly encourage you to read and properly configure the Mattermost :ref:`LiveIndexingBatchSize ` configuration setting. - -Limitations ------------- - -1. Elasticsearch uses a standard selection of "stop words" to keep search results relevant. Results for the following words will not be returned: "a", "an", "and", "are", "as", "at", "be", "but", "by", "for", "if", "in", "into", "is", "it", "no", "not", "of", "on", "or", "such", "that", "the", "their", "then", "there", "these", "they", "this", "to", "was", "will", and "with". -2. Searching stop words in quotes returns more results than just the searched terms (`ticket `__). -3. Search results are limited to a user's team and channel membership. This is enforced by the Mattermost server. The entities are indexed in Elasticsearch in a way that allows Mattermost to filter them when querying, so the Mattermost server narrows down the results on every Elasticsearch request applying those filters. - -Frequently asked questions (FAQ) --------------------------------- - -Do I need to use Elasticsearch? -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The Elasticsearch engine is designed for large Enterprise deployments to run highly efficient database searches in a cluster environment. The default Mattermost database search starts to show performance degradation at around 2.5 million posts, depending on the specifications for the database server. If you expect your Mattermost server to have more than 2.5 million posts, we recommend using Elasticsearch for optimum search performance. - -Should I install Elasticsearch on the same machine as Mattermost Server? -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -No. We strongly recommend that you install Elasticsearch on a different machine than the Mattermost Server. - -What types of indexes are created? -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Mattermost creates three types of indexes: users, channels, and posts. Users and channels have one index each. Posts are aggregated by date, into multiple indexes. - -Can I pause an Elasticsearch indexing job? -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Yes. From Mattermost v6.7, the Elasticsearch indexing job is resumable. Stopping a server while the Elasticsearch indexing job is running puts the job in pending status. The job resumes when the server restarts. System admins can cancel an indexing job through the System Console. - -Can an index rollover policy be defined? -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The :ref:`AggregatePostsAfterDays ` configuration setting defines a cutoff value. All posts preceding this value are reindexed and aggregated into new and bigger indexes. The default setting is 365 days. - -Are there any new search features offered with Elasticsearch? -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -The current implementation of Elasticsearch matches the search features currently available with database search. The Mattermost team is working on extending the Elasticsearch feature set with file name and content search, date filters, and operators and modifiers. - -Are my files stored in Elasticsearch? -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -No, files and attachments are not stored. - -How do I monitor system health of an Elasticsearch server? -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -You can use this Prometheus exporter to monitor `various metrics `__ about Elasticsearch: `justwatchcom/elasticsearch_exporter `__. - -You can also refer to this `article about Elasticsearch performance monitoring `__. It's not written specifically for Prometheus, which :doc:`Mattermost's performance monitoring ` system uses, but has several tips and best practices. - -What form of data is sent to Elasticsearch? -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Mattermost communicates with Elasticsearch through its REST API using JSON messages for indexing and querying entities. - -How much data is sent to Elasticsearch and when? -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Every time a message is published, a channel is created, or a user changes, (either because their properties change e.g.: change of the first name or because they join/leave a channel), the data associated with that event is sent to Elasticsearch. - -If search via Elasticsearch is enabled, every search will generate a query. If autocompletion is enabled, every user or channel autocompletion associated with writing a message or user search will generate a query. - -How do I know if an Elasticsearch job fails? -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Mattermost provides the status of each Elasticsearch indexing job in **System Console > Environment > Elasticsearch**. Here you can see if the job succeeded or failed, including the details of the error. - -Failures are returned in the server logs. The error log begins with the string ``Failed job`` and includes a job_id key/value pair. Elasticsearch job failures are identified with worker name ``EnterpriseElasticsearchAggregator`` and ``EnterpriseElasticsearchIndexer``. You can optionally create a script that programmatically queries for such failures and notifies the appropriate system. - -My Elasticsearch indexes won't complete, what should I do? -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -If you have an Elasticsearch indexing job that's paused, it's likely your Elasticsearch server has restarted. If you restart your Elasticsearch server, you must also restart Mattermost to ensure jobs are completed. If restarting the Mattermost server does not resolve the issue, `contact Mattermost Support `__. - -Do I need to purge first then bulk index each time? -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Yes. - -Required Permissions For Mattermost Service Account ---------------------------------------------------- -In "least privilege" environments you may need to further constrain the service account permissions to limit the access your Elasticsearch service account has. -The following JSON provides an example of a "least privilege" permission set that allows Mattermost to operate correctly with Elasticsearch: - - .. code-block:: json - - { - "cluster_permissions": [ - "cluster:monitor/*", - "indices:admin/template/put", - "indices:data/write/bulk" - ], - "index_permissions": [ - { - "index_patterns": [ - "\*" - ], - "allowed_actions": [ - "indices:admin/get", - "indices:admin/create", - "indices:admin/delete", - "indices:admin/mapping/put", - "indices:admin/mappings/fields/get*", - "indices:data/read*", - "indices:data/write*" - ] - } - ] - } - -A simpler, more flexible, and resilient variant of the above would be: - -.. code-block:: json - - { - "cluster_permissions": [ - "cluster:monitor/*", - "indices:admin/template/put", - "indices:data/write/bulk" - ], - "index_permissions": [ - { - "index_patterns": [ - "\*" - ], - "allowed_actions": [ - "indices:*" - ] - } - ] - } diff --git a/source/scale/enterprise-search.rst b/source/scale/enterprise-search.rst new file mode 100644 index 00000000000..e3aeed6dc09 --- /dev/null +++ b/source/scale/enterprise-search.rst @@ -0,0 +1,189 @@ +Enterprise search +================== + +.. include:: ../_static/badges/ent-cloud-selfhosted.rst + :start-after: :nosearch: + +Mattermost database search starts to show performance degradation at around 2 million posts, on a server with 32 GB RAM and 4 CPUs. If you anticipate your Mattermost server reaching more than 2.5 million posts, we recommend enabling `Elasticsearch <#elasticsearch>`__ or `AWS OpenSearch Service <#aws-opensearch-service>`__ for optimum search performance **before** reaching 3 million posts. Both tools are highly capable and can handle enterprise-scale workloads. The choice between them depends on the following factors: + +- **Licensing and cost**: OpenSearch may be preferable for organizations avoiding proprietary licensing or opting for cost-effective solutions. +- **Feature requirements**: Elasticsearch's proprietary features (e.g., advanced analytics, security suites) may be preferred by organizations needing powerful out-of-the-box functionality. +- **Infrastructure alignment**: OpenSearch aligns well with AWS-centric environments, while Elasticsearch offers broader integrations. + +.. toctree:: + :maxdepth: 1 + :hidden: + :titlesonly: + + Elasticsearch setup + AWS OpenSearch setup + +Elasticsearch +------------- + +Elasticsearch is a well-established and widely used search engine with a large ecosystem and community support that provides enterprise-scale deployments with optimized search performance, dedicated indexing, and usage resourcing via cluster support for fast, predicable search results. + +Mattermost's implementation uses `Elasticsearch `_ as a distributed, RESTful search engine supporting highly efficient database searches in a :doc:`cluster environment `. Learn more about :doc:`setting up and configuring Mattermost for an Elasticsearch server `. + +AWS OpenSearch Service +----------------------- + +AWS OpenSearch Service is the official path forward from Elasticsearch v7.10.x for AWS customers. It's a fully managed service that makes it easy to deploy, operate, and scale OpenSearch clusters in the AWS Cloud to provide a simple and cost-effective way to search, analyze, and visualize data in real time. + +The AWS OpenSearch Service is built on the open-source OpenSearch project, which is a community-driven fork of Elasticsearch. Learn more about :doc:`setting up and configuring Mattermost for an OpenSearch server `. + +Supported paths +---------------- + +Review the following support paths for enterprise search based on the version you're using: + +.. tab:: Elasticsearch v8 + + `Elasticsearch v8 `__ is supported from Mattermost v9.11. While Mattermost supports Elasticsearch v7.17+, we recommend upgrading your Elasticsearch v7 instance to v8.x. See the `Elasticsearch upgrade `_ documentation for details. + + When using Elasticsearch v8, ensure you set ``action.destructive_requires_name`` to ``false`` in ``elasticsearch.yml`` to allow for wildcard operations to work. + +.. tab:: AWS OpenSearch Service + + AWS OpenSearch Service is the official path forward from Elasticsearch v7.10.x for AWS customers to provide a simple and cost-effective way to search, analyze, and visual data in real time. It's essentially a continuation of Elasticsearch v7.10.x but maintained as open source by AWS. It provides long-term support, active development, and compatibility with AWS clients, libraries, and managed services. + + See the **AWS Elasticsearch v7.10.x** tab on this page for details on upgrading to AWS OpenSearch, and see the :doc:`AWS OpenSearch setup ` documentation for details on configuring your Mattermost deployment to use AWS OpenSearch. + +.. tab:: AWS Elasticsearch v7.10.x + + If you're using Elasticsearch v7.10.x under AWS’s managed services, you can't use newer Elasticsearch clients like the v8 client without changing backend infrastructure. If you're using AWS Elasticsearch v7.10.x, you must `upgrade to AWS OpenSearch `_ for future compatibility. + + The migration path from Elasticsearch v7.10.x to OpenSearch has been designed to be straightforward, minimizing effort: + + 1. Disable "compatibility mode" in OpenSearch. + 2. Upgrade Mattermost server. + 3. Update the Mattermost ``ElasticsearchSettings.Backend`` configuration setting value from ``elasticsearch`` to ```opensearch``` manually or using :ref:`mmctl `. This value cannot be changed using the System Console. See the Mattermost search :ref:`backend type ` configuration setting documentation for additional details. + 4. Restart the Mattermost server. + +Frequently asked questions (FAQ) +-------------------------------- + +Do I need to use Elasticsearch or AWS OpenSearch? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +No. Enterprise search engines are designed for large Enterprise deployments to run highly efficient database searches in a cluster environment. The default Mattermost database search starts to show performance degradation at around 2.5 million posts, depending on the specifications for the database server. If you expect your Mattermost server to have more than 2.5 million posts, we recommend using Elasticsearch or AWS OpenSearch for optimum search performance. + +Should I install Elasticsearch or OpenSearch on the same machine as Mattermost Server? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +No. We strongly recommend that you install Elasticsearch or AWS OpenSearch on a dedicated machine that's separate from the Mattermost server. + +What types of indexes are created? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Mattermost creates three types of indexes: users, channels, and posts. Users and channels have one index each. Posts are aggregated by date, into multiple indexes. + +Can I pause an search indexing job? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Yes. From Mattermost v6.7, the search indexing job is resumable. Stopping a server while the search indexing job is running puts the job in pending status. The job resumes when the server restarts. System admins can cancel an indexing job through the System Console. + +Can an index rollover policy be defined? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The :ref:`AggregatePostsAfterDays ` configuration setting defines a cutoff value. All posts preceding this value are reindexed and aggregated into new and bigger indexes. The default setting is 365 days. + +Are there any new search features offered with Elasticsearch? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The current implementation of Elasticsearch matches the search features currently available with database search. The Mattermost team is working on extending the Elasticsearch feature set with file name and content search, date filters, and operators and modifiers. + +Are my files stored in Elasticsearch or OpenSearch? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +No, files and attachments are not stored. + +How do I monitor system health of an Elasticsearch server? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can use this Prometheus exporter to monitor `various metrics `__ about Elasticsearch: `justwatchcom/elasticsearch_exporter `__. + +You can also refer to this `article about Elasticsearch performance monitoring `__. It's not written specifically for Prometheus, which :doc:`Mattermost's performance monitoring ` system uses, but has several tips and best practices. + +What form of data is sent to Elasticsearch or OpenSearch? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Mattermost communicates with Elasticsearch or OpenSearch through its REST API using JSON messages for indexing and querying entities. + +How much data is sent to Elasticsearch or OpenSearch and when? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Every time a message is published, a channel is created, or a user changes, (either because their properties change e.g.: change of the first name or because they join/leave a channel), the data associated with that event is sent to Elasticsearch or OpenSearch. + +If search via Elasticsearch or OpenSearch is enabled, every search will generate a query. If autocompletion is enabled, every user or channel autocompletion associated with writing a message or user search will generate a query. + +How do I know if a search job fails? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Mattermost provides the status of each Elasticsearch or OpenSearch indexing job in **System Console > Environment > Elasticsearch**. Here you can see if the job succeeded or failed, including the details of the error. + +Failures are returned in the server logs. The error log begins with the string ``Failed job`` and includes a job_id key/value pair. Search job failures are identified with worker name ``EnterpriseElasticsearchAggregator`` and ``EnterpriseElasticsearchIndexer``. You can optionally create a script that programmatically queries for such failures and notifies the appropriate system. + +My search indexes won't complete, what should I do? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you have an search indexing job that's paused, it's likely your Elasticsearch or OpenSearch server has restarted. If you restart that server, you must also restart Mattermost to ensure jobs are completed. If restarting the Mattermost server does not resolve the issue, `commercial customers can contact Mattermost Support `__ for assistance. + +Do I need to purge first then bulk index each time? +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Yes. + +Required Permissions For Mattermost Service Account +--------------------------------------------------- +In "least privilege" environments you may need to further constrain the service account permissions to limit the access your Elasticsearch or AWS OpenSearch service account has. + +The following JSON provides an example of a "least privilege" permission set that allows Mattermost to operate correctly with Elasticsearch or AWS OpenSearch: + + .. code-block:: json + + { + "cluster_permissions": [ + "cluster:monitor/*", + "indices:admin/template/put", + "indices:data/write/bulk" + ], + "index_permissions": [ + { + "index_patterns": [ + "\*" + ], + "allowed_actions": [ + "indices:admin/get", + "indices:admin/create", + "indices:admin/delete", + "indices:admin/mapping/put", + "indices:admin/mappings/fields/get*", + "indices:data/read*", + "indices:data/write*" + ] + } + ] + } + +A simpler, more flexible, and resilient variant of the above would be: + +.. code-block:: json + + { + "cluster_permissions": [ + "cluster:monitor/*", + "indices:admin/template/put", + "indices:data/write/bulk" + ], + "index_permissions": [ + { + "index_patterns": [ + "\*" + ], + "allowed_actions": [ + "indices:*" + ] + } + ] + } diff --git a/source/scale/opensearch-setup.rst b/source/scale/opensearch-setup.rst new file mode 100644 index 00000000000..12a9d32cd94 --- /dev/null +++ b/source/scale/opensearch-setup.rst @@ -0,0 +1,61 @@ +AWS OpenSearch server setup +============================ + +.. include:: ../_static/badges/ent-cloud-selfhosted.rst + :start-after: :nosearch: + +AWS OpenSearch Service allows you to search large volumes of data quickly, in near real-time, by creating and managing an index of post data. The indexing process can be managed from the System Console after setting up and connecting an OpenSearch server. The post index is stored on the OpenSearch server and updated constantly after new posts are made. In order to index existing posts, a bulk index of the entire post database must be generated. + +Deploying AWS OpenSearch includes the following two steps: `setting up AWS OpenSearch <#set-up-aws-opensearch>`__, and `configuring Mattermost <#configure-mattermost>`_. + +Set up AWS OpenSearch +---------------------- + +From Mattermost v9.11, beta support is available for `AWS OpenSearch v1.x and v2.x `_. We highly recommend that you set up an AWS OpenSearch server on a separate machine from the Mattermost server. + +1. Download and install the latest release of AWS OpenSearch. See the `OpenSearch `_ documentation for installation details. + + + +Configure Mattermost +--------------------- + +Follow these steps to configure Mattermost to use AWS OpenSearch server and to generate the post index: + +1. Go to **System Console > Environment > Elasticsearch**. +2. Set **Enable Elasticsearch Indexing** to ``true`` to enable the other the settings on the page. Once the configuration is saved, new posts made to the database are automatically indexed on the OpenSearch server. +3. Change the Mattermost ``ElasticsearchSettings.Backend`` configuration setting value to ``opensearch``. +4. Set the AWS OpenSearch server connection details: + + a. Enter **Server Connection Address** for the AWS OpenSearch server you set up earlier. + b. (Optional) Enter **Server Username** used to access the OpenSearch server. For OpenSearch, leave this field blank. + c. (Optional) Enter **Server Password** associated with the username. For OpenSearch, leave this field blank. + d. Set **Enable Cluster Sniffing** (Optional). Sniffing finds and connects to all data nodes in your cluster automatically. For OpenSearch, this field should be set to ``false``. + +.. tip:: + + Optional CA and client certificate configuration settings are available for use with basic auth credentials or to replace them. See the :ref:`Elasticsearch configuration settings ` documentation for details. + +5. Select **Test Connection** and then select **Save**. If the server connection is unsuccessful you won't be able to save the configuration or enable searching with OpenSearch. + +6. Select **Build Index** to build the post index of existing posts. This process can take up to a few hours depending on the size of the post database and number of messages. The progress percentage can be seen as the index is created. To avoid downtime, set **Enable Elasticsearch for search queries** to ``false`` so that database search is available during the indexing process. + + .. important:: + + Complete bulk indexing before enabling OpenSearch in the next step. Otherwise, search results will be incomplete. + +7. Enable Elasticsearch by setting **Enable Elasticsearch for search queries** to ``true``, and setting **Enable Elasticsearch for autocomplete** to ``true``. + +8. Save your configuration updates and restart the Mattermost server. + +.. note:: + + - Additional advanced Elasticsearch settings for large deployments can be configured outside the System Console in the ``config.json`` file. Read the :ref:`Elasticsearch configuration settings ` documentation to learn more. + - If your deployment has a large number of posts (typically in excess of one million but not strictly defined), the reindexing progress percentage may stay at 99% for a long time. The size of the data to be indexed is estimated, and on large databases, estimations can become inaccurate. While progress estimates may be inaccurate, and the progress percentage may appear stuck at near completion, indexing will continue behind the scenes until complete. + - Search results for files shared before upgrading to Mattermost Server v5.35 may be incomplete until an extraction command is run using the :ref:`mmctl `. After running this command, the search index must be rebuilt. Go to **System Console > Environment > Elasticsearch > Bulk Indexing**, then select **Index Now** to rebuild the search index to include older file contents. + - For high post volume deployments, we strongly encourage you to read and properly configure the Mattermost :ref:`LiveIndexingBatchSize ` configuration setting. + +Limitations +------------ + +TBD \ No newline at end of file diff --git a/source/scale/scaling-for-enterprise.rst b/source/scale/scaling-for-enterprise.rst index 96050658012..acd0a1e9783 100644 --- a/source/scale/scaling-for-enterprise.rst +++ b/source/scale/scaling-for-enterprise.rst @@ -9,10 +9,12 @@ Mattermost is designed to scale from small teams hosted on a single server to la Server requirements vary based on usage and we highly recommend that you run a pilot before an enterprise-wide deployment in order to estimate full scale usage based on your specific organizational needs. -Elasticsearch --------------- +Enterprise search +------------------ -:doc:`Elasticsearch ` provides enterprise-scale deployments with optimized search performance, dedicated indexing and usage resourcing via cluster support without performance degradation and timeouts, resulting in faster, more predicable search results. +We highly recommend a dedicated server for large enterprise deployments to run highly efficient database searches in a cluster environment. + +For deployments with over 5 million posts, :doc:`Enterprise search ` using :ref:`Elasticsearch ` or :ref:`AWS OpenSearch Service ` is required for optimized search performance, dedicated indexing and usage resourcing via cluster support without performance degradation and timeouts, resulting in faster, more predicable search results. High availability ----------------- @@ -27,7 +29,7 @@ Available reference architectures :hidden: :titlesonly: - Elasticsearch + Enterprise search High availability Scale up to 200 users Scale up to 2000 users diff --git a/source/upgrade/admin-onboarding-tasks.rst b/source/upgrade/admin-onboarding-tasks.rst index b8c0beb8e81..8caeb6a13ab 100644 --- a/source/upgrade/admin-onboarding-tasks.rst +++ b/source/upgrade/admin-onboarding-tasks.rst @@ -143,6 +143,6 @@ Then, enable batched email notifications by setting **System Console > Notificat **9. Enable Elasticsearch** -Mattermost Enterprise customers can enable :doc:`Elasticsearch ` for optimized search performance at enterprise-scale. Elasticsearch solves many known issues with full text database search, such as dots, dashes, and email addresses returning unexpected results. +Mattermost Enterprise customers can enable :doc:`enterprise search ` for optimized search performance at enterprise-scale. Both Elasticsearch and AWS OpenSearch solve many known issues with full text database search, such as dots, dashes, and email addresses returning unexpected results. -Enable Elasticsearch by setting **System Console > Elasticsearch > Enable Indexing** to **true**. See the :ref:`Elasticsearch ` configuration settings documentation for details. Enabling Elasticsearch requires :ref:`setting up an Elasticsearch server `. +Enable Elasticsearch by setting **System Console > Elasticsearch > Enable Indexing** to **true**. See the :ref:`Elasticsearch ` configuration settings documentation for details. Enabling Elasticsearch requires :ref:`setting up an Elasticsearch server `. diff --git a/source/upgrade/important-upgrade-notes.rst b/source/upgrade/important-upgrade-notes.rst index 645d3a37b91..bb3a2ea6f2b 100644 --- a/source/upgrade/important-upgrade-notes.rst +++ b/source/upgrade/important-upgrade-notes.rst @@ -315,17 +315,17 @@ Important Upgrade Notes | +------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | | Deprecated the experimental Strict CSRF token enforcement. This feature will be fully removed in Mattermost v11. | +----------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+ -| v9.11 | Added support for Elasticsearch v8. Also added Beta support for Opensearch v1.x and v2.x. A new config setting ``ElasticsearchSettings.Backend`` has been | -| | added to differentiate between Elasticsearch and Opensearch. The default value is ``elasticsearch`` which breaks support for AWS Elasticsearch v7.10.x | +| v9.11 | Added support for Elasticsearch v8. Also added Beta support for OpenSearch v1.x and v2.x. A new config setting ``ElasticsearchSettings.Backend`` has been | +| | added to differentiate between Elasticsearch and OpenSearch. The default value is ``elasticsearch`` which breaks support for AWS Elasticsearch v7.10.x | | | since the official v8 client only works from Elasticsearch v7.11+ versions. | | | | | | .. note:: | | | | -| | - For AWS customers on Opensearch, you must modify Mattermost configuration from ``elasticsearch`` to ``opensearch`` and disable compatibility mode. | -| | See the `Opensearch documentation `_ for details on upgrading. | +| | - For AWS customers on OpenSearch, you must modify Mattermost configuration from ``elasticsearch`` to ``opensearch`` and disable compatibility mode. | +| | See the `OpenSearch documentation `_ for details on upgrading. | | | - After upgrading the Mattermost server, use :ref:`mmctl ` or edit the config manually, then restart the | | | Mattermost server. | -| | - If you are using Opensearch, you **must** set the backend to ``opensearch``. Otherwise Mattermost will not work. | +| | - If you are using OpenSearch, you **must** set the backend to ``opensearch``. Otherwise Mattermost will not work. | | | | | | If you are using Elasticsearch v8, be sure to set ``action.destructive_requires_name`` to ``false`` in ``elasticsearch.yml`` to allow for wildcard operations to | | | work. | @@ -915,7 +915,7 @@ Important Upgrade Notes | | :ref:`here `. | +----------------------------------------------------+------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | v6.2 | Channel results in the channel autocomplete will include private channels. Customers using :doc:`Bleve ` or | -| | :doc:`Elasticsearch ` for autocomplete will have to reindex their data to get the new results. Since this can | +| | :doc:`Elasticsearch ` for autocomplete will have to reindex their data to get the new results. Since this can | | | take a long time, we suggest disabling autocomplete and running indexing in the background. When this is complete, re-enable autocomplete. | | | | | | .. note:: | From 7b1f15da3d172cdb8604471f8014b329e165d8f1 Mon Sep 17 00:00:00 2001 From: "Carrie Warner (Mattermost)" <74422101+cwarnermm@users.noreply.github.com> Date: Tue, 29 Apr 2025 16:52:21 -0400 Subject: [PATCH 2/5] ES config settings apply equally to OS --- source/conf.py | 3 +- .../environment-configuration-settings.rst | 170 +++++++++--------- ...igure-mattermost-for-enterprise-search.rst | 43 +++++ source/scale/elasticsearch-setup.rst | 54 ++---- source/scale/enterprise-search.rst | 6 +- source/scale/opensearch-setup.rst | 41 +---- source/upgrade/admin-onboarding-tasks.rst | 2 +- .../upgrade/enterprise-roll-out-checklist.rst | 2 +- 8 files changed, 147 insertions(+), 174 deletions(-) create mode 100644 source/scale/common-configure-mattermost-for-enterprise-search.rst diff --git a/source/conf.py b/source/conf.py index 49da0867265..6aca9afb26b 100644 --- a/source/conf.py +++ b/source/conf.py @@ -1947,7 +1947,8 @@ def setup(_: Sphinx): "https://docs.mattermost.com/deploy/server/server-deployment-planning.html", "configure/using-outbound-proxy.html": "https://docs.mattermost.com/deploy/server/server-deployment-planning.html", - +"configure/environment-configuration-settings.html#elasticsearch": + "https://docs.mattermost.com/configure/environment-configuration-settings.html#enterprise-search", # Deploy redirects "deploy/mobile-apps-faq.html": diff --git a/source/configure/environment-configuration-settings.rst b/source/configure/environment-configuration-settings.rst index a801f400f70..2ecd7a2c029 100644 --- a/source/configure/environment-configuration-settings.rst +++ b/source/configure/environment-configuration-settings.rst @@ -8,7 +8,7 @@ Review and manage the following environmental configuration options in the Syste - `Web server <#web-server>`__ - `Database <#database>`__ -- `Elasticsearch <#elasticsearch>`__ +- `Enterprise search <#enterprise-search>`__ - `File storage <#file-storage>`__ - `Image proxy <#image-proxy>`__ - `SMTP <#smtp>`__ @@ -1212,15 +1212,17 @@ Replica monitor interval (seconds) ---- -Elasticsearch -------------- +Enterprise search +----------------- .. include:: ../_static/badges/ent-selfhosted.rst :start-after: :nosearch: -Elasticsearch provides enterprise-scale deployments with optimized search performance and prevents performance degradation and timeouts. Learn more about :doc:`Elasticsearch ` in our product documentation. +Core database search happens in a relational database and is intended for deployments under about 2–3 million posts and file entries. Beyond that scale, enabling enterprise search with Elasticsearch or AWS OpenSearch is highly recommended for optimum search performance before reaching 3 million posts. -You can configure the Elasticsearch environment in which Mattermost is deployed in **System Console > Environment > Elasticsearch**. You can also edit the ``config.json`` file as described in the following tables. Changes to configuration settings in this section require a server restart before taking effect. +For deployments with over 3 million posts, Elasticsearch or AWS OpenSearch is required to avoid significant performance issues, such as timeouts, with :doc:`message searches ` and :doc:`@mentions `. + +You can configure Mattermost enterprise search by going to **System Console > Environment > Elasticsearch**. The following configuration settings apply to both Elasticsearch and AWS OpenSearch. You can also edit the ``config.json`` file as described in the following tables. Changes to configuration settings in this section require a server restart before taking effect. .. config:setting:: enable-elasticsearch-indexing :displayname: Enable Elasticsearch indexing (Elasticsearch) @@ -1239,33 +1241,31 @@ Enable Elasticsearch indexing | Configure Mattermost to index new posts automatically. | - System Config path: **Environment > Elasticsearch** | | | - ``config.json`` setting: ``".Elasticsearchsettings.EnableIndexing: false",`` | | - **true**: Indexing of new messages occurs automatically. | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_ENABLEINDEXING`` | -| - **false**: **(Default)** Elasticsearch indexing is disabled | | -| and new messages aren't indexed. | | +| - **false**: **(Default)** Indexing of new messages is | | +| disabled, and new messages aren't indexed. | | +---------------------------------------------------------------+--------------------------------------------------------------------------------+ .. note:: - - Core search happens in a relational database and is intended for deployments under about 2–3 million posts and file entries. Beyond that scale, enabling `enabling Elasticsearch for search queries <#enable-elasticsearch-for-search-queries>`__ for search queries is highly recommended. - - If you anticipate your Mattermost server reaching more than 2.5 million posts and file entries, we recommend enabling Elasticsearch for optimum search performance before reaching 3 million posts. - - For deployments with over 3 million posts, Elasticsearch is required to avoid significant performance issues, such as timeouts, with :doc:`message searches ` and :doc:`@mentions `. - - If indexing is disabled and then re-enabled after an index is created, purge and rebuild the index to ensure complete search results. + If indexing is disabled and then re-enabled after an index is created, purge and rebuild the index to ensure complete search results. .. config:setting:: backend-type :displayname: Elasticsearch backend type (Elasticsearch) :systemconsole: Environment > Elasticsearch :configjson: .Elasticsearchsettings.Backend :environment: MM_ELASTICSEARCHSETTINGS_BACKEND - :description: Set the type of search backend as either Elasticsearch or OpenSearch. + :description: Set the type of search backend as either Elasticsearch or AWS OpenSearch. Backend type ~~~~~~~~~~~~~ +Both :doc:`Elasticsearch ` and :doc:`AWS OpenSearch ` provide enterprise-scale deployments with optimized search performance and prevents performance degradation and timeouts. Learn more about :doc:`enterprise search ` in our product documentation. + +----------------------------------------------------+-----------------------------------------------------------------------------------+ | The type of search backend. | - System Config path: **Environment > Elasticsearch** | | | - ``config.json`` setting: ``".Elasticsearchsettings.Backend: elasticsearch",`` | | - ``elasticsearch`` - (**Default**) | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_BACKEND`` | -| - ``opensearch`` - Required for AWS OpenSearch | | -| customers. | | +| - ``opensearch`` - Required for AWS OpenSearch. | | +----------------------------------------------------+-----------------------------------------------------------------------------------+ Learn more about :ref:`enterprise search version support `. @@ -1275,14 +1275,14 @@ Learn more about :ref:`enterprise search version support Elasticsearch :configjson: .Elasticsearchsettings.ConnectionUrl :environment: MM_ELASTICSEARCHSETTINGS_CONNECTIONURL - :description: The address of the Elasticsearch server. + :description: The address of the Elasticsearch or AWS OpenSearch server. Server connection address ~~~~~~~~~~~~~~~~~~~~~~~~~ +----------------------------------------------------+--------------------------------------------------------------------------+ -| The address of the Elasticsearch server. | - System Config path: **Environment > Elasticsearch** | -| | - ``config.json`` setting: ``".Elasticsearchsettings.ConnectionUrl",`` | +| The address of the Elasticsearch or AWS | - System Config path: **Environment > Elasticsearch** | +| OpenSearch server. | - ``config.json`` setting: ``".Elasticsearchsettings.ConnectionUrl",`` | | | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_CONNECTIONURL`` | +----------------------------------------------------+--------------------------------------------------------------------------+ @@ -1291,20 +1291,20 @@ Server connection address :systemconsole: Environment > Elasticsearch :configjson: .Elasticsearchsettings.CA :environment: MM_ELASTICSEARCHSETTINGS_CA - :description: Optional path to the Custom Certificate Authority certificates for the Elasticsearch server. + :description: Optional path to the Custom Certificate Authority certificates for the Elasticsearch or AWS OpenSearch server. CA path ~~~~~~~ +----------------------------------------------------+--------------------------------------------------------------------------+ | Optional path to the Custom Certificate Authority | - System Config path: **Environment > Elasticsearch** | -| certificates for the Elasticsearch server. | - ``config.json`` setting: ``".Elasticsearchsettings.CA",`` | -| | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_CA`` | +| certificates for the Elasticsearch or AWS | - ``config.json`` setting: ``".Elasticsearchsettings.CA",`` | +| OpenSearch server. | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_CA`` | +----------------------------------------------------+--------------------------------------------------------------------------+ .. note:: - - Available from Mattermost v7.8. The certificate path should be ``/opt/mattermost/data/elasticsearch/`` and configured in the System Console as ``./elasticsearch/cert.pem``. + - Available from Mattermost v7.8. The certificate path should be ``/opt/mattermost/data/elasticsearch/`` or ``/opt/mattermost/data/opensearch`` and configured in the System Console as ``./elasticsearch/cert.pem`` or ``./opensearch/cert.pem``. - Can be used in conjunction with basic authentication credentials or can replace them. Leave this setting blank to use the default Certificate Authority certificates for the operating system. .. config:setting:: client-certificate-path @@ -1312,7 +1312,7 @@ CA path :systemconsole: Environment > Elasticsearch :configjson: .Elasticsearchsettings.ClientCert :environment: MM_ELASTICSEARCHSETTINGS_CLIENTCERT - :description: Optional client certificate for the connection to the Elasticsearch server in PEM format. + :description: Optional client certificate for the connection to the Elasticsearch or AWS OpenSearch server in PEM format. Client certificate path ~~~~~~~~~~~~~~~~~~~~~~~ @@ -1321,8 +1321,8 @@ Available from Mattermost v7.8. Can be used in conjunction with basic auth crede +----------------------------------------------------+--------------------------------------------------------------------------+ | Optional client certificate for the connection to | - System Config path: **Environment > Elasticsearch** | -| the Elasticsearch server in the PEM format. | - ``config.json`` setting: ``".Elasticsearchsettings.ClientCert",`` | -| | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_CLIENTCERT`` | +| the Elasticsearch or AWS OpenSearch server in | - ``config.json`` setting: ``".Elasticsearchsettings.ClientCert",`` | +| the PEM format. | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_CLIENTCERT`` | +----------------------------------------------------+--------------------------------------------------------------------------+ .. config:setting:: client-certificate-key-path @@ -1361,7 +1361,7 @@ Skip TLS verification | | - ``config.json`` setting: ``".Elasticsearchsettings.SkipTLSVerification: false",`` | | - **true**: Skips the certificate verification step for | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_SKIPTLSVERIFICATION`` | | TLS connections. | | -| - **false**: **(Default)** Mattermost does not skip | | +| - **false**: **(Default)** Mattermost requires | | | certificate verification. | | +---------------------------------------------------------------+-------------------------------------------------------------------------------------+ @@ -1370,14 +1370,14 @@ Skip TLS verification :systemconsole: Environment > Elasticsearch :configjson: .Elasticsearchsettings.UserName :environment: MM_ELASTICSEARCHSETTINGS_USERNAME - :description: (Optional) The username to authenticate to the Elasticsearch server. + :description: (Optional) The username to authenticate to the Elasticsearch or AWS OpenSearch server. Server username ~~~~~~~~~~~~~~~ +---------------------------------------------------------------+--------------------------------------------------------------------------+ | (Optional) The username to authenticate to the | - System Config path: **Environment > Elasticsearch** | -| Elasticsearch server. | - ``config.json`` setting: ``".Elasticsearchsettings.UserName",`` | +| Elasticsearch or AWS OpenSearch server. | - ``config.json`` setting: ``".Elasticsearchsettings.UserName",`` | | | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_USERNAME`` | | String input. | | +---------------------------------------------------------------+--------------------------------------------------------------------------+ @@ -1387,14 +1387,14 @@ Server username :systemconsole: Environment > Elasticsearch :configjson: .Elasticsearchsettings.Password :environment: MM_ELASTICSEARCHSETTINGS_PASSWORD - :description: (Optional) The password to authenticate to the Elasticsearch server. + :description: (Optional) The password to authenticate to the Elasticsearch or AWS OpenSearch server. Server password ~~~~~~~~~~~~~~~ +---------------------------------------------------------------+--------------------------------------------------------------------------+ | (Optional) The password to authenticate to the | - System Config path: **Environment > Elasticsearch** | -| Elasticsearch server. | - ``config.json`` setting: ``".Elasticsearchsettings.Password",`` | +| Elasticsearch or AWS OpenSearch server. | - ``config.json`` setting: ``".Elasticsearchsettings.Password",`` | | | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_PASSWORD`` | | String input. | | +---------------------------------------------------------------+--------------------------------------------------------------------------+ @@ -1421,7 +1421,7 @@ Enable cluster sniffing | - **false**: **(Default)** Cluster sniffing is disabled. | | +----------------------------------------------------------------+--------------------------------------------------------------------------+ -Select the **Test Connection** button in the System Console to validate the connection between Mattermost and the Elasticsearch server. +Select the **Test Connection** button in the System Console to validate the connection between Mattermost and the Elasticsearch or AWS OpenSearch server. .. config:setting:: bulk-indexing :displayname: Bulk indexing (Elasticsearch) @@ -1443,7 +1443,7 @@ Bulk indexing - Always `purge indexes <#purge-indexes>`__ before bulk indexing. - Select the **Index Now** button in the System Console to start a bulk index of all posts, and review all index jobs in progress. - - Elasticsearch is available during indexing, but search results may be incomplete until the indexing job is complete. + - Elasticsearch or AWS OpenSearch is available during indexing, but search results may be incomplete until the indexing job is complete. - If an in-progress indexing job is canceled, the index and search results will be incomplete. .. config:setting:: rebuild-channels-index @@ -1469,7 +1469,7 @@ Select the **Rebuild Channels Index** button in the System Console to purge the :systemconsole: Environment > Elasticsearch :configjson: N/A :environment: N/A - :description: Purge the entire Elasticsearch index by selecting Purge Indexes before creating a new index. + :description: Purge the entire Elasticsearch or AWS OpenSearch index by selecting Purge Indexes before creating a new index. Purge indexes ~~~~~~~~~~~~~ @@ -1505,28 +1505,22 @@ Indexes to skip while purging :systemconsole: Environment > Elasticsearch :configjson: .Elasticsearchsettings.EnableSearching :environment: MM_ELASTICSEARCHSETTINGS_ENABLESEARCHING - :description: Configure Mattermost to use Elasticsearch for all search queries using the latest index. + :description: Configure Mattermost to use Elasticsearch or AWS OpenSearch for all search queries using the latest index. - - **true**: Elasticsearch is used for all search queries using the latest index. Search results may be incomplete until a bulk index of the existing message database is completed. + - **true**: Elasticsearch or AWS OpenSearch is used for all search queries using the latest index. Search results may be incomplete until a bulk index of the existing message database is completed. - **false**: **(Default)** Relational database search is used for search queries. Enable Elasticsearch for search queries ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -.. important:: - - - Core search happens in a relational database and is intended for deployments under about 2-3 million posts and file entries. Beyond that scale, enabling Elasticsearch for search queries is highly recommended. - - If you anticipate your Mattermost server reaching more than 2.5 million posts and file entries, we recommend enabling Elasticsearch for optimum search performance **before** reaching 3 million posts. - - For deployments with over 3 million posts, Elasticsearch with :ref:`dedicated indexing ` and scaled usage resourcing through :doc:`cluster support ` is required to avoid significant performance issues, such as timeouts, with :doc:`message searches ` and :doc:`@mentions `. - +---------------------------------------------------------------+---------------------------------------------------------------------------------+ -| Configure Mattermost to use Elasticsearch for all search | - System Config path: **Environment > Elasticsearch** | -| queries using the latest index. | - ``config.json`` setting: ``".Elasticsearchsettings.EnableSearching: false",`` | +| Configure Mattermost to use Elasticsearch or AWS OpenSearch | - System Config path: **Environment > Elasticsearch** | +| for all search queries using the latest index. | - ``config.json`` setting: ``".Elasticsearchsettings.EnableSearching: false",`` | | | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_ENABLESEARCHING`` | -| - **true**: Elasticsearch is used for all search queries | | -| using the latest index. Search results may be incomplete | | -| until a bulk index of the existing message database is | | -| completed. | | +| - **true**: Elasticsearch or AWS OpenSearch is used for all | | +| search queries using the latest index. Search results may | | +| be incomplete until a bulk index of the existing message | | +| database is completed. | | | - **false**: **(Default)** Database search is used for | | | search queries. | | +---------------------------------------------------------------+---------------------------------------------------------------------------------+ @@ -1547,12 +1541,13 @@ Enable Elasticsearch for autocomplete queries ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +---------------------------------------------------------------+------------------------------------------------------------------------------------+ -| Configure Mattermost to use Elasticsearch for all | - System Config path: **Environment > Elasticsearch** | -| autocompletion queries on users and channels using the | - ``config.json`` setting: ``".Elasticsearchsettings.EnableAutocomplete: false",`` | -| latest index. | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_ENABLEAUTOCOMPLETE`` | +| Configure Mattermost to use Elasticsearch or AWS OpenSearch | - System Config path: **Environment > Elasticsearch** | +| for all autocompletion queries on users and channels using | - ``config.json`` setting: ``".Elasticsearchsettings.EnableAutocomplete: false",`` | +| the latest index. | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_ENABLEAUTOCOMPLETE`` | | | | -| - **true**: Elasticsearch will be used for all autocompletion | | -| queries on users and channels using the latest index. | | +| - **true**: Elasticsearch or AWS OpenSearch will be used for | | +| all autocompletion queries on users and channels using the | | +| latest index. | | | - **false**: **(Default)** Database autocomplete is used. | | +---------------------------------------------------------------+------------------------------------------------------------------------------------+ @@ -1578,7 +1573,7 @@ Post index replicas - If this setting is changed, the changed configuration only applies to newly-created indexes. To apply the change to existing indexes, purge and rebuild the index after changing this setting. - If there are ``n`` data nodes, the number of replicas per shard for each index should be ``n-1``. - - If the number of nodes in an Elasticsearch cluster changes, this configuration setting, as well as `Channel Index Replicas <#channel-index-replicas>`__ and `User Index Replicas <#user-index-replicas>`__ must also be updated accordingly. + - If the number of nodes in an Elasticsearch or AWS OpenSearch cluster changes, this configuration setting, as well as `Channel Index Replicas <#channel-index-replicas>`__ and `User Index Replicas <#user-index-replicas>`__ must also be updated accordingly. .. config:setting:: post-index-shards :displayname: Post index shards (Elasticsearch) @@ -1618,7 +1613,7 @@ Channel index replicas .. note:: - If there are ``n`` data nodes, the number of replicas per shard for each index should be ``n-1``. If the number of nodes in an Elasticsearch cluster changes, this configuration setting, as well as `Post Index Replicas <#post-index-shards>`__ and `User Index Replicas <#user-index-replicas>`__ must also be updated accordingly. + If there are ``n`` data nodes, the number of replicas per shard for each index should be ``n-1``. If the number of nodes in an Elasticsearch or AWS OpenSearch cluster changes, this configuration setting, as well as `Post Index Replicas <#post-index-shards>`__ and `User Index Replicas <#user-index-replicas>`__ must also be updated accordingly. .. config:setting:: channel-index-shards :displayname: Channel index shards (Elasticsearch) @@ -1654,7 +1649,7 @@ User index replicas .. note:: - If there are ``n`` data nodes, the number of replicas per shard for each index should be ``n-1``. If the number of nodes in an Elasticsearch cluster changes, this configuration setting, as well as `Post Index Replicas <#post-index-shards>`__ and `User Index Replicas <#user-index-replicas>`__ must also be updated accordingly. + If there are ``n`` data nodes, the number of replicas per shard for each index should be ``n-1``. If the number of nodes in an Elasticsearch or AWS OpenSearch cluster changes, this configuration setting, as well as `Post Index Replicas <#post-index-shards>`__ and `User Index Replicas <#user-index-replicas>`__ must also be updated accordingly. .. config:setting:: user-index-shards :displayname: User index shards (Elasticsearch) @@ -1677,22 +1672,22 @@ User index shards :systemconsole: N/A :configjson: .Elasticsearchsettings.AggregatePostsAfterDays :environment: MM_ELASTICSEARCHSETTINGS_AGGREGATEPOSTSAFTERDAYS - :description: Elasticsearch indexes older than the age specified by this setting, in days, will be aggregated during the daily scheduled job. Default is **365** days. + :description: Elasticsearch or AWS OpenSearch indexes older than the age specified by this setting, in days, will be aggregated during the daily scheduled job. Default is **365** days. Aggregate search indexes ~~~~~~~~~~~~~~~~~~~~~~~~ +---------------------------------------------------------------+----------------------------------------------------------------------------------------+ -| Elasticsearch indexes older than the age specified by this | - System Config path: N/A | -| setting, in days, will be aggregated during the daily | - ``config.json`` setting: ``".Elasticsearchsettings.AggregatePostsAfterDays: 365",`` | -| scheduled job. | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_AGGREGATEPOSTSAFTERDAYS`` | +| Elasticsearch or AWS OpenSearch indexes older than the age | - System Config path: N/A | +| specified by this setting, in days, will be aggregated during | - ``config.json`` setting: ``".Elasticsearchsettings.AggregatePostsAfterDays: 365",`` | +| the daily scheduled job. | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_AGGREGATEPOSTSAFTERDAYS`` | | | | | Numerical input. Default is **365** days. | | +---------------------------------------------------------------+----------------------------------------------------------------------------------------+ .. note:: - If you’re using :doc:`data retention ` and :doc:`Elasticsearch `, configure this with a value greater than your data retention policy. + If you’re using :doc:`data retention ` and :doc:`enterprise search `, configure this with a value greater than your data retention policy. .. config:setting:: post-aggregator-start-time :displayname: Post aggregator start time (Elasticsearch) @@ -1718,35 +1713,35 @@ Post aggregator start time :systemconsole: N/A :configjson: .Elasticsearchsettings.IndexPrefix :environment: MM_ELASTICSEARCHSETTINGS_INDEXPREFIX - :description: The prefix added to the Elasticsearch index name. + :description: The prefix added to the Elasticsearch or AWS OpenSearch index name. Index prefix ~~~~~~~~~~~~ +---------------------------------------------------------------+--------------------------------------------------------------------------+ -| The prefix added to the Elasticsearch index name. | - System Config path: N/A | -| | - ``config.json`` setting: ``".Elasticsearchsettings.IndexPrefix",`` | +| The prefix added to the Elasticsearch or AWS OpenSearch | - System Config path: N/A | +| index name. | - ``config.json`` setting: ``".Elasticsearchsettings.IndexPrefix",`` | | | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_INDEXPREFIX`` | +---------------------------------------------------------------+--------------------------------------------------------------------------+ .. note:: - When this setting is used, all Elasticsearch indexes created by Mattermost are given this prefix. You can set different prefixes so that multiple Mattermost deployments can share an Elasticsearch cluster without the index names colliding. + When this setting is used, all Elasticsearch or AWS OpenSearch indexes created by Mattermost are given this prefix. You can set different prefixes so that multiple Mattermost deployments can share an Elasticsearch or AWS OpenSearch cluster without the index names colliding. .. config:setting:: live-indexing-batch-size :displayname: Live indexing batch size (Elasticsearch) :systemconsole: N/A :configjson: .Elasticsearchsettings.LiveIndexingBatchSize :environment: MM_ELASTICSEARCHSETTINGS_LIVEINDEXINGBATCHSIZE - :description: The number of new posts batched together before they're added to the Elasticsearch index. Default is **1**. + :description: The number of new posts batched together before they're added to the Elasticsearch or AWS OpenSearch index. Default is **1**. Live indexing batch size ~~~~~~~~~~~~~~~~~~~~~~~~ +---------------------------------------------------------------+-----------------------------------------------------------------------------------+ | The number of new posts needed before those posts are added | - System Config path: N/A | -| to the Elasticsearch index. Once added to the Index, | - ``config.json`` setting: ``".Elasticsearchsettings.LiveIndexingBatchSize: 1",`` | -| the post becomes searchable. | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_LIVEINDEXINGBATCHSIZE`` | +| to the Elasticsearch or AWS OpenSearch index. Once added to | - ``config.json`` setting: ``".Elasticsearchsettings.LiveIndexingBatchSize: 1",`` | +| the index, the post becomes searchable. | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_LIVEINDEXINGBATCHSIZE`` | | | | | On servers with more than 1 post per second, we suggest | | | setting this value to the average number of posts over a | | @@ -1758,11 +1753,11 @@ Live indexing batch size .. note:: - It may be necessary to increase this value to avoid hitting the rate limit or resource limit of your Elasticsearch cluster on installs handling more than 1 post per second. + It may be necessary to increase this value to avoid hitting the rate limit or resource limit of your Elasticsearch or AWS OpenSearch cluster on installs handling more than 1 post per second. **What exactly happens when I increase this value?** -The primary impact is that a post will be indexed into Elasticsearch after the threshold of posts is met, which then makes the posts searchable within Mattermost. So, if you set this based on recommendations for larger servers, and you make a post, you cannot find it via search for ~10–20 seconds, on average. Realistically, no users should see or feel this impact due to the limited number of users who are actively **searching** for a post this quickly. You can set this value to a lower or higher average depending on your Elasticsearch server specifications. +The primary impact is that a post will be indexed into Elasticsearch or AWS OpenSearch after the threshold of posts is met, which then makes the posts searchable within Mattermost. So, if you set this based on recommendations for larger servers, and you make a post, you cannot find it via search for ~10–20 seconds, on average. Realistically, no users should see or feel this impact due to the limited number of users who are actively **searching** for a post this quickly. You can set this value to a lower or higher average depending on your Elasticsearch or AWS OpenSearch server specifications. During busy periods, this delay will be faster as more traffic is occurring, causing more posts and a quicker time to hit the index number. During slower periods, expect the reverse. @@ -1786,7 +1781,7 @@ During busy periods, this delay will be faster as more traffic is occurring, cau GROUP BY date_trunc('minute', to_timestamp(createat/1000)) ) as ppm; -2. Decide the acceptable index window for your environment, and divide your average posts per minute by that. We suggest 10-20 seconds. Assuming you have ``600`` posts per minute on average, and you want to index every 20 seconds (``60 seconds / 20 seconds = 3```) you would calculate ``600 / 3`` to come to the number ``200``. After 200 posts, Mattermost will index the posts into Elasticsearch. So, on average, there would be a 20-second delay in searchability. +2. Decide the acceptable index window for your environment, and divide your average posts per minute by that. We suggest 10-20 seconds. Assuming you have ``600`` posts per minute on average, and you want to index every 20 seconds (``60 seconds / 20 seconds = 3```) you would calculate ``600 / 3`` to come to the number ``200``. After 200 posts, Mattermost will index the posts into Elasticsearch or AWS OpenSearch. So, on average, there would be a 20-second delay in searchability. 3. Edit the ``config.json`` or run mmctl to modify the ``LiveIndexingBatchSize`` setting @@ -1836,15 +1831,16 @@ Batch size :systemconsole: N/A :configjson: .Elasticsearchsettings.RequestTimeoutSeconds :environment: MM_ELASTICSEARCHSETTINGS_REQUESTTIMEOUTSECONDS - :description: The timeout, in seconds, for Elasticsearch calls. Default is **30** seconds. + :description: The timeout, in seconds, for Elasticsearch or AWS OpenSearch calls. Default is **30** seconds. Request timeout ~~~~~~~~~~~~~~~ +---------------------------------------------------------------+------------------------------------------------------------------------------------+ -| The timeout, in seconds, for Elasticsearch calls. | - System Config path: N/A | -| | - ``config.json`` setting: ``".Elasticsearchsettings.RequestTimeoutSeconds :30",`` | -| Numerical input in seconds. Default is **30** seconds. | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_REQUESTTIMEOUTSECONDS`` | +| The timeout, in seconds, for Elasticsearch or AWS OpenSearch | - System Config path: N/A | +| calls. | - ``config.json`` setting: ``".Elasticsearchsettings.RequestTimeoutSeconds :30",`` | +| | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_REQUESTTIMEOUTSECONDS`` | +| Numerical input in seconds. Default is **30** seconds. | | +---------------------------------------------------------------+------------------------------------------------------------------------------------+ .. config:setting:: trace @@ -1852,27 +1848,27 @@ Request timeout :systemconsole: N/A :configjson: .Elasticsearchsettings.Trace :environment: MM_ELASTICSEARCHSETTINGS_TRACE - :description: Options for printing Elasticsearch trace errors. + :description: Options for printing Elasticsearch or AWS OpenSearch trace errors. - - **error**: Creates the error trace when initializing the Elasticsearch client and prints any template creation or search query that returns an error as part of the error message. + - **error**: Creates the error trace when initializing the Elasticsearch or AWS OpenSearch client and prints any template creation or search query that returns an error as part of the error message. - **all**: Creates the three traces (error, trace and info) for the driver and doesn’t print the queries because they will be part of the trace log level of the driver. - **not specified**: **(Default)** No error trace is created. Trace ~~~~~ -+---------------------------------------------------------------+--------------------------------------------------------------------------+ -| Options for printing Elasticsearch trace errors. | - System Config path: N/A | -| | - ``config.json`` setting: ``".Elasticsearchsettings.Trace",`` | -| - **error**: Creates the error trace when initializing | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_TRACE`` | -| the Elasticsearch client and prints any template creation | | -| or search query that returns an error as part of the | | -| error message. | | -| - **all**: Creates the three traces (error, trace and info) | | -| for the driver and doesn’t print the queries because they | | -| will be part of the trace log level of the driver. | | -| - **not specified**: **(Default)** No error trace is created. | | -+---------------------------------------------------------------+--------------------------------------------------------------------------+ ++---------------------------------------------------------------------+--------------------------------------------------------------------------+ +| Options for printing Elasticsearch or AWS OpenSearch trace errors. | - System Config path: N/A | +| | - ``config.json`` setting: ``".Elasticsearchsettings.Trace",`` | +| - **error**: Creates the error trace when initializing | - Environment variable: ``MM_ELASTICSEARCHSETTINGS_TRACE`` | +| the Elasticsearch or AWS OpenSearch client and prints any | | +| template creation or search query that returns an error as part | | +| of the error message. | | +| - **all**: Creates the three traces (error, trace and info) | | +| for the driver and doesn’t print the queries because they | | +| will be part of the trace log level of the driver. | | +| - **not specified**: **(Default)** No error trace is created. | | ++---------------------------------------------------------------------+--------------------------------------------------------------------------+ ---- diff --git a/source/scale/common-configure-mattermost-for-enterprise-search.rst b/source/scale/common-configure-mattermost-for-enterprise-search.rst new file mode 100644 index 00000000000..885d6374088 --- /dev/null +++ b/source/scale/common-configure-mattermost-for-enterprise-search.rst @@ -0,0 +1,43 @@ +.. meta:: + :name: robots + :content: noindex + +:orphan: +:nosearch: + +.. This page intentionally not accessible via the LHS navigation pane because it's included in other pages + + +4. Set the server connection details for Elasticsearch or AWS OpenSearch: + + a. Enter **Server Connection Address** for the enterprise search server you set up earlier. + b. (Optional) Enter **Server Username** used to access the enterprise search server. For AWS Elasticsearch, leave this field blank. + c. (Optional) Enter **Server Password** associated with the username. For AWS Elasticsearch, leave this field blank. + d. Set **Enable Cluster Sniffing** (Optional). Sniffing finds and connects to all data nodes in your cluster automatically. For AWS Elasticsearch, this field should be set to ``false``. + +.. tip:: + + From Mattermost v7.8, optional CA and client certificate configuration settings are available for use with basic auth credentials or to replace them. See the :ref:`Enterprise search configuration settings ` documentation for details. + +4. Select **Test Connection** and then select **Save**. If the server connection is unsuccessful you won't be able to save the configuration or enable searching with Elasticsearch or AWS OpenSearch. + +5. Select **Build Index** to build the post index of existing posts. This process can take up to a few hours depending on the size of the post database and number of messages. The progress percentage can be seen as the index is created. To avoid downtime, set **Enable Elasticsearch for search queries** to ``false`` so that database search is available during the indexing process. + + .. important:: + + Complete bulk indexing before enabling Elasticsearch or AWS OpenSearch in the next step. Otherwise, search results will be incomplete. + +6. Enable enterprise search by setting **Enable Elasticsearch for search queries** to ``true``, and setting **Enable Elasticsearch for autocomplete** to ``true``. + +7. Save your configuration updates and restart the Mattermost server. + +.. note:: + + For high post volume deployments, we strongly encourage you to read and properly configure the Mattermost :ref:`LiveIndexingBatchSize ` configuration setting. + +Limitations +------------ + +1. Elasticsearch and AWS OpenSearch uses a standard selection of "stop words" to keep search results relevant. Results for the following words will not be returned: "a", "an", "and", "are", "as", "at", "be", "but", "by", "for", "if", "in", "into", "is", "it", "no", "not", "of", "on", "or", "such", "that", "the", "their", "then", "there", "these", "they", "this", "to", "was", "will", and "with". +2. Searching stop words in quotes returns more results than just the searched terms (`ticket `__). +3. Search results are limited to a user's team and channel membership. This is enforced by the Mattermost server. The entities are indexed in Elasticsearch or AWS OpenSearch in a way that allows Mattermost to filter them when querying, so the Mattermost server narrows down the results on every Elasticsearch or AWS OpenSearch request applying those filters. \ No newline at end of file diff --git a/source/scale/elasticsearch-setup.rst b/source/scale/elasticsearch-setup.rst index ac69d2e50d1..1043e27afb1 100644 --- a/source/scale/elasticsearch-setup.rst +++ b/source/scale/elasticsearch-setup.rst @@ -43,14 +43,16 @@ We highly recommend that you set up Elasticsearch server on a dedicated machine 6. In this file, replace the ``network.host`` value of ``_eth0_`` with your network interface name, and save your changes. -7. Restart Elasticsearch by running the following commands: +7. When using Elasticsearch v8, ensure you set ``action.destructive_requires_name`` to ``false`` in ``elasticsearch.yml`` to allow for wildcard operations to work. + +8. Restart Elasticsearch by running the following commands: .. code-block:: sh sudo systemctl stop elasticsearch sudo systemctl start elasticsearch -8. Confirm the ports are listenings by running the following command: +9. Confirm the ports are listenings by running the following command: .. code-block:: sh @@ -58,15 +60,15 @@ We highly recommend that you set up Elasticsearch server on a dedicated machine You should see the following ports, including the ones listening on ports 9200 and 9300. Confirm these are listening on your server's IP address. -9. Create an Elasticsearch directory and give it the proper permissions. +10. Create an Elasticsearch directory and give it the proper permissions. -10. Install the `icu-analyzer plugin `__ to the ``/usr/share/elasticsearch/plugins`` directory by running the following command: +11. Install the `icu-analyzer plugin `__ to the ``/usr/share/elasticsearch/plugins`` directory by running the following command: .. code-block:: sh sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install analysis-icu -11. Test the connection from Mattermost to Elasticsearch by running the following command: +12. Test the connection from Mattermost to Elasticsearch by running the following command: .. code-block:: sh @@ -77,45 +79,9 @@ Configure Mattermost Follow these steps to configure Mattermost to use your Elasticsearch server and to generate the post index: -.. tip:: - - Advanced Elasticsearch configuration settings for large deployments can be configured outside the System Console in the ``config.json`` file. See the :ref:`Elasticsearch configuration settings ` documentation to learn more. - 1. Go to **System Console > Environment > Elasticsearch**. 2. Set **Enable Elasticsearch Indexing** to ``true`` to enable the other the settings on the page. Once the configuration is saved, new posts made to the database are automatically indexed on the Elasticsearch server. -3. Set the Elasticsearch server connection details: - - a. Enter **Server Connection Address** for the Elasticsearch server you set up earlier. - b. (Optional) Enter **Server Username** used to access the Elasticsearch server. For AWS Elasticsearch, leave this field blank. - c. (Optional) Enter **Server Password** associated with the username. For AWS Elasticsearch, leave this field blank. - d. Set **Enable Cluster Sniffing** (Optional). Sniffing finds and connects to all data nodes in your cluster automatically. For AWS Elasticsearch, this field should be set to ``false``. - -.. tip:: - - From Mattermost v7.8, optional CA and client certificate configuration settings are available for use with basic auth credentials or to replace them. See the :ref:`Elasticsearch configuration settings ` documentation for details. - -4. Select **Test Connection** and then select **Save**. If the server connection is unsuccessful you won't be able to save the configuration or enable searching with Elasticsearch. - -5. Select **Build Index** to build the post index of existing posts. This process can take up to a few hours depending on the size of the post database and number of messages. The progress percentage can be seen as the index is created. To avoid downtime, set **Enable Elasticsearch for search queries** to ``false`` so that database search is available during the indexing process. - - .. important:: - - Complete bulk indexing before enabling Elasticsearch in the next step. Otherwise, search results will be incomplete. - -6. Enable Elasticsearch by setting **Enable Elasticsearch for search queries** to ``true``, and setting **Enable Elasticsearch for autocomplete** to ``true``. - -7. Save your configuration updates and restart the Mattermost server. - -.. note:: - - - Additional advanced Elasticsearch settings for large deployments can be configured outside the System Console in the ``config.json`` file. Read the :ref:`Elasticsearch configuration settings ` documentation to learn more. - - If your deployment has a large number of posts (typically in excess of one million but not strictly defined), the reindexing progress percentage may stay at 99% for a long time. The size of the data to be indexed is estimated, and on large databases, estimations can become inaccurate. While progress estimates may be inaccurate, and the progress percentage may appear stuck at near completion, indexing will continue behind the scenes until complete. - - Search results for files shared before upgrading to Mattermost Server v5.35 may be incomplete until an extraction command is run using the :ref:`mmctl `. After running this command, the search index must be rebuilt. Go to **System Console > Environment > Elasticsearch > Bulk Indexing**, then select **Index Now** to rebuild the search index to include older file contents. - - For high post volume deployments, we strongly encourage you to read and properly configure the Mattermost :ref:`LiveIndexingBatchSize ` configuration setting. - -Limitations ------------- +3. Ensure **Backend type** is set to ``elasticsearch``. -1. Elasticsearch uses a standard selection of "stop words" to keep search results relevant. Results for the following words will not be returned: "a", "an", "and", "are", "as", "at", "be", "but", "by", "for", "if", "in", "into", "is", "it", "no", "not", "of", "on", "or", "such", "that", "the", "their", "then", "there", "these", "they", "this", "to", "was", "will", and "with". -2. Searching stop words in quotes returns more results than just the searched terms (`ticket `__). -3. Search results are limited to a user's team and channel membership. This is enforced by the Mattermost server. The entities are indexed in Elasticsearch in a way that allows Mattermost to filter them when querying, so the Mattermost server narrows down the results on every Elasticsearch request applying those filters. \ No newline at end of file +.. include:: /scale/common-configure-mattermost-for-enterprise-search.rst + :start-after: :nosearch: \ No newline at end of file diff --git a/source/scale/enterprise-search.rst b/source/scale/enterprise-search.rst index e3aeed6dc09..c51cebb4792 100644 --- a/source/scale/enterprise-search.rst +++ b/source/scale/enterprise-search.rst @@ -6,9 +6,9 @@ Enterprise search Mattermost database search starts to show performance degradation at around 2 million posts, on a server with 32 GB RAM and 4 CPUs. If you anticipate your Mattermost server reaching more than 2.5 million posts, we recommend enabling `Elasticsearch <#elasticsearch>`__ or `AWS OpenSearch Service <#aws-opensearch-service>`__ for optimum search performance **before** reaching 3 million posts. Both tools are highly capable and can handle enterprise-scale workloads. The choice between them depends on the following factors: -- **Licensing and cost**: OpenSearch may be preferable for organizations avoiding proprietary licensing or opting for cost-effective solutions. +- **Licensing and cost**: AWS OpenSearch may be preferable for organizations avoiding proprietary licensing or opting for cost-effective solutions. - **Feature requirements**: Elasticsearch's proprietary features (e.g., advanced analytics, security suites) may be preferred by organizations needing powerful out-of-the-box functionality. -- **Infrastructure alignment**: OpenSearch aligns well with AWS-centric environments, while Elasticsearch offers broader integrations. +- **Infrastructure alignment**: AWS OpenSearch aligns well with AWS-centric environments, while Elasticsearch offers broader integrations. .. toctree:: :maxdepth: 1 @@ -41,8 +41,6 @@ Review the following support paths for enterprise search based on the version yo `Elasticsearch v8 `__ is supported from Mattermost v9.11. While Mattermost supports Elasticsearch v7.17+, we recommend upgrading your Elasticsearch v7 instance to v8.x. See the `Elasticsearch upgrade `_ documentation for details. - When using Elasticsearch v8, ensure you set ``action.destructive_requires_name`` to ``false`` in ``elasticsearch.yml`` to allow for wildcard operations to work. - .. tab:: AWS OpenSearch Service AWS OpenSearch Service is the official path forward from Elasticsearch v7.10.x for AWS customers to provide a simple and cost-effective way to search, analyze, and visual data in real time. It's essentially a continuation of Elasticsearch v7.10.x but maintained as open source by AWS. It provides long-term support, active development, and compatibility with AWS clients, libraries, and managed services. diff --git a/source/scale/opensearch-setup.rst b/source/scale/opensearch-setup.rst index 12a9d32cd94..e2ff9c3104c 100644 --- a/source/scale/opensearch-setup.rst +++ b/source/scale/opensearch-setup.rst @@ -20,42 +20,11 @@ From Mattermost v9.11, beta support is available for `AWS OpenSearch v1.x and v2 Configure Mattermost --------------------- -Follow these steps to configure Mattermost to use AWS OpenSearch server and to generate the post index: +Follow these steps to configure Mattermost to use your AWS OpenSearch server and to generate the post index: 1. Go to **System Console > Environment > Elasticsearch**. -2. Set **Enable Elasticsearch Indexing** to ``true`` to enable the other the settings on the page. Once the configuration is saved, new posts made to the database are automatically indexed on the OpenSearch server. -3. Change the Mattermost ``ElasticsearchSettings.Backend`` configuration setting value to ``opensearch``. -4. Set the AWS OpenSearch server connection details: +2. Set **Enable Elasticsearch Indexing** to ``true`` to enable the other the settings on the page. Once the configuration is saved, new posts made to the database are automatically indexed on the AWS OpenSearch server. +3. Ensure **Backend type** is set to ``opensearch``. - a. Enter **Server Connection Address** for the AWS OpenSearch server you set up earlier. - b. (Optional) Enter **Server Username** used to access the OpenSearch server. For OpenSearch, leave this field blank. - c. (Optional) Enter **Server Password** associated with the username. For OpenSearch, leave this field blank. - d. Set **Enable Cluster Sniffing** (Optional). Sniffing finds and connects to all data nodes in your cluster automatically. For OpenSearch, this field should be set to ``false``. - -.. tip:: - - Optional CA and client certificate configuration settings are available for use with basic auth credentials or to replace them. See the :ref:`Elasticsearch configuration settings ` documentation for details. - -5. Select **Test Connection** and then select **Save**. If the server connection is unsuccessful you won't be able to save the configuration or enable searching with OpenSearch. - -6. Select **Build Index** to build the post index of existing posts. This process can take up to a few hours depending on the size of the post database and number of messages. The progress percentage can be seen as the index is created. To avoid downtime, set **Enable Elasticsearch for search queries** to ``false`` so that database search is available during the indexing process. - - .. important:: - - Complete bulk indexing before enabling OpenSearch in the next step. Otherwise, search results will be incomplete. - -7. Enable Elasticsearch by setting **Enable Elasticsearch for search queries** to ``true``, and setting **Enable Elasticsearch for autocomplete** to ``true``. - -8. Save your configuration updates and restart the Mattermost server. - -.. note:: - - - Additional advanced Elasticsearch settings for large deployments can be configured outside the System Console in the ``config.json`` file. Read the :ref:`Elasticsearch configuration settings ` documentation to learn more. - - If your deployment has a large number of posts (typically in excess of one million but not strictly defined), the reindexing progress percentage may stay at 99% for a long time. The size of the data to be indexed is estimated, and on large databases, estimations can become inaccurate. While progress estimates may be inaccurate, and the progress percentage may appear stuck at near completion, indexing will continue behind the scenes until complete. - - Search results for files shared before upgrading to Mattermost Server v5.35 may be incomplete until an extraction command is run using the :ref:`mmctl `. After running this command, the search index must be rebuilt. Go to **System Console > Environment > Elasticsearch > Bulk Indexing**, then select **Index Now** to rebuild the search index to include older file contents. - - For high post volume deployments, we strongly encourage you to read and properly configure the Mattermost :ref:`LiveIndexingBatchSize ` configuration setting. - -Limitations ------------- - -TBD \ No newline at end of file +.. include:: /scale/common-configure-mattermost-for-enterprise-search.rst + :start-after: :nosearch: \ No newline at end of file diff --git a/source/upgrade/admin-onboarding-tasks.rst b/source/upgrade/admin-onboarding-tasks.rst index 8caeb6a13ab..45854de7d93 100644 --- a/source/upgrade/admin-onboarding-tasks.rst +++ b/source/upgrade/admin-onboarding-tasks.rst @@ -145,4 +145,4 @@ Then, enable batched email notifications by setting **System Console > Notificat Mattermost Enterprise customers can enable :doc:`enterprise search ` for optimized search performance at enterprise-scale. Both Elasticsearch and AWS OpenSearch solve many known issues with full text database search, such as dots, dashes, and email addresses returning unexpected results. -Enable Elasticsearch by setting **System Console > Elasticsearch > Enable Indexing** to **true**. See the :ref:`Elasticsearch ` configuration settings documentation for details. Enabling Elasticsearch requires :ref:`setting up an Elasticsearch server `. +Enable Elasticsearch by setting **System Console > Elasticsearch > Enable Indexing** to **true**. See the :ref:`Elasticsearch ` configuration settings documentation for details. Enabling Elasticsearch requires :ref:`setting up an Elasticsearch server `. diff --git a/source/upgrade/enterprise-roll-out-checklist.rst b/source/upgrade/enterprise-roll-out-checklist.rst index 552ee6d4246..e832693e55c 100644 --- a/source/upgrade/enterprise-roll-out-checklist.rst +++ b/source/upgrade/enterprise-roll-out-checklist.rst @@ -180,7 +180,7 @@ Much of the preparation work is focused on ensuring the environment is deployed - Upload your valid Enterprise License under Edition and License - Ensure site URL is set appropriately for your production, dev and staging environments - Add your database configuration to **System Console > Environment > Database** - - Add your Elasticsearch configuration to **System Console > Environment > Elasticsearch** + - Add your Elasticsearch or AWS OpenSearch configuration to **System Console > Environment > Elasticsearch** - Add your file storage system configuration to **System Console > Environment > File Storage** - Add your proxy configuration to **System Console > Environment > Image Proxy** - Add your SMTP configuration to **System Console > Environment > SMTP** From f43b9ca31d332ae69ef92a38bd23507787288a4f Mon Sep 17 00:00:00 2001 From: "Carrie Warner (Mattermost)" <74422101+cwarnermm@users.noreply.github.com> Date: Tue, 29 Apr 2025 16:53:14 -0400 Subject: [PATCH 3/5] build fixes --- source/configure/environment-configuration-settings.rst | 2 +- source/upgrade/admin-onboarding-tasks.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/source/configure/environment-configuration-settings.rst b/source/configure/environment-configuration-settings.rst index 2ecd7a2c029..090dc2c2229 100644 --- a/source/configure/environment-configuration-settings.rst +++ b/source/configure/environment-configuration-settings.rst @@ -1520,7 +1520,7 @@ Enable Elasticsearch for search queries | - **true**: Elasticsearch or AWS OpenSearch is used for all | | | search queries using the latest index. Search results may | | | be incomplete until a bulk index of the existing message | | -| database is completed. | | +| database is completed. | | | - **false**: **(Default)** Database search is used for | | | search queries. | | +---------------------------------------------------------------+---------------------------------------------------------------------------------+ diff --git a/source/upgrade/admin-onboarding-tasks.rst b/source/upgrade/admin-onboarding-tasks.rst index 45854de7d93..eccc71ac2eb 100644 --- a/source/upgrade/admin-onboarding-tasks.rst +++ b/source/upgrade/admin-onboarding-tasks.rst @@ -145,4 +145,4 @@ Then, enable batched email notifications by setting **System Console > Notificat Mattermost Enterprise customers can enable :doc:`enterprise search ` for optimized search performance at enterprise-scale. Both Elasticsearch and AWS OpenSearch solve many known issues with full text database search, such as dots, dashes, and email addresses returning unexpected results. -Enable Elasticsearch by setting **System Console > Elasticsearch > Enable Indexing** to **true**. See the :ref:`Elasticsearch ` configuration settings documentation for details. Enabling Elasticsearch requires :ref:`setting up an Elasticsearch server `. +Enable Elasticsearch by setting **System Console > Elasticsearch > Enable Indexing** to **true**. See the :ref:`Elasticsearch ` configuration settings documentation for details. Enabling Elasticsearch requires :ref:`setting up an Elasticsearch server `. From a99b2846b8ba5dd34e3448b54f2f205de12a7676 Mon Sep 17 00:00:00 2001 From: "Carrie Warner (Mattermost)" <74422101+cwarnermm@users.noreply.github.com> Date: Wed, 30 Apr 2025 08:33:01 -0400 Subject: [PATCH 4/5] Incorporated reviewer feedback --- .../common-configure-mattermost-for-enterprise-search.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/source/scale/common-configure-mattermost-for-enterprise-search.rst b/source/scale/common-configure-mattermost-for-enterprise-search.rst index 885d6374088..1ae617263bb 100644 --- a/source/scale/common-configure-mattermost-for-enterprise-search.rst +++ b/source/scale/common-configure-mattermost-for-enterprise-search.rst @@ -11,9 +11,9 @@ 4. Set the server connection details for Elasticsearch or AWS OpenSearch: a. Enter **Server Connection Address** for the enterprise search server you set up earlier. - b. (Optional) Enter **Server Username** used to access the enterprise search server. For AWS Elasticsearch, leave this field blank. - c. (Optional) Enter **Server Password** associated with the username. For AWS Elasticsearch, leave this field blank. - d. Set **Enable Cluster Sniffing** (Optional). Sniffing finds and connects to all data nodes in your cluster automatically. For AWS Elasticsearch, this field should be set to ``false``. + b. (Optional) Enter **Server Username** used to access the enterprise search server. + c. (Optional) Enter **Server Password** associated with the username. + d. Set **Enable Cluster Sniffing** (Optional). Sniffing finds and connects to all data nodes in your cluster automatically. .. tip:: From ba0a36f8e601afde44fc9c0f206ac25b2ea78243 Mon Sep 17 00:00:00 2001 From: "Carrie Warner (Mattermost)" <74422101+cwarnermm@users.noreply.github.com> Date: Wed, 30 Apr 2025 08:38:17 -0400 Subject: [PATCH 5/5] Formatting fixes --- .../common-configure-mattermost-for-enterprise-search.rst | 4 ++-- source/scale/enterprise-search.rst | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/source/scale/common-configure-mattermost-for-enterprise-search.rst b/source/scale/common-configure-mattermost-for-enterprise-search.rst index 1ae617263bb..b05e61b783c 100644 --- a/source/scale/common-configure-mattermost-for-enterprise-search.rst +++ b/source/scale/common-configure-mattermost-for-enterprise-search.rst @@ -23,9 +23,9 @@ 5. Select **Build Index** to build the post index of existing posts. This process can take up to a few hours depending on the size of the post database and number of messages. The progress percentage can be seen as the index is created. To avoid downtime, set **Enable Elasticsearch for search queries** to ``false`` so that database search is available during the indexing process. - .. important:: +.. important:: - Complete bulk indexing before enabling Elasticsearch or AWS OpenSearch in the next step. Otherwise, search results will be incomplete. + Complete bulk indexing before enabling Elasticsearch or AWS OpenSearch in the next step. Otherwise, search results will be incomplete. 6. Enable enterprise search by setting **Enable Elasticsearch for search queries** to ``true``, and setting **Enable Elasticsearch for autocomplete** to ``true``. diff --git a/source/scale/enterprise-search.rst b/source/scale/enterprise-search.rst index c51cebb4792..9b282f3e835 100644 --- a/source/scale/enterprise-search.rst +++ b/source/scale/enterprise-search.rst @@ -39,7 +39,7 @@ Review the following support paths for enterprise search based on the version yo .. tab:: Elasticsearch v8 - `Elasticsearch v8 `__ is supported from Mattermost v9.11. While Mattermost supports Elasticsearch v7.17+, we recommend upgrading your Elasticsearch v7 instance to v8.x. See the `Elasticsearch upgrade `_ documentation for details. + `Elasticsearch v8 `__ is supported from Mattermost v9.11. While Mattermost supports Elasticsearch v7.17+, we recommend upgrading your Elasticsearch v7 instance to v8.x. See the `Elasticsearch upgrade `_ documentation for upgrade details, and see the :doc:`Elasticsearch setup ` documentation for details on configuring your Mattermost deployment to use Elasticsearch. .. tab:: AWS OpenSearch Service