Skip to content

Commit

Permalink
Merge pull request #4524 from himeshsiriwardana/elk-fixes
Browse files Browse the repository at this point in the history 
Fixed elk errors
  • Loading branch information
@himeshsiriwardana
himeshsiriwardana authored Mar 6, 2024
2 parents 0c56cab + 51f879d commit 500dd16
Show file tree
Hide file tree
Showing 14 changed files with 182 additions and 130 deletions.
Binary file modified en/identity-server/7.0.0/docs/assets/img/elk-analytics/alerting/elk-alerting-1.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
63 changes: 39 additions & 24 deletions en/identity-server/7.0.0/docs/deploy/elk-analytics-installation-guide.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,28 +6,28 @@ This guide shows you how to configure ELK-based Analytics for WSO2 Identity Serv

Follow the steps below to enable ELK-based analytics in WSO2 Identity Server.

1. Download and install WSO2 Identity Server. For detailed information
on how to install WSO2 IS, see [Install WSO2 Identity Server]({{base_path}}/deploy/get-started/install/).
1. Download and install WSO2 Identity Server.

2. Navigate to the `{IS_HOME}/repository/conf/` directory and open the `deployment.toml` file.
!!! note
For detailed information, see [Install WSO2 Identity Server]({{base_path}}/deploy/get-started/install/).

3. Add the following configuration to the `deployment.toml` file.
2. Open the `deployment.toml` file found in the `{IS_HOME}/repository/conf/` directory and add the following configuration.

```
[analytics.elk]
enable=true
```
## Enable Logs in WSO2 Identity Server
3. Restart {{product_name}}.
1. Navigate to the `<IS_HOME>/repository/conf` directory and open the `log4j2.properties` file.
## Enable Logs in WSO2 Identity Server
2. Add the following configurations to the `log4j2.properties` file.
1. Open the `log4j2.properties` file found in the `<IS_HOME>/repository/conf` directory and add the following configurations.
- Add `ANALYTICS_EVENT_LOGFILE` to the list of all appenders as follows: <br />
`appenders = {other appenders} , ANALYTICS_EVENT_LOGFILE`
- Add the following appender configs:
- Add the following appender configurations:
```
appender.ANALYTICS_EVENT_LOGFILE.type = RollingFile
Expand All @@ -53,46 +53,57 @@ Follow the steps below to enable ELK-based analytics in WSO2 Identity Server.
- Add `org.wso2.carbon.event.output.adapter.logger.LoggerEventAdapter` to the list of all loggers as follows:
```
loggers = {existing loggers}, org-wso2-carbon-event.output-adapter-logger-LoggerEventAdapter
loggers = {other loggers}, org-wso2-carbon-event.output-adapter-logger-LoggerEventAdapter
```
- Add the following logger configs
- Add the following logger configurations.
```
logger.org-wso2-carbon-event.output-adapter-logger-LoggerEventAdapter.name=org.wso2.carbon.event.output.adapter.logger.LoggerEventAdapter
logger.org-wso2-carbon-event.output-adapter-logger-LoggerEventAdapter.level=INFO
logger.org-wso2-carbon-event.output-adapter-logger-LoggerEventAdapter.appenderRef.CUSTOM_LOGFILE.ref=ANALYTICS_EVENT_LOGFILE
```
!!! note
The `analytics_events.log` file will be rolled each day or when the log size reaches the limit of 1000 MB by default. Furthermore, only 10 revisions will be kept and older revisions will be deleted automatically. You can change these configurations by updating the configurations provided in step 2 given above in this. section.
The `analytics_events.log` file will be rolled each day or when the log size reaches the limit of 1000 MB by default. Furthermore, only 10 revisions will be kept and older revisions will be deleted automatically. You can change these configurations by updating the appender configurations above.
2. Restart {{product_name}}.
## Configure ELK
Follow the guides mentioned below to install the ELK components.
### Install Elasticsearch
1. [Install Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html) according to your operating system.
1. [Install Elasticsearch](https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html){:target="_blank"} according to your operating system.
2. Make sure Elasticsearch is up and running.
!!! note
Take note of the password generated for the `elastic` user.
### Install Filebeat
1. [Install Filebeat](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation-configuration.html#installation) according to your operating system.
1. [Install Filebeat](https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-installation-configuration.html#installation){:target="_blank"} according to your operating system.
2. Open the **filebeat.yml** file in the root directory and enter these [configurations](https://github.com/wso2-extensions/identity-elk-integration/blob/main/filebeat/filebeat.yml).
2. Open the **filebeat.yml** file in the root directory and add this [configuration](https://github.com/wso2-extensions/identity-elk-integration/blob/main/filebeat/filebeat.yml){:target="_blank"}.
!!! info
- Replace `<IS_HOME>` with the location of your WSO2 Identity Server installation.
- To configure Filebeat to mask sensitive information in logs, follow this [guide]({{base_path}}/deploy/monitor/elk-mask-sensitive-information-in-logs).
### Install Logstash
1. [Install Logstash](https://www.elastic.co/guide/en/logstash/current/installing-logstash.html) according to your operating system.
2. In the Logstash directory, create a file with the **.conf** extension and add these [configurations](https://github.com/wso2-extensions/identity-elk-integration/blob/main/logstash/logstash-filebeat.conf).
3. [Start the logstash server](https://www.elastic.co/guide/en/logstash/8.1/running-logstash-command-line.html#running-logstash-command-line) with the `-f` flag followed by the location of the configuration file you created.
1. [Install Logstash](https://www.elastic.co/guide/en/logstash/current/installing-logstash.html){:target="_blank"} according to your operating system.
2. In the Logstash directory, create a file with the **.conf** extension and add these [configurations](https://github.com/wso2-extensions/identity-elk-integration/blob/main/logstash/logstash-filebeat.conf){:target="_blank"}.
!!! info
- [Set a password](https://www.elastic.co/guide/en/elasticsearch/reference/current/reset-password.html){: target="_blank"} for the `elastic` super user.
- Replace `<ELASTICSEARCH_HOME>`, `<ELASTICSEARCH_USERNAME>`, `<ELASTICSEARCH_USER_PASSWORD>` with the corresponding values.
3. [Start the logstash server](https://www.elastic.co/guide/en/logstash/8.1/running-logstash-command-line.html#running-logstash-command-line){:target="_blank"} with the `-f` flag followed by the location of the configuration file you created.
### Installing Kibana
1. [Install Kibana](https://www.elastic.co/guide/en/elastic-stack-get-started/current/get-started-elastic-stack.html#install-kibana) according to your operating system and do this [one time configuration](https://www.elastic.co/guide/en/elasticsearch/reference/8.2/configuring-stack-security.html#stack-start-with-security).
1. [Install Kibana](https://www.elastic.co/guide/en/kibana/current/install.html){:target="_blank"} according to your operating system and do this [one time configuration](https://www.elastic.co/guide/en/elasticsearch/reference/8.2/configuring-stack-security.html#stack-start-with-security){:target="_blank"}.
2. Once Kibana is running, enter its web interface using the following address:
Expand All @@ -102,7 +113,7 @@ Follow the steps below to enable ELK-based analytics in WSO2 Identity Server.
## Configure ELK Analytics Dashboards
1. Navigate to Kibana installation folder and run the following command to install the [Kibana Enhanced Table](https://github.com/fbaligand/kibana-enhanced-table) plugin.
1. Navigate to the Kibana installation folder and run the following command to install the [Kibana Enhanced Table](https://github.com/fbaligand/kibana-enhanced-table){: target="_blank"} plugin.
```
./bin/kibana-plugin install https://github.com/fbaligand/kibana-enhanced-table/releases/download/vA.B.C/enhanced-table-A.B.C_X.Y.Z.zip
Expand All @@ -113,7 +124,9 @@ Follow the steps below to enable ELK-based analytics in WSO2 Identity Server.
2. Restart Kibana service and log in to Kibana.
3. Navigate to **Stack Management** > **Index Management** and select the **Index Templates** tab.
3. On the left navigation panel, under **Management** click **Stack Management**.
4. Under **Stack Management** > **Data**, click **Index Management** and go to its **Index Templates** tab.
4. Under the **Index patterns** column, if you have any index patterns created under the following names, delete them before moving to the next step.
Expand All @@ -125,8 +138,10 @@ Follow the steps below to enable ELK-based analytics in WSO2 Identity Server.
5. Download the artifact file [here]({{base_path}}/assets/img/elk-analytics/kibana-8-x-auth-and-session.ndjson).
6. Navigate to **Stack Management** > **Saved Objects**.
6. Under **Stack Management** > **Kibana**, click **Saved Objects**.
7. Click **Import**, add the downloaded artifact file as an import object, and click **Import**.
7. Click **Import**, add the downloaded artifact file as an import object, and import.
8. Once import is complete, click **Done**.
9. Navigate to the **Dashboard** section of Kibana to view the created **Auth** and **Session** dashboards.
9. On the left navigation panel, under **Analytics**, click **Dashboards** to view the created **Auth** and **Session** dashboards.
12 changes: 7 additions & 5 deletions en/identity-server/7.0.0/docs/deploy/elk-analytics-sso-guide.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ To enable Elastic stack SSO with WSO2 Identity Server, follow the steps given be

## Configure OIDC Realm in Elasticsearch

To configure single sign-on in the Elastic Stack using OpenID connect, follow the steps given [here](https://www.elastic.co/guide/en/elasticsearch/reference/7.16/oidc-guide.html).
To configure single sign-on in the Elastic Stack using OpenID connect, follow the steps given [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/oidc-guide.html){: target="_blank"}.

A sample OpenID connect realm is as follows.

Expand All @@ -63,13 +63,13 @@ A sample OpenID connect realm is as follows.

## Configure role mappings

Once the above steps are completed, role mapping needs to be configured in Kibana to allow WSO2 Identity Server users to access the dashboards in Kibana. For that follow the steps below.
Once the above steps are completed, role mapping needs to be configured in Kibana to allow {{product_name}} users to access the dashboards in Kibana. To do so, follow the steps below.

### Create Users and Roles in WSO2 Identity Server
### Create Users and Roles in {{product_name}}

Follow the steps below to create a role.

1. Login to the WSO2 Identity Server console.
1. Login to the {{product_name}} Console.

2. Navigate to **User Management** > **Roles**.

Expand All @@ -94,7 +94,9 @@ Follow the steps below to associate Kibana roles with the roles you created in W

1. Login to Kibana using basic authentication.

2. Click **Stack Management** on the left panel and then click **Role Mappings** under the **Security** section.
2. On the left navigation panel, under **Management** click **Stack Management**.

3. Under **Stack Management** > **Security**, click **Role Mappings**.

3. Click **Create Role Mapping** and add a new role mapping by giving a **Mapping name**.

Expand Down
Loading

0 comments on commit 500dd16

Please sign in to comment.