Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Become: false for operation /tmp/opensearch-nodecerts #51

Merged
merged 5 commits into from
May 2, 2022
Merged

Become: false for operation /tmp/opensearch-nodecerts #51

merged 5 commits into from
May 2, 2022

Conversation

patsevanton
Copy link
Contributor

Signed-off-by: Anton Patsev patsev.anton@gmail.com

Description

Become: false for operation /tmp/opensearch-nodecerts

Issues Resolved

Fix #45

Check List

  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@patsevanton
Become: false for operation /tmp/opensearch-nodecerts
1934b1a 
Signed-off-by: Anton Patsev <patsev.anton@gmail.com>
@patsevanton patsevanton requested a review from a team as a code owner April 8, 2022 03:59
@patsevanton patsevanton mentioned this pull request Apr 8, 2022
Closed
1 task
@saravanan30erd
Copy link
Collaborator

saravanan30erd commented Apr 8, 2022 •
edited
Loading

@patsevanton There is one more local_action which we missed.
https://github.com/opensearch-project/ansible-playbook/blob/main/roles/linux/opensearch/tasks/security.yml#L151
Please add become: false for that task too. Otherwise we will face the below error,

TASK [linux/opensearch : Security Plugin configuration | Cleanup local temporary directory] **********************************************************************
fatal: [os1 -> localhost]: FAILED! => {"changed": false, "module_stderr": "sudo: a password is required\n", "module_stdout": "", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

Additionally please add how to run the playbook using non-root in README.

For running with non-root user (which have sudo privileges), we need to to below changes
In hosts file, change the user details

os1 ansible_host=10.0.0.1  ansible_user=ec2-user ip=10.0.0.1

and run the playbook using --become

ansible-playbook -i inventories/opensearch/hosts opensearch.yml --extra-vars "admin_password=Test@123 kibanaserver_password=Test@6789" --become

Please add this details in README (Don't forget to sign the commit :)- ) @patsevanton

@peterzhuamazon I tested with both users using single-node cluster type, root user and non-root ec2-user user with sudo privileges after fixing the issue here and it works fine.

Please test it with multi-node after that fix, then this PR is good to merge.

@patsevanton
become: false for Cleanup local temporary directory
baf3929 
Signed-off-by: Anton Patsev <patsev.anton@gmail.com>
@patsevanton
readme: run the playbook as ec2-user, add --become
ba91102 
Signed-off-by: Anton Patsev <patsev.anton@gmail.com>
@patsevanton
Copy link
Contributor Author

Added become: false to Cleanup local temporary directory
Added ansible_user as user
Added --become for run playbook

Cannot test multi-node because #49

@peterzhuamazon
Copy link
Member

@saravanan30erd could you approve?
Thanks.

@saravanan30erd
Copy link
Collaborator

@saravanan30erd could you approve? Thanks.

Will do. I need to test it with multi-node setup.

@peterzhuamazon
Copy link
Member

@saravanan30erd could you approve? Thanks.

Will do. I need to test it with multi-node setup.

Hi @saravanan30erd let me know the status of this.
Thanks.

@saravanan30erd
Copy link
Collaborator

saravanan30erd commented Apr 22, 2022 •
edited
Loading

@peterzhuamazon I tested for multiple node setup with non-root user (ec2-user in EC2), it works fine.
Already tested with root user and with non-root user for single-node setup here

We can merge this now.

Logs % ansible-playbook -i inventories/opensearch/hosts opensearch.yml --extra-vars "admin_password=Test@123 kibanaserver_password=Test@6789" --become [WARNING]: Invalid characters were found in group names but not replaced, use -vvvv to see details [DEPRECATION WARNING]: "include" is deprecated, use include_tasks/import_tasks instead. This feature will be removed in version 2.16. Deprecation warnings can be disabled by setting deprecation_warnings=False in ansible.cfg.

PLAY [opensearch installation & configuration] *****************************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************
[WARNING]: Platform linux on host os1 is using the discovered Python interpreter at /usr/bin/python3.7, but future installation of another Python
interpreter could change the meaning of that path. See https://docs.ansible.com/ansible-core/2.12/reference_appendices/interpreter_discovery.html
for more information.
ok: [os1]
[WARNING]: Platform linux on host os2 is using the discovered Python interpreter at /usr/bin/python3.7, but future installation of another Python
interpreter could change the meaning of that path. See https://docs.ansible.com/ansible-core/2.12/reference_appendices/interpreter_discovery.html
for more information.
ok: [os2]
[WARNING]: Platform linux on host os5 is using the discovered Python interpreter at /usr/bin/python3.7, but future installation of another Python
interpreter could change the meaning of that path. See https://docs.ansible.com/ansible-core/2.12/reference_appendices/interpreter_discovery.html
for more information.
ok: [os5]
[WARNING]: Platform linux on host os4 is using the discovered Python interpreter at /usr/bin/python3.7, but future installation of another Python
interpreter could change the meaning of that path. See https://docs.ansible.com/ansible-core/2.12/reference_appendices/interpreter_discovery.html
for more information.
ok: [os4]
[WARNING]: Platform linux on host os3 is using the discovered Python interpreter at /usr/bin/python3.7, but future installation of another Python
interpreter could change the meaning of that path. See https://docs.ansible.com/ansible-core/2.12/reference_appendices/interpreter_discovery.html
for more information.
ok: [os3]

TASK [linux/opensearch : hostname] *****************************************************************************************************************
changed: [os1]
changed: [os4]
changed: [os5]
changed: [os2]
changed: [os3]

TASK [linux/opensearch : Disable the selinux] ******************************************************************************************************
skipping: [os1]
skipping: [os2]
skipping: [os3]
skipping: [os4]
skipping: [os5]

TASK [linux/opensearch : Hosts | populate inventory into hosts file] *******************************************************************************
changed: [os2]
changed: [os5]
changed: [os1]
changed: [os4]
changed: [os3]

TASK [linux/opensearch : Set vm.max_map_count in sysctl.conf] **************************************************************************************
changed: [os4]
changed: [os2]
changed: [os3]
changed: [os5]
changed: [os1]

TASK [linux/opensearch : Set open files limit in sysctl.conf] **************************************************************************************
changed: [os1]
changed: [os3]
changed: [os4]
changed: [os5]
changed: [os2]

TASK [linux/opensearch : OpenSearch Install | Download opensearch 1.3.1] ***************************************************************************
changed: [os1]
changed: [os2]
changed: [os3]
changed: [os4]
changed: [os5]

TASK [linux/opensearch : OpenSearch Install | Create opensearch user] ******************************************************************************
changed: [os2]
changed: [os3]
changed: [os1]
changed: [os5]
changed: [os4]

TASK [linux/opensearch : OpenSearch Install | Create home directory] *******************************************************************************
changed: [os4]
changed: [os3]
changed: [os2]
changed: [os1]
changed: [os5]

TASK [linux/opensearch : OpenSearch Install | Extract the tar file] ********************************************************************************
changed: [os2]
changed: [os3]
changed: [os4]
changed: [os5]
changed: [os1]

TASK [linux/opensearch : OpenSearch Install | Copy Configuration File] *****************************************************************************
changed: [os1]
changed: [os2]
changed: [os3]
changed: [os4]
changed: [os5]

TASK [linux/opensearch : OpenSearch Install | Copy jvm.options File for Instance] ******************************************************************
changed: [os5]
changed: [os3]
changed: [os4]
changed: [os2]
changed: [os1]

TASK [linux/opensearch : OpenSearch Install | create systemd service] ******************************************************************************
changed: [os2]
changed: [os1]
changed: [os3]
changed: [os5]
changed: [os4]

TASK [linux/opensearch : Security Plugin configuration | Create local temporary directory for certificates generation] *****************************
changed: [os1 -> localhost]

TASK [linux/opensearch : Security Plugin configuration | Download certificates generation tool] ****************************************************
changed: [os1 -> localhost]

TASK [linux/opensearch : Security Plugin configuration | Extract the certificates generation tool] *************************************************
changed: [os1 -> localhost]

TASK [linux/opensearch : Security Plugin configuration | Make the executable file] *****************************************************************
ok: [os1 -> localhost]

TASK [linux/opensearch : Security Plugin configuration | Prepare the certificates generation template file] ****************************************
changed: [os1 -> localhost]

TASK [linux/opensearch : Security Plugin configuration | Generate the node & admin certificates in local] ******************************************
changed: [os1 -> localhost]

TASK [linux/opensearch : Security Plugin configuration | Copy the node & admin certificates to opensearch nodes] ***********************************
changed: [os3] => (item=root-ca.pem)
changed: [os2] => (item=root-ca.pem)
changed: [os1] => (item=root-ca.pem)
changed: [os5] => (item=root-ca.pem)
changed: [os4] => (item=root-ca.pem)
changed: [os3] => (item=root-ca.key)
changed: [os2] => (item=root-ca.key)
changed: [os4] => (item=root-ca.key)
changed: [os5] => (item=root-ca.key)
changed: [os1] => (item=root-ca.key)
changed: [os3] => (item=os3.key)
changed: [os2] => (item=os2.key)
changed: [os4] => (item=os4.key)
changed: [os5] => (item=os5.key)
changed: [os1] => (item=os1.key)
changed: [os3] => (item=os3.pem)
changed: [os2] => (item=os2.pem)
changed: [os4] => (item=os4.pem)
changed: [os1] => (item=os1.pem)
changed: [os5] => (item=os5.pem)
changed: [os3] => (item=os3_http.key)
changed: [os2] => (item=os2_http.key)
changed: [os4] => (item=os4_http.key)
changed: [os1] => (item=os1_http.key)
changed: [os5] => (item=os5_http.key)
changed: [os3] => (item=os3_http.pem)
changed: [os2] => (item=os2_http.pem)
changed: [os4] => (item=os4_http.pem)
changed: [os1] => (item=os1_http.pem)
changed: [os5] => (item=os5_http.pem)
changed: [os3] => (item=admin.key)
changed: [os2] => (item=admin.key)
changed: [os4] => (item=admin.key)
changed: [os1] => (item=admin.key)
changed: [os5] => (item=admin.key)
changed: [os3] => (item=admin.pem)
changed: [os2] => (item=admin.pem)
changed: [os4] => (item=admin.pem)
changed: [os1] => (item=admin.pem)
changed: [os5] => (item=admin.pem)

TASK [linux/opensearch : Security Plugin configuration | Copy the security configuration file 1 to cluster] ****************************************
changed: [os1]
changed: [os2]
changed: [os5]
changed: [os4]
changed: [os3]

TASK [linux/opensearch : Security Plugin configuration | Copy the security configuration file 2 to cluster] ****************************************
changed: [os1]
changed: [os3]
changed: [os4]
changed: [os2]
changed: [os5]

TASK [linux/opensearch : Security Plugin configuration | Prepare the opensearch security configuration file] ***************************************
changed: [os2]
changed: [os1]
changed: [os5]
changed: [os3]
changed: [os4]

TASK [linux/opensearch : Security Plugin configuration | Set the file ownerships] ******************************************************************
changed: [os2]
changed: [os1]
changed: [os3]
changed: [os4]
changed: [os5]

TASK [linux/opensearch : Security Plugin configuration | Set the folder permission] ****************************************************************
changed: [os1]
changed: [os2]
changed: [os3]
changed: [os5]
changed: [os4]

TASK [linux/opensearch : Security Plugin configuration | Restart opensearch with security configuration] *******************************************
changed: [os1]
changed: [os2]
changed: [os4]
changed: [os3]
changed: [os5]

TASK [linux/opensearch : Pause for 3 seconds to provide sometime for OpenSearch start] *************************************************************
Pausing for 3 seconds
(ctrl+C then 'C' = continue early, ctrl+C then 'A' = abort)
ok: [os1]

TASK [linux/opensearch : Security Plugin configuration | Copy the opensearch security internal users template] *************************************
changed: [os1]

TASK [linux/opensearch : Security Plugin configuration | Set the Admin user password] **************************************************************
changed: [os1]

TASK [linux/opensearch : Security Plugin configuration | Set the kibanaserver user pasword] ********************************************************
changed: [os1]

TASK [linux/opensearch : Security Plugin configuration | Initialize the opensearch security index in opensearch] ***********************************
changed: [os1]

TASK [linux/opensearch : Security Plugin configuration | Cleanup local temporary directory] ********************************************************
changed: [os1 -> localhost]

TASK [linux/opensearch : Remove cluster.initial_master_nodes setting from configuration] *********************************************************
changed: [os1]
changed: [os3]
changed: [os4]
changed: [os5]
changed: [os2]

TASK [linux/opensearch : Make sure opensearch is started] ******************************************************************************************
ok: [os5]
ok: [os4]
ok: [os3]
ok: [os1]
ok: [os2]

TASK [linux/opensearch : Get all the installed ES plugins] *****************************************************************************************
changed: [os1]
changed: [os5]
changed: [os3]
changed: [os4]
changed: [os2]

TASK [linux/opensearch : Show all the installed ES plugins] ****************************************************************************************
ok: [os1] => {
"msg": "opensearch-alerting\nopensearch-anomaly-detection\nopensearch-asynchronous-search\nopensearch-cross-cluster-replication\nopensearch-index-management\nopensearch-job-scheduler\nopensearch-knn\nopensearch-ml\nopensearch-observability\nopensearch-performance-analyzer\nopensearch-reports-scheduler\nopensearch-security\nopensearch-sql"
}
ok: [os2] => {
"msg": "opensearch-alerting\nopensearch-anomaly-detection\nopensearch-asynchronous-search\nopensearch-cross-cluster-replication\nopensearch-index-management\nopensearch-job-scheduler\nopensearch-knn\nopensearch-ml\nopensearch-observability\nopensearch-performance-analyzer\nopensearch-reports-scheduler\nopensearch-security\nopensearch-sql"
}
ok: [os3] => {
"msg": "opensearch-alerting\nopensearch-anomaly-detection\nopensearch-asynchronous-search\nopensearch-cross-cluster-replication\nopensearch-index-management\nopensearch-job-scheduler\nopensearch-knn\nopensearch-ml\nopensearch-observability\nopensearch-performance-analyzer\nopensearch-reports-scheduler\nopensearch-security\nopensearch-sql"
}
ok: [os4] => {
"msg": "opensearch-alerting\nopensearch-anomaly-detection\nopensearch-asynchronous-search\nopensearch-cross-cluster-replication\nopensearch-index-management\nopensearch-job-scheduler\nopensearch-knn\nopensearch-ml\nopensearch-observability\nopensearch-performance-analyzer\nopensearch-reports-scheduler\nopensearch-security\nopensearch-sql"
}
ok: [os5] => {
"msg": "opensearch-alerting\nopensearch-anomaly-detection\nopensearch-asynchronous-search\nopensearch-cross-cluster-replication\nopensearch-index-management\nopensearch-job-scheduler\nopensearch-knn\nopensearch-ml\nopensearch-observability\nopensearch-performance-analyzer\nopensearch-reports-scheduler\nopensearch-security\nopensearch-sql"
}

TASK [linux/opensearch : Wait for opensearch to startup] *******************************************************************************************
ok: [os1]
ok: [os3]
ok: [os5]
ok: [os4]
ok: [os2]

TASK [linux/opensearch : Check the opensearch status] **********************************************************************************************
changed: [os1]
changed: [os4]
changed: [os3]
changed: [os5]
changed: [os2]

TASK [linux/opensearch : Show the opensearch status] ***********************************************************************************************
ok: [os1] => {
"msg": {
"active_primary_shards": 2,
"active_shards": 7,
"active_shards_percent_as_number": 100.0,
"cluster_name": "development-cluster",
"delayed_unassigned_shards": 0,
"discovered_master": true,
"initializing_shards": 0,
"number_of_data_nodes": 5,
"number_of_in_flight_fetch": 0,
"number_of_nodes": 5,
"number_of_pending_tasks": 0,
"relocating_shards": 0,
"status": "green",
"task_max_waiting_in_queue_millis": 0,
"timed_out": false,
"unassigned_shards": 0
}
}
ok: [os2] => {
"msg": {
"active_primary_shards": 2,
"active_shards": 7,
"active_shards_percent_as_number": 100.0,
"cluster_name": "development-cluster",
"delayed_unassigned_shards": 0,
"discovered_master": true,
"initializing_shards": 0,
"number_of_data_nodes": 5,
"number_of_in_flight_fetch": 0,
"number_of_nodes": 5,
"number_of_pending_tasks": 0,
"relocating_shards": 0,
"status": "green",
"task_max_waiting_in_queue_millis": 0,
"timed_out": false,
"unassigned_shards": 0
}
}
ok: [os3] => {
"msg": {
"active_primary_shards": 2,
"active_shards": 7,
"active_shards_percent_as_number": 100.0,
"cluster_name": "development-cluster",
"delayed_unassigned_shards": 0,
"discovered_master": true,
"initializing_shards": 0,
"number_of_data_nodes": 5,
"number_of_in_flight_fetch": 0,
"number_of_nodes": 5,
"number_of_pending_tasks": 0,
"relocating_shards": 0,
"status": "green",
"task_max_waiting_in_queue_millis": 0,
"timed_out": false,
"unassigned_shards": 0
}
}
ok: [os4] => {
"msg": {
"active_primary_shards": 2,
"active_shards": 7,
"active_shards_percent_as_number": 100.0,
"cluster_name": "development-cluster",
"delayed_unassigned_shards": 0,
"discovered_master": true,
"initializing_shards": 0,
"number_of_data_nodes": 5,
"number_of_in_flight_fetch": 0,
"number_of_nodes": 5,
"number_of_pending_tasks": 0,
"relocating_shards": 0,
"status": "green",
"task_max_waiting_in_queue_millis": 0,
"timed_out": false,
"unassigned_shards": 0
}
}
ok: [os5] => {
"msg": {
"active_primary_shards": 2,
"active_shards": 7,
"active_shards_percent_as_number": 100.0,
"cluster_name": "development-cluster",
"delayed_unassigned_shards": 0,
"discovered_master": true,
"initializing_shards": 0,
"number_of_data_nodes": 5,
"number_of_in_flight_fetch": 0,
"number_of_nodes": 5,
"number_of_pending_tasks": 0,
"relocating_shards": 0,
"status": "green",
"task_max_waiting_in_queue_millis": 0,
"timed_out": false,
"unassigned_shards": 0
}
}

TASK [linux/opensearch : Verify the roles of opensearch cluster nodes] *****************************************************************************
changed: [os1]

TASK [linux/opensearch : Show the roles of opensearch cluster nodes] *******************************************************************************
ok: [os1] => {
"msg": "ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name\n10.0.0.55 8 95 11 0.23 0.15 0.05 di - os4\n10.0.0.209 7 96 11 0.17 0.15 0.06 di - os5\n10.0.0.193 10 91 18 0.18 0.18 0.08 dm - os1\n10.0.0.229 7 95 11 0.27 0.22 0.09 dm - os3\n10.0.0.220 8 97 19 0.47 0.22 0.09 dm * os2"
}

PLAY [opensearch dashboards installation & configuration] ******************************************************************************************

TASK [Gathering Facts] *****************************************************************************************************************************
[WARNING]: Platform linux on host dashboards1 is using the discovered Python interpreter at /usr/bin/python3.7, but future installation of another
Python interpreter could change the meaning of that path. See https://docs.ansible.com/ansible-
core/2.12/reference_appendices/interpreter_discovery.html for more information.
ok: [dashboards1]

TASK [linux/dashboards : hostname] *****************************************************************************************************************
changed: [dashboards1]

TASK [linux/dashboards : Disable the selinux] ******************************************************************************************************
skipping: [dashboards1]

TASK [linux/dashboards : Hosts | populate inventory into hosts file] *******************************************************************************
changed: [dashboards1]

TASK [linux/dashboards : Set open files limit in sysctl.conf] **************************************************************************************
changed: [dashboards1]

TASK [linux/dashboards : Dashboards Install | Download opensearch dashbaord 1.3.1] *****************************************************************
changed: [dashboards1]

TASK [linux/dashboards : Dashboards Install | Create opensearch user] ******************************************************************************
changed: [dashboards1]

TASK [linux/dashboards : Dashboards Install | Create home directory] *******************************************************************************
changed: [dashboards1]

TASK [linux/dashboards : Dashboards Install | Extract the tar file] ********************************************************************************
changed: [dashboards1]

TASK [linux/dashboards : Dashboards Install | Copy Configuration File] *****************************************************************************
changed: [dashboards1]

TASK [linux/dashboards : Dashboards Install | create systemd service] ******************************************************************************
changed: [dashboards1]

TASK [linux/dashboards : Make sure opensearch dashboards is started] *******************************************************************************
changed: [dashboards1]

TASK [linux/dashboards : Get all the installed dashboards plugins] *********************************************************************************
changed: [dashboards1]

TASK [linux/dashboards : Show all the installed dashboards plugins] ********************************************************************************
ok: [dashboards1] => {
"msg": "alertingDashboards@1.3.1.0\nanomalyDetectionDashboards@1.3.1.0\nganttChartDashboards@1.3.1.0\nindexManagementDashboards@1.3.1.0\nobservabilityDashboards@1.3.1.0\nqueryWorkbenchDashboards@1.3.1.0\nreportsDashboards@1.3.1.0\nsecurityDashboards@1.3.1.0"
}

PLAY RECAP *****************************************************************************************************************************************
dashboards1 : ok=13 changed=11 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
os1 : ok=40 changed=32 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
os2 : ok=26 changed=21 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
os3 : ok=26 changed=21 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
os4 : ok=26 changed=21 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0
os5 : ok=26 changed=21 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0

@saravanan30erd saravanan30erd mentioned this pull request Apr 22, 2022
Open
@peterzhuamazon
Copy link
Member

peterzhuamazon commented Apr 23, 2022 •
edited
Loading

@saravanan30erd

You can approve now as you are maintainer 😄

saravanan30erd
saravanan30erd approved these changes Apr 24, 2022
View reviewed changes
@patsevanton
Add examples: hosts_ec2-user and hosts_root_user
83c575e 
Signed-off-by: Anton Patsev <patsev.anton@gmail.com>
peterzhuamazon
peterzhuamazon requested changes Apr 25, 2022
View reviewed changes
Copy link
Member

@peterzhuamazon peterzhuamazon left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably use non-root user than use ec2-user as the latter is specifically for AL2 AMI.
CentOS7 AMI uses centos and Ubuntu uses ubuntu and more.
Please let me know if this makes sense to you.

Thanks.

@patsevanton
Add examples hosts for centos and ubuntu
f81e8e6 
Signed-off-by: Anton Patsev <patsev.anton@gmail.com>
@patsevanton
Copy link
Contributor Author

I created new commit:
Add examples hosts for centos and ubuntu

@peterzhuamazon peterzhuamazon merged commit 2cb49ff into opensearch-project:main May 2, 2022
@peterzhuamazon
Copy link
Member

Thanks @patsevanton for the patience your PR is merged now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@saravanan30erd saravanan30erd saravanan30erd approved these changes

@prudhvigodithi prudhvigodithi prudhvigodithi approved these changes

@peterzhuamazon peterzhuamazon peterzhuamazon approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG][opensearch] sudo: a password is required
4 participants
@patsevanton @saravanan30erd @peterzhuamazon @prudhvigodithi