Skip to content

Latest commit

 

History

History
201 lines (150 loc) · 7.37 KB

setup-terraform.md

File metadata and controls

201 lines (150 loc) · 7.37 KB

Setup Terraform

The terraform backend can be used to validate workloads on a remote cluster, On-Premises or on Cloud.

Setup Terraform for Cloud Validation

  • Follow the instructions in the WSF Cloud Setup to setup the development host.
  • The terraform backend supports Cloud vendors such as aws, gcp, azure, tencent, alicloud, and oracle. Each vendor has a corresponding configuration file: script/terraform/terraform-config.<SUT>.tf, where <SUT> is the Cloud vendor name. You can customize as needed.

Configure Cloud Account

If this is your first time, run the terraform build command:

Note: mentioned below commands should be executed from a build folder such as ~/workload-services-framework/build# make build_terraform

make build_terraform

Then proceed with the Cloud account setup as follows:

make aws           # or make -C ../.. aws, if under build/workload/<workload>
$ aws configure    # please specify a region and output format as json
$ exit
make azure         # or make -C ../.. azure, if under build/workload/<workload>
$ az login
# if you have multiple subscriptions in your account, please set
# az account set --subscription="<subscription-id>"
$ exit
make gcp           # or make -C ../.. gcp, if under build/workload/<workload>
$ gcloud init --no-launch-browser
$ gcloud auth application-default login --no-launch-browser # Please make sure quota-project-id is set
$ gcloud auth activate-service-account --key-file <keyfile> # required only for running workloads in Google Kubernetes Engine
$ exit
make tencent       # or make -C ../.. tencent, if under build/workload/<workload>
$ tccli configure  # please specify a region
$ exit
make alicloud      # make -C ../.. alicloud, if under build/workload/<workload>
$ aliyun configure # please specify a region
$ exit
make oracle        # make -C ../.. oracle, if under build/workload/<workload>
$ oci setup config # please specify a compartment id and upload public key to the oracle cloud console 
$ exit

Run Workload(s) Through Terraform

cd workload/<workload>
make
./ctest.sh -N

Cleanup Cloud Resources

If your terraform validation is interrupted for any reason, the Cloud resource may remain active. You can explicitly cleanup any Cloud resources as follows:

Note: mentioned below commands should be executed from the required workload folder such as ~/workload-services-framework/build/workload/OpenSSL3-RSAMB# make -C ../.. azure

make -C ../.. [aws|gcp|azure|tencent|alicloud]
$ cleanup
$ exit

Setup Terraform for On-Premises Validation

Under script/terraform, you can create any terraform-config.mysut.tf out of terraform-config.static.tf, where mysut is your sut name. Use cmake -DTERRAFORM_SUT=mysut .. to configure it.

Now you can run any workload as follows:

cd build
cmake -DTERRAFORM_SUT=static -DBENCHMARK=workload/<workload-name> ..
make
./ctest.sh -N

Setup Terraform for KVM Validation

Now you can run any workload as follows:

cd workload/<workload>
make
./ctest.sh -N

Telemetry Trace and Publishing Options

See Trace Module for available trace options. You can enable telemetry trace modules during the workload validation as follows:

cmake -DTERRAFORM_OPTIONS=--collectd ..
cd workload/<workload>
./ctest.sh -N

Additionally, you can use --svrinfo to the TERRAFORM_OPTIONS to automatically detect the platform information as follows:

cmake -DTERRAFORM_OPTIONS=--svrinfo ..
cd workload/<workload>
./ctest.sh -N

See also: Publishing Module Options.

Debugging

While the workload evaluation is in progress, you can logon to the remote instances to debug any encountered issues. As terraform engine runs inside a container, you need to first login to the container as follows:

./debug.sh

The script will bring you to the container shell where you can perform, from the current directory or /opt/workspace, additional operations such as examining the workload execution logs and logging onto the workload instances.

Files of interest:

  • cluster-config.yaml: The workload cluster configuration definition.
  • terraform-config.yaml: The workload terraform entry point.
  • workload-config.yaml: The workload configuration parameters.
  • kubernetes-config.yaml[.mod.yaml]: The kubernetes deployment script (for containerized workloads.)
  • cluster.yaml: The ansible playbook to initialize the VM instances.
  • deployment.yaml: The ansible playbook to run the workload.
  • inventory.yaml: The provisioned VM information.
  • tfplan.logs: The process logs.
  • ssh_access.key[.pub]: The SSH keys for accessing to the VM instances.
  • template/*: Source code used to provision VMs and evaluate workloads.
$ cat inventory.yaml
...
        worker-0:
          ansible_host: 35.92.225.114
          ansible_user: ubuntu
...
$ ssh -i ssh_access.key ubuntu@35.92.225.114

HINT: while working within the terraform container, if you need to ssh or scp data to public VMs, use the ssh identity ssh_access.key. If you are not under the default directory /opt/workspace, please use the absolute path.

For Example:

$ scp -r -i ssh_access.key /tmp/test.log <username>@<public_ip>:/tmp/
Warning: Permanently added '<public_ip>' (ED25519) to the list of known hosts.
test.log                                                                                                                                                                                                                                                    100%    5     0.1KB/s   00:00

Setting Breakpoint(s)

You can set one or many breakpoints by specifying the wl_debug option in TERRAFORM_OPTIONS or terraform-config.<sut>.tf:

cmake -DTERRAFORM_OPTIONS=--wl_debug=<BreakPoint>[,<BreakPoint>] ..

The following <BreakPoint>s are supported:

  • PrepareStage: Pause when the workload is about to setup the host environment.
  • RunStage: Pause when the workload is about to start the workload execution.
  • CleanupStage: Pause when the workload is about to cleanup.

When a breakpoint is reached, the execution is paused for an hour (as specified by the wl_debug_timeout value.) You can explicitly resume the execution by creating a signaling file under /opt/workspace, as follows:

./debug.sh
$ touch ResumeRunStage
$ exit