Snapshot Creation Verification

Verify that a snapshot can be created and used for metadata and backfill scenarios.

Install the Elasticsearch S3 Repository Plugin

The snapshot needs to be stored in a location that the Migration Assistant can access. We use AWS S3 as that location, and the Migration Assistant creates an S3 bucket for this purpose. Therefore, it is necessary to install the Elasticsearch S3 Repository Plugin on your source nodes as described here.

Additionally, ensure that the plugin has been configured with AWS credentials that allow it to read and write to AWS S3. If your Elasticsearch cluster is running on EC2 or ECS instances with an execution IAM Role, include the necessary S3 permissions. Alternatively, you can store the credentials in the Elasticsearch Key Store as described here.

Verifying S3 Repository Plugin Configuration

You can verify that the S3 Repository Plugin is configured correctly by creating a test snapshot.

Create an S3 bucket for the snapshot using the following AWS CLI command:

aws s3api create-bucket --bucket <your-bucket-name> --region <your-aws-region>

Register a new S3 Snapshot Repository on your source cluster using this curl command:

curl -X PUT "http://<your-source-cluster>:9200/_snapshot/test_s3_repository" -H "Content-Type: application/json" -d '{
  "type": "s3",
  "settings": {
    "bucket": "<your-bucket-name>",
    "region": "<your-aws-region>"
  }
}'

You should receive a response like: {"acknowledged":true}.

Create a test snapshot that captures only the cluster's metadata:

curl -X PUT "http://<your-source-cluster>:9200/_snapshot/test_s3_repository/test_snapshot_1" -H "Content-Type: application/json" -d '{
  "indices": "",
  "ignore_unavailable": true,
  "include_global_state": true
}'

Example Response

You should receive a response like: {"accepted":true}.

Check the AWS Console to confirm that your bucket contains the snapshot. It will appear similar to this:

Cleaning Up After Verification

To remove the resources created during verification:

Delete the snapshot:

curl -X DELETE "http://<your-source-cluster>:9200/_snapshot/test_s3_repository/test_snapshot_1?pretty"

Delete the snapshot repository:

curl -X DELETE "http://<your-source-cluster>:9200/_snapshot/test_s3_repository?pretty"

Delete the S3 bucket and its contents:

aws s3 rm s3://<your-bucket-name> --recursive
aws s3api delete-bucket --bucket <your-bucket-name> --region <your-aws-region>

Troubleshooting

Access Denied Error (403)

If you encounter an error like AccessDenied (Service: Amazon S3; Status Code: 403), verify the following:

• The IAM role assigned to your Elasticsearch cluster has the necessary S3 permissions.
• The bucket name and region provided in the snapshot configuration match the actual S3 bucket you created.

Related Links

See the Elasticsearch S3 Repository Plugin Configuration Guide for details on setting up AWS credentials for the plugin.