Send Data Stream Data to Elasticsearch

Hands-on to use Elasticsearch to receive data from Azion Data Stream.

View as Markdown

While configuring streams, you need to set up a specific endpoint to stream your Azion data.

Continue reading for a step by step on how to connect an Elasticsearch endpoint to receive data from Data Stream.

Elasticsearch requirements

You can run an Elasticsearch instance on any cloud computing platform. To set it up, look for the specific documentation for your chosen platform.

For the next steps, you’ll use the Elasticsearch cloud. Follow the next steps to continue your Elasticsearch Cloud configuration:

  1. Create an account at Elasticsearch Cloud.

  2. Access your cloud account.

  3. Create a deployment. This process takes about five minutes.

  4. To create a deploy, you have to provide the following information:

    • A name.
    • A cloud provider.
    • A region for your cloud.
    • A hardware profile.
    • The version you’ll use.
    • Optionally, you can configure advanced settings.

  5. Click the Create deployment button. The creation process takes about five minutes.

  6. Go to your dashboard on Elastic Cloud. You’ll see the created deployment with the name you’ve chosen.

  7. Click on the gear on the right corner of the deploy to access the configuration page for the deployment.

  8. Under Applications, click Copy endpoint according to the endpoint you want to use to get your URL.

  9. On the left menu, click API console > create an API key following the Elasticsearch instructions.

Configuring the endpoint in Data Stream

Next, follow these steps to configure the new endpoint you created in Elasticsearch in your Azion Data Stream.

You can find detailed steps for the entire configuration on the How to use Data Stream guide.

All key fields will be masked and an icon that allows revealing the field content will be only displayed to users with the Edit Data Stream permission. Users without the Edit Data Stream permission can only view the settings of the existing stream but not edit or create new ones.

In the Destination configurations:

  1. On the Connector dropdown menu, select Elasticsearch.
  2. On URL, add the URL of your endpoint you copied on Elasticsearch. Example: https://my-deploy-pg4by0.es.us-central1.gcp.cloud.es.io
  3. On Encoded API Key, add the Base64 key corresponding to the "encoded" value generated while creating the API Key in ElasticSearch. Example: VnVhQ2ZHY0JDZGJrUW0tZTVhT3g6dWkybHAyYXhUTm1zeWFrdzl0dk5udw==
  4. Make sure the Active switch is on.
  5. Click the Save button.

After saving the configurations, your data will be streamed to the newly configured endpoint.

You can keep track of the calls made by Data Stream to Elasticsearch on Real-Time Events. To do so, select Data Source > Data Stream and choose the filters options as you wish.

Trademarks

Elasticsearch is a registered trademark of Elasticsearch B.V. in the United States and other countries.