Send Data Stream Data to Amazon Kinesis

Hands-on to use Amazon Kinesis Data Firehose to receive data from Azion Data Stream.

View as Markdown

While configuring streams, you need to set up a specific endpoint to stream your Azion data.

Continue reading for a step by step on how to connect an Amazon Kinesis Data Firehose endpoint to receive data from Data Stream.

Amazon Kinesis Data Firehose requirements

To start using Amazon Kinesis Data Firehose, follow the steps:

  1. Create an account on Amazon AWS.
  2. In the AWS dashboard, create an instance.
  3. Fill in the requested information.
  4. Create an Amazon S3 bucket.

You’ll also need to create and save the following information:

  • Stream Name
  • Region
  • Access Key
  • Secret Key

See how to obtain them while still on the AWS platform:

Stream Name

  1. Go to the Amazon Kinesis console homepage.
  2. On Get started, select Kinesis Data Firehose as an option.
  3. Click the Create delivery stream button. You’ll be redirected to your stream creation page.
  4. In the Choose source and destination section, on Source, select Direct PUT.
  5. On Destination, select Amazon S3. The rest of the configuration options will load.
  6. In the Delivery stream name section, choose a delivery stream name according to what you find most convenient.
  7. Keep the other options with the default value.
  8. In the Destination Settings section, on S3 bucket, click browse and select a previously created S3 bucket. You can also click the Create button and create a new one now.
  9. Keep the other options with the default value.
  10. Click on Create a delivery flow.

You can find more details on the AWS Creating the Kinesis Data Firehose delivery stream documentation.

Region

This is the region where your AWS Kinesis instance is running.

  1. On the top bar of your dashboard, right next to the help button with the ’?’ icon, you find a dropdown menu, the Regions selector, where you can see the name of the region and its code.
  2. Find the code and keep it to use it later. It looks similar to this: us-east-1

You can find more details on the AWS Regions and Zones documentation.

Access Key and Secret Key

  1. Access the Identify and Access Management (IAM) panel > click the Create new access key button.
  2. A pop-up will appear on the screen with your key information.
  3. Click on Display Access Key to display your key information as indicated below:
    • Access Key ID, with a format similar to AKIA5LVCRTER524RAIA3
    • Secret Access Key, with a format similar to N9UTjunb299M5ryfKdG2cKh2+Pe2mBiGZXaBpVff
  4. Save the two keys information in a safe place to use them later on Azion Data Stream.

You can also download the EMP file with the information for the keys.

Configuring the endpoint in Data Stream

Next, follow these steps to configure the new endpoint you created in Amazon Kinesis Data Firehose in your Azion Data Stream.

You can find detailed steps for the entire configuration on the How to use Data Stream guide.

All key fields will be masked and an icon that allows revealing the field content will be only displayed to users with the Edit Data Stream permission. Users without the Edit Data Stream permission can only view the settings of the existing stream but not edit or create new ones.

In the Destination configurations:

  1. On the Connector dropdown menu, select AWS Kinesis Data Firehose.
  2. On Stream Name, add the name you gave your delivery Kinesis Firehose stream.
  3. On Region, add the region configured on your stream.
  4. On Access Key, add the access key you created for your stream.
  5. On Secret Key, add the secret key you created for your stream.
  6. Make sure the Active switch is on.
  7. Click the Save button.

After saving the configurations, your data will be streamed to the newly configured endpoint.

You can keep track of the calls made by Data Stream to Amazon Kinesis Data Firehose on Real-Time Events. To do so, select Data Source > Data Stream and choose the filters options as you wish.

Watch a video tutorial on how to use Amazon Kinesis Data Firehose with Data Stream on Azion’s YouTube channel:

Trademarks

Amazon Web Services, AWS, Amazon Kinesis and affiliates are registered trademarks of Amazon Inc in the United States and other countries.