delivery_streams

Creates, updates, deletes or gets a delivery_stream resource or lists delivery_streams in a region

Overview

Name	`delivery_streams`
Type	Resource
Description	Resource Type definition for AWS::KinesisFirehose::DeliveryStream
Id	`aws.kinesisfirehose.delivery_streams`

Fields

Name	Datatype	Description
`arn`	`string`
`delivery_stream_encryption_configuration_input`	`object`
`delivery_stream_name`	`string`
`delivery_stream_type`	`string`
`elasticsearch_destination_configuration`	`object`
`amazonopensearchservice_destination_configuration`	`object`
`amazon_open_search_serverless_destination_configuration`	`object`
`extended_s3_destination_configuration`	`object`
`kinesis_stream_source_configuration`	`object`
`msk_source_configuration`	`object`
`redshift_destination_configuration`	`object`
`s3_destination_configuration`	`object`
`splunk_destination_configuration`	`object`
`http_endpoint_destination_configuration`	`object`
`snowflake_destination_configuration`	`object`
`tags`	`array`
`region`	`string`	AWS region.

Methods

Name	Accessible by	Required Params
`create_resource`	`INSERT`	`region`
`delete_resource`	`DELETE`	`data__Identifier, region`
`update_resource`	`UPDATE`	`data__Identifier, data__PatchDocument, region`
`list_resources`	`SELECT`	`region`
`get_resource`	`SELECT`	`data__Identifier, region`

`SELECT` examples

Gets all delivery_streams in a region.

SELECT
region,
arn,
delivery_stream_encryption_configuration_input,
delivery_stream_name,
delivery_stream_type,
elasticsearch_destination_configuration,
amazonopensearchservice_destination_configuration,
amazon_open_search_serverless_destination_configuration,
extended_s3_destination_configuration,
kinesis_stream_source_configuration,
msk_source_configuration,
redshift_destination_configuration,
s3_destination_configuration,
splunk_destination_configuration,
http_endpoint_destination_configuration,
snowflake_destination_configuration,
tags
FROM aws.kinesisfirehose.delivery_streams
WHERE region = 'us-east-1';

Gets all properties from an individual delivery_stream.

SELECT
region,
arn,
delivery_stream_encryption_configuration_input,
delivery_stream_name,
delivery_stream_type,
elasticsearch_destination_configuration,
amazonopensearchservice_destination_configuration,
amazon_open_search_serverless_destination_configuration,
extended_s3_destination_configuration,
kinesis_stream_source_configuration,
msk_source_configuration,
redshift_destination_configuration,
s3_destination_configuration,
splunk_destination_configuration,
http_endpoint_destination_configuration,
snowflake_destination_configuration,
tags
FROM aws.kinesisfirehose.delivery_streams
WHERE region = 'us-east-1' AND data__Identifier = '<DeliveryStreamName>';

`INSERT` example

Use the following StackQL query and manifest file to create a new delivery_stream resource, using stack-deploy.

Required Properties
All Properties
Manifest

/*+ create */
INSERT INTO aws.kinesisfirehose.delivery_streams (
 DeliveryStreamEncryptionConfigurationInput,
 DeliveryStreamName,
 DeliveryStreamType,
 ElasticsearchDestinationConfiguration,
 AmazonopensearchserviceDestinationConfiguration,
 AmazonOpenSearchServerlessDestinationConfiguration,
 ExtendedS3DestinationConfiguration,
 KinesisStreamSourceConfiguration,
 MSKSourceConfiguration,
 RedshiftDestinationConfiguration,
 S3DestinationConfiguration,
 SplunkDestinationConfiguration,
 HttpEndpointDestinationConfiguration,
 SnowflakeDestinationConfiguration,
 Tags,
 region
)
SELECT 
'{{ DeliveryStreamEncryptionConfigurationInput }}',
 '{{ DeliveryStreamName }}',
 '{{ DeliveryStreamType }}',
 '{{ ElasticsearchDestinationConfiguration }}',
 '{{ AmazonopensearchserviceDestinationConfiguration }}',
 '{{ AmazonOpenSearchServerlessDestinationConfiguration }}',
 '{{ ExtendedS3DestinationConfiguration }}',
 '{{ KinesisStreamSourceConfiguration }}',
 '{{ MSKSourceConfiguration }}',
 '{{ RedshiftDestinationConfiguration }}',
 '{{ S3DestinationConfiguration }}',
 '{{ SplunkDestinationConfiguration }}',
 '{{ HttpEndpointDestinationConfiguration }}',
 '{{ SnowflakeDestinationConfiguration }}',
 '{{ Tags }}',
'{{ region }}';

/*+ create */
INSERT INTO aws.kinesisfirehose.delivery_streams (
 DeliveryStreamEncryptionConfigurationInput,
 DeliveryStreamName,
 DeliveryStreamType,
 ElasticsearchDestinationConfiguration,
 AmazonopensearchserviceDestinationConfiguration,
 AmazonOpenSearchServerlessDestinationConfiguration,
 ExtendedS3DestinationConfiguration,
 KinesisStreamSourceConfiguration,
 MSKSourceConfiguration,
 RedshiftDestinationConfiguration,
 S3DestinationConfiguration,
 SplunkDestinationConfiguration,
 HttpEndpointDestinationConfiguration,
 SnowflakeDestinationConfiguration,
 Tags,
 region
)
SELECT 
 '{{ DeliveryStreamEncryptionConfigurationInput }}',
 '{{ DeliveryStreamName }}',
 '{{ DeliveryStreamType }}',
 '{{ ElasticsearchDestinationConfiguration }}',
 '{{ AmazonopensearchserviceDestinationConfiguration }}',
 '{{ AmazonOpenSearchServerlessDestinationConfiguration }}',
 '{{ ExtendedS3DestinationConfiguration }}',
 '{{ KinesisStreamSourceConfiguration }}',
 '{{ MSKSourceConfiguration }}',
 '{{ RedshiftDestinationConfiguration }}',
 '{{ S3DestinationConfiguration }}',
 '{{ SplunkDestinationConfiguration }}',
 '{{ HttpEndpointDestinationConfiguration }}',
 '{{ SnowflakeDestinationConfiguration }}',
 '{{ Tags }}',
 '{{ region }}';

version: 1
name: stack name
description: stack description
providers:
  - aws
globals:
  - name: region
    value: '{{ vars.AWS_REGION }}'
resources:
  - name: delivery_stream
    props:
      - name: DeliveryStreamEncryptionConfigurationInput
        value:
          KeyARN: '{{ KeyARN }}'
          KeyType: '{{ KeyType }}'
      - name: DeliveryStreamName
        value: '{{ DeliveryStreamName }}'
      - name: DeliveryStreamType
        value: '{{ DeliveryStreamType }}'
      - name: ElasticsearchDestinationConfiguration
        value:
          BufferingHints:
            IntervalInSeconds: '{{ IntervalInSeconds }}'
            SizeInMBs: '{{ SizeInMBs }}'
          CloudWatchLoggingOptions:
            Enabled: '{{ Enabled }}'
            LogGroupName: '{{ LogGroupName }}'
            LogStreamName: '{{ LogStreamName }}'
          DomainARN: '{{ DomainARN }}'
          IndexName: '{{ IndexName }}'
          IndexRotationPeriod: '{{ IndexRotationPeriod }}'
          ProcessingConfiguration:
            Enabled: '{{ Enabled }}'
            Processors:
              - Parameters:
                  - ParameterName: '{{ ParameterName }}'
                    ParameterValue: '{{ ParameterValue }}'
                Type: '{{ Type }}'
          RetryOptions:
            DurationInSeconds: '{{ DurationInSeconds }}'
          RoleARN: '{{ RoleARN }}'
          S3BackupMode: '{{ S3BackupMode }}'
          S3Configuration:
            BucketARN: '{{ BucketARN }}'
            BufferingHints:
              IntervalInSeconds: '{{ IntervalInSeconds }}'
              SizeInMBs: '{{ SizeInMBs }}'
            CloudWatchLoggingOptions: null
            CompressionFormat: '{{ CompressionFormat }}'
            EncryptionConfiguration:
              KMSEncryptionConfig:
                AWSKMSKeyARN: '{{ AWSKMSKeyARN }}'
              NoEncryptionConfig: '{{ NoEncryptionConfig }}'
            ErrorOutputPrefix: '{{ ErrorOutputPrefix }}'
            Prefix: '{{ Prefix }}'
            RoleARN: '{{ RoleARN }}'
          ClusterEndpoint: '{{ ClusterEndpoint }}'
          TypeName: '{{ TypeName }}'
          VpcConfiguration:
            RoleARN: '{{ RoleARN }}'
            SubnetIds:
              - '{{ SubnetIds[0] }}'
            SecurityGroupIds:
              - '{{ SecurityGroupIds[0] }}'
          DocumentIdOptions:
            DefaultDocumentIdFormat: '{{ DefaultDocumentIdFormat }}'
      - name: AmazonopensearchserviceDestinationConfiguration
        value:
          BufferingHints:
            IntervalInSeconds: '{{ IntervalInSeconds }}'
            SizeInMBs: '{{ SizeInMBs }}'
          CloudWatchLoggingOptions: null
          DomainARN: '{{ DomainARN }}'
          IndexName: '{{ IndexName }}'
          IndexRotationPeriod: '{{ IndexRotationPeriod }}'
          ProcessingConfiguration: null
          RetryOptions:
            DurationInSeconds: '{{ DurationInSeconds }}'
          RoleARN: '{{ RoleARN }}'
          S3BackupMode: '{{ S3BackupMode }}'
          S3Configuration: null
          ClusterEndpoint: '{{ ClusterEndpoint }}'
          TypeName: '{{ TypeName }}'
          VpcConfiguration: null
          DocumentIdOptions: null
      - name: AmazonOpenSearchServerlessDestinationConfiguration
        value:
          BufferingHints:
            IntervalInSeconds: '{{ IntervalInSeconds }}'
            SizeInMBs: '{{ SizeInMBs }}'
          CloudWatchLoggingOptions: null
          IndexName: '{{ IndexName }}'
          ProcessingConfiguration: null
          RetryOptions:
            DurationInSeconds: '{{ DurationInSeconds }}'
          RoleARN: '{{ RoleARN }}'
          S3BackupMode: '{{ S3BackupMode }}'
          S3Configuration: null
          CollectionEndpoint: '{{ CollectionEndpoint }}'
          VpcConfiguration: null
      - name: ExtendedS3DestinationConfiguration
        value:
          BucketARN: '{{ BucketARN }}'
          BufferingHints: null
          CloudWatchLoggingOptions: null
          CompressionFormat: '{{ CompressionFormat }}'
          CustomTimeZone: '{{ CustomTimeZone }}'
          DataFormatConversionConfiguration:
            Enabled: '{{ Enabled }}'
            InputFormatConfiguration:
              Deserializer:
                HiveJsonSerDe:
                  TimestampFormats:
                    - '{{ TimestampFormats[0] }}'
                OpenXJsonSerDe:
                  CaseInsensitive: '{{ CaseInsensitive }}'
                  ColumnToJsonKeyMappings: {}
                  ConvertDotsInJsonKeysToUnderscores: '{{ ConvertDotsInJsonKeysToUnderscores }}'
            OutputFormatConfiguration:
              Serializer:
                OrcSerDe:
                  BlockSizeBytes: '{{ BlockSizeBytes }}'
                  BloomFilterColumns:
                    - '{{ BloomFilterColumns[0] }}'
                  BloomFilterFalsePositiveProbability: null
                  Compression: '{{ Compression }}'
                  DictionaryKeyThreshold: null
                  EnablePadding: '{{ EnablePadding }}'
                  FormatVersion: '{{ FormatVersion }}'
                  PaddingTolerance: null
                  RowIndexStride: '{{ RowIndexStride }}'
                  StripeSizeBytes: '{{ StripeSizeBytes }}'
                ParquetSerDe:
                  BlockSizeBytes: '{{ BlockSizeBytes }}'
                  Compression: '{{ Compression }}'
                  EnableDictionaryCompression: '{{ EnableDictionaryCompression }}'
                  MaxPaddingBytes: '{{ MaxPaddingBytes }}'
                  PageSizeBytes: '{{ PageSizeBytes }}'
                  WriterVersion: '{{ WriterVersion }}'
            SchemaConfiguration:
              CatalogId: '{{ CatalogId }}'
              DatabaseName: '{{ DatabaseName }}'
              Region: '{{ Region }}'
              RoleARN: '{{ RoleARN }}'
              TableName: '{{ TableName }}'
              VersionId: '{{ VersionId }}'
          DynamicPartitioningConfiguration:
            Enabled: '{{ Enabled }}'
            RetryOptions:
              DurationInSeconds: '{{ DurationInSeconds }}'
          EncryptionConfiguration: null
          ErrorOutputPrefix: '{{ ErrorOutputPrefix }}'
          FileExtension: '{{ FileExtension }}'
          Prefix: '{{ Prefix }}'
          ProcessingConfiguration: null
          RoleARN: '{{ RoleARN }}'
          S3BackupConfiguration: null
          S3BackupMode: '{{ S3BackupMode }}'
      - name: KinesisStreamSourceConfiguration
        value:
          KinesisStreamARN: '{{ KinesisStreamARN }}'
          RoleARN: '{{ RoleARN }}'
      - name: MSKSourceConfiguration
        value:
          MSKClusterARN: '{{ MSKClusterARN }}'
          TopicName: '{{ TopicName }}'
          AuthenticationConfiguration:
            RoleARN: '{{ RoleARN }}'
            Connectivity: '{{ Connectivity }}'
      - name: RedshiftDestinationConfiguration
        value:
          CloudWatchLoggingOptions: null
          ClusterJDBCURL: '{{ ClusterJDBCURL }}'
          CopyCommand:
            CopyOptions: '{{ CopyOptions }}'
            DataTableColumns: '{{ DataTableColumns }}'
            DataTableName: '{{ DataTableName }}'
          Password: '{{ Password }}'
          ProcessingConfiguration: null
          RetryOptions:
            DurationInSeconds: '{{ DurationInSeconds }}'
          RoleARN: '{{ RoleARN }}'
          S3BackupConfiguration: null
          S3BackupMode: '{{ S3BackupMode }}'
          S3Configuration: null
          Username: '{{ Username }}'
      - name: S3DestinationConfiguration
        value: null
      - name: SplunkDestinationConfiguration
        value:
          CloudWatchLoggingOptions: null
          HECAcknowledgmentTimeoutInSeconds: '{{ HECAcknowledgmentTimeoutInSeconds }}'
          HECEndpoint: '{{ HECEndpoint }}'
          HECEndpointType: '{{ HECEndpointType }}'
          HECToken: '{{ HECToken }}'
          ProcessingConfiguration: null
          RetryOptions:
            DurationInSeconds: '{{ DurationInSeconds }}'
          S3BackupMode: '{{ S3BackupMode }}'
          S3Configuration: null
          BufferingHints:
            IntervalInSeconds: '{{ IntervalInSeconds }}'
            SizeInMBs: '{{ SizeInMBs }}'
      - name: HttpEndpointDestinationConfiguration
        value:
          RoleARN: '{{ RoleARN }}'
          EndpointConfiguration:
            Url: '{{ Url }}'
            AccessKey: '{{ AccessKey }}'
            Name: '{{ Name }}'
          RequestConfiguration:
            ContentEncoding: '{{ ContentEncoding }}'
            CommonAttributes:
              - AttributeName: '{{ AttributeName }}'
                AttributeValue: '{{ AttributeValue }}'
          BufferingHints: null
          CloudWatchLoggingOptions: null
          ProcessingConfiguration: null
          RetryOptions: null
          S3BackupMode: '{{ S3BackupMode }}'
          S3Configuration: null
      - name: SnowflakeDestinationConfiguration
        value:
          AccountUrl: '{{ AccountUrl }}'
          PrivateKey: '{{ PrivateKey }}'
          KeyPassphrase: '{{ KeyPassphrase }}'
          User: '{{ User }}'
          Database: '{{ Database }}'
          Schema: '{{ Schema }}'
          Table: '{{ Table }}'
          SnowflakeRoleConfiguration:
            Enabled: '{{ Enabled }}'
            SnowflakeRole: '{{ SnowflakeRole }}'
          DataLoadingOption: '{{ DataLoadingOption }}'
          MetaDataColumnName: '{{ MetaDataColumnName }}'
          ContentColumnName: '{{ ContentColumnName }}'
          SnowflakeVpcConfiguration:
            PrivateLinkVpceId: '{{ PrivateLinkVpceId }}'
          CloudWatchLoggingOptions: null
          ProcessingConfiguration: null
          RoleARN: '{{ RoleARN }}'
          RetryOptions:
            DurationInSeconds: '{{ DurationInSeconds }}'
          S3BackupMode: '{{ S3BackupMode }}'
          S3Configuration: null
      - name: Tags
        value:
          - Key: '{{ Key }}'
            Value: '{{ Value }}'

`DELETE` example

/*+ delete */
DELETE FROM aws.kinesisfirehose.delivery_streams
WHERE data__Identifier = '<DeliveryStreamName>'
AND region = 'us-east-1';

Permissions

To operate on the delivery_streams resource, the following permissions are required:

Create

firehose:CreateDeliveryStream,
firehose:DescribeDeliveryStream,
iam:GetRole,
iam:PassRole,
kms:CreateGrant,
kms:DescribeKey

Read

firehose:DescribeDeliveryStream,
firehose:ListTagsForDeliveryStream

Update

firehose:UpdateDestination,
firehose:DescribeDeliveryStream,
firehose:StartDeliveryStreamEncryption,
firehose:StopDeliveryStreamEncryption,
firehose:ListTagsForDeliveryStream,
firehose:TagDeliveryStream,
firehose:UntagDeliveryStream,
kms:CreateGrant,
kms:RevokeGrant,
kms:DescribeKey

Delete

firehose:DeleteDeliveryStream,
firehose:DescribeDeliveryStream,
kms:RevokeGrant,
kms:DescribeKey

List

firehose:ListDeliveryStreams

Overview​

Fields​

Methods​

SELECT examples​

INSERT example​

DELETE example​

Permissions​

Create​

Read​

Update​

Delete​

List​