location_hdfs

Creates, updates, deletes or gets a location_hdf resource or lists location_hdfs in a region

Overview

Name	location_hdfs
Type	Resource
Description	Resource schema for AWS::DataSync::LocationHDFS.
Id	aws.datasync.location_hdfs

Fields

Name	Datatype	Description
name_nodes	array	An array of Name Node(s) of the HDFS location.
block_size	integer	Size of chunks (blocks) in bytes that the data is divided into when stored in the HDFS cluster.
replication_factor	integer	Number of copies of each block that exists inside the HDFS cluster.
kms_key_provider_uri	string	The identifier for the Key Management Server where the encryption keys that encrypt data inside HDFS clusters are stored.
qop_configuration	object	Configuration information for RPC Protection and Data Transfer Protection. These parameters can be set to AUTHENTICATION, INTEGRITY, or PRIVACY. The default value is PRIVACY.
authentication_type	string	The authentication mode used to determine identity of user.
simple_user	string	The user name that has read and write permissions on the specified HDFS cluster.
kerberos_principal	string	The unique identity, or principal, to which Kerberos can assign tickets.
kerberos_keytab	string	The Base64 string representation of the Keytab file.
kerberos_krb5_conf	string	The string representation of the Krb5Conf file, or the presigned URL to access the Krb5.conf file within an S3 bucket.
tags	array	An array of key-value pairs to apply to this resource.
agent_arns	array	ARN(s) of the agent(s) to use for an HDFS location.
subdirectory	string	The subdirectory in HDFS that is used to read data from the HDFS source location or write data to the HDFS destination.
location_arn	string	The Amazon Resource Name (ARN) of the HDFS location.
location_uri	string	The URL of the HDFS location that was described.
region	string	AWS region.

Methods

Name	Accessible by	Required Params
create_resource	INSERT	NameNodes, AuthenticationType, AgentArns, region
delete_resource	DELETE	data__Identifier, region
update_resource	UPDATE	data__Identifier, data__PatchDocument, region
list_resources	SELECT	region
get_resource	SELECT	data__Identifier, region

SELECT examples

Gets all location_hdfs in a region.

SELECT
region,
name_nodes,
block_size,
replication_factor,
kms_key_provider_uri,
qop_configuration,
authentication_type,
simple_user,
kerberos_principal,
kerberos_keytab,
kerberos_krb5_conf,
tags,
agent_arns,
subdirectory,
location_arn,
location_uri
FROM aws.datasync.location_hdfs
WHERE region = 'us-east-1';

Gets all properties from an individual location_hdf.

SELECT
region,
name_nodes,
block_size,
replication_factor,
kms_key_provider_uri,
qop_configuration,
authentication_type,
simple_user,
kerberos_principal,
kerberos_keytab,
kerberos_krb5_conf,
tags,
agent_arns,
subdirectory,
location_arn,
location_uri
FROM aws.datasync.location_hdfs
WHERE region = 'us-east-1' AND data__Identifier = '<LocationArn>';

INSERT example

Use the following StackQL query and manifest file to create a new location_hdf resource, using stack-deploy.

Required Properties

/*+ create */
INSERT INTO aws.datasync.location_hdfs (
 NameNodes,
 AuthenticationType,
 AgentArns,
 region
)
SELECT 
'{{ NameNodes }}',
 '{{ AuthenticationType }}',
 '{{ AgentArns }}',
'{{ region }}';

All Properties

/*+ create */
INSERT INTO aws.datasync.location_hdfs (
 NameNodes,
 BlockSize,
 ReplicationFactor,
 KmsKeyProviderUri,
 QopConfiguration,
 AuthenticationType,
 SimpleUser,
 KerberosPrincipal,
 KerberosKeytab,
 KerberosKrb5Conf,
 Tags,
 AgentArns,
 Subdirectory,
 region
)
SELECT 
 '{{ NameNodes }}',
 '{{ BlockSize }}',
 '{{ ReplicationFactor }}',
 '{{ KmsKeyProviderUri }}',
 '{{ QopConfiguration }}',
 '{{ AuthenticationType }}',
 '{{ SimpleUser }}',
 '{{ KerberosPrincipal }}',
 '{{ KerberosKeytab }}',
 '{{ KerberosKrb5Conf }}',
 '{{ Tags }}',
 '{{ AgentArns }}',
 '{{ Subdirectory }}',
 '{{ region }}';

Manifest

version: 1
name: stack name
description: stack description
providers:
  - aws
globals:
  - name: region
    value: '{{ vars.AWS_REGION }}'
resources:
  - name: location_hdf
    props:
      - name: NameNodes
        value:
          - Hostname: '{{ Hostname }}'
            Port: '{{ Port }}'
      - name: BlockSize
        value: '{{ BlockSize }}'
      - name: ReplicationFactor
        value: '{{ ReplicationFactor }}'
      - name: KmsKeyProviderUri
        value: '{{ KmsKeyProviderUri }}'
      - name: QopConfiguration
        value:
          RpcProtection: '{{ RpcProtection }}'
          DataTransferProtection: '{{ DataTransferProtection }}'
      - name: AuthenticationType
        value: '{{ AuthenticationType }}'
      - name: SimpleUser
        value: '{{ SimpleUser }}'
      - name: KerberosPrincipal
        value: '{{ KerberosPrincipal }}'
      - name: KerberosKeytab
        value: '{{ KerberosKeytab }}'
      - name: KerberosKrb5Conf
        value: '{{ KerberosKrb5Conf }}'
      - name: Tags
        value:
          - Key: '{{ Key }}'
            Value: '{{ Value }}'
      - name: AgentArns
        value:
          - '{{ AgentArns[0] }}'
      - name: Subdirectory
        value: '{{ Subdirectory }}'

DELETE example

/*+ delete */
DELETE FROM aws.datasync.location_hdfs
WHERE data__Identifier = '<LocationArn>'
AND region = 'us-east-1';

Permissions

To operate on the location_hdfs resource, the following permissions are required:

Create

datasync:CreateLocationHdfs,
datasync:DescribeLocationHdfs,
datasync:TagResource,
datasync:ListTagsForResource

Read

datasync:DescribeLocationHdfs,
datasync:ListTagsForResource

Update

datasync:UpdateLocationHdfs,
datasync:DescribeLocationHdfs,
datasync:ListTagsForResource,
datasync:TagResource,
datasync:UntagResource

Delete

datasync:DeleteLocation

List

datasync:ListLocations