jobs

Creates, updates, deletes or gets a job resource or lists jobs in a region

Overview

Name	jobs
Type	Resource
Description	Resource schema for AWS::DataBrew::Job.
Id	aws.databrew.jobs

Fields

Name	Datatype	Description
dataset_name	string	Dataset name
encryption_key_arn	string	Encryption Key Arn
encryption_mode	string	Encryption mode
name	string	Job name
type	string	Job type
log_subscription	string	Log subscription
max_capacity	integer	Max capacity
max_retries	integer	Max retries
outputs	array
data_catalog_outputs	array
database_outputs	array
output_location	object	Output location
project_name	string	Project name
recipe	object	Resource schema for AWS::DataBrew::Recipe.
role_arn	string	Role arn
tags	array
timeout	integer	Timeout
job_sample	object	Job Sample
profile_configuration	object	Profile Job configuration
validation_configurations	array	Data quality rules configuration
region	string	AWS region.

Methods

Name	Accessible by	Required Params
create_resource	INSERT	Name, RoleArn, Type, region
delete_resource	DELETE	data__Identifier, region
update_resource	UPDATE	data__Identifier, data__PatchDocument, region
list_resources	SELECT	region
get_resource	SELECT	data__Identifier, region

SELECT examples

Gets all jobs in a region.

SELECT
region,
dataset_name,
encryption_key_arn,
encryption_mode,
name,
type,
log_subscription,
max_capacity,
max_retries,
outputs,
data_catalog_outputs,
database_outputs,
output_location,
project_name,
recipe,
role_arn,
tags,
timeout,
job_sample,
profile_configuration,
validation_configurations
FROM aws.databrew.jobs
WHERE region = 'us-east-1';

Gets all properties from an individual job.

SELECT
region,
dataset_name,
encryption_key_arn,
encryption_mode,
name,
type,
log_subscription,
max_capacity,
max_retries,
outputs,
data_catalog_outputs,
database_outputs,
output_location,
project_name,
recipe,
role_arn,
tags,
timeout,
job_sample,
profile_configuration,
validation_configurations
FROM aws.databrew.jobs
WHERE region = 'us-east-1' AND data__Identifier = '<Name>';

INSERT example

Use the following StackQL query and manifest file to create a new job resource, using stack-deploy.

Required Properties

/*+ create */
INSERT INTO aws.databrew.jobs (
 Name,
 Type,
 RoleArn,
 region
)
SELECT 
'{{ Name }}',
 '{{ Type }}',
 '{{ RoleArn }}',
'{{ region }}';

All Properties

/*+ create */
INSERT INTO aws.databrew.jobs (
 DatasetName,
 EncryptionKeyArn,
 EncryptionMode,
 Name,
 Type,
 LogSubscription,
 MaxCapacity,
 MaxRetries,
 Outputs,
 DataCatalogOutputs,
 DatabaseOutputs,
 OutputLocation,
 ProjectName,
 Recipe,
 RoleArn,
 Tags,
 Timeout,
 JobSample,
 ProfileConfiguration,
 ValidationConfigurations,
 region
)
SELECT 
 '{{ DatasetName }}',
 '{{ EncryptionKeyArn }}',
 '{{ EncryptionMode }}',
 '{{ Name }}',
 '{{ Type }}',
 '{{ LogSubscription }}',
 '{{ MaxCapacity }}',
 '{{ MaxRetries }}',
 '{{ Outputs }}',
 '{{ DataCatalogOutputs }}',
 '{{ DatabaseOutputs }}',
 '{{ OutputLocation }}',
 '{{ ProjectName }}',
 '{{ Recipe }}',
 '{{ RoleArn }}',
 '{{ Tags }}',
 '{{ Timeout }}',
 '{{ JobSample }}',
 '{{ ProfileConfiguration }}',
 '{{ ValidationConfigurations }}',
 '{{ region }}';

Manifest

version: 1
name: stack name
description: stack description
providers:
  - aws
globals:
  - name: region
    value: '{{ vars.AWS_REGION }}'
resources:
  - name: job
    props:
      - name: DatasetName
        value: '{{ DatasetName }}'
      - name: EncryptionKeyArn
        value: '{{ EncryptionKeyArn }}'
      - name: EncryptionMode
        value: '{{ EncryptionMode }}'
      - name: Name
        value: '{{ Name }}'
      - name: Type
        value: '{{ Type }}'
      - name: LogSubscription
        value: '{{ LogSubscription }}'
      - name: MaxCapacity
        value: '{{ MaxCapacity }}'
      - name: MaxRetries
        value: '{{ MaxRetries }}'
      - name: Outputs
        value:
          - CompressionFormat: '{{ CompressionFormat }}'
            Format: '{{ Format }}'
            FormatOptions:
              Csv:
                Delimiter: '{{ Delimiter }}'
            PartitionColumns:
              - '{{ PartitionColumns[0] }}'
            Location:
              Bucket: '{{ Bucket }}'
              Key: '{{ Key }}'
            Overwrite: '{{ Overwrite }}'
            MaxOutputFiles: '{{ MaxOutputFiles }}'
      - name: DataCatalogOutputs
        value:
          - CatalogId: '{{ CatalogId }}'
            DatabaseName: '{{ DatabaseName }}'
            TableName: '{{ TableName }}'
            S3Options:
              Location: null
            DatabaseOptions:
              TempDirectory: null
              TableName: '{{ TableName }}'
            Overwrite: '{{ Overwrite }}'
      - name: DatabaseOutputs
        value:
          - GlueConnectionName: '{{ GlueConnectionName }}'
            DatabaseOutputMode: '{{ DatabaseOutputMode }}'
            DatabaseOptions: null
      - name: OutputLocation
        value:
          Bucket: '{{ Bucket }}'
          Key: '{{ Key }}'
          BucketOwner: '{{ BucketOwner }}'
      - name: ProjectName
        value: '{{ ProjectName }}'
      - name: Recipe
        value:
          Description: '{{ Description }}'
          Name: '{{ Name }}'
          Steps:
            - Action:
                Operation: '{{ Operation }}'
                Parameters: null
              ConditionExpressions:
                - Condition: '{{ Condition }}'
                  Value: '{{ Value }}'
                  TargetColumn: '{{ TargetColumn }}'
          Tags:
            - Key: '{{ Key }}'
              Value: '{{ Value }}'
      - name: RoleArn
        value: '{{ RoleArn }}'
      - name: Tags
        value:
          - null
      - name: Timeout
        value: '{{ Timeout }}'
      - name: JobSample
        value:
          Mode: '{{ Mode }}'
          Size: '{{ Size }}'
      - name: ProfileConfiguration
        value:
          DatasetStatisticsConfiguration:
            IncludedStatistics:
              - '{{ IncludedStatistics[0] }}'
            Overrides:
              - Statistic: null
                Parameters: {}
          ProfileColumns:
            - Regex: '{{ Regex }}'
              Name: '{{ Name }}'
          ColumnStatisticsConfigurations:
            - Selectors:
                - null
              Statistics: null
          EntityDetectorConfiguration:
            EntityTypes:
              - '{{ EntityTypes[0] }}'
            AllowedStatistics:
              Statistics:
                - null
      - name: ValidationConfigurations
        value:
          - RulesetArn: '{{ RulesetArn }}'
            ValidationMode: '{{ ValidationMode }}'

DELETE example

/*+ delete */
DELETE FROM aws.databrew.jobs
WHERE data__Identifier = '<Name>'
AND region = 'us-east-1';

Permissions

To operate on the jobs resource, the following permissions are required:

Create

databrew:CreateProfileJob,
databrew:CreateRecipeJob,
databrew:TagResource,
databrew:UntagResource,
iam:PassRole

Read

databrew:DescribeJob,
databrew:ListTagsForResource,
iam:ListRoles

Update

databrew:UpdateProfileJob,
databrew:UpdateRecipeJob,
iam:PassRole

Delete

databrew:DeleteJob

List

databrew:ListJobs,
databrew:ListTagsForResource,
iam:ListRoles