data_quality_job_definitions

Creates, updates, deletes or gets a data_quality_job_definition resource or lists data_quality_job_definitions in a region

Overview

Name	data_quality_job_definitions
Type	Resource
Description	Resource Type definition for AWS::SageMaker::DataQualityJobDefinition
Id	aws.sagemaker.data_quality_job_definitions

Fields

Name	Datatype	Description
job_definition_arn	string	The Amazon Resource Name (ARN) of job definition.
job_definition_name	string	The name of the job definition.
data_quality_baseline_config	object	Baseline configuration used to validate that the data conforms to the specified constraints and statistics.
data_quality_app_specification	object	Container image configuration object for the monitoring job.
data_quality_job_input	object	The inputs for a monitoring job.
data_quality_job_output_config	object	The output configuration for monitoring jobs.
job_resources	object	Identifies the resources to deploy for a monitoring job.
network_config	object	Networking options for a job, such as network traffic encryption between containers, whether to allow inbound and outbound network calls to and from containers, and the VPC subnets and security groups to use for VPC-enabled jobs.
endpoint_name	string	The name of the endpoint used to run the monitoring job.
role_arn	string	The Amazon Resource Name (ARN) of an IAM role that Amazon SageMaker can assume to perform tasks on your behalf.
stopping_condition	object	Specifies a time limit for how long the monitoring job is allowed to run.
tags	array	An array of key-value pairs to apply to this resource.
creation_time	string	The time at which the job definition was created.
region	string	AWS region.

Methods

Name	Accessible by	Required Params
create_resource	INSERT	DataQualityAppSpecification, DataQualityJobInput, DataQualityJobOutputConfig, JobResources, RoleArn, region
delete_resource	DELETE	data__Identifier, region
list_resource	SELECT	region
get_resource	SELECT	data__Identifier, region

SELECT examples

List all data_quality_job_definitions in a region.

SELECT
region,
job_definition_arn
FROM aws.sagemaker.data_quality_job_definitions
WHERE region = 'us-east-1';

Gets all properties from a data_quality_job_definition.

SELECT
region,
job_definition_arn,
job_definition_name,
data_quality_baseline_config,
data_quality_app_specification,
data_quality_job_input,
data_quality_job_output_config,
job_resources,
network_config,
endpoint_name,
role_arn,
stopping_condition,
tags,
creation_time
FROM aws.sagemaker.data_quality_job_definitions
WHERE region = 'us-east-1' AND data__Identifier = '<JobDefinitionArn>';

INSERT example

Use the following StackQL query and manifest file to create a new data_quality_job_definition resource, using stack-deploy.

Required Properties

/*+ create */
INSERT INTO aws.sagemaker.data_quality_job_definitions (
 DataQualityAppSpecification,
 DataQualityJobInput,
 DataQualityJobOutputConfig,
 JobResources,
 RoleArn,
 region
)
SELECT 
'{{ DataQualityAppSpecification }}',
 '{{ DataQualityJobInput }}',
 '{{ DataQualityJobOutputConfig }}',
 '{{ JobResources }}',
 '{{ RoleArn }}',
'{{ region }}';

All Properties

/*+ create */
INSERT INTO aws.sagemaker.data_quality_job_definitions (
 JobDefinitionName,
 DataQualityBaselineConfig,
 DataQualityAppSpecification,
 DataQualityJobInput,
 DataQualityJobOutputConfig,
 JobResources,
 NetworkConfig,
 EndpointName,
 RoleArn,
 StoppingCondition,
 Tags,
 region
)
SELECT 
 '{{ JobDefinitionName }}',
 '{{ DataQualityBaselineConfig }}',
 '{{ DataQualityAppSpecification }}',
 '{{ DataQualityJobInput }}',
 '{{ DataQualityJobOutputConfig }}',
 '{{ JobResources }}',
 '{{ NetworkConfig }}',
 '{{ EndpointName }}',
 '{{ RoleArn }}',
 '{{ StoppingCondition }}',
 '{{ Tags }}',
 '{{ region }}';

Manifest

version: 1
name: stack name
description: stack description
providers:
  - aws
globals:
  - name: region
    value: '{{ vars.AWS_REGION }}'
resources:
  - name: data_quality_job_definition
    props:
      - name: JobDefinitionName
        value: '{{ JobDefinitionName }}'
      - name: DataQualityBaselineConfig
        value:
          BaseliningJobName: '{{ BaseliningJobName }}'
          ConstraintsResource:
            S3Uri: '{{ S3Uri }}'
          StatisticsResource:
            S3Uri: null
      - name: DataQualityAppSpecification
        value:
          ContainerArguments:
            - '{{ ContainerArguments[0] }}'
          ContainerEntrypoint:
            - '{{ ContainerEntrypoint[0] }}'
          ImageUri: '{{ ImageUri }}'
          PostAnalyticsProcessorSourceUri: null
          RecordPreprocessorSourceUri: null
          Environment: {}
      - name: DataQualityJobInput
        value:
          EndpointInput:
            EndpointName: '{{ EndpointName }}'
            LocalPath: '{{ LocalPath }}'
            S3DataDistributionType: '{{ S3DataDistributionType }}'
            S3InputMode: '{{ S3InputMode }}'
            ExcludeFeaturesAttribute: '{{ ExcludeFeaturesAttribute }}'
          BatchTransformInput:
            DataCapturedDestinationS3Uri: '{{ DataCapturedDestinationS3Uri }}'
            DatasetFormat:
              Csv:
                Header: '{{ Header }}'
              Json:
                Line: '{{ Line }}'
              Parquet: '{{ Parquet }}'
            LocalPath: '{{ LocalPath }}'
            S3DataDistributionType: '{{ S3DataDistributionType }}'
            S3InputMode: '{{ S3InputMode }}'
            ExcludeFeaturesAttribute: '{{ ExcludeFeaturesAttribute }}'
      - name: DataQualityJobOutputConfig
        value:
          KmsKeyId: '{{ KmsKeyId }}'
          MonitoringOutputs:
            - S3Output:
                LocalPath: '{{ LocalPath }}'
                S3UploadMode: '{{ S3UploadMode }}'
                S3Uri: '{{ S3Uri }}'
      - name: JobResources
        value:
          ClusterConfig:
            InstanceCount: '{{ InstanceCount }}'
            InstanceType: '{{ InstanceType }}'
            VolumeKmsKeyId: '{{ VolumeKmsKeyId }}'
            VolumeSizeInGB: '{{ VolumeSizeInGB }}'
      - name: NetworkConfig
        value:
          EnableInterContainerTrafficEncryption: '{{ EnableInterContainerTrafficEncryption }}'
          EnableNetworkIsolation: '{{ EnableNetworkIsolation }}'
          VpcConfig:
            SecurityGroupIds:
              - '{{ SecurityGroupIds[0] }}'
            Subnets:
              - '{{ Subnets[0] }}'
      - name: EndpointName
        value: null
      - name: RoleArn
        value: '{{ RoleArn }}'
      - name: StoppingCondition
        value:
          MaxRuntimeInSeconds: '{{ MaxRuntimeInSeconds }}'
      - name: Tags
        value:
          - Value: '{{ Value }}'
            Key: '{{ Key }}'

DELETE example

/*+ delete */
DELETE FROM aws.sagemaker.data_quality_job_definitions
WHERE data__Identifier = '<JobDefinitionArn>'
AND region = 'us-east-1';

Permissions

To operate on the data_quality_job_definitions resource, the following permissions are required:

Create

sagemaker:CreateDataQualityJobDefinition,
sagemaker:DescribeDataQualityJobDefinition,
sagemaker:AddTags,
iam:PassRole

Delete

sagemaker:DeleteDataQualityJobDefinition

Read

sagemaker:DescribeDataQualityJobDefinition

List

sagemaker:ListDataQualityJobDefinitions,
sagemaker:ListTags