jobs

Creates, updates, deletes or gets a job resource or lists jobs in a region

Overview

Name	jobs
Type	Resource
Description	Resource Type definition for AWS::Glue::Job
Id	aws.glue.jobs

Fields

Name	Datatype	Description
connections	object	Specifies the connections used by a job
max_retries	number	The maximum number of times to retry this job after a JobRun fails
description	string	A description of the job.
timeout	integer	The maximum time that a job run can consume resources before it is terminated and enters TIMEOUT status.
allocated_capacity	number	The number of capacity units that are allocated to this job.
name	string	The name you assign to the job definition
role	string	The name or Amazon Resource Name (ARN) of the IAM role associated with this job.
default_arguments	object	The default arguments for this job, specified as name-value pairs.
notification_property	object	Specifies configuration properties of a notification.
worker_type	string	TThe type of predefined worker that is allocated when a job runs.
execution_class	string	Indicates whether the job is run with a standard or flexible execution class.
log_uri	string	This field is reserved for future use.
command	object	The code that executes a job.
glue_version	string	Glue version determines the versions of Apache Spark and Python that AWS Glue supports.
execution_property	object	The maximum number of concurrent runs that are allowed for this job.
security_configuration	string	The name of the SecurityConfiguration structure to be used with this job.
number_of_workers	integer	The number of workers of a defined workerType that are allocated when a job runs.
tags	object	The tags to use with this job.
max_capacity	number	The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs.
non_overridable_arguments	object	Non-overridable arguments for this job, specified as name-value pairs.
maintenance_window	string	Property description not available.
job_mode	string	Property description not available.
job_run_queuing_enabled	boolean	Property description not available.
region	string	AWS region.

For more information, see AWS::Glue::Job.

Methods

Name	Accessible by	Required Params
create_resource	INSERT	Role, Command, region
delete_resource	DELETE	data__Identifier, region
update_resource	UPDATE	data__Identifier, data__PatchDocument, region
list_resources	SELECT	region
get_resource	SELECT	data__Identifier, region

SELECT examples

Gets all jobs in a region.

SELECT
region,
connections,
max_retries,
description,
timeout,
allocated_capacity,
name,
role,
default_arguments,
notification_property,
worker_type,
execution_class,
log_uri,
command,
glue_version,
execution_property,
security_configuration,
number_of_workers,
tags,
max_capacity,
non_overridable_arguments,
maintenance_window,
job_mode,
job_run_queuing_enabled
FROM aws.glue.jobs
WHERE region = 'us-east-1';

Gets all properties from an individual job.

SELECT
region,
connections,
max_retries,
description,
timeout,
allocated_capacity,
name,
role,
default_arguments,
notification_property,
worker_type,
execution_class,
log_uri,
command,
glue_version,
execution_property,
security_configuration,
number_of_workers,
tags,
max_capacity,
non_overridable_arguments,
maintenance_window,
job_mode,
job_run_queuing_enabled
FROM aws.glue.jobs
WHERE region = 'us-east-1' AND data__Identifier = '<Name>';

INSERT example

Use the following StackQL query and manifest file to create a new job resource, using stack-deploy.

Required Properties

/*+ create */
INSERT INTO aws.glue.jobs (
 Role,
 Command,
 region
)
SELECT 
'{{ Role }}',
 '{{ Command }}',
'{{ region }}';

All Properties

/*+ create */
INSERT INTO aws.glue.jobs (
 Connections,
 MaxRetries,
 Description,
 Timeout,
 AllocatedCapacity,
 Name,
 Role,
 DefaultArguments,
 NotificationProperty,
 WorkerType,
 ExecutionClass,
 LogUri,
 Command,
 GlueVersion,
 ExecutionProperty,
 SecurityConfiguration,
 NumberOfWorkers,
 Tags,
 MaxCapacity,
 NonOverridableArguments,
 MaintenanceWindow,
 JobMode,
 JobRunQueuingEnabled,
 region
)
SELECT 
 '{{ Connections }}',
 '{{ MaxRetries }}',
 '{{ Description }}',
 '{{ Timeout }}',
 '{{ AllocatedCapacity }}',
 '{{ Name }}',
 '{{ Role }}',
 '{{ DefaultArguments }}',
 '{{ NotificationProperty }}',
 '{{ WorkerType }}',
 '{{ ExecutionClass }}',
 '{{ LogUri }}',
 '{{ Command }}',
 '{{ GlueVersion }}',
 '{{ ExecutionProperty }}',
 '{{ SecurityConfiguration }}',
 '{{ NumberOfWorkers }}',
 '{{ Tags }}',
 '{{ MaxCapacity }}',
 '{{ NonOverridableArguments }}',
 '{{ MaintenanceWindow }}',
 '{{ JobMode }}',
 '{{ JobRunQueuingEnabled }}',
 '{{ region }}';

Manifest

version: 1
name: stack name
description: stack description
providers:
  - aws
globals:
  - name: region
    value: '{{ vars.AWS_REGION }}'
resources:
  - name: job
    props:
      - name: Connections
        value:
          Connections:
            - '{{ Connections[0] }}'
      - name: MaxRetries
        value: null
      - name: Description
        value: '{{ Description }}'
      - name: Timeout
        value: '{{ Timeout }}'
      - name: AllocatedCapacity
        value: null
      - name: Name
        value: '{{ Name }}'
      - name: Role
        value: '{{ Role }}'
      - name: DefaultArguments
        value: {}
      - name: NotificationProperty
        value:
          NotifyDelayAfter: '{{ NotifyDelayAfter }}'
      - name: WorkerType
        value: '{{ WorkerType }}'
      - name: ExecutionClass
        value: '{{ ExecutionClass }}'
      - name: LogUri
        value: '{{ LogUri }}'
      - name: Command
        value:
          Name: '{{ Name }}'
          PythonVersion: '{{ PythonVersion }}'
          Runtime: '{{ Runtime }}'
          ScriptLocation: '{{ ScriptLocation }}'
      - name: GlueVersion
        value: '{{ GlueVersion }}'
      - name: ExecutionProperty
        value:
          MaxConcurrentRuns: null
      - name: SecurityConfiguration
        value: '{{ SecurityConfiguration }}'
      - name: NumberOfWorkers
        value: '{{ NumberOfWorkers }}'
      - name: Tags
        value: {}
      - name: MaxCapacity
        value: null
      - name: NonOverridableArguments
        value: {}
      - name: MaintenanceWindow
        value: '{{ MaintenanceWindow }}'
      - name: JobMode
        value: '{{ JobMode }}'
      - name: JobRunQueuingEnabled
        value: '{{ JobRunQueuingEnabled }}'

DELETE example

/*+ delete */
DELETE FROM aws.glue.jobs
WHERE data__Identifier = '<Name>'
AND region = 'us-east-1';

Permissions

To operate on the jobs resource, the following permissions are required:

Create

iam:GetRole,
iam:PassRole,
glue:CreateJob,
glue:GetJob,
glue:TagResource

Read

glue:GetJob,
glue:GetTags

Delete

glue:DeleteJob,
glue:GetJob,
glue:UntagResource

Update

iam:GetRole,
iam:PassRole,
glue:UpdateJob,
glue:UntagResource,
glue:TagResource

List

glue:ListJobs