pipelines

Creates, updates, deletes or gets a pipeline resource or lists pipelines in a region

Overview

Name	pipelines
Type	Resource
Description	An example resource schema demonstrating some basic constructs and validation rules.
Id	aws.datapipeline.pipelines

Fields

Name	Datatype	Description
activate	boolean	Indicates whether to validate and start the pipeline or stop an active pipeline. By default, the value is set to true.
description	string	A description of the pipeline.
name	string	The name of the pipeline.
parameter_objects	array	The parameter objects used with the pipeline.
parameter_values	array	The parameter values used with the pipeline.
pipeline_objects	array	The objects that define the pipeline. These objects overwrite the existing pipeline definition. Not all objects, fields, and values can be updated. For information about restrictions, see Editing Your Pipeline in the AWS Data Pipeline Developer Guide.
pipeline_tags	array	A list of arbitrary tags (key-value pairs) to associate with the pipeline, which you can use to control permissions. For more information, see Controlling Access to Pipelines and Resources in the AWS Data Pipeline Developer Guide.
pipeline_id	string
region	string	AWS region.

Methods

Name	Accessible by	Required Params
create_resource	INSERT	Name, region
delete_resource	DELETE	data__Identifier, region
update_resource	UPDATE	data__Identifier, data__PatchDocument, region
list_resources	SELECT	region
get_resource	SELECT	data__Identifier, region

SELECT examples

Gets all pipelines in a region.

SELECT
region,
activate,
description,
name,
parameter_objects,
parameter_values,
pipeline_objects,
pipeline_tags,
pipeline_id
FROM aws.datapipeline.pipelines
WHERE region = 'us-east-1';

Gets all properties from an individual pipeline.

SELECT
region,
activate,
description,
name,
parameter_objects,
parameter_values,
pipeline_objects,
pipeline_tags,
pipeline_id
FROM aws.datapipeline.pipelines
WHERE region = 'us-east-1' AND data__Identifier = '<PipelineId>';

INSERT example

Use the following StackQL query and manifest file to create a new pipeline resource, using stack-deploy.

Required Properties

/*+ create */
INSERT INTO aws.datapipeline.pipelines (
 Name,
 region
)
SELECT 
'{{ Name }}',
'{{ region }}';

All Properties

/*+ create */
INSERT INTO aws.datapipeline.pipelines (
 Activate,
 Description,
 Name,
 ParameterObjects,
 ParameterValues,
 PipelineObjects,
 PipelineTags,
 region
)
SELECT 
 '{{ Activate }}',
 '{{ Description }}',
 '{{ Name }}',
 '{{ ParameterObjects }}',
 '{{ ParameterValues }}',
 '{{ PipelineObjects }}',
 '{{ PipelineTags }}',
 '{{ region }}';

Manifest

version: 1
name: stack name
description: stack description
providers:
  - aws
globals:
  - name: region
    value: '{{ vars.AWS_REGION }}'
resources:
  - name: pipeline
    props:
      - name: Activate
        value: '{{ Activate }}'
      - name: Description
        value: '{{ Description }}'
      - name: Name
        value: '{{ Name }}'
      - name: ParameterObjects
        value:
          - Attributes:
              - Key: '{{ Key }}'
                StringValue: '{{ StringValue }}'
            Id: '{{ Id }}'
      - name: ParameterValues
        value:
          - Id: '{{ Id }}'
            StringValue: '{{ StringValue }}'
      - name: PipelineObjects
        value:
          - Fields:
              - Key: '{{ Key }}'
                RefValue: '{{ RefValue }}'
                StringValue: '{{ StringValue }}'
            Id: '{{ Id }}'
            Name: '{{ Name }}'
      - name: PipelineTags
        value:
          - Key: '{{ Key }}'
            Value: '{{ Value }}'

DELETE example

/*+ delete */
DELETE FROM aws.datapipeline.pipelines
WHERE data__Identifier = '<PipelineId>'
AND region = 'us-east-1';

Permissions

To operate on the pipelines resource, the following permissions are required:

Create

datapipeline:CreatePipeline,
datapipeline:PutPipelineDefinition,
datapipeline:GetPipelineDefinition,
datapipeline:DescribePipelines,
datapipeline:ValidatePipelineDefinition,
datapipeline:ActivatePipeline,
datapipeline:AddTags,
iam:PassRole

Read

datapipeline:GetPipelineDefinition,
datapipeline:DescribePipelines

Update

datapipeline:PutPipelineDefinition,
datapipeline:AddTags,
datapipeline:RemoveTags,
datapipeline:DeactivatePipeline,
datapipeline:GetPipelineDefinition,
datapipeline:ActivatePipeline,
datapipeline:ValidatePipelineDefinition,
datapipeline:DescribePipelines,
datapipeline:AddTags,
datapipeline:RemoveTags,
iam:PassRole

Delete

datapipeline:DeletePipeline,
datapipeline:DescribePipelines,
datapipeline:GetPipelineDefinition,
datapipeline:RemoveTags

List

datapipeline:ListPipelines