⏰ Pipeline Triggers & Scheduling¶

Master pipeline scheduling with schedule triggers, tumbling window triggers, and event-based triggers for automated data workflows.

📋 Table of Contents¶

Trigger Types
Schedule Trigger
Tumbling Window Trigger
Event-Based Trigger
Next Steps

🎯 Trigger Types¶

Trigger Type	Use Case	Backfill	Dependency
Schedule	Fixed time execution	No	No
Tumbling Window	Time-series processing	Yes	Yes
Storage Event	File arrival	No	No
Custom Event	External events	No	No

📅 Schedule Trigger¶

Execute pipelines at specific times.

Daily Trigger¶

{
  "name": "DailyMidnightTrigger",
  "properties": {
    "type": "ScheduleTrigger",
    "typeProperties": {
      "recurrence": {
        "frequency": "Day",
        "interval": 1,
        "startTime": "2025-01-01T00:00:00Z",
        "timeZone": "UTC",
        "schedule": {
          "hours": [0],
          "minutes": [0]
        }
      }
    },
    "pipelines": [
      {
        "pipelineReference": {
          "referenceName": "DailyETLPipeline",
          "type": "PipelineReference"
        },
        "parameters": {
          "ProcessDate": "@trigger().scheduledTime"
        }
      }
    ]
  }
}

Hourly Trigger¶

{
  "name": "HourlyTrigger",
  "properties": {
    "type": "ScheduleTrigger",
    "typeProperties": {
      "recurrence": {
        "frequency": "Hour",
        "interval": 1,
        "startTime": "2025-01-01T00:00:00Z"
      }
    }
  }
}

🔄 Tumbling Window Trigger¶

Process data in fixed-size, non-overlapping time windows.

{
  "name": "HourlyTumblingWindowTrigger",
  "properties": {
    "type": "TumblingWindowTrigger",
    "typeProperties": {
      "frequency": "Hour",
      "interval": 1,
      "startTime": "2025-01-01T00:00:00Z",
      "maxConcurrency": 3,
      "retryPolicy": {
        "count": 3,
        "intervalInSeconds": 30
      }
    },
    "pipeline": {
      "pipelineReference": {
        "referenceName": "IncrementalLoadPipeline",
        "type": "PipelineReference"
      },
      "parameters": {
        "WindowStart": "@trigger().outputs.windowStartTime",
        "WindowEnd": "@trigger().outputs.windowEndTime"
      }
    }
  }
}

📁 Event-Based Trigger¶

Trigger based on file arrival or custom events.

Storage Event Trigger¶

{
  "name": "BlobEventTrigger",
  "properties": {
    "type": "BlobEventsTrigger",
    "typeProperties": {
      "blobPathBeginsWith": "/input/data/blobs/",
      "blobPathEndsWith": ".csv",
      "ignoreEmptyBlobs": true,
      "events": ["Microsoft.Storage.BlobCreated"],
      "scope": "/subscriptions/xxx/resourceGroups/xxx/providers/Microsoft.Storage/storageAccounts/xxx"
    },
    "pipelines": [
      {
        "pipelineReference": {
          "referenceName": "ProcessNewFilePipeline",
          "type": "PipelineReference"
        },
        "parameters": {
          "FilePath": "@triggerBody().folderPath",
          "FileName": "@triggerBody().fileName"
        }
      }
    ]
  }
}

📚 Additional Resources¶

🚀 Next Steps¶

→ 14. Dependency Management

Module Progress: 13 of 18 complete

Tutorial Version: 1.0 Last Updated: January 2025