Monitoring Setup Guide¶

Home > Monitoring > Monitoring Setup

This guide covers the setup and configuration of monitoring solutions for Azure Synapse Analytics, including Azure Monitor, Log Analytics, and alerting.

📊 Monitoring Architecture¶

Implement comprehensive monitoring for your Azure Synapse Analytics environment to ensure optimal performance, security, and reliability.

Reference Architecture¶

Initial Setup¶

Create a dedicated Log Analytics workspace for all Synapse-related monitoring to centralize analysis.

Follow these steps to set up the monitoring foundation:

1. Create Log Analytics Workspace:

az monitor log-analytics workspace create \
  --resource-group myResourceGroup \
  --workspace-name synapse-monitoring \
  --sku PerGB2018

1. Create Action Groups for notifications:

az monitor action-group create \
  --resource-group myResourceGroup \
  --name synapse-critical-alerts \
  --short-name synapseAlert \
  --email-receiver name=opsTeam email=ops@contoso.com

1. Enable Microsoft Insights Provider:

az provider register --namespace Microsoft.Insights

1. Assign Monitoring Contributor Role:

az role assignment create \
  --assignee "monitoring-service-principal-id" \
  --role "Monitoring Contributor" \
  --scope "/subscriptions/{subscription-id}/resourceGroups/{resource-group}"

Metrics Collection¶

Configure metrics collection for Azure Synapse Analytics components:

# Enable metrics collection on Synapse workspace
az monitor diagnostic-settings create \
  --name synapse-metrics \
  --resource "/subscriptions/{subscription-id}/resourceGroups/{resource-group}/providers/Microsoft.Synapse/workspaces/{workspace}" \
  --workspace "/subscriptions/{subscription-id}/resourceGroups/{resource-group}/providers/Microsoft.OperationalInsights/workspaces/{la-workspace}" \
  --metrics '[{"category": "AllMetrics", "enabled": true, "retentionPolicy": {"days": 90, "enabled": true}}]'

Log Analytics Setup¶

Configure appropriate retention periods based on your compliance requirements and budget considerations.

Configure Log Analytics to collect Synapse diagnostic logs:

1. Enable Diagnostic Settings:

# PowerShell example for configuring diagnostic settings
$workspace = Get-AzOperationalInsightsWorkspace -ResourceGroupName "myResourceGroup" -Name "synapse-monitoring"

Set-AzDiagnosticSetting -ResourceId "/subscriptions/{subscription-id}/resourceGroups/{resource-group}/providers/Microsoft.Synapse/workspaces/{workspace}" `
  -WorkspaceId $workspace.ResourceId `
  -Enabled $true `
  -Category SQLSecurityAuditEvents,SynapseRbacOperations,GatewayApiRequests,BuiltinSqlReqsEnded,IntegrationPipelineRuns,IntegrationActivityRuns,IntegrationTriggerRuns

1. Log Categories to Enable:

1. Create Custom Log Queries:

// Query for failed pipeline runs
SynapseIntegrationPipelineRuns
| where Status == "Failed"
| project TimeGenerated, PipelineName, RunId, Parameters, SystemParameters, ErrorMessage
| order by TimeGenerated desc

Alerting Setup¶

Configure these essential alerts for Azure Synapse Analytics:

1. Performance Alerts:
2. SQL Pool DWU/cDWU utilization > 90% for 30 minutes
3. Spark job failures > 5 in an hour

4. Query duration > 60 seconds consistently

5. Availability Alerts:

6. Workspace availability < 99.9%
7. Failed connectivity attempts > 10 in 15 minutes

8. Integration runtime availability issues

9. Security Alerts:

10. Multiple failed authentication attempts
11. Firewall rule changes
12. Permission changes to sensitive data

{
  "location": "Global",
  "tags": {},
  "properties": {
    "description": "Alert when SQL pool is near capacity",
    "severity": 2,
    "enabled": true,
    "scopes": [
      "/subscriptions/{subscription-id}/resourceGroups/{resource-group}/providers/Microsoft.Synapse/workspaces/{workspace}/sqlPools/{pool-name}"
    ],
    "evaluationFrequency": "PT5M",
    "windowSize": "PT15M",
    "criteria": {
      "allOf": [
        {
          "metricName": "DWULimit",
          "metricNamespace": "Microsoft.Synapse/workspaces/sqlPools",
          "operator": "GreaterThan",
          "threshold": 90,
          "timeAggregation": "Average"
        }
      ]
    },
    "actions": [
      {
        "actionGroupId": "/subscriptions/{subscription-id}/resourceGroups/{resource-group}/providers/microsoft.insights/actionGroups/{action-group-name}"
      }
    ]
  }
}

Monitoring Dashboards¶

Create custom dashboards using Azure Monitor Workbooks:

1. Executive Dashboard - High-level overview of workspace health and usage
2. Operational Dashboard - Detailed performance and availability metrics
3. Security Dashboard - Authentication events and security alerts
4. Cost Management Dashboard - Resource utilization and cost analytics

Pin the most important metrics and logs to your Azure portal dashboard for quick access.

{
  "lenses": {
"0": {
  "order": 0,
  "parts": {
"0": {
  "position": {
"x": 0,
"y": 0,
"colSpan": 6,
"rowSpan": 4
  },
  "metadata": {
"inputs": [
  {
"name": "resourceTypeMode",
"value": "workspace"
  },
  {
"name": "ComponentId",
"value": "/subscriptions/{subscription-id}/resourceGroups/{resource-group}/providers/Microsoft.OperationalInsights/workspaces/{workspace-name}"
  }
],
"type": "Extension/Microsoft_OperationsManagementSuite_Workspace/PartType/LogsDashboardPart",
"settings": {
  "content": {
"Query": "SynapseIntegrationPipelineRuns\n| where Status == \"Failed\"\n| summarize FailedCount=count() by bin(TimeGenerated, 1h), PipelineName\n| render timechart",
"PartTitle": "Failed Pipeline Runs"
  }
}
  }
}
  }
}
  }
}

Automation and Integration¶

Extend your monitoring setup with these automations:

1. Automated Remediation - Use Azure Automation to automatically resolve common issues
2. Monitoring as Code - Deploy monitoring configurations using ARM templates
3. Integration with ITSM - Connect alerts to ServiceNow or other ITSM systems
4. Power BI Integration - Create rich visualizations from monitoring data

Implementation Checklist¶

• Create Log Analytics workspace for centralized monitoring
• Configure diagnostic settings for all Synapse components
• Set up action groups for notifications
• Create custom log queries for common scenarios
• Configure performance, availability, and security alerts
• Create monitoring dashboards for different stakeholders
• Implement automation for common remediation tasks
• Document monitoring architecture and alert procedures

Related Resources¶

• Azure Monitor for Synapse Analytics
• Log Analytics Query Examples
• Azure Monitor Workbooks