Study Databricks DE-PRO System Tables and Event Logs: key concepts, common traps, and exam decision cues.
Most monitoring misses come from choosing the wrong signal source. DE-PRO wants you to know what each Databricks observability tool is best at.
| Need | Better first signal |
|---|---|
| account or workspace telemetry, auditing, and cost | system tables |
| query bottlenecks and operator behavior | query profile |
| declarative pipeline lifecycle and quality events | event log |
| low-level execution detail on a Spark workload | Spark UI |
| Scope | Better first signal |
|---|---|
| account- or workspace-wide telemetry | system tables |
| one query’s execution behavior | query profile |
| one declarative pipeline’s lifecycle events | event log |
| stage or task execution detail | Spark UI |
The exam often puts all four in the answer set. The correct move is to classify scope first.
| If the stem says… | Strong reading |
|---|---|
| “resource utilization or auditing” | system tables |
| “pipeline event history” | event log |
| “query bottleneck” | query profile |
| “monitor workload” | choose the signal source that matches the scope |
Professional diagnosis is mostly about picking the smallest useful signal:
Using the wrong layer first usually delays the fix and increases noise.
| Trap | Better rule |
|---|---|
| using query profile for account-wide telemetry | it is too narrow for that |
| using system tables to explain one specific pipeline event | event logs may be the better first source |
| treating all observability tools as interchangeable | each one answers a different layer of the question |
| Scenario clue | Stronger answer shape |
|---|---|
| “cost, audit, workspace telemetry” | system tables |
| “pipeline event history or expectations outcome” | event log |
| “query operator bottleneck” | query profile |
| “task-level stage behavior” | Spark UI |
Monitoring questions usually hinge on telemetry scope. If the question is about broad cost, audit, and platform workload signals, think system tables. If it is about one Lakeflow pipeline’s lifecycle and quality events, think event log. If it is about one SQL operator bottleneck, system tables are too broad and query analysis is stronger. DE-PRO rewards choosing the narrowest signal source that answers the question well.