Use this cheat sheet for Google Cloud Professional Data Engineer (PDE) after you know the service names and need faster data-path decisions. PDE questions reward choosing the right ingestion, storage, transformation, governance, serving, and monitoring pattern for the data requirement.
Read every PDE question in this order
- Identify the data requirement: batch, streaming, analytics, transactional, ML feature, governance, or reporting.
- Name the data shape: structured, semi-structured, unstructured, time series, event stream, or relational.
- Choose storage based on query pattern, consistency, latency, scale, and cost.
- Choose processing based on latency, complexity, managed-service fit, and operational burden.
- Add quality, lineage, privacy, access, monitoring, and failure handling.
PDE answer sequence
Use this when the stem mixes ingestion, storage, transformation, governance, serving, or monitoring.
flowchart TD
S["Scenario"] --> D["Define the data requirement"]
D --> S2["Name the data shape"]
S2 --> S3["Choose storage by query pattern and cost"]
S3 --> P["Choose processing by latency and complexity"]
P --> G["Add quality, lineage, privacy, and monitoring"]
Ingestion chooser
| Requirement |
Start with |
Watch for |
| event stream |
Pub/Sub plus streaming processing |
ordering, duplicates, replay, dead-letter handling |
| batch file load |
Cloud Storage to BigQuery or processing pipeline |
schema, partition, quality, and load cadence |
| database replication |
managed transfer or change data capture pattern |
consistency, downtime, schema change, and permissions |
| external SaaS data |
transfer connector or API ingestion |
freshness, quota, retries, and ownership |
| IoT or telemetry |
streaming pipeline |
volume, late data, windowing, and monitoring |
| one-time migration |
transfer service based on size, timeline, and network path |
validation and reconciliation |
Storage and serving chooser
| Workload |
Better fit |
| SQL analytics at scale |
BigQuery |
| object and raw zone storage |
Cloud Storage |
| relational app workload |
Cloud SQL or AlloyDB-style relational pattern |
| global relational consistency |
Spanner-style distributed database pattern |
| low-latency wide-column access |
Bigtable-style pattern |
| search or operational serving |
choose based on access pattern, latency, and query shape |
| BI dashboard |
governed BigQuery model, semantic definitions, and refresh plan |
Processing and orchestration
| Need |
Start with |
| stream or batch data processing |
Dataflow-style pipeline |
| Spark/Hadoop ecosystem |
Dataproc-style managed cluster pattern |
| SQL-first transformations |
BigQuery SQL and Dataform-style transformation workflow |
| workflow scheduling |
Composer or managed orchestration pattern |
| reproducible transformation |
versioned SQL/code, tests, lineage, and deployment process |
| pipeline failure handling |
retries, dead-letter path, alerts, and replay plan |
BigQuery exam traps
| Trap |
Better instinct |
| query scans too much |
partition, cluster, filter early, and avoid unnecessary columns |
| access is too broad |
dataset/table access, row/column controls, authorized views, and policy tags |
| dashboard is slow |
model query shape, materialization, BI Engine-like acceleration where appropriate, and cache behavior |
| schema evolves |
manage schema change, validation, and downstream compatibility |
| cost surprise |
bytes processed, storage, slots/reservations where applicable, and user behavior |
Governance and quality
| Risk |
Control |
| unknown data meaning |
catalog, glossary, owner, and lineage |
| sensitive fields |
classification, DLP, masking, encryption, and access policy |
| bad data enters pipeline |
validation, schema checks, constraints, anomaly detection, and quarantine |
| compliance retention |
lifecycle, deletion, legal hold where required, and audit logs |
| unclear report trust |
source-of-truth definitions, freshness, and quality score |
| uncontrolled sharing |
IAM, service accounts, dataset controls, and review process |
Reliability and optimization
| Symptom |
First checks |
| late data |
windowing, watermark, source delay, retry path, and event time |
| duplicate records |
idempotency key, deduplication, delivery semantics, and merge logic |
| pipeline breaks on schema |
compatibility, validation, and staged rollout |
| slow processing |
partitioning, parallelism, hot keys, worker sizing, and query shape |
| high cost |
storage lifecycle, query scan, cluster use, streaming cost, and waste |
| unreliable dashboards |
pipeline health, data freshness, failed loads, and semantic definitions |
Common traps
| Trap |
Better instinct |
| streaming because it sounds modern |
use streaming only when latency requirement justifies it |
| one database for every problem |
match storage to access pattern and consistency requirement |
| governance after ingestion |
design access, classification, lineage, and retention up front |
| scaling before tuning |
inspect partitioning, query shape, hot keys, and pipeline design |
| scanner-style quality |
quality needs rule, owner, threshold, and remediation path |
Final 15-minute review
| If the stem says… |
Start here |
| real-time events |
Pub/Sub, stream processing, windows, duplicates, dead letters |
| analytics |
BigQuery, partitioning, clustering, governance, query cost |
| raw data lake |
Cloud Storage, catalog, lifecycle, access, quality |
| pipeline failure |
retry, replay, dead-letter, monitoring, idempotency |
| sensitive data |
classification, IAM, DLP, encryption, row/column controls |
| BI trust |
definitions, source quality, freshness, lineage, dashboard audience |
Practice fit
Use IT Mastery for the exact product route, practice status, spaced review when available, and close-answer explanation practice as coverage expands.
One-line decision rule
PDE answers should follow the data path: ingest correctly, store for the access pattern, transform reliably, govern explicitly, serve clearly, and optimize from evidence.