Black Friday50% off your first month on any monthly plan with EARLY50 See plans
All 33 lessons
  1. Start here: cloud and data basics

    1. 1. Start here: the exam, the plan and what you already know
    2. 2. Cloud in plain words: SaaS, PaaS, IaaS and where Fabric fits
    3. 3. Data basics: files, tables, formats and the lakehouse idea
    4. 4. A tour of Microsoft Fabric: OneLake, workspaces and items
    5. 5. The three languages: SQL, PySpark and KQL side by side
    6. 6. Getting data in: the map of Fabric tools
  2. Store, ingest and transform batch data (exam area 2, 30-35%)

    1. 7. Choosing a data store and a transformation tool
    2. 8. Shortcuts and mirroring: use data without copying it
    3. 9. Ingest data with pipelines and Copy job
    4. 10. Transform with PySpark and SQL: join, denormalize, aggregate
    5. 11. Messy data: duplicates, missing values and late arrivals
    6. 12. Full and incremental loads, and loading a star schema
  3. Orchestrate and stream (exam areas 1 and 2)

    1. 13. Pipeline or notebook? Choosing how to orchestrate
    2. 14. Schedules and event-based triggers
    3. 15. Parameters and dynamic expressions
    4. 16. Streaming in Fabric: Eventstreams, Spark streaming and KQL
    5. 17. KQL and windowing functions for real-time data
  4. Set up and secure a Fabric solution (exam area 1, 30-35%)

    1. 18. Workspace settings: Spark, domains, OneLake and data workflows
    2. 19. Version control with Git and database projects
    3. 20. Deployment pipelines: development, test and production
    4. 21. Who can see what: workspace roles and item permissions
    5. 22. Fine-grained security: rows, columns, objects, files and masking
    6. 23. Governance: sensitivity labels and endorsement
  5. Monitor, fix and optimize (exam area 3, 30-35%)

    1. 24. Monitoring hub, alerts and Activator
    2. 25. Fixing errors in pipelines, dataflows and notebooks
    3. 26. Fixing errors in Eventhouse, Eventstream and T-SQL
    4. 27. Optimize lakehouse tables: V-Order, OPTIMIZE, VACUUM and partitions
    5. 28. Optimize Spark, pipelines and warehouses
    6. 29. Optimize eventstreams, eventhouses and queries
  6. Exam practice and final review

    1. 30. Exam day: question types, timing and how to think
    2. 31. Mock exam 1
    3. 32. Mock exam 2
    4. 33. Final review: every decision on one page
See the leaderboard

20 min

Workspace settings: Spark, domains, OneLake and data workflows

Configure the workspace settings the exam asks about and know what each one changes.

By the end of this lesson you can

  • Choose between a starter pool and a custom Spark pool
  • Use environments and high concurrency to control libraries, start time and cost
  • Explain who creates domains and who can assign a workspace to one
  • Pick the right OneLake and Apache Airflow job setting for a scenario

From earlier lessons · 1 of 3

Which KQL line counts events in 5-minute buckets?

Pick one answer.

From earlier lessons · 2 of 3

Every minute, a tile shows trips over the previous 10 minutes. Which window is this?

Pick one answer.

From earlier lessons · 3 of 3

What does a OneLake shortcut to an Amazon S3 bucket do?

Pick one answer.

Northwind Rides' data team has four complaints this week. Notebooks take three minutes to start. Every notebook needs the same geospatial library installed by hand. Finance wants its own area in the data catalog. And the cloud bill shows a jump in charges for reading the partner's S3 bucket. All four are fixed in one place: workspace settings.

[4]

Who can change workspace settings

Exam depth: know it cold.

Workspace settings hold the configuration for one workspace. To open them, select More options next to the workspace name, then Workspace settings.

Only a workspace Admin can change them. Members and Contributors can't change the settings in this lesson.

The settings this lesson covers live in four places:

  • Data Engineering/Science > Spark settings: pools, environment, jobs and high concurrency.
  • OneLake: shortcut caching and diagnostics.
  • Data Factory > Apache Airflow Runtime Settings: the default Airflow pool.
  • Domain assignment, covered below.

One setting needs a second person. A custom Spark pool also needs a capacity admin to allow Customized workspace pools.

[39][152][154]

Starter pool or custom pool

Exam depth: know it cold.

A Spark pool is the set of machines (nodes) that runs your notebooks and Spark jobs.

Starter poolCustom pool
CreatedAutomatically, with every workspace.By a workspace admin, with New pool.
Node sizeMedium, fixed.Small, Medium, Large, X-Large or XX-Large.
Session startAbout 5-10 seconds, using machines kept warm.About 3 minutes, unless kept warm as a custom live pool.
You can tuneMax nodes, autoscale, dynamic executors.Node size, autoscale min and max, dynamic executors.

Autoscale adds and removes nodes as load changes. Dynamic allocation does the same for executors, the Spark processes that do the work.

Node sizes run from Small (4 vCores, or virtual processor cores, and 32 GB) to XX-Large (64 vCores, 512 GB). X-Large and XX-Large need a paid capacity, not a trial.

Decision rule: fast start and default sizing means the starter pool; specific node sizes or scaling means a custom pool.

[152][153][154][155]

The Create new pool page in workspace settings, under Data Engineering/Science > Spark settings. It shows Spark pool name, Node family set to Memory optimized, and the Node size list from Small to XX-Large, with Enable dynamic allocation checked.
Microsoft Learn, CC BY 4.0

[152]

You are a data engineer at Northwind Rides. A nightly job joins a year of trips and runs out of memory on Medium nodes. The workspace runs on a paid F64 capacity. You need larger nodes for this workload. What should you do?

Pick one answer.

[152][154]

Environments: runtime, compute and libraries in one item

Exam depth: know it cold.

An environment is a Fabric item that bundles Spark settings for notebooks and Spark job definitions. It holds three things:

  • Spark compute: the Spark runtime (a version of Spark with built-in libraries), the pool, and driver and executor cores and memory.
  • Libraries: public packages, such as from PyPI, the public Python package index, or custom files you upload.
  • Resources: small shared files.

You attach an environment to a notebook from its Environment menu. A workspace admin can also set a default environment on the Environment tab of Spark settings. Every notebook that doesn't pick its own then gets those libraries.

The Customize compute configuration for items switch decides if environments may change cores and memory. When it's off, every environment uses the default pool's compute.

[152][156][157][159]

[157][158]

High concurrency and job settings

Exam depth: know it cold.

High concurrency lets several notebooks share one running Spark session. Later notebooks skip the start-up wait, and only the notebook that started the session is billed. Sharing needs the same user, the same default lakehouse and the same Spark compute settings. One session holds up to 5 notebooks by default, or up to 50 with spark.highConcurrency.max.

It has two switches under High concurrency: For notebooks and For pipeline running multiple notebooks.

The Jobs tab controls job admission, the rule for starting Spark jobs on a busy capacity. By default, optimistic job admission starts a job with its minimum cores. Reserve maximum cores for active Spark jobs turns that off. The tab also sets the Spark session timeout, 20 minutes by default.

[152][160][161][162]

You are a data engineer at Northwind Rides. A pipeline runs five notebooks that share a default lakehouse and Spark settings. Each notebook waits for its own Spark session, and each is billed. You need to cut start time and cost with no code changes. What should you do?

Pick one answer.

[160][161]

Five notebooks, five Spark sessions, five bills. Turn on high concurrency: five notebooks, one session. Picture: Claymation-style scene, two halves.

Domains and subdomains

Exam depth: know it cold.

A domain groups workspaces by business area, such as Finance or Operations. Items in those workspaces get a domain tag, so people can filter the OneLake catalog by domain. Some tenant settings can also be handed down to a domain. A subdomain splits a domain further, such as Operations > Fleet.

RoleCan do
Fabric adminCreate, rename and delete domains, and name domain admins.
Domain adminEdit the description, add contributors and assign workspaces. Can't delete or rename the domain.
Domain contributorA workspace admin allowed to assign their own workspace, from its workspace settings.

Subdomains are created by a Fabric admin or domain admin, and use the parent's domain admins. A domain doesn't grant access to data; workspace roles and item permissions do.

[50]

Rendering diagram…
Three roles, three jobs. Only a workspace admin can put a workspace into a domain from the workspace side.

[50]

You are a data engineer at Northwind Rides. A Fabric admin has created a Finance domain and listed you as a domain contributor. You are a Member, not an Admin, of the Payments workspace. You need Payments assigned to Finance. What must happen first?

Pick one answer.

OneLake workspace settings

Exam depth: know it cold.

The OneLake tab has two settings a workspace admin can change.

  • Enable cache for shortcuts: OneLake keeps a copy of files read through an external shortcut. Later reads come from the cache, which cuts egress cost, the fee a cloud charges for data leaving it. It works for Google Cloud Storage, S3, S3 compatible and on-premises gateway shortcuts. Set a Retention Period of 1-28 days, which resets each time a file is read. Files over 1 GB aren't cached. Reset cache empties it.
  • Add diagnostic events to a lakehouse: logs who accessed what data into a lakehouse on the same capacity.

Trap: the switches that allow OneLake file explorer or apps outside Fabric are tenant settings. A workspace admin can't change them.

[74][163][164]

Workspace settings with the OneLake tab selected. Under Shortcut Settings, Enable cache for shortcuts is switched On, Retention Period is set to 28 days, and a Reset cache button sits below.
Microsoft Learn, CC BY 4.0

[74]

Apache Airflow job settings

Exam depth: recognize it.

An Apache Airflow job runs Airflow workflows written in Python, called DAGs. Older pages call it Data workflows. Its pool is set under Data Factory > Apache Airflow Runtime Settings, in Default Data Workflow Setting.

Starter poolCustom pool
UptimeShuts down after 20 minutes idle.Always on until you pause it.
Best forDevelopment and testing.Production schedules.
OptionsNone.Node size, autoscale, extra nodes.

Decision rule: a DAG that must run on a schedule around the clock needs a custom pool.

[165][166]

Try it in Fabric

You need the Admin role on a workspace. A trial capacity is enough.

  1. Open the workspace and select Workspace settings.
  2. Expand Data Engineering/Science and select Spark settings.
  3. On the Pool tab, open the Default pool for workspace list. Select StarterPool to see its node limits.
  4. Open the list again and select New pool.
  5. Enter a name, keep Memory optimized, and pick Small as the node size.
  6. Keep autoscale and dynamic allocation on, then select Create.
  7. Open the Jobs tab, find Spark session timeout and note the 20-minute default.

[152][153]

Exam-style practice

You are a data engineer at Northwind Rides. Notebooks read the partner's maintenance files through an Amazon S3 shortcut many times a day. Egress charges are rising. The files must stay in S3. You need to cut the charges with the least effort. What should you do?

Pick one answer.

[74]

You are a data engineer at Northwind Rides. Every notebook in the workspace needs the same geospatial library. New notebooks keep missing it. You are a workspace admin. You need the library available by default to all notebooks. What should you do?

Pick one answer.

[152][159]

You are a data engineer at Northwind Rides. An Apache Airflow job must run its DAGs on a schedule every hour, day and night. Runs are missed after quiet periods on the default pool. You need reliable scheduled runs. What should you configure?

Pick one answer.

[165]

FlashcardsIndex cardsFlashcards ⚡

Card 1 of 15

[50][74][152][154][156][160][162][165]

Worth keeping

  1. Only workspace Admins change workspace settings.
  2. Starter pool for fast start; custom pool when you need node size or scaling control.
  3. Libraries and Spark properties live in an environment, set as the workspace default.
  4. High concurrency shares one session across notebooks and bills only the first.
  5. Domains organize and delegate but grant no access; shortcut caching cuts cross-cloud reads.

On Monday someone edited the shared environment and three notebooks broke. Nobody knows what changed, and there's no way back. Next lesson connects the workspace to Git, so every change has a history and an undo.

Leaderboard

Loading…

50 XP for each lesson you finish, 10 for each right answer. Names show only for learners who turn on Show me on leaderboards.

Sources for this lesson (19)

Each fact in the lesson is marked with the number of the page it comes from.

  1. [4]Microsoft Learn: Study Guide for Exam DP-700
  2. [39]Microsoft Learn: Workspaces in Microsoft Fabric and Power BI
  3. [50]Microsoft Learn: Fabric domains
  4. [74]Microsoft Learn: OneLake shortcuts
  5. [152]Microsoft Learn: Data Engineering workspace administration settings in Microsoft Fabric
  6. [153]Microsoft Learn: Configure starter pools in Microsoft Fabric
  7. [154]Microsoft Learn: Create custom Spark pools in Microsoft Fabric
  8. [155]Microsoft Learn: Overview of Apache Spark compute in Microsoft Fabric
  9. [156]Microsoft Learn: Create, configure, and use an environment in Fabric
  10. [157]Microsoft Learn: Spark compute configuration settings in Fabric environments
  11. [158]Microsoft Learn: Migrate workspace libraries and Spark properties to a default environment
  12. [159]Microsoft Learn: Manage Apache Spark libraries in Microsoft Fabric
  13. [160]Microsoft Learn: High concurrency mode in Apache Spark for Fabric
  14. [161]Microsoft Learn: Configure high concurrency mode for Fabric notebooks in pipelines
  15. [162]Microsoft Learn: Job admission in Apache Spark for Microsoft Fabric
  16. [163]Microsoft Learn: OneLake diagnostics
  17. [164]Microsoft Learn: OneLake tenant settings
  18. [165]Microsoft Learn: Apache Airflow Job workspace settings
  19. [166]Microsoft Learn: Apache Airflow compute in Fabric