What exactly is Microsoft Fabric Data Factory?
Is it just Azure Data Factory with a new name
or something completely different?
If you've heard the term and wondered what it actually means,
you're not alone.
By the end of this episode, you'll know what it is,
how it's different from the classic ADF
and the core pieces you actually need to understand.
If you're dealing with data movement in Fabric,
this is the engine that makes it happen.
And it's simpler than you think.
We'll look at four key building blocks
and see how they fit together.
But first, let's define what we're talking about.
Here's the simplest definition.
Fabric Data Factory is the engine inside Microsoft Fabric
that moves and integrates your data.
Think of it as the replacement for Azure Data Factory
and Synapse Pipelines, all in a single SAS platform.
That's the first thing to understand.
It's not a separate service you manage.
It's built right into Fabric connected to one lake.
Let's use an analogy.
If Fabric is an office building, Data Factory is the mail room
and the conveyor belts that move documents between floors.
You don't think about the mail room every day.
But without it, nothing gets from point A to point B.
Same here, Data Factory handles the movement behind the scenes.
The big difference from the old world,
no extra storage configuration needed.
Your data lands right in one lake.
In Azure Data Factory, you had to set up a storage account,
create containers, and configure linked services.
In Fabric, that's all gone.
One lake is already there, so you just point your pipeline at it
and go.
That sounds familiar, right?
So how is this different from the Azure Data Factory
you already know?
How it's different from Azure Data Factory?
A lot of people think Fabric Data Factory is just Azure Data
Factory with a new label, but it's not.
Let's clear up a few myths.
Pricing is where things really change.
ADF charges per activity run, so every copy activity
stored procedure and pipeline execution adds to your bill.
Fabric Data Factory uses capacity-based pricing instead.
You buy a Fabric capacity, F2, F4, F16, whatever,
and all your workloads share that compute pool.
Data Factory Power BI, Lakehouse, the whole platform.
If you already have Fabric capacity,
running pipelines costs very little extra.
If you don't, you need to weigh the total Fabric cost
against your current ADF bill.
Connections are simpler too.
In ADF, you had to create a linked service for every source
and destination, then data sets on top of that,
then configure the integration run time.
With Fabric, connections are defined in line
and one lake is automatic.
You pick your source, pick your destination,
and the connection is set up on the spot, much less clicking.
There are also new built-in activities.
You can send emails or teams messages directly
from your pipeline without needing logic apps
or custom web hooks.
It's just a drag and drop activity,
and that alone saves a lot of time.
And deployment is better now.
ADF had deployment pipelines,
but they were clunky.
Fabric has built-in deployment pipelines
with workspace-level promotion,
so you can move from dev to test to production in a few clicks.
No manual scripts needed.
Now let's be honest about what's missing.
SSIS support and managed VNet aren't here yet.
If you're heavily invested in SQL server integration services,
you'll need to wait.
For most people, that's fine.
The vast majority of data integration work
doesn't need SSIS.
So here's a simple rule of thumb.
If your ADF bill is under $1,500 a month,
and you're not using Fabric,
stick with ADF because it's cheaper for low volume workloads.
If you're already in Fabric, data factories included, so use it.
And if your ADF costs a higher than that,
it's worth doing the math on a Fabric capacity.
Building block one, data pipelines.
Now let's look at the first building block, data pipelines.
Pipelines are the orchestrators.
They decide what runs, when, and in what order,
and everything else plugs into them.
Think of a pipeline like a recipe.
You have steps, you can branch, and you can loop back.
First, you preheat the oven, then mix the ingredients,
and if the batter looks too dry, you add more milk before baking.
A pipeline works the same way.
First, copy data from a source, then check if the file exists,
and if it does, run a transformation.
If it doesn't send an alert, you control the flow.
The core activities you'll use most often are
copy data for each if condition get metadata and look up.
Copy data moves data from point A to point B,
for each loop through a list, say 10 files in a folder.
If condition checks whether something is true or false,
get metadata asks questions about your data,
how big is this file?
When was it last modified?
Look up reads a configuration file
and brings the values into your pipeline.
That's the starter kit.
The whole thing is low code.
You drag and drop activities onto a canvas, connect them
with arrows, set the properties, and you're done.
No coding needed for most tasks.
It looks like a flow chart, because that's exactly what it is.
Here's a real example.
Imagine pulling sales data from an API every morning.
Your pipeline starts with a copy data activity
that lands it in a lake house.
Then a data flow, Gen 2 step, cleans the data, removes nulls,
fixes data formats, then a notebook enriches it
with customer segments.
Finally, a team's activity sends a message to your team.
Sales data updated.
That's one pipeline doing the work that
used to require multiple tools.
You can also make pipelines reusable.
Say you have the same logic for 10 different data sources.
Instead of building 10 pipelines, you build one
and use parameters, source URL, table name, destination folder,
all as variables.
One pipeline, 10 different jobs.
That's parameterization.
And here's something new.
You can ask co-pilot to write pipeline expressions
in plain English.
Type add two days to this date,
and it generates the expression for you
no more memorizing function syntax or googling
how to format a timestamp.
Just say what you need.
Building block two, data flow, Gen 2.
So what happens when you need to clean up data,
but don't want to write code?
That's the exact problem data flow, Gen 2 solves.
Data flow, Gen 2 is the low code transformation tool
inside fabric data factory.
If you've used Power Query in Excel or Power BI before,
this will feel very familiar.
It has the same interface, the same 300 plus transformations
and the same engine.
Just running in the cloud with a lake house as the destination.
Think of it this way.
If pipelines are the assembly line that moves boxes around,
data flow, Gen 2 is the kitchen where you actually chop, season,
and cook the raw ingredients.
Pipelines handle the logistics.
Data flow, Gen 2 handles the actual work
of making the data useful.
When should you reach for it?
When you need to clean data, change types,
merge tables or filter rows, all without touching spark
or SQL.
If your transformation fits on a whiteboard,
it probably fits in data flow, Gen 2.
Here's a quick example.
You import a CSV file full of customer orders.
There are duplicate rows.
Some dates are in the wrong format.
A few columns have nulls.
You open data flow, Gen 2, click a few buttons
to remove duplicates, change the date column type
and fill in missing values.
Then you join it with a product table from another source.
The whole thing takes 10 minutes.
No code, the output lands in a lake house table, ready for reporting.
You can also see everything visually.
There's a diagram view that shows each step as a node.
Source goes into a filter node.
Filter feeds into a merge node.
Merge feeds into the output.
You can see the whole flow at a glance.
That's really helpful when you're debugging
or when you need to explain the logic to someone else.
Scheduling is flexible too.
You can run data flow, Gen 2 on its own,
set it to refresh every morning at 6am.
Or you can call it from a parent pipeline
as one step in a larger workflow.
Most production setups do the latter.
The pipeline handles the orchestration
and data flow, Gen 2, handles the transformation step inside it.
But here's an honest note.
For really heavy transformations,
think billions of rows, complex window functions
or custom machine learning logic,
you might still use notebooks with Spark.
Data flow Gen 2 isn't built for every scenario,
but for the vast majority of what most teams need,
cleaning, shaping, joining, aggregating,
it covers about 80% of the work.
That's a lot of code you don't have to write.
Building block three, copy jobs.
Now let's talk about moving data.
What if you needed to flow in continuously
from a database?
That's where copy jobs come in.
A copy job is a simplified way to copy data
from a source to one lake with built-in change tracking.
No complex pipeline configuration, no custom scripts.
You pick a source, pick a destination,
and tell it how often you want it to run.
That's really it.
The key feature is change data capture or CDC.
Copy jobs automatically track inserts, updates,
and deletes at the source.
You don't need to build manual watermarking logic
that thing where you save a timestamp and query
for newer records.
Copy jobs handle that behind the scenes.
It just works without any extra effort.
There's more.
Copy jobs support SCD type 2, slowly changing dimensions type 2.
That means it keeps full history.
When a customer changes their address,
the old address stays in the table with an end date.
The new address gets a start date.
You can see exactly how a record changed over time.
That's huge for auditing and historical reporting.
So when do you use copy jobs
versus the standard copy activity?
Here's the rule.
Copy jobs are for set it and forget it continues ingestion.
You want data flowing in automatically,
day after day, with minimal maintenance.
Copy activities are for one-off or scheduled bulk loads
where you need more control over the logic,
the mappings, and the error handling.
Think of it this way.
A copy job is like having a librarian
who automatically spots new books being added
and puts them on the shelf.
You don't need to check the catalog every time
because the librarian handles it.
A copy activity is more like you walking into the library,
picking the books yourself, and deciding exactly where they go.
Both get the job done, but copy jobs save you the effort
for routine work.
Set up is simple.
You pick the source, say a SQL server database.
You pick the destination, a lake house table.
You map the fields.
You tell it if you want incremental refresh or full refresh,
and that covers it.
The job runs on your schedule
and keeps your lake house in sync automatically.
Building block 4 mirroring.
Now what if you need real-time replication?
That's where mirroring comes in.
Here's the simplest definition.
Mirroring is a way to replicate your operational database
into one lake continuously without writing a single line of code.
You set it up once and it runs in the background.
The data stays fresh in near real time.
No pipelines to build, no data flows to maintain.
It just works.
How does it actually work?
Mirroring reads the transaction log of your source database.
Every change, every insert, update, and delete
gets picked up and appears in one lake within minutes.
This isn't batch processing.
It's continuous replication.
The moment something changes in your source database,
it shows up in fabric.
Let's talk about sources.
Right now, you can mirror as your SQL database,
as your Cosmos DB, Snowflake, and others.
More are coming like private network mirroring
for databases behind firewalls,
which was announced at build 2026.
Why would you use this?
Because you get a live copy of your database
in one lake without building any ETL pipelines.
You don't need copy activities.
You don't need change tracking scripts.
You don't need incremental load logic.
The system handles all of that for you.
Here's an analogy to help it stick.
Think of mirroring as having a real-time clone
of your database sitting inside fabric.
Update the source once and the clone updates instantly.
No code required to keep them in sync.
The system does the heavy lifting.
There's also a cost benefit worth mentioning.
Mirroring comes with free compute and storage
up to a certain limit.
You only pay for the fabric capacity you already have.
For many organizations, that makes it cheaper than building
and maintaining custom replication pipelines.
And here's a practical use case.
You can build Power BI reports directly on mirror data
without affecting the source database.
Your operational system keeps running at full speed.
Your analysts get fresh data.
Everyone wins.
How these pieces work together.
So you have pipelines, data flows, copy jobs, and mirroring.
How do they fit together?
Most real-world implementations follow a simple pattern.
Ingest, clean, enrich, serve.
It's called the medallion architecture.
Bronze, silver, gold.
Think of it as three layers.
Bronze holds raw data exactly as it arrived.
Silver is cleaned and de-duplicated.
Gold is enriched and ready for reporting.
Here's what that looks like in practice.
A mirroring or copy job lands raw data
into a bronze lake house table.
That's the ingest step.
Then a data flow gen 2 cleans and transforms it into silver.
Removes nulls, standardizes formats, joins related tables.
Finally, a pipeline orchestrates the whole thing.
It triggers the copy job, waits for it to finish,
runs the data flow, and sends a team's notification
when it's done.
That's the full flow.
The real magic isn't any single tool.
It's how they all talk to each other through one lake.
Your mirror data lives in one lake.
Your data flow output lands in one lake.
Your pipeline reads from one lake.
Everything shares the same storage layer.
You can even share data between workspaces using shortcuts.
No copying needed.
There's a dependency chain worth understanding.
Pipelines can call data flow gen 2 as a step inside the pipeline.
They can also include notebooks, DBT jobs, and even Azure data
factory pipelines if you're migrating gradually.
One pipeline can orchestrate them all.
Here's the thing.
You don't need to stitch together five different services
anymore.
One platform, one storage layer, one set of permissions.
That's the point of fabric.
And it works in the real world.
A UAE based distributor used this exact pattern.
Copy jobs for SAP data.
Data flow gen 2 for cleaning pipelines for orchestration.
They cut their months in close from nine days to two.
That's not a demo.
That's production.
Practical guidance and cost.
So who should actually use fabric data factory?
If you need to move or transform data inside fabric,
this is your tool that covers data engineers building
pipelines, analytics engineers modeling data,
and even business analysts using data flow gen 2
for self-service transformations.
Who should skip it?
If you only need one time file uploads,
start with Lake House directly.
Drag and drop a CSV done.
If you're heavily invested in SSIS, wait for that support.
And if your ADF bill is under 1500 a month
and you're not using fabric, staying put makes sense for now.
Let's talk cost.
Fabric data factory uses capacity units or CUs.
Your pipelines consume CUs based on activity complexity
and data volume.
A simple copy job uses fewer than a flow
with joins and aggregations.
You don't need to track every penny
because the fabric capacity metrics app shows you
what's consuming what.
Here's the rule of thumb.
If you already have fabric capacity like F2 or F4,
running pipelines cost very little extra
since the capacity is already paid for.
If you don't have fabric capacity,
compare the total cost to your current ADF bill.
For most organizations with moderate pipeline volumes,
fabric comes out ahead.
Start small with the free 60 day trial.
Create a workspace, build one pipeline
that copies a CSV to a Lake House and see how it feels.
If you've used ADF or Power Query,
the learning curve is shallow.
If you haven't, this is the simplest entry point
to modern data integration.
Fabric data factory isn't ADF in a new box.
It's a simpler, sass-native engine
that makes moving data feel like a built-in feature,
not a separate project.
Start with one pipeline that copies a file to a Lake House.
That's your first step.
Share this with someone still confused about the difference
and subscribe for more knowledge nuggets.