Hello everyone and welcome to another episode of Microsoft Knowledge Nuggets here on M365.
FM, I'm Mirko Peters. Imagine you need one virtual machine for a small app.
You pick a region, choose a VM size, see the price and create it.
Simple, right? Now imagine that same app suddenly needs 2,000 VMs overnight.
You might be running simulations, processing a huge data set or testing software with
thousands of short-lived workers. Suddenly that one easy VM choice turns into a real headache.
The size you picked might not have enough capacity in your region.
A different size could work, but the price shifts. Spot VMs cost way less, but Azure can take
them back at any time. You could build multiple virtual machine scale sets and write custom scripts
to hunt for capacity, but that quickly becomes a management nightmare. Here's the question,
Azure Compute Fleet is built to answer. Why should one exact VM size keep a large deployment
from starting at all? By the end of this episode, you'll know what Compute Fleet actually does,
what it doesn't do, and where it fits alongside virtual machine scale sets.
Compute Fleet in plain English. Azure Compute Fleet is a managed service designed to get
you a large pool of VM compute capacity. Notice I didn't say it creates one specific machine.
You tell Azure what kind of capacity you can use, how much you need, where you're willing to run it,
and whether you want standard on-demand VMs, lower cost spot VMs, or both.
Then Azure scans for capacity across those acceptable options. Think of it like a modern office
building. You don't walk in demanding desk 27 on floor 8, in one exact room facing one exact window.
Instead, you tell the building manager, "I need 500 desks for my team." These floors work,
these room layouts are fine. Here's our budget. The manager looks at what's free and gets everyone
seated. Compute Fleet works the same way. Your request might say, "I need a thousand workers,
they need this much CPU and memory. These VM sizes are acceptable. These availability zones are
acceptable. Use on-demand for part of the workload and spot where it makes sense."
That wider net makes it much easier to find available capacity. A VM SKU is just a particular size
and type a virtual machine. It describes things like CPU, memory, local storage, and processor
family. If you ask for only one SKU, Azure can only search for that one exact shape. If that capacity
is busy, your deployment waits or fails. With Compute Fleet, you list multiple acceptable VM sizes.
Azure can then place some workers on one size and others on another as long as your workload runs
on both. For big workloads, that flexibility is the whole point. Compute Fleet can handle up to 10,000
VMs in a single deployment. You no longer need to create thousands of machines one by one or build
separate scripts for every acceptable VM size. Instead, you create one fleet request. Behind the scenes,
Azure checks the capacity choices you allowed, finds a workable mix, and creates the VM groups
that make up the fleet. You still have to provide the VM image, the network setup, and the software
each worker should run. Fleet doesn't understand your business job on its own. It doesn't know whether
a worker is calculating financial risk, rendering a video frame, testing software, or processing a
genome sample. It also doesn't root web traffic, replace a load balancer, or fix code inside a VM.
Fleet's role is narrower. It helps you acquire and manage a large pool of compute capacity when
more than one path can get you there. That's the shift. Instead of handing Azure a rigid shopping
list that says, "I need 1,000 of this exact machine." You give it a practical request that says,
"I need this much compute, and here are the choices that work." The next part is where that
request becomes useful, because you need to decide how much freedom you're willing to give Azure.
The three choices that give Azure room to work. The first choice is the list of VM sizes you'll
accept. Most people start with a name like D8's, F16, or some exact SKU that makes sense when you're
running one server and you know exactly what it needs. But at Fleet scale, the better question
isn't "which processor name do I want?" It's, "What does my work actually need?" Imagine a risk
model that runs thousands of separate calculations. Each worker needs enough, CPU and memory to process
one slice of the data. It might run just as well on several compute-focused VM families,
even if the underlying processor is different. The job cares about finishing calculations correctly.
It doesn't care whether every worker has the same label on the box. So instead of allowing one VM
size, you can allow several sizes that meet the same practical need that gives as you
have more capacity options to search through, while your workers still get the resources they need.
There is a limit to this flexibility though. Your software has to run correctly on every VM type you allow.
If one family needs a special driver, a different processor design or a particular
local disk setup, don't add it just because it looks cheaper. A wider list only helps when every
option you include is a real option. The second choice is what you want Azure to favor when it finds
those options. Sometimes finishing the work matters more than finding the lowest possible price.
Maybe a team meets to complete an overnight calculation before the market opens. In that case,
you'd lean toward a capacity-focused choice. Your telling Azure find available compute that meets my
rules. The job needs to start. This doesn't mean Azure creates capacity from nowhere. It means Azure
can search through the sizes, zones and purchase choices you allowed, then favor the options with the
best chance of getting your workers running. Other jobs can take a different path. Maybe you're creating
test environments, running a large simulation that can retry later, or processing a backlog of
files with no hard deadline. Cost may matter more than speed in those cases. A price-focused choice
tells Azure to favor lower cost eligible capacity. That can reduce your compute bill, but it can
also mean waiting longer for enough workers. The least expensive choice isn't always available when
you need it, and it may not give you the full amount of capacity immediately. You've probably seen
this outside Azure. A cheap flight is perfect if your schedule is flexible. If you need to be
somewhere for an important meeting tomorrow morning, the lowest fare probably isn't the decision
that helps you. Compute fleet lets you make the same trade-off with compute. Then there's the middle
ground. Many teams don't want to pay the highest price for every worker, but they also can't accept a
fleet that never reaches a useful size. A balanced approach gives Azure a room to consider both price
and available capacity. You set the boundaries and Azure works inside them. That's a much more
honest design than pretending every workload has one simple goal. Some jobs need a deadline,
some jobs need a budget, most need both. The third choice is how you describe the machines themselves.
You can list exact SKUs, which works well when you've already tested a small set of sizes,
but Azure also has an attribute-based approach, currently available in Preview,
where you describe the shape of compute you need. For example, you might describe a worker
that needs a certain amount of CPU, memory, local storage, or an accelerator like a GPU. Azure can
then find VM sizes that match those requirements. Think about the difference. An SKU list says,
use these named models. An attribute request says, "Find machines with these physical needs."
The second approach can reduce the work of updating your fleet definition when newer VM
generations become available, but you still need to test your image and software against the kinds
of machines as you may choose. Don't confuse flexibility with handing over every decision.
You define the safe boundaries and Azure selects within them. So those are the three choices,
which VM sizes or shapes will work, whether capacity or price comes first,
and how broadly you describe the compute you need. Lower cost capacity can make a fleet far cheaper,
but it comes with one rule that every team needs to plan for. Sometimes Azure can take a worker away.
Spot and on demand, cheap seats and reserved seats. So let's talk about the two ways you can
pay for the VMs in a fleet. On demand VMs are the stable option. You ask Azure for a VM,
you pay the normal rate while it runs, and Azure doesn't reclaim it just because another customer needs
that capacity. For work that must keep moving, on demand capacity gives you a dependable base.
Think of it like booking confirmed seats for the people who must be at the event. Your job
controller might run there, your queue service might depend on it, or maybe you need a minimum number
of workers running all night because missing the morning deadline isn't an option. Spot VMs work
differently. Azure sells unused compute capacity at a lower price, sometimes much lower than the normal
on demand rate. That makes Spot useful when you have a lot of work to process and can handle
interruptions. But the lower price comes with a condition. Azure can reclaim a Spot VM when it
needs that capacity back. The VM can disappear while it's working. That doesn't mean Spot is
unreliable in a bad way. It means you need to use it honestly. A Spot VM is not the place for the only
copy of an important database, a single user facing server, or a job that loses hours of progress
when one worker stops. You plan for the worker to leave. This is why a mixed fleet often makes sense.
You keep enough on demand VMs for the work that must continue. Then you add Spot VMs for the
extra processing power. When Spot capacity is available, the fleet can process much more work for
less money. When some Spot workers disappear, the baseline on demand workers keep going. That setup
gives you a practical balance. The stable workers protect your minimum throughput while the Spot
workers help you finish sooner or process more work when Azure has spare capacity.
Compute Fleet helps manage the capacity side of that mix. It can use the allocation choices you
set, look for eligible Spot or on-demand options, and react when Spot VMs are evicted.
Still, Fleet cannot protect work that exists only inside of VM. Your workload design has to do that.
Imagine a video rendering job. One worker renders frame one through frame one hundred.
Another handles the next group. If a Spot worker disappears halfway through a frame,
the system should mark that frame as unfinished and send it back to the queue.
The next worker starts it again. No drama, no loss project. That same pattern works for simulations,
data extraction and transformation jobs, software builds, automated tests, development
environments and genomics processing where many samples can run separately. These jobs work
well because they can split into smaller pieces. One worker can process one file, one worker can
calculate one scenario, one worker can test one build if the worker goes away, another worker can
pick up that small piece later. But a single database server is a very different job.
If the database stores its only copy of the data on a Spot VM, an eviction can interrupt the
service and put recovery at risk. The same applies to stateful software that cannot restart
cleanly or a customer facing application that depends on one VM staying online. Spot doesn't
turn those designs into safe designs. For a Spot-friendly workload, keep progress somewhere durable
outside the worker VM. That might mean a storage account, a database or another service that
records what has finished and what still needs work. Use a durable queue so workers pull tasks one at a time.
Save checkpoints during long running jobs. Make each task safe to retry, so running it again doesn't
create duplicate results or corrupt data. That way, when Azure reclaims a Spot VM, you lose only
the small piece it was processing, not the entire job. This can save a lot of money but cost alone
doesn't tell you whether compute fleet belongs in your design. The next question is whether you need
to fleet at all or whether a virtual machine scale set fits the job better. Compute fleet versus
virtual machine scale sets. You might be wondering, doesn't Azure already have a service called virtual
machine scale sets? It does and the two are related because they both help you run groups of Azure VMs
instead of managing each one by hand. But here's the thing, they solve different problems starting from
different questions. A virtual machine scale set or VMSS for short starts with your application.
Let's say you have a web app, an API or an internal business service, you create a group of VMs with
a similar setup, put them behind a load balancer and let the group grow or shrink as demand changes.
Traffic rises, so the scale set adds more application servers, traffic falls and it removes servers,
so you aren't paying for machines you no longer need. That's a very normal pattern for a long running
application tier. Your VMs usually run the same image, the same application and a closely related
VM setup. Azure can use health checks to see whether the app is responding and it can add or
remove instances based on CPU use, a schedule or another metric you choose. Think of a VM scale set
like a store manager planning a regular team. Every morning the manager needs cashiers, shelf
stockers and people at the service desk. They know the jobs, the location and the basic tools each
person needs. When the store gets busy, they call in more people from the same team. That's what VM
assess does well. It runs a known application pattern and adjusts the number of workers as demand
changes. Compute fleets start somewhere else. Instead of asking out how many web servers does my app
need right now? It asks how can I get a very large amount of acceptable compute capacity for this job?
That distinction changes how you design the request. With a fleet the VM size can vary across the
workers as long as each worker can perform the task. You might accept several machine families,
use different purchase options and let Azure choose a workable mix based on your rules.
Think of compute fleet as a staffing agency trying to fill thousands of shifts before a deadline.
The agency doesn't care whether every worker wears the same shoe size, it cares whether each
person can do the work, whether enough people arrive on time and what the full shift costs.
That's a much better fit for a job that can split across many independent workers, a large data run,
a simulation, a rendering project or a big set of automated tests. The difference isn't that one
service is better than the other, they solve different problems. So choose virtual machine scale
sets when you run a steady application tier that receives traffic. That could be a customer
facing website, an API, a line of business app or a group of application servers behind Azure
load balancer or application gateway. In those cases you usually care about application health
and traffic handling. You need instances with a known setup, you need a way to add servers when
users arrive and remove them when demand drops. VMSS fits that model. Choose compute fleet when the
main challenge is getting a lot of compute for work that can spread out. The work might last
for hours, days or only a short burst but it doesn't depend on one identical application server
receiving user requests. Fleet works well when a worker can take a task, complete it,
save the result and move on. That's common in high performance computing, batch processing,
large simulations, data processing and large temporary worker pools. There's also a practical point
that trips people up. Compute fleet does not replace your application design. It doesn't replace
the queue that hands out work, the checkpoints that save progress or a load balancer for a web app.
And it doesn't turn a single stateful server into a fault tolerant system. As you can help acquire
the VMS but your software still needs to know what each VMS should do, where its data lives,
and what should happen if a worker stops halfway through a task. That shared responsibility matters.
As your manages the physical servers and the fleet capacity rules, you design the workload so
it can survive the normal events of cloud computing, including a VM restart, a failed task or
an interrupted worker. So a simple decision guide looks like this. If you're scaling an application
because user traffic changes, start with virtual machine scale sets. If you're gathering a large
amount of flexible compute because a job needs many workers, look at compute fleet. And if your
architecture includes both needs, you can use both services. A VM scale set can run the customer
facing application tier, while compute fleet supplies temporary workers for the heavy background job.
Picture a team that needs thousands of workers overnight to finish a calculation before the next
business date. That's where these choices stop being product names and start changing the outcome.
A fleet example. Finishing a large job without betting on one VM size. Imagine a finance team that
runs risk calculations every night. They need to process a huge amount of market data,
test thousands of possible outcomes and produce results before people arrive the next morning.
The team has one clear goal. Finish the calculations by morning without paying the normal
on-demand rate for every single worker. Their first attempt looks simple. They choose one compute
focused VM size, request a large number of workers, and start the job. Then as you cannot find enough
of that exact size in the region. Nothing is wrong with the VM size or the code. There just
isn't enough capacity for that one exact choice when the team needs it. If their whole design
depends on that one SKU, the job starts late. A late start means fewer results by morning or a rushed
and expensive move to another setup. So the team changes the request. Instead of demanding one VM size,
they identify several CPU focused sizes that their risk software can use. Each option has enough
memory for a task, enough CPU for a calculation, and works with the same VM image. They also allow more
than one availability zone where their data and workload can run. That gives Azure more places to
find workers. Next, the team decides how much capacity must stay available no matter what. They choose
a group of on-demand VMs as their reliable base. Those workers give the job a minimum processing rate
throughout the night. Even if lower cost capacity becomes scarce, the run continues. Above that
base they allow spot VMs. Those spot workers do the extra work. When Azure has spare capacity,
the team can process many more calculations without using on-demand VMs for every task.
The request now says something like this. Give us enough stable workers to guarantee a minimum
result, then add as many lower cost workers as you can from this approved list of VM sizes.
That changes the situation. The team isn't betting the whole overnight run on one size, one zone,
or one price type. But there's another part of the design that matters just as much as the fleet
request. The risk calculations cannot live only inside the worker VMs. Before the fleet starts,
a job system breaks the large calculation into small pieces. One task might calculate the outcome
for one market scenario. Another might handle a different portfolio or time range. Each task enters a queue.
A worker takes one task, processes it, writes the result to durable storage,
and then asks for another task. Progress lives outside the worker. That means a worker can
disappear without taking the whole job with it. Supposer spot VM receives an eviction
notice halfway through a calculation. The VM stops. The unfinished task remains marked as incomplete
in the queue. Another worker, perhaps a new spot VM or one of the on-demand workers, can take that
task and run it again. The team loses one small piece of work, but they don't lose the whole overnight
calculation. And boom, that's the real benefit of a fleet-friendly design. Azure handles finding
and managing the compute capacity. The team's job system handles the work, retries, and saved results.
Neither side can do the other side's job. Compute fleet cannot decide whether rerunning a calculation
creates the wrong financial result. The team has to make tasks safe to retry and prevent duplicate
processing. At the same time, the team doesn't need custom scripts that search through every approved
VM size and keep trying to create workers one group at a time. They define the acceptable choices
and let Azure work within those rules. By morning, the exact number of spot workers may have changed
during the night. That's expected. The stable workers kept processing. Extra spot workers increased
throughput when capacity existed, interrupted work returned to the queue. The final cost stayed
under more control than an all-on-demand run. This isn't a promise that every task will finish
at the lowest price. It's a way to avoid tying the whole job to one narrow capacity choice.
The same pattern appears in life sciences. A research team may need to process a large set of
genome samples. Each sample can move through the same analysis pipeline, but each sample is separate
from the next. The team stores input data and results outside the worker VMs. A worker processes one
sample or one stage of a sample, saves the output and moves on. If that worker stops, another worker
resumes from the last save point. Compute fleet can supply the large, flexible worker pool,
but the pipeline still needs to track what finished, what failed, and what must run again.
Before you put a fleet into a real design though, there are a few decisions that prevent you from
choosing it for the wrong kind of workload. When to use it and what to decide first. What exactly
is a compute fleet workload? It's work that can spread across many independent workers. That's the
starting point. Here's the simple question you need to answer. If one worker stops can another worker
take over without you having to fix the job by hand? If the answer is yes, you probably have a good
fleet workload, batch processing, test workers, analysis jobs, and large file processing often fit
because the work splits into separate units that run in parallel. Start with the workload behavior,
not the VM name. Think of it like a team of freelancers. If one drops out, another picks up the task
without losing progress. Can the work run without keeping its state inside one VM? If it needs state,
can it save progress somewhere durable and restart from that point? Can the same task run again
safely if a worker never reports back? Those answers tell you far more than any pricing page ever will.
Now decide the minimum capacity that must stay available. This is the part of the job you cannot
afford to lose if lower cost capacity becomes unavailable or gets interrupted. Use on-demand VMs for
that baseline then decide how much extra capacity you are willing to add through spot VMs. That number
should come from your deadline, your job backlog, and your budget. Not from a guest that spot will always
be there. After that define your VM choices carefully. You might approve several related VM families
that your software already supports. Or if the preview feature fits your design, you can describe
the CPU, memory, storage, or accelerator needs and let azure pick matching machines keep the list
practical. A narrow list limits the places as you can search. An overly broad list can put your
workload on machines that behave differently from the ones you tested. Every allowed choice needs
to work with your image, your network setup, and your software. You also need a clear ceiling.
Set the target capacity for the job then set a maximum cost exposure before launching.
Large compute jobs can grow fast when a queue contains more work than expected. Or when a deadline
pushes more work onto on-demand capacity. A budget is not just a fine and setting, it is a design
boundary. Where your workload and region supported spread the fleet across availability zones.
This gives azure more placement choices and reduces the chance that one local capacity issue affects
every worker. Keep an eye on the job, not just the VM count. Monitor the capacity fleet has actually
acquired. Watch the queue backlog, failed tasks, retry count, eviction events, completion rate,
and cost. A fleet with many healthy VMs can still be failing the business goal if the queue is growing
or results are not being saved. Here's the mistake most people make. They treat compute fleet as
automatic application recovery. It does not do that. It manages capacity. Your job system still
needs to recover the work. Once those pieces are in place, one flexible request can replace a
pile of manual skew checks, separate spot plans and scripts that retry capacity one VM group at a time.
So pick one workload that already runs as a batch job. Or one that you can safely retry.
Write down five VM choices that can genuinely run it. Not five names you found in a list,
but five tested options your software supports. Then next to that list write down four more things.
The minimum on demand capacity you need. The extra spot capacity you can use. Where progress is saved.
And exactly how an unfinished task returns for retry. Add a cost limit before you launch.
That small design exercise turns compute fleet from a product name into a real plan.
It changes the request from I need this exact VM to I need this work completed and these compute
choices are safe for long running application tiers that serve user traffic continue with virtual
machine scale sets next Azure can find the workers but your workload design decides what happens when
one disappears.