Your copilot can talk, it understands your context,
and it can even reason through multi-step problems
to give you a response that feels human.
But the moment you ask it to do something computationally
heavy, everything falls apart.
If you needed to validate data across three legacy systems,
transform that data, and apply specific business rules
before giving an answer, it hits a wall.
That wall is the connector architecture.
Standard out of the box connectors are great
for moving data from point A to point B,
because that is exactly what they were designed to do.
However, they were never built for heavy computation
or complex orchestration.
They definitely weren't built to handle the deep business
logic that separates a demo-friendly chatbot
from a system that actually gets work done.
The real gap here isn't between what copilot can ask
and what your data can answer.
In reality, the gap is between what the bot can say
and what your business actually needs to do.
We are solving that problem today.
We are moving away from a bot that just talks
and building a bot that actually executes.
The bridge to get there is pro-code as your functions,
but they have to be architected the right way to work.
Why standard connectors fail at scale?
Most teams don't see this problem until it is too late
to easily fix it.
You start with a use case that looks simple on the surface,
like a user asking about a specific invoice.
To answer, the bot has to check three different systems,
validate the information and apply your business rules.
Naturally, you try to build this using standard connectors.
The first issue is that these connectors
are built for simple data movement
rather than complex logic.
They are perfectly fine if you just need to get a record
or update a single field.
They are not fine when you need to transform data
and check it against multiple systems
to return a structured result.
When your copilot needs to perform real work,
these standard connectors quickly become a massive bottleneck.
You find yourself chaining them together
where one connector calls another
and then that one branches off based on a dozen conditions.
You end up nesting logic, deeper and deeper
and every single new connection adds more latency to the process.
This is what I call the latency tax.
Every time a connector makes a call,
you pay for the network, roundtrip,
the authentication and the overhead of moving that data.
If you have a complex workflow,
it can easily take 10 seconds or more for the user to get an answer.
To the person waiting, it feels like the bot has crashed
and usually they just give up and leave.
Then you have to deal with governance and DLP policies.
These rules are designed to stop data
from moving between approved and blocked connectors,
which is security doing its job.
But this also creates silos
where your finance tools can't talk to your HR tools
without breaking a policy.
You are constantly hitting walls
that make sense for protection,
but absolutely poison your technical architecture.
At the same time, the low-code expressions in Power Automate
have very real limits on what they can execute.
You can handle basic string changes or simple logic,
but they aren't built for heavy lifting
or consuming external APIs efficiently.
They simply weren't designed for the high-level work
that your business actually requires to function.
Because of these limits,
teams start building messy workarounds.
They create nested flows that call other flows
or hide data in queues to try and delay the work.
These integrations are brittle
and they break the second your business requirements change.
You are essentially trying to solve a connector problem
by stacking one workaround on top of another.
The reality is that none of this is maintainable or scalable
because it isn't a real engineering solution.
The core problem is that you are trying to do pro-code work
using low-code tools.
Connectors aren't bad,
but they are simply the wrong tool for this specific job.
What you actually need is total control over your environment.
You need type safety, enterprise libraries,
and the ability to use caching strategies or connection pooling.
You need retry logic that actually understands your business
and monitoring that catches a bug before a user ever sees it.
That is why we use Azure Functions.
It gives you a blank canvas where you use your own language
and your own libraries.
Your logic sits right next to the power platform
but it stays connected through a hardened secure gateway.
On paper, this architecture looks simple
but it is often very hard to execute in practice.
That is exactly what we are going to fix right now.
The architecture choice, where the rubber meets the road.
So what is the actual alternative here?
It is not about replacing the power platform entirely
which is the exact mistake most teams make the moment they hit a wall.
They assume the only answer is to rip everything out
and rewrite the whole system in net.
But that is not the right move.
The real answer is to extend the power platform
with the very things it was never designed to handle in the first place.
Azure Functions sit entirely outside that low-code perimeter.
They are not part of the power platform ecosystem.
They are not governed by those same restrictive rules
and they are not throttled by those frustrating execution limits.
Think of them as a blank, C-bath canvas
where you can write real code that is type safe,
fully testable, and connected to your existing enterprise libraries.
They provide everything that power-automate expressions simply cannot be.
But here is the critical part of the strategy.
These functions are not disconnected from the power platform either.
They plug right back into it through a clean interface
using open API specs and custom connectors,
which acts as the handshake between these two different worlds.
When your co-pilot asks the power platform to perform a task,
the platform does not struggle to do it with standard connectors.
Instead it calls your function.
Your function handles the heavy computation
and returns structured data
and then the power platform passes that result back to co-pilot.
The entire process feels completely seamless to the person using it.
This is what we call the fusion developer model
and it is quickly becoming the standard for enterprise AI
because it is the only architecture that actually works when you try to scale.
Now as your functions come in a few different flavors
and choosing the right one is the first real decision you have to make.
You have the consumption plan, which is the original serverless model
where the system scales all the way down to zero.
You only pay for what you actually use,
but the cold starts are absolutely brutal
and we will come back to that specific problem in just a minute.
Then you have the premium plan,
which uses always on instances that stay warm,
so cold starts are never an issue.
The downside is that you are paying a baseline cost
regardless of how much traffic you actually have.
If you are building a co-pilot plugin
that only gets cold during normal business hours,
that becomes a very expensive way to operate.
Finally you have Flex consumption,
which is the new default as of 2026
and the right choice for almost every co-pilot architecture.
Flex gives you the best of both worlds
because it is still serverless and scales dynamically,
but you can configure always ready instances that stay warm.
This creates a small pool of workers
that are always initialized and ready to serve requests immediately.
The baseline cost for an always ready instance is about $0.000004
per gigabyte second,
which is six and a half times cheaper than on-demand execution time.
You get those warm starts without the massive premium price tag.
Why does a warm start matter so much for a co-pilot?
It matters because cold start latency is catastrophic for interactive AI.
When a user types a question, co-pilot processes it and calls your function,
but then your function has to wake up.
It has to load the runtime, initialize every dependency,
and de-serialize the request before it even starts running your code.
This entire process can take five seconds or more from a cold start,
leaving the user staring at a loading spinner
until they eventually switch their mental context.
By the time your function finally returns a result,
they have already moved on to something else.
Flex with always ready completely eliminates that friction.
Because your instances are already warm,
the runtime is loaded,
and your dependencies are already initialized.
When a request comes in, your code runs and the response goes out
with sub-100 millisecond latency.
The user perceives the interaction as instant,
which is exactly what you want.
Using C-plus in an isolated worker model gives you a level of control
you just cannot find anywhere else in the power platform ecosystem.
You get type safety from the compiler,
testability through dependency injection,
and access to any new get package you might need.
You can use the exact same libraries,
your backend teams are already using like entity framework
for database queries, poly for resilience,
or open telemetry for instrumentation.
You can even manage your secrets through Key Vault,
providing everything a real production system
actually needs to survive.
The connection between your co-pilot and your function
lives and dies by the open API spec.
This is a critical piece of the puzzle
that we will spend a lot of time on later.
The open API spec is not just a piece of documentation,
but rather the semantic contract that tells the LLM
when your function is relevant and exactly how to call it.
If you write a poor spec,
co-pilot will not use your function even when it should,
but a well-written spec ensures co-pilot calls it reliably
and passes the correct parameters every single time.
Your function lives in Azure while your co-pilot lives
in the power platform,
and they communicate through open API and custom connectors.
You write all the complex logic you need in CYs
while the power platform handles the orchestration
and the conversation flow.
This clear division of labor is what separates
a working professional plugin from a collection
of janky workarounds.
It might not be the sexiest solution,
and it is not purely low code or purely pro code,
but it is the real answer to the problem.
The cold start problem and why flex consumption wins.
Cold start is not just a minor latency issue,
but rather a massive invisible overhead
that triggers every single time your function wakes up
from a nap.
To understand why this happens,
we have to look at what occurs when a function goes cold.
Because nobody has called the function recently,
Azure has scaled your instance all the way down to zero
to save resources.
When a new request finally arrives,
the platform has to allocate a brand new worker,
start the net runtime and load your assemblies from the disk.
It then has to perform GT compilation on your code
and initialize your entire dependency injection container.
It still has to establish connections to your database,
your cache, and any other external services
before a single line of your business logic
even starts to run.
On the original consumption plan,
this entire sequence happens on every single request
whenever your traffic is sparse.
You are paying for compute time
from the very moment the request arrives
until your code actually finishes executing.
For a typical net function using a few enterprise libraries,
that overhead usually lasts between one and five seconds.
If your application has heavy initialization requirements,
like loading machine learning models
or building large dependency graphs,
that wait time can easily stretch to 10 seconds or more.
The timeline for the user is just brutal.
They type a question, co-pilot processes it,
and then your function takes two full seconds just to wake up.
Even if your business logic runs in a lightning fast 100 milliseconds,
the total perceived latency is still over two seconds.
The user notices that delay, the bot feels sluggish,
and the frustrating part is that nothing in your actual code is slow.
The problem is entirely tied to the initialization process.
This is the exact point where flex consumption diverges
from the old way of doing things.
Flex introduces the concept of always ready instances,
where you configure a minimum number of instances
that stay perpetually initialized.
These instances never scale to zero,
so the runtime stays loaded, your dependencies stay ready,
and your database connections stay pooled.
When a request arrives, there's no startup overhead at all
because your code runs immediately.
We start measuring latency in tens of milliseconds instead of seconds.
The cost structure is what makes this approach economically sensible
for most businesses.
Always ready instances are built at a baseline rate
of approximately 0.000004 dollars per gigabyte second,
even when they are sitting idle.
That is incredibly cheap.
If you run a single always ready instance
with 512 megabytes of memory,
you are looking at a bill of roughly $15 per month.
When you compare that to the premium plan,
where the smallest tier keeps instances running
at a much higher baseline cost,
flex becomes the obvious choice for anyone who is cost conscious.
Here is the key insight you need to remember.
Always ready instances do not eliminate the cold start problem
for every single scenario,
but they do eliminate it for your baseline traffic.
If your co-pilot generates 10 concurrent calls
and you have two always ready instances,
those first two requests will get warm performance.
The remaining eight requests will trigger new instances
that have to go through a cold start.
In the real world, most concurrent co-pilot calls
stay within your always ready capacity
and feel instantaneous.
You only see minor start-up latency
when your demand genuinely exceeds that provisioned baseline.
The premium plan takes a much more aggressive approach
by keeping minimum instances running at all times
without any scale to zero behavior.
Cold start is eliminated across all of your traffic
because those instances never go idle,
but you definitely pay for that guarantee.
You are stuck paying an hourly rate
for reserved compute regardless of whether your function
handles zero requests or a thousand.
If you are running a use case that only happens
during business hours, that pricing model
simply does not make sense for your budget.
Flex with one or two always ready instances
hits the perfect sweet spot for most enterprise teams.
Your baseline co-pilot traffic gets handled
with sub-hundred millisecond start-up latency
while your burst traffic scales dynamically
without you having to pay a massive premium.
Your monthly cost stays reasonable
and you have successfully eliminated
the single biggest performance problem
that plagues serverless AI applications today.
This is the very first technical decision
that separates a real production architecture
from a simple prototype.
Open API specs, the bridge between code and AI.
Now we get to the craft
that actually separates a working plug-in from a broken one.
Your as-your-function might be fast,
your always ready instances are warm
and your seawars code is solid,
but none of that matters if co-pilot never calls
your function in the first place.
That is the job of open API.
It tells the AI when your function is relevant
and exactly how to invoke it.
But here is the problem.
Most developers treat open API like documentation
or something you generate after the fact
just to keep the API docs in sync with the code.
They see it as a boring checkbox on a deployment checklist.
But that is not what open API is anymore.
In reality, open API is the semantic contract
that the language model uses to reason about your function.
It is the specification the AI reads
to decide whether to call you at all
and it serves as the instruction manual
for filling in parameters and interpreting results.
The distinction matters
because documentation is written for humans.
While open API specs for co-pilot
need to be written for models.
Those are not the same thing.
Power platform currently standardizes on open API 2.0,
which you might know as swagger, though support for 3.0
is expanding through 2026.
If you are building a production plug in today,
you should target 2.0 for broad compatibility.
The concepts we are discussing apply to both versions,
but the tooling landscape still leans heavily on swagger.
It starts with the operation ID.
This is the identifier for each endpoint in your spec
and it needs to be stable and verb based.
It should describe what actually happens
rather than what the internal code does.
If you compare validate invoice against process V2,
one tells the model exactly what to expect
while the other tells the model nothing.
The model will try to choose correctly based on context,
but if you force it to guess, it will eventually guess wrong.
When that happens, your entire plug and fail silently.
Descriptions are where most specs fall apart
because developers write descriptions for other developers.
They explain how the endpoint works,
the response format or the parameter validation rules,
but the model does not read like a developer reads.
The model needs to know when to use this operation.
What user intent triggers it and what specific problem it solves.
Compare two descriptions of the same function.
One says returns invoice data from the accounting system
while the other says use this when the user asks
about a specific invoice to retrieve details
including amount, date, vendor and payment status.
The first tells the model nothing useful,
but the second tells the model exactly when to call this operation.
Parameter descriptions work the same way.
You are not writing for developers who can infer the intent,
but for a model that will pass your description to fill in values.
You need to be explicit about format constraints.
Instead of just saying date, you should say invoice date
in ISO 8601 format and do not use relative terms
like yesterday or last quarter.
The model will try to send relative terms
if you do not explicitly forbid them.
And you want to prevent that failure before it happens.
You also need to include examples.
Do not just provide one, but use multiple examples
showing realistic inputs and outputs.
The model uses these as demonstrations of correct usage
and a well-crafted example is worth 1,000 words of explanation.
Response schemas also need to be complete.
You should define every status code your function can return,
including 200 for success, 400 for validation errors,
404 when the resource is missing,
and 429 when rate limits are hit.
For each response, you must describe what it means.
The model uses error schemas to understand failure modes.
So if you do not define what a 400 error looks like,
the model cannot learn to anticipate failure.
It cannot adjust its behavior
and it cannot explain to the user why something went wrong.
This is the bridge between your code and the AI.
If you get it right, co-pilot will use your function reliably,
but if you get it wrong, your function will sit unused
even when it is the perfect tool for the job.
The irony is that this spec is often more important
than the code itself.
Writing open API for LLM reasoning.
Most teams skip this step because they assume
that having a technically correct open API spec is enough.
It isn't.
The model reads your description like a user reads documentation.
So if your language is vague, the model makes vague inferences.
It misses context and calls your operation when it shouldn't
or worse, it doesn't call your operation
when it absolutely should.
One level deeper, we have the problem of operation collisions.
This happens when you have two endpoints
that sound similar from the model's perspective,
like get customer by name and search customers.
Both retrieve customer information
and both accept a name parameter.
So the model has to choose which one to call
based on your descriptions.
If those descriptions are interchangeable,
the model will pick randomly and half your queries
will go to the wrong operation.
The fix isn't to rename everything,
but to be explicit about the scenario each operation handles.
Get customer by name should say, use this
when the user provides an exact customer name
and you need to retrieve a specific record.
Meanwhile, search customers should say,
use this when the user is asking about customers
but hasn't specified a particular name
such as when they ask to see customers
from a specific region.
Now the model can differentiate and it will choose correctly.
Parameter examples are also non-negotiable,
while standard documentation might include examples,
open API specs for co-pilot must include them.
The model uses examples as demonstrations of correct usage.
When you write an example of 2020-4015 for a date parameter,
you are teaching the model that this is the format you expect.
You are preventing it from sending yesterday
or last month or any other natural language expression.
A single well chosen example prevents
entire categories of failures.
Error schemers get overlooked constantly
because developers focus on the happy path.
But the model needs to understand failure parts too
and it needs structured error information to do that.
A consistent error schema with a code, a message,
and details tells the model how to interpret failures.
It learns to recognize when a call failed and why,
which allows it to adjust its behavior
or explain the failure to the user.
Without that structure, errors become opaque
and the model cannot learn from them.
Consider a practical example where your invoice validation
endpoint returns a 400 status when the invoice number
doesn't exist.
If your error response is just a message saying not found,
that is technically correct, but semantically useless.
The model sees a 400 error and knows something went wrong,
but it doesn't know to ask the user for clarification
or suggest trying a different number.
Compare that to a structured response
that includes an invoice not found code
and a suggestion to check the number and try again.
Now the model understands the failure mode
and can recover intelligently.
Finally, you must curate your surface area.
The temptation is to expose every endpoint you have built
because it feels comprehensive,
but in reality that is a disaster.
Every operation you expose adds cognitive load to the model
and every extra operation increases the chance
of a wrong selection.
Every unused endpoint in your spec is just technical debt.
You should remove operations that aren't high value
and hide internal administrative endpoints.
Only expose what co-pilot actually needs to do its job.
This curation process is where your architecture
becomes intentional instead of accidental.
C# Azure Functions, the engine.
Now let's talk about the actual code
that powers this architecture.
Your co-pilot is fast and your open API spec is semantic
and clear, but the function itself is where precision matters.
This is the code that actually runs.
This is where the theory meets the work.
HTTP triggered functions are your standard entry point.
Co-pilot calls an endpoint, your function receives the request,
it processes data and then it returns a response.
This request response pattern is fundamental.
Everything else in this section is about making that cycle
as efficient as possible.
The isolated worker model in Net6 and above
gives you control that you don't get in earlier versions.
You own the startup sequence and manage the middleware,
which means you configure dependency injection
at the application level rather than the function level.
You control what gets initialized
once per application lifetime versus once per request.
This control is exactly how you eliminate startup waste.
Here's the critical pattern.
Use Async and await throughout your entire stack.
This shouldn't just be in your function handler,
but in every layer from database queries
to HTTP calls and cache operations.
The reason is simple.
Blocking calls waste your always ready capacity.
When your code blocks while waiting for a database query,
that thread just sits there idle.
A thread is a scarce resource
and your instance can only handle a limited number
of concurrent requests before blocking starts
queuing up the work with Async and await
that thread returns to the pool.
So your instance can handle the next request.
Your always ready capacity suddenly
serves 10 times the throughput.
This isn't just a performance nicety.
It is architectural.
Without Async, your cold start latency
won't even matter because your warm latency will be terrible anyway.
Next, let's look at singletons.
Your HTTP client, your database connection pool,
and your caching client should all be initialized
once when your function app starts.
Most developers get this right for the HTTP client,
but they miss it for other clients.
A database connection takes time to establish,
and a rate is client needs to authenticate.
So these operations are expensive on every new initialization.
They are only cheap on the second call
because pooling handles the heavy lifting.
Dependency injection handles this elegantly
if you register your clients as singletons
and inject them into your function handler.
Every invocation then reuses the same instance,
but there's a counter-intuitive pattern here.
Use lazy initialization for expensive dependencies.
If your function loads a heavy ML model on startup,
that model consumes memory and CPU while it sits there unused.
If only half your invocations actually need that model,
you're wasting always ready capacity on something
that isn't helping you.
The solution is the lazy pattern in Seahorse.
The dependency initializes on the first actual use
rather than at app startup.
Subsequent uses hit the cached instance instantly.
This is how you keep cold starts low
while still having access to heavy dependencies
when they are needed.
Ready to run publishing is another overlooked lever.
It's a simple flag you set during the build process.
Instead of your net assemblies being jade compiled
at run time on the first use,
they're pre-compiled ahead of time.
Your function starts and all the code is already ready
to execute.
There is no jit overhead and no compilation delay.
For latency critical functions,
ready to run saves hundreds of milliseconds
on a cold start.
It also provides faster warm execution.
It's a pure win and the trade-off
of a slightly larger deployment package
is irrelevant compared to the latency gain.
These patterns are not optional for production
co-pilot plugins.
Using async throughout singletons for clients,
lazy initialization for expensive work
and ready to run publishing is the difference
between a function that takes two seconds
and one that takes 100 milliseconds every time.
The actual business logic is usually straightforward.
You validate invoices query databases
or call downstream APIs.
But the infrastructure around that logic determines
whether your plugin feels instant or sluggish.
This is where developers who understand serverless architecture
separate themselves from developers
who just run code in the cloud.
Securing the function identity and network.
A fast function that leaks data is worse
than a slow function that stays secure.
Performance is meaningless without a hardened perimeter.
Your function lives in Azure, handles requests
from power platform and calls downstream systems.
It touches sensitive business data.
Every connection point is an attack service.
Every decision you make about authentication
and network topology determines whether your plugin
is secure or vulnerable.
Start with identity.
Managed identities are the foundational security pattern
for Azure workloads.
Instead of storing API keys or connection strings
in your configuration,
your function app gets an identity issued
by Microsoft EntraID.
When your function needs to authenticate
to Azure SQL or Azure Storage,
it proves its identity cryptographically.
There are no secrets to rotate and no keys embedded in code.
There are no credential leaks waiting to happen
because the Azure platform handles all of it.
Your only job is to request the identity
and assign it the minimum permissions it needs.
This pattern extends beyond Azure services.
If your function needs to call a REST API
that supports EntraID,
your function simply authenticates as itself
using that managed identity.
It requests a token from EntraID
and passes that token in the authorization header.
The downstream service validates the token
and grants access based on the permissions assigned
to your functions identity.
The entire handshake is cryptographic
so no secrets ever touch your code.
Now we have to consider who can call your function.
By default, an HTTP triggered function in Azure
is protected by a function key.
It's a simple bearer token that you pass
as a query parameter.
Azure validates it and while this works for internal tools,
it doesn't work for production co-pilot plugins.
A key is a secret and secrets get leaked or shared.
Eventually, secrets stop being secret.
Microsoft EntraID authentication is the production pattern you need.
You register your function app as an application in EntraID
and configure it to accept tokens issued for that application.
Power Platform's custom connector
creates an app registration in your tenant
and you grant that app permission to your function.
When the connector calls your function,
it includes a JWT token that proves it's authorized.
As your validates the token signature,
which is cryptographically bound to your function.
Only calls from Power Platform with a valid token can reach your code.
Unauthorized callers get rejected at the platform level
before your code even runs.
Network topology matters just as much.
By default, your function is publicly accessible
and the internet can reach it.
That's convenient for testing, but it's a disaster for production.
Private endpoints change this equation.
You configure a private endpoint for your function
so traffic routes through Azure's internal network.
Your function has no public IP address,
which means there's no way to reach it from the internet.
It exists only inside your vnet
and is accessible only to resources
that have authorized access to that vnet.
This private topology creates a hardened perimeter.
Power Platform reaches your function through a private connection
and the downstream systems your function calls
are also behind private endpoints.
Traffic never leaves the Microsoft network.
There's no exposure window and no internet route
where packets could be intercepted.
The on-behave for flow is a subtler but crucial pattern.
When a user interacts with a co-pilot,
they're authenticated to Power Platform
under their own identity.
When your function calls a downstream system,
it should act on behalf of that user rather than itself.
The obi-o-flow handles this by receiving a token
that represents the authenticated user.
It exchanges that token for a downstream token
that preserves the user's identity.
The downstream system logs the action under the user's account,
which is how you maintain audit trails
and enforce user-level authorization.
API management sits in front of your function
to apply rate, limiting, and throttling.
A single misbehaving flow can't overwhelm your backend
because API am enforces quotas at the platform level.
Your function doesn't need to implement this itself
or check rate limits.
It doesn't need to decide whether to accept or reject
a request because APM does that
before traffic ever reaches your function.
Logging completes the picture.
Every request should be logged with a correlation ID
that traces it through your system.
User context and operation outcomes matter
and application insights is where this telemetry lives.
It's queryable, it's retentable, and it's auditable.
These patterns are the foundation
of a secure production plugin.
Managed identities, enter ID authentication,
private endpoints, obo flows, and comprehensive logging
aren't optional if you want to build something professional.
The DLP bypass risk and how to mitigate it.
Your function is secure, your custom connector is authenticated,
and your network is hardened.
But there is a structural vulnerability sitting right
in front of you that most teams simply do not see
until it becomes a massive problem.
Custom connectors can be weaponized
to bypass your DLP policies.
This is not just a theoretical risk or a what-if scenario.
It is documented.
Security research has shown that a custom connector
can internally reach a service that is explicitly blocked
by your organization's DLP policies.
The blocked connector stays blocked at the policy level,
but the functionality becomes accessible anyway,
because it is being proxy through something
that is not on the block list.
Here is how that actually happens in a real environment.
Your organization has sensitive data in Salesforce,
and your CFO makes a policy decision
that Salesforce cannot connect to public cloud storage.
A DLP rule is created, Salesforce moves to the blocked category
and suddenly nobody can create flows
that move data from Salesforce to Dropbox or OneDrive.
The policy is enforced, and on the surface, everything looks fine.
But then someone builds a custom connector
that calls a webhook hosted in their personal Azure account.
The custom connector sends Salesforce data to that webhook
and the webhook writes it to Dropbox.
The entire path goes around the DLP rule
because the blocked connector never appears
in the flow definition.
The policy audit shows nothing,
the custom connector is marked as business approved,
and everything looks legitimate.
Now, your sensitive data is sitting in a storage account
that is completely outside of your organization's control.
This is the structural vulnerability you have to face.
Custom connectors are integration points
that can proxy traffic to restricted destinations
and unless you treat them as privileged assets,
they become escape hatches from your entire governance model.
Your first instinct might be to ban custom connectors entirely,
but that is the wrong move,
that kills legitimate use cases
and stops your developers from integrating
with systems that Power Platform does not natively support.
Baning them does not solve the problem.
It just creates pressure for people
to find even more dangerous workarounds.
The right answer is architecture.
The custom connector should never have direct access
to your downstream systems.
Instead, it should root through a hardened gateway
like API management.
When you put APM in front of your Azure functions,
it enforces your policies before any traffic ever reaches
your function code.
APIM becomes your central control point.
It validates that the caller is authorized,
it checks rate limits,
and it logs every single request that comes through.
But critically, it can enforce endpoint filtering.
You define exactly which domains the API is allowed to reach
and everything else gets blocked.
Your custom connector can call your function a thousand times,
but if that function tries to reach a blocked service,
APIM rejects it.
The policy is enforced at the gateway level
rather than at the individual connector.
This requires a fundamental mindset shift.
You have to stop thinking of the custom connector
as a black box that you trust to do the right thing,
start thinking of it as a potential attack surface
that is privileged, restricted, and constantly monitored.
Operationally, this means you need real governance
around who can create these connectors.
You cannot let every developer build them.
You need a small, specialized team.
You also need to limit who can modify them
and require formal approval
before anything is deployed to production.
Every connector must be treated as a tracked asset.
The open API spec for your function
becomes part of your enforcement strategy.
You do not expose operations
that would let someone call restricted services
or upload files to arbitrary storage.
You do not expose operations that bypass authentication.
You design your functions contract
with the assumption that the caller might be hostile.
This might sound paranoid to some,
but it is actually just realistic.
Most people do not intentionally try to break the rules,
but they do find creative solutions to business problems
without understanding the security implications.
A hardened perimeter prevents those good intentions
from turning into serious security incidents.
API management as the security front door.
This is where your architecture starts to get sophisticated.
API management is the component
that takes a collection of scattered security concerns
and transforms them into a unified enforcement layer.
In the operational reality of the setup,
API management sits directly between power platform
and your Azure functions.
It is the only thing your custom connector can actually reach.
Your function does not have a public endpoint
and there is no direct route from the connector
to the function code.
Everything has to flow through APM first.
This is a decisive architectural choice
because it makes APM your single point of control.
Every request is inspected, logged,
and evaluated against your policies
before it ever touches your back end.
Think about how this simplifies authentication.
Your custom connector might use API keys,
while your downstream services use
"EntraID" with certificates
and your internal API is "Use OAuth".
APIM can accept all three of these types at the edge,
normalize them into a standard format
and then pass a clean request to your function.
Your function does not need to understand
the complexity of certificate validation or OAuth flows.
It just receives a header saying the request
is from a trusted caller and proceeds with the work.
This decoupling makes your entire architecture
much easier to maintain over time.
The real power of APM emerges when you start using policies.
A policy is just a rule that runs on inbound
or outbound traffic.
You can use them to inject headers, validate JWT tokens
or transform request bodies by stripping out fields
your function does not need.
You can log every request to a security system
for auditing or enforce schema validation
so your function never sees malformed data.
This is what we call preventive security.
Your function does not get garbage input
because APM rejects it at the door.
It does not get called by unauthorized users
because APM validates credentials before forwarding the request.
You do not have to write basic security checks
into your code because the gateway has already handled them.
Rate limiting and quota management also live at this layer.
You can define exactly how many requests per minute
an API key can send, which prevents
a single misbehaving flow and power platform
from overwhelming your backend.
If a flow starts sending millions of requests,
APM throttles it with a too many requests response.
This forces the flow to back off
and ensures your function stays available for everyone else.
Without this, a simple bug in a flow
becomes a denial of service attack
against your own infrastructure.
APM also allows you to route traffic intelligently.
You can set up rules to send traffic
from specific IP addresses to a canary end point for testing.
You can run A/B tests by sending a small percentage
of traffic to a new version of your function
while the rest stays on the stable version.
This gives you granular control over roll outs.
Instead of switching versions all at once
and hoping for the best, you can move slowly and reduce risk.
You should always publish your open API spec from APIM
rather than the function itself.
Your function's actual URL might be hidden deep in Azure
and power platform should never see it.
When power platform imports the spec,
it only sees the APIM endpoint.
This ensures the spec always matches the enforced behavior.
You cannot accidentally bypass the gateway
because the connector has no idea the function endpoint
even exists.
This architecture finally answers the big question
of how to stop custom connectors from becoming bypass mechanisms.
You eliminate the bypass entirely.
The connector cannot reach anything except the gateway
and the gateway enforces policies
that are immutable to the average user.
There is no side channel and no workaround.
APIM is the enforcement surface
that turns your governance intentions into a technical reality.
It is the difference between having a policy on paper
and actually preventing data from leaving your environment.
This is the point where security stops being a document
and starts being part of your infrastructure.
Durable functions for stateful workflows.
Most of the patterns we have discussed so far
follow a very simple logic where a request comes in,
the function processes it and a response goes out immediately.
The single execution cycle works perfectly
for things like data transformation or basic validation
but real enterprise workflows almost never fit into that tiny box.
Imagine your co-pilot user needs a multi-step approval
for a new project.
First, a request is created and rooted to a manager
who then takes a few hours to approve or reject it.
If they say yes, the request moves to the finance department
for a second review and only after that final decision
is the resource actually provisioned or archived.
This entire sequence could easily take three days to finish
and no HTTP request on Earth is going to stay open
for 72 hours.
Functions aren't designed to sit idle and wait for days
and if you try to force them to,
the system will simply time out and fail.
This is the exact point where standard HTTP triggered functions
hit a wall because they are completely stateless by design.
Every time a function runs, it starts with a blank slate
and has no memory of what happened five minutes ago
or what the previous step accomplished.
When your workflow needs to pause at step two
to wait for a human to click a button,
a standard function has nowhere to store that progress.
You could try to build your own state management
using external databases and complex timers,
but that approach is usually brittle, hard to maintain
and prone to breaking under pressure.
Durable functions were built to solve this specific problem
by using a collection of specialized functions
that work together as a team.
The orchestrator function acts as the brain of the operation
and defines the entire workflow like a state machine.
It decides which step comes next, waits for external conditions
to be met and handles retries of a specific task fails.
Because the orchestrator is deterministic
and has no side effects, it doesn't make API calls itself,
but instead coordinates the entire execution from start to finish.
The actual heavy lifting is handled by activity functions
which behave like the standard functions you are already used to.
These activities are the ones that call downstream systems,
validate your data or send out notification emails to the team.
If an activity function fails or times out,
the orchestrator sees that failure immediately
and decides whether to try again or trigger a rollback.
Everything starts with a client function,
which is the entry point that triggers the entire orchestration.
When a user asks for that approval workflow,
the client function kicks off the process
and hands back an instance ID right away.
The user sees a message saying their request was submitted
with a specific tracking number, giving them instant feedback
while the actual work happens asynchronously in the background.
This architecture completely flips the traditional model
on its head.
Instead of making a user wait for a complex process to finish,
the workflow moves forward on its own timeline
without blocking the interface.
The user can check the status whenever they want
or the system can send a web hook callback
once the final step is complete.
One of the biggest advantages is that the orchestrator
treats the entire workflow as a piece of data that it can track.
It knows exactly which activities finished
and where the process stands at any given second.
If the function app crashes or as you
ammoves your instance to a different server,
the orchestrator has a full record of the state
and can pick up right where it left off.
You never lose work and you never have to restart a three-day process
from the very beginning just because of a minor system update.
Resilience is also much easier to manage
because retryser handled automatically by the platform.
If a service is temporarily down or hitting a rate limit,
you can set a retry policy with exponential back off
at the orchestration level.
This means every single activity gets
that same level of protection without you
having to write custom error handling code
for every individual function.
Durable functions really show their value
when you need to handle fan out and fan in scenarios.
If your workflow needs to run five different validation
checks at the same time, the orchestrator
can spawn all five activities simultaneously
rather than waiting for them to finish one by one.
Once every parallel task reports back with the result,
the orchestrator fans back in and moves to the next phase of the project.
This type of parallel execution cuts down on total latency
and makes your enterprise workflows feel significantly faster.
Status tracking is built directly into the runtime
so you don't need to build custom tables
to see if a process is still running.
You can simply query the system for a specific instance ID
to see which activities are done and what the results were.
This is the professional way to handle the 80% of business logic
that doesn't fit into a simple request response window.
You stop fighting against the limitations of HTTP
and start using a tool that was actually designed
for long running, stateful work, latency optimization,
every millisecond counts.
While durable functions are great for those long running background tasks,
most of your co-pilot interactions need to happen in real time.
When a user types a question into a chat box,
they expect the response to appear almost instantly.
In these synchronous moments, your ability
to shave off every possible millisecond
becomes your biggest competitive advantage.
For simple operations like a quick database lookup or a validation check,
you should be aiming for a warm invocation latency of under 100 milliseconds.
If the task is more complex and involves calling multiple systems
or aggregating large amounts of data,
you should still try to stay under 300 milliseconds.
Once you cross that half-second mark,
users start to feel the lag
and if a response takes longer than a full second,
you have likely lost their attention entirely.
These numbers aren't just guesses,
but are actually based on how the human brain perceives the system as being responsive.
We expect an immediate acknowledgement of our input within about a tenth of a second
and anything slower than that creates a feeling of friction.
If your architecture can't deliver that sub-100 millisecond response on the happy path,
your users will start to doubt the reliability of the entire tool.
The foundation of a fast system is smart connection management.
As we discussed earlier, you should never create a brand new database connection
for every single request because the overhead of handshakes and authentication is far too high.
By using a pooled connection that stays alive between requests,
your code can start running the actual query immediately
without waiting for the network setup.
The same logic applies to your HTTP clients,
where reusing a single instance for all outbound calls eliminates the need for repeated TLS handshakes.
Where you put your infrastructure matters just as much as the code you write.
If your function is sitting in a data center in Virginia,
but your database is located in London,
every single query has to travel through undersea cables across the Atlantic.
That physical distance adds 50 to 100 milliseconds of pure travel time
before your database even sees the request.
By co-locating your functions, storage, and databases in the same region,
you can drop that roundtrip time down to single digits
and save a massive amount of time for free.
Catching is another way to amplify your speed by avoiding the network entirely.
Reference data and configuration settings that don't change often should be stored
in memory as static fields within your function code,
accessing a variable in memory takes microseconds,
which is thousands of times faster than making a roundtrip to a database.
For data that needs to be shared across multiple instances,
a distributed cache like Redis is a great middle ground
that is still much faster than a traditional disk-based query.
It is important to remember that latency is never a single,
consistent number for every user.
Some requests will hit the cache and be lightning fast,
while others will miss and have to fetch fresh data from a slow downstream service.
This is why you cannot rely on averages,
because an average of 300 milliseconds might be hiding a few requests
that take five seconds to finish.
Users don't remember the average experience,
but they definitely remember the one time the system felt broken and slow.
You need to focus your optimization efforts on tail latency,
specifically looking at your P95 and P99 metrics.
If your P99 latency is five seconds,
it means one out of every hundred requests is failing the user experience,
which happens about one's per conversation for an active user.
When a user hits that slow path,
they don't care that the other 99 requests were fast.
They just feel like the system is unreliable.
To fix this, you need deep instrumentation
that shows you exactly how long each dependency and code path is taking.
Once you can see which specific API call or database index is causing the slowdown,
you can actually do something about it.
Without that visibility, you are just guessing and hoping for the best.
This level of detail is what separates a basic function
that technically works from a co-pilot experience that feels truly instant.
Monitoring and observability.
Blind optimization is a waste of time.
You need eyes inside your system.
Application insights is where those eyes live.
It serves as the standard telemetry platform for Azure workloads
and it becomes your primary diagnostic tool for understanding
how your co-pilot plug-in actually behaves in production.
Not how you think it behaves,
not how it behaves in your test environment.
You need to see how it actually behaves when real users interact with it.
Send execution time data to application insights.
How long does each function invocation take from start to finish?
And how long does the database query or the downstream API call take?
The more granular you go, the better your visibility.
You are building a map of where time actually gets spent.
This level of detail is how you find the bottleneck that is limiting your whole system.
Sometimes the problem is obvious like a database query that takes two seconds
but other times it is subtle, like a cache miss that cascades into five sequential lookups.
Without instrumentation, you are just guessing.
With it, you see the actual behavior.
Correlation IDs are how you track a single user request through your entire system.
Your co-pilot user types a question
and that question generates a unique correlation ID that travels through power platform
and into your custom connector.
That ID enters your function and your function logs that ID with every operation it performs.
When your function calls your database, that query includes the correlation ID
and when it calls a downstream API, that request includes it too.
Later when you are investigating why a user request failed,
you search for that correlation ID across all your logs.
You see the entire journey, you see exactly where it broke.
This is how you move from something went wrong to this specific API call,
returned a 500 at 1432 17 UTC.
Custom metrics go beyond raw performance data.
You care about business outcomes, not just technical metrics,
track invoices validated per minute,
or approval workflows completed today,
or even your cache hit rate percentage.
These metrics answer the question of whether your system is actually working for your users.
A 100 millisecond response time is great,
but if only 60% of validation requests succeed, your system is failing.
Business metrics surface the problems that matter.
Alerting strategy separates operational maturity from reactive firefighting.
Alert on P95 and P99 latency spikes,
or when your error rate exceeds a threshold, or even on cold start frequency.
But do not alert on averages.
An average latency of 300 milliseconds tells you nothing,
because it masks the fact that half your requests take 100 milliseconds,
and the other half take 500.
Alert on the tail.
Alert when something changes unexpectedly.
Alert when your system violates a known pattern.
This is how you catch problems early instead of discovering them when users start to complain.
Diagnostic settings on your function app export logs to log analytics.
Application insights captures application telemetry,
while log analytics captures platform level logs for when instances scale,
when deployments happen, or when quotas are hit.
Together they give you complete visibility.
Logs live in log analytics for years,
which means you can query historical data or build long term trend analysis.
You can prove that a change you made six months ago improves stability.
Azure Monitor Workbooks tie this together into dashboards.
A workbook is a canvas where you build custom visualizations.
You add charts showing latency trends, gauges showing current error rates,
and tables listing your slowest operations.
You can even add heat maps showing traffic patterns by time of day.
These workbooks become the operational brain of your system.
When something feels off you check the workbook,
you see the actual behavior, you adjust.
The discipline here is measurement without obsession.
You need visibility, but you do not need to track everything.
Start with latency per operation, error rate and cold start frequency.
Add business metrics relevant to your domain and refine from there.
The goal is to catch degradation before it becomes an outage.
This is how you move from hoping your system works to knowing it works.
Testing the integration from unit to end to end.
Theory breaks on contact with reality.
Everything we have built from the function and the open API spec to the security infrastructure
needs to survive actual usage.
That is what testing does.
It bridges the gap between this should work and this actually works.
Start with unit tests.
Your C business logic needs full coverage.
Validate invoices transform data and apply business rules.
These tests run fast and they run locally.
They mock external dependencies.
So you are not actually calling your database or hitting external APIs.
Every time you run the test suite, this is standard practice.
But for serverless functions, unit tests become your primary validation tool.
You cannot easily debug a cold start issue in production
and you cannot step through your code with a debugger
while it is processing a live copilot request.
Unit tests catch logic errors before they escape.
Integration tests are where most teams stumble.
They test the contract between your code and the external world.
Here is what matters.
Call your function via HTTP and invoke it exactly the way the custom connector will invoke it.
Send a real HTTP request with real headers and a real payload, then capture the response.
Validate that the response schema matches your open API spec.
This is crucial because your open API spec defines what downstream systems expect.
If your function returns a field with the wrong type or misses a required field
or includes extra fields that the spec does not document, the custom connector might break.
Integration tests catch this.
They validate the contract, not the internal implementation.
Load testing reveals what unit tests hide.
Your function handles one concurrent request perfectly.
But what about 50 or 500?
Load testing tools like K6 or Azure.
Load testing simulate realistic concurrent load.
You define a scenario where 50 users each make 10 requests with realistic think time between them
and then the tool hammers your function.
It shows you where your latency degrades.
It shows you where you hit resource limits.
It shows you whether your rate limiting works.
It shows you your actual system behavior under stress, not imaginary behavior in a lab environment.
Here is the critical part.
Test with real user language, not API syntax.
Do not test by calling your function with a perfectly formatted JSON payload.
Test by simulating how co-pilot actually invokes your function.
Co-pilot sends natural language and the LLM interprets it and fills in parameters.
Sometimes it interprets ambiguously and sometimes it misses context.
You want to find these problems in testing, not in production.
A co-pilot plug-in that works perfectly with structured input but fails with natural language variations is broken.
Load testing that does not capture this reality is just theater.
The handshake between power platform and Azure is a specific test.
Your custom connector maps to your function.
It transforms input and it handles errors.
You need to validate that this mapping works.
Create a test flow in power automate or power apps call your custom connector
and verify that the call succeeds.
Verify that the response is processed correctly by the low-code layer.
This test is part integration test and part sanity check.
It catches configuration problems that code tests miss.
A firewall rule that blocks your connector from reaching your function will not show up in a unit test
but it will show up in a connector test.
Deployment slots let you test in production without affecting users.
You deploy a new version to a staging slot.
You run your tests against staging and you validate the behavior.
You do load testing against staging and then you flip the production slot to the staged version.
Now your production traffic roots to the new code.
If something breaks you flip back to the previous version in seconds.
This is continuous deployment.
It is how you iterate safely.
The testing discipline separates teams that ship stable copilot plugins
from teams that debug issues in production at 2 in the morning.
Testing is not about passing tests.
It is about confidence.
It is about knowing that your plugin will work when real users depend on it.
Versioning and evolution.
Your first copilot plugin is finally in production
and everything is working exactly as planned.
Users are starting to depend on it
and teams have already built their daily workflows around the tool.
But now you need to add a new feature
or maybe you found a bug that needs fixing.
This is the moment where versioning disciplines
separates a professional maintainable system
from one that becomes fragile and broken.
Here is the hard truth about software.
Once your function is in production,
it doesn't really belong to you anymore.
It belongs to the flows and systems that call it.
Those flows have specific dependencies
and they expect your parameters to mean the exact same thing every single time.
You cannot casually change those expectations without consequences.
Every update you make either respects those existing dependencies
or it completely breaks them.
The operation ID is your contract with the rest of the world.
When you define an operation in your Open API spec
with an ID-like validate invoice,
that name becomes your permanent anchor point.
Power Platform flows reference that specific ID to function
so if you change it, those flows will break silently.
They will try to call an operation that no longer exists
and the users will only see a failure they don't understand.
They will blame the system and stop trusting your tool,
which is why you should never change an operation ID.
It must stay stable for the entire life of that operation.
If the behavior changes fundamentally,
you aren't modifying an old tool.
You are building a new one with a new ID.
This is where semantic versioning becomes your best friend.
Your API needs a version number starting at 1.0.0.0.
The first number represents the major version,
the second is the minor version and the third is for patches.
When you add a new operation,
you bump that minor version to 1.1.0.
Clients can upgrade whenever they feel like it
because nothing broke since you only added features instead of removing them.
When you fix a bug in the code without changing the contract,
you just update the patch version.
However, if you break the contract or change how a flow needs to behave,
you must move to version 2.0.0.0.
This tells everyone downstream that something big changed
and they need to review their implementation.
Using separate function apps or deployment slots
lets you run multiple API versions at the exact same time.
Your version 1 app stays stable
and serves your current production traffic
while your version 2 app gets all the new features.
You can deploy the new version to a staging slot to validate it
before running both versions side by side in production.
The connector for version 1 keeps talking to the old app
and the connector for version 2 reaches the new one.
This lets teams migrate at their own pace
and nobody is forced to upgrade until they are actually ready.
This gradual transition is the only way to move the platform forward
without breaking the system's people rely on.
The open API spec itself is a piece of history
that needs proper version control.
You should store it and get right alongside your function code
so you can track every change in your commits.
When you release a new version of your API,
make sure to tag the commit that contains that specific spec.
Later on when a teammate asks what change between the old and new versions
you will actually have the answer.
You can compare the files and see exactly which operations were added or modified.
This is how you communicate clearly with other teams
and maintain a record of how the system evolved.
Deplication paths are where most teams start to struggle.
Eventually you will have an operation
that is no longer the best way to solve a problem
because a better version exists.
You might want to delete the old one immediately
but you have to remember that people are still using it.
Instead of deleting it,
market is deprecated in your spec and explain why.
Suggest a new alternative
and give everyone a clear timeline
for when the old version will disappear.
If you tell teams the operation will be gone by December,
they have months to plan their migration.
When that date finally arrives, they are ready for the change
and the transition happens without any drama.
This level of discipline turns your API
from a source of constant frustration into a reliable foundation.
It is the difference between being afraid to touch your code
and having the confidence to evolve.
You can move faster because you have a strategy
that respects the people using your work.
Your function is never truly finished
but it should evolve with purpose instead of moving in a state of chaos.
That is what makes a system professional.
Cost optimization and right sizing.
Performance and security are always the top priorities
but they have to live inside a real-world budget.
Every choice we have made so far
from using always-ready instances
to setting up comprehensive logging comes with a price tag.
The real question isn't whether you can afford this setup
but whether you are paying for it in the most efficient way possible.
Flex consumption billing is calculated using two main factors
how long your code runs and how many times it gets called.
You are essentially paying for gigabyte seconds.
If you run a function with 512 megabytes of memory for two seconds
that counts as one gigabyte second.
You also pay a small fee for every million times the function is triggered
which is usually around 40 cents.
If you optimize both of these areas, you can keep your bill under control
but if you ignore them, your costs will start to spiral.
Always-ready instances provide a baseline cost that you need to account for.
Let's look at the actual numbers to see how this works.
A single instance with two gigabytes of memory
costs a tiny fraction of a cent per second
but that idle time adds up fast.
Keeping one instance warm all day and night
costs about $288 every month.
If you need two instances that jumps to $576.
For that same price you could almost run a premium plan with a dedicated instance.
If you are constantly running multiple always-ready instances
the math changes and premium might actually be the cheaper path.
Most teams make a huge mistake by picking their memory size based on a gut feeling.
They assume a small function only needs 512 megabytes
because the data it processes is tiny.
What they miss is that as you atties your CPU power directly to how much memory you allocate
that small instance might take three full seconds to finish a task
because it has very little processing powers.
If you bump that up to two gigabytes,
the extra CPU might finish the same task in 600 milliseconds.
Even though the larger instance costs more per second,
the faster finish time can actually save you money on your baseline costs.
You end up with a lower bill and a response time that is three times faster.
This is the strange reality of serverless economics
where the more expensive option is sometimes the cheaper one.
Scaling up to a bigger instance can be more cost effective
than scaling out to many small ones.
You have to run the actual math for your specific workload instead of just guessing.
Azure Cost Management is the best tool for doing this math correctly.
It lets you break down your spending by specific operations or time periods
so you can see exactly where the money is going.
You can find the functions that run all the time,
but finish quickly, which are usually very cheap.
You can also spot the rare functions that take forever to finish
and cost a fortune every time they run.
Once you identify those expensive operations,
you can target them for optimization.
Faster code reduces your bill and better caching means you don't have to run the code as often.
Batching is your most powerful weapon when it comes to saving money.
If your copilot needs to check 10 different invoices,
do not make 10 separate calls to your function.
You should send all 10 invoices in one single request
so the function only has to run once.
This means you only deal with one cold start and one unit of billing instead of 10.
A single batch operation is almost always faster than 10 separate round trips over the network.
It pays for itself immediately by cutting down on invocation charges and execution time.
Reserved capacity is something you should consider once your traffic becomes predictable.
If you know for a fact that you will run 5,000 calls every day,
you can stop paying the standard on demand rates.
As your savings plans allow you to commit to a certain amount of usage in exchange for a 20% discount.
You are trading away some flexibility for a lower price,
so the math only works if you actually use what you promised.
If you overestimate, you are paying for air, but if you get it right,
it is the cheapest way to run at scale.
Optimizing your costs isn't something you do once and then forget about.
It is a constant cycle of measuring your results and making small adjustments.
Improvements in performance almost always lead to a lower bill
and looking at your costs often reveals where your code is slow.
These two goals actually work together.
Usually the fastest function is also the cheapest one to run,
so you should always be chasing both.
Deployment and CI CD.
Getting code from your laptop to production safely is only half the battle
because the other half is making sure you don't break everything once it's actually live.
This is where deployment discipline separates the high performers from the teams
that only ship once a quarter because they are terrified of a system crash.
Everything starts with infrastructure as code.
Your function apps, your key vaults and your storage accounts are all defined as code
rather than being clicked into existence.
You might use bicep if you are staying native to Microsoft
or you might choose terraform for a multi-cloud approach,
but the core principle never changes.
You aren't clicking buttons in the Azure portal to provision resources anymore.
Instead, you declare your infrastructure in code,
check it into Git, and version it just like your application logic.
When you deploy, that code drives the environment,
which makes your deployments repeatable across dev, test and production.
By using the same template for every stage,
you ensure that if a feature works in your test environment,
it will work in production because the underlying infrastructure is identical.
Automation is what makes this whole process reliable.
When you push code to Git, a pipeline in GitHub actions or Azure DevOps
triggers automatically to run your unit tests.
If those tests fail, the deployment stops immediately,
so you never ship broken code to your users.
If they pass, the machine runs integration tests against a staging environment
and builds your deployment package.
It validates your infrastructure code and checks for common mistakes
before anything touches a live server.
This entire flow is automatic, repeatable and auditable.
A deployment isn't a person running manual commands on a Friday afternoon,
but a machine following a strictly defined process.
This shift to automation is actually where your risk decreases.
Manual deployments are where mistakes happen
like when someone misses a configuration step
or accidentally deploys the wrong version of a file.
Automated pipelines remove that human element entirely.
The same process runs every single time,
so you know exactly what is happening under the hood.
Deployment slots give you an extra layer of safety.
Your production slot handles the live traffic
while your staging slot acts as a perfect mirror where you can test new code.
You deploy the new version to staging,
run your full load tests,
and validate that the system is stable.
Once you are ready, you perform a swap.
Staging becomes production and production becomes staging in an instantaneous transition.
If something goes wrong after the swap, you just flip it back.
The old code is still sitting there in the staging slot,
so you can revert in seconds without an outage.
This is how you ship with confidence.
Security is non-negotiable,
which means secrets never go in your code.
If a database password or a connection string
ends up in your git repository,
your entire system is compromised.
Anyone with access to that repo now has the keys to your data.
You should keep those secrets in Azure Key Vault
and let your function authenticate using a managed identity.
The secret never touches your source code
and it never appears in your logs.
It stays protected by Azure's encryption and access controls
at all times.
Environment variables are different
because they represent configuration that is safe to share.
Your function needs to know which database to talk to,
but that answer changes depending on where the code is running.
In dev the variable points to a test database,
but in production it points to the real thing.
The code stays exactly the same,
but the behavior changes based on the environment variables you've set.
Your rollback procedure is more important than the deployment itself.
If you use slots, a rollback is just a 30 second swap,
but if you don't have a tested plan,
you'll end up improvising under pressure.
When the team is stressed during an outage,
they make mistakes that make the downtime even longer.
You need to document your rollback steps and run a dry run every quarter.
You have to know exactly what to do before a crisis actually hits.
This level of discipline turns deployment from a high-stakes gamble
into a boring routine operation.
You end up shipping more often while breaking things less
and when something does go wrong, you recover before anyone notices.
Real-world scenario, invoice validation plugin.
Architectural principles can feel abstract
until they meet a real business problem.
Let's look at how a finance team builds an invoice validation plugin
to see how these different pieces actually work together.
The finance team usually spends hours answering the same basic questions
about vendor payments and approval thresholds.
Employees want to know if an invoice is valid
or if a specific vendor has been paid yet.
Right now, answering those questions means logging into an old accounting system
and digging through spreadsheets,
which takes about 10 minutes per request.
The team is essentially drowning in manual, low-value work.
A co-pilot plugin changes that dynamic entirely.
An employee can simply ask co-pilot if a specific invoice is valid
and the system roots that request to the validation plugin.
The plugin returns a clear, structured answer in less than 500 milliseconds.
The employee gets what they need instantly without bothering the finance department.
The open API spec for this plugin defines three specific operations.
You have validate invoice to check status, get invoice details
to pull record information, and flag invoice for review for manual audits.
These names are chosen carefully as verb-known pairs
so the LLM can easily tell them apart.
When a user asks for information, the model roots to the details operation
but when they ask for a status check, it hits the validation logic.
The engine behind this is a CPI's function running on flex consumption
with always-ready instances.
Instead of running all the logic in one place,
it uses durable functions to coordinate the workflow.
An orchestrator function starts by calling three separate activities at the same time.
One checks the accounting system, another verifies the vendor credentials
and the third looks for duplicate submissions.
Because these checks run in parallel,
the orchestrator only waits for the slowest one to finish before moving on.
If the invoice passes every check, the system calculates the approval threshold
and returns the result.
If it fails anywhere, the process stops and reports the error immediately.
The orchestrator packages all of this into structured data
that flows back to the co-pilot.
The user doesn't just see a yes or no,
but a clear explanation of why an invoice was rejected
or why it needs a manual review.
API management sits in front of the whole operation to keep things stable.
The custom connector talks to API M instead of hitting the function directly,
which allows you to enforce rate limits.
This prevents a single user from overwhelming the back end with too many requests.
API M also handles the heavy lifting of security
by validating tokens and injecting correlation IDs into every request.
It even checks the data coming back from the function
to make sure it matches the expected schema.
If the function returns something messy,
API M catches it before the co-pilot tries to read it.
Speed is the main goal here,
so the team uses several optimization tricks.
Since vendor credentials don't change very often,
they are cached directly in memory on the always ready instance.
The first request might be a bit slower,
but every request after that hits the cache for near instant results.
Connection pooling keeps the database links open,
so the system doesn't have to waste time creating new ones for every query.
By running the three main checks in parallel,
the total latency drops from three seconds down to about 300 milliseconds.
Monitoring keeps a close eye on three specific areas.
If the validation failure rate climbs above 5%,
an alert tells the team that the accounting system might be down.
If the P99 latency goes over one second,
the team knows they have a cache issue or a system slowdown.
They also track cold starts to make sure the always ready instances are doing their job.
If startup time starts to creep up,
it means a deployment changed something that made the app too heavy.
The result is a system with total traceability.
Every single request has a correlation ID that follows it through API M,
the function, and the database.
If a user asks why their invoice was denied three weeks later,
the finance team can pull the logs and see exactly which check failed.
This is a plug and build to scale.
It is fast, it is auditable,
and it is easy to maintain because every component has a specific job.
When you combine already instances with durable functions and API M governance,
the whole system becomes much stronger than its individual parts.
Common pitfalls and how to avoid them.
Most teams don't fail because they don't understand the architecture,
but rather because they skip steps that seem optional
until the moment they become catastrophic.
These pitfalls are essentially the tax you pay for cutting corners during development.
The first major mistake is writing your open API spec for the wrong audience.
You're a developer who understands your own code,
so you write clear documentation for other humans,
but the model doesn't read like a programmer.
It reads like a user.
When you write returns customer records filtered by status,
you've said something technically correct,
but the model just sees a generic operation for retrieving data.
When three different operations in your spec all do similar things like search,
get and fetch, the model gets confused and starts picking between them at random.
Your copilot calls the wrong operation, the flow fails,
and the user blames the bot for being broken.
What actually happened is your spec was too ambiguous for the LLM to navigate.
You need to write descriptions that explain when to use an operation instead of just what it does.
If you write use this when you need to find customers by their name or company,
the model finally has the language it needs to root the request correctly.
Choosing not to use flex always ready is like leaving your car running in the driveway all day
just to avoid a two second startup delay.
You might be solving the cold start problem,
but you're also burning money and wasting resources for no reason.
Classic consumption scales to zero, which means
if your plug-ins sits idle for an hour and a user finally makes a request,
the function has to spin up from scratch.
The user experiences a massive lag spike while they wait for the system to wake up.
Flex with always ready keeps a small instance warm at a fraction of the cost of a premium plan,
making it the obvious choice for any professional setup.
Teams often skip this because they think they'll optimize the performance later,
but later never actually comes.
They end up living with five second startup delays that compound into thousands of lost conversations
all while an always ready budget would have only cost about $200 a month.
Embedding secrets directly in your code is a fast way to lose your job.
Your function needs a database password,
so you hard-code it and check that code into Git,
which allows anyone who clones the repository to read your credentials.
They can connect to your production database and export your customer data to the internet
before you even realize there's a leak.
You didn't mean to expose the data, but you took a shortcut that bypassed basic security.
As your key vault exists for this exact reason, and you need to use it.
Your function should authenticate to the key vault using a managed identity,
so the secret is returned securely, without ever touching your source code.
Never hard-code your secrets because this isn't just a best practice.
It's a fundamental requirement for staying employed.
Blocking calls in your functions will completely waste your investment in always ready.
When your function calls an external API and the thread sits there waiting for a response,
that always ready instance is being consumed while doing zero actual work.
If the next request comes in while the thread is blocked,
there's no available instance to handle it, so a new one has to spin up.
This creates a cold start and a latency spike that defeats the entire purpose of keeping the instance warm in the first place.
You need to use async and a wait to make your API calls asynchronously so the thread can yield.
This allows another request to use that same thread while the first one waits for its data to come back.
Concurrency is what multiplies your capacity and blocking code just throws that multiplier away.
Flying without monitoring or logging is like flying a plane blind.
Your function might work perfectly in a test environment and the deployment might go fine,
but two weeks later, users start reporting that requests are failing silently.
Your function isn't throwing any obvious errors, it's just returning empty results and without logs,
you have no way to know why.
You'll spend three days guessing at the cause when you could have seen the answer in 30 seconds.
With proper logs, you would immediately see that a downstream dependency changed its response format and broke your passing logic.
Monitoring is not an optional add-on, it's insurance against spending your entire week debugging problems
that should have been visible from the start.
If you don't version your open API spec, your evolution will eventually turn into total chaos.
The moment you modify an operation or change a parameter type, you break every existing flow that depends on the old structure.
The person who created that flow has no idea what changed or why their automation suddenly stopped working,
so they file a support ticket and wait for an apology.
You end up reverting the change and realizing you haven't actually made any progress.
You need to version your spec and track every change you make.
When you modify a feature, bump the version number and communicate that shift to your users.
This gives teams the time they need to adapt so you can evolve the system without breaking everything in your path.
These aren't subtle problems that only show up for experts.
They are common, preventable and incredibly expensive if you choose to ignore them.
Scaling from pilot to enterprise.
Your invoice validation plugin is finally live and stable and the finance teams are actually using it.
But now the CRM team wants something similar, the HR team wants a bot for employee queries and the logistics team is asking for shipment tracking.
One single plugin proves that your architecture works but now you suddenly need 10 of them.
If you don't have a scaling strategy, you're going to spend the next year reinventing the same wheels over and over again.
You'll keep making the same decisions about APM configurations and function structures while making different mistakes in every single implementation.
By the time you get to plugin number five, you'll be drowning in maintenance.
This is the point where patent libraries separate the mature organizations from the chaotic ones.
A patent library is really just a collection of reusable templates that keep you from starting at zero every time.
Since you've already standardized the invoice validation plugin, you should extract that logic into a template.
Create a generic scaffolding that includes your HTTP triggers, your durable functions orchestrators and your standard error handling.
When a developer needs to build a new plugin, they start from this template instead of a blank screen.
They can focus entirely on the business logic because all the plumbing is already handled and correct.
Your open API templates should follow the same logic. Since you've already learned what the LLM understands, you can create a canonical structure for all new operations.
This ensures your specs stay consistent, your naming conventions remain the same, and the LLM never runs into any surprises.
Bicep modules allow you to encapsulate your infrastructure patterns so they can be deployed reliably.
Instead of every developer trying to write their own key vault setup or APM configuration, they should just call a pre-made module.
The module handles the complex details while the developer just provides the parameters.
If a dev needs a new plugin that handles a thousand requests a day, the module creates the right instant size and applies the necessary security policies automatically.
The developer doesn't have to write a single line of infrastructure code because they just declared what they needed and the system delivered it.
True consistency only emerges when you make reuse the easiest path forward.
Centralizing your authentication and authorization removes a massive amount of friction from the development process.
You shouldn't have every single plugin managing its own identity system instead you should create a shared authentication policy within APM so that when a request arrives the security check happens exactly once.
The request then carries a trusted identity token to every downstream service.
This centralization makes your governance much simpler because you have one single place to audit access or revoke credentials.
If compliance suddenly requires multi-factor authentication, you only have to implement it in one spot.
Teams won't try to bypass the security layer because you've made it the most convenient way to build. Implementing governance for custom connectors is the only way to prevent shadow integrations from taking over.
Without a process in place, a developer might create a custom connector for a new API and deploy it to production without anyone ever seeing it.
You'll have no idea it exists, even though it might be reaching a blocked service or exposing sensitive company data.
You need a simple approval workflow where developers submit their connector requests for a quick security review.
Once the connector is approved, it gets added to the pattern library for other teams to reuse.
This doesn't actually slow down development, it actually speeds things up because developers can build on top of approved patterns instead of making up ad hoc solutions on the fly.
A self-service portal can let developers request new functions without having to wait for your team to manually intervene.
The portal is just a simple form where a user describes what the function needs to validate and how much traffic they expect.
That form feeds directly into a pipeline that builds the function from your template, generates the spec, and deploys it to a deep environment.
This gives the developers instant scaffolding to work with while giving your team total visibility into what's being built.
It allows everyone to move faster without sacrificing the quality of the underlying system.
Runbooks are your operational insurance for when things go wrong in the middle of the night.
When a plug-in fails at 2 in the morning, the engineer on call shouldn't have to understand every detail of the architecture to fix it.
The runbook should tell them exactly what to do if latency spikes or if a cache hit rate drops below a certain level.
If the vendor credential cache expires, the instructions tell them to restart the always ready instances.
The engineer follows the steps and fixes the problem in 10 minutes instead of spending 2 hours investigating the root cause.
Runbooks capture the collective knowledge of everyone who has ever debugged the system and they are the difference between reactive firefighting and a confident response.
This is the path for scaling from a single pilot plug-in to a full enterprise platform.
You don't get there by hoping every team figures out the same lessons on their own.
You get there by building those lessons directly into the infrastructure and the processes that scale with you.
The strategic shift from low-code toys to enterprise engines.
We have walked through the entire architecture together.
You understand the hosting choices, the security perimeter, the monitoring discipline and the deployment rigor.
But these are not just abstract concepts.
They represent a fundamental shift in how enterprise AI actually gets built.
Low-code platforms change the game by democratizing automation.
Before Power Automate existed, you needed enterprise architects to design integration solutions at astronomical hourly rates.
Low-code put that power into the hands of business teams.
It was genuine innovation. It reduced friction and accelerated delivery.
But it also created a dangerous assumption that low-code can do everything.
The assumption is that you can automate anything with enough power, automate flows.
That assumption breaks the moment you hit sophisticated business logic.
This is where the fusion developer model emerges.
It is not about low-code replacing professionals.
And it is not about professionals dismissing low-code.
Instead, professionals use low-code where it excels.
For orchestration, coordination and connecting systems.
Then they use pro-code where it actually matters.
For heavy computation, complex transformations and strict business rules.
The co-pilot plug-in architecture you have learned is the physical manifestation of that fusion.
Power Platform handles what it was designed for.
As your functions handles what requires real engineering.
Teams that master this architecture build things their competitors cannot replicate.
A competitor might see your co-pilot plug-in working smoothly and decide they want one too.
They can build a basic version in a few weeks using standard connectors.
But if your plug-in handles edge cases through sophisticated C-POS logic, they cannot copy that quickly.
If it maintains sub-second latency through flex always ready configurations.
Or governs data access through APM policies and managed identities, it will take them months to catch up.
By the time they figure it out, you have already iterated.
You have added new features. You have become reliable.
You have gained a competitive advantage that compounds over time.
This advantage does not come from raw intelligence or some secret technology.
It comes from discipline.
It comes from understanding that every architectural decision serves a specific purpose.
Choosing always ready over cold starts or using durable functions instead of nested flows is intentional design.
It is not accidental emergence.
The investment multiplies across every plug-in you build.
When you build your first plug-in with proper infrastructure, you invest heavily in open API spec design.
APM configuration and monitoring. The initial bill is substantial.
But the second plug-in reuses everything.
The infrastructure is already there. The patterns are documented.
And the team understands the playbook.
The second plug-in costs a fraction of the first.
By the time you reach the tenth plug-in, you are deploying at a cadence that smaller, less disciplined teams simply cannot match.
Infrastructure investments compound.
Security and governance are not after thoughts in this model.
They are built in. You are not trying to bolt on compliance after the architecture already exists.
You are designing policies into APM from the very start.
You are not wondering who has access to what?
You are using managed identities so every access point is auditable and revocable.
You are not hoping someone remembers to log important events.
You are capturing correlation IDs and streaming them to application insights automatically.
The cost of these controls is low when they are part of the foundation.
The cost of retrofitting them later is astronomical.
Cost optimization becomes a continuous discipline rather than a crisis event.
You are not discovering six months down the line that your baseline costs are too high.
You are tracking costs weekly. You are right-sizing memory based on actual execution patterns.
You are using cash data to reduce expensive downstream calls.
These micro-optimizations compound. Over a year they save tens of thousands of dollars in infrastructure spend.
The real value is not any individual plug-in. It is the reusable infrastructure and the patterns that enable rapid innovation across dozens of plug-ins.
You have built a platform. New capabilities that used to take three months to deliver now take three weeks.
Because the foundation is solid. The foundation handles the scaling, the security and the monitoring.
Teams building on top can finally focus on business logic instead of infrastructure.
This is what separates organizations that just use AI tools from organizations that build AI platforms.
You have the blueprint. You understand how flex consumption eliminates cold starts.
You know how to write open API specs that the LLM actually understands.
You have learned to secure your work through APM and managed identities.
You have seen how to scale from one plug-in to an entire enterprise platform.
Start with one high value workflow. Do not pick the easiest problem.
Pick the one that matters most to your business. Validate the architecture on something real.
That first plug-in will teach you what works and what does not.
It will show you the operational reality behind these design principles.
Invest in open API specs and API management early.
They might feel like overhead when you are shipping your first plug-in.
By your third plug-in they will have paid for themselves through reduced maintenance and faster iteration.
Monitor everything obsessively. You cannot optimize what you cannot measure.
Latency, errors, cold starts and cache, hit rates all matter.
If it matters operationally you need to measure it.
Let the data guide your next move. Connect with the community.
Share your patterns and learn from the mistakes of others.
The Fusion developer model is still emerging. Teams are figuring this out in parallel right now.
What you learn becomes valuable to others.
And what they learn will accelerate your next iteration.
Subscribe to the M36-DiFM podcast for the latest on co-pilot, Azure and the future of enterprise AI.