“Why did my pipeline break?”

Every time, someone from the platform team had to stop what they were doing, read the logs, figure out whose problem it was, and explain it.

Sometimes it was on the developer’s side.

Sometimes it was on the DevOps’ side.

So I built a workflow that reads the logs, figures out whose problem it is, opens a PR with a fix, and if the change is small enough, merges it automatically. The developer finds out their pipeline broke and was already fixed. And if the workflow does a great job, the platform team doesn’t find out at all.

Here’s how we built it at Port.

Step 1: Create your context lake

The agentic workflow we’re going to build needs three things from a context lake to do its job: the logs from the failed run, ownership data to know who’s responsible, and a skill file that tells it how you want it to behave.

• GitHub: gives the agent live read access to your workflow runs: the failed step, the full log output, the error message, and the code that triggered the run. This is the core or the triage decision.

• A service catalog with pipeline ownership: at minimum, a mapping of “this pipeline belongs to this team.” The agent needs to answer: given this failed pipeline, who do I assign the PR to?

• A skill file: This is where you can tell your agent things like what counts as a developer failure vs a platform failure, how to write the PR description, or when it’s safe to auto-merge.

Here’s a minimal example:

## CI Failure Classification Rules

** Developer issue (assign PR to developer): **

- Build step failed → platform owns the Dockerfile, not what’s inside it

- Linter or format check failed → standard violation

- Pre-merge test suite failed → new code broke the test

** Platform issue (assign PR to platform team): **

- Push to GitHub failed → credential, API, or network issue

- Deploy to cloud failed → infrastructure misconfiguration

- Secret or env var missing → platform configuration

**Auto-merge if:**

- Change is a single-line config fix

- No infrastructure files modified

- Failure pattern has been seen and resolved before

**PR description format:**

- One sentence explaining what failed and why

- One sentence explaining what the fix does

- Link to the relevant standard or runbook if applicable

Learn more about skills in Port here

For better performance, you might also want to give it access to:

• Deployment history: recent deployments from the same service help the agent understand if a code change caused the failure

• CI job failure history: past failures on this pipeline let the agent recognize known patterns and skip the investigation for things it’s seen before

Learn more about building a context lake here

Step 2: Build the agentic workflow

Once the context lake is set up, you’re ready to build the workflow.

Set up a trigger: listen for failed CI jobs

You can listen for a failed job from anywhere: GitHub Actions, Jenkins, GitLab CI, ArgoCD, wherever you want

When a CI job status changes to failed, the workflow starts.

I would recommend not triggering it on every pipeline failure. That could be expensive and noisy. Start with a small part like your pre-merge test suite on production. If it fails, the PR reopens and the merge is blocked. It only runs maybe 10-20 times a day, not thousands, but it’s urgent to fix.

Other good starting points: self-serve infra pipelines (create a database, provision a new user), or anything where a failure has a clear owner and the stakes are obvious.

Build an agent that triages the failure

The first agent reads what failed and classifies it. It has access to everything you set up in step 1: the live logs via GitHub MCP, the ownership data from your catalog, the skill file, and the failure history if you’ve connected it.

Any engineer who’s done this for a while can look at a failed pipeline and tell you whose problem it is. The agent does the same thing using the context lake and the skill you wrote for it.

The agent cross-references with the skill file to apply your org’s classification rules. For example:

• Logs show npm run build failed with Type error: Property ‘id’ does not exist on type ‘User’ → build step, code issue, developer’s problem

• Logs show docker push failed with unauthorized: authentication required → push step, credential expired, DevOp’ problem

• Logs show eslint failed with no-unused-vars on line 34 → linter step, standard violation, developer’s problem

• Logs show terraform apply failed with Error: AccessDenied → deploy step, IAM misconfiguration, DevOps’ problem

If you’ve connected CI failure history, it checks whether this exact pattern has come up before and how it was resolved. If it has, it skips the analysis and goes straight to the known fix.

The output is a classification. Is it a developer issue or platform issue? Plus it adds a plain-language summary of what failed and why.

Add a Slack message node

The triage agent sends a Slack message with the result: what failed, why, and whose problem it is. The developer or platform engineer sees the classification before a PR ever lands in their inbox.

Send the suggested fix to a coding agent

The triage agent from the previous step injects a prompt into a coding agent like Cursor or Claude Code. The coding agent reads the classification, looks at the relevant code, and opens a PR with a suggested fix.

Add an agent that will assign the PR to the right person (or auto-merge)

The next question is: who is assigned to the PR?

If the developer should be assigned, it looks at who triggered the run.

But if it’s not, the agent uses the context lake to decide who is assigned to the PR.

Build step failed → developer. The platform owns the Dockerfile, not the code inside it.

Push to GitHub failed → platform. That's a credential or network issue, not the developer's code.

Linter failed → developer. They broke a standard.

Deploy to cloud failed → platform. Something is misconfigured in the infrastructure.

Pre-merge test suite failed → usually the developer, since new code broke the test, not the environment.

But for certain failures (like a config typo, a minor standard deviation, a known-safe pattern), the agent assesses the risk of the fix and merges the PR itself. The CI job reruns and nobody reviews it.

For anything riskier (larger diffs, infrastructure changes, new failure patterns), it assigns to a human and waits.

And then you get this in Slack:

Learn more about workflows in Port here

Tracking the ROI of self-healing CI pipelines

The immediate payoff here is fewer Slack pings and faster fixes.

The longer-term payoff is that the context lake gets smarter. Every fix the agent opens gets stored as a resolved failure. That way the next time the same pattern shows up, the agent recognizes it, skips the investigation, and goes straight to the fix. The failure history you connected in step 1 is what makes this possible. It becomes like a memory for this agent.

Build it yourself

Now you know how to build it yourself.

As a reminder, you need three things:

1. A context lake that tracks your services, deployments, and CI/CD jobs

2. Access to the failed CI step

3. An agentic workflow that triggers on status: failed

And of course AI.

Start with one high-value, low-frequency pipeline. Build the triage agent first. See the classification work on real failures. Then layer in PR creation, assignment logic, and the auto-merge decision.

A customer told us about how theirs goes.

Every few weeks they freeze their code and essentially lock themselves in a room and argue about how to manage the next release.

It got me thinking. Surely an agent can help smooth out the release process.

How? With an agent that assesses risk for you for upcoming releases.

And there are lots of risk that can compound:

So I decided to build an agent that can help.

Let me show you how it works.

Step 1: Build your context lake

To build reliable agents, you first need to build a context lake - the system of records and system of actions for any agent to get reliable data and a library of actions it can take.

Specifically for a release management agent, you’ll want to bring in test coverage from your CI pipeline (GitHub Actions, Jenkins, GitLab CI, SonarQube), health status and error rates from your observability stack (Datadog, New Relic, Prometheus/Grafana, Splunk), incident and on-call data (PagerDuty, OpsGenie), and deployment history from your CI/CD.

Then, make sure your each service has the right information to make it useful for a release agent:

• Service tier: tier-1 (revenue-critical), tier-2, or tier-3

• Health status: healthy, degraded, or down right now.

• Test coverage: actual percentage, pulled from CI

• Runbook: yes or no

• On-call rotation: yes or no
  And the most important one:

• Dependency graph: what does this service depend on, and what depends on it

If you want to provide your agent even more context, you can start working on more things that could help the agent assess the risk, like:

• Change failure rate: what percentage of past deployments caused a failure or rollback

• Time since last deployment: services that haven’t shipped in months carry accumulated risk. Less familiarity with the process, more untested surface area.

• Concurrent deployments: what else is going out in the same window that might affect this release

• Deployment timing: Friday afternoon, end of quarter, peak traffic hours

Learn more about building a context lake here

Step 2: Encode your risk tolerance in a skill file

Now that your context lake has all the data an agent needs, you have to tell it how to think about that data. Mostly you want to tell it what kind of risk tolerance you have. And ultimately what you are looking for is a score to tell you how much risk there is in a particular release.

Leading up to a release in large companies, there’s usually a checklist. Those checklists usually have items that fall into three categories: hard gates, scoring factors, and contextual factors.

1. Hard gates

The release doesn’t proceed until these are met.

Some examples:

• Tests passing across unit, integration, and e2e

• Runbook exists for common failure modes

• On-call assigned and aware of the deployment

• Rollback steps defined and tested

• No critical or high-severity security vulnerabilities (from Snyk, SonarQube, or similar)

You can include these in a skill but generally they should be hardcoded into your workflow as human-in-the-loop gates or scorecards. The agent may or may not obey these gates but it’s worth trying to see how it works for you.

2. Scoring factors

These factors contribute to the overall risk score but don’t block on their own.

Some examples:

• Test coverage

• Recent incident history

• Change failure rate

• Blast radius (how many services are downstream of what’s changing?)

• Change size

In the skill, you should assign point values to thresholds for each of these.

3. Contextual factors

These don’t block or score but shift the recommendation:

• Friday deploy or day before a public holiday

• Multiple releases going out in the same window

• Service hasn’t been deployed in several months

• Business-critical period: end of quarter, major customer event, product launch week

In the skill, these become warnings: “Note these in the summary and factor them into the strategy recommendation.”

Here’s what a well-researched release risk skill looks like:

# 🚦 Release Risk Assessment Skill

Use this skill before approving or deploying any release. It turns catalog signals into a clear rollout decision.

---

## 🎯 Outcome

- Identify impacted services, products, and customers

- Detect active incidents in the blast radius

- Evaluate quality signals: tests, scorecards, coverage, incident history

- Recommend one rollout strategy: Canary, Blue-Green, Feature Flags, or Full Send

---

## 📥 Inputs to Collect from Port

| Signal | Source |

|--------|--------|

| Services in release | `release.includes_services` + `release.deployments → deployment.service` |

| Blast radius | `product.releases`, `product.services`, `product.customers` |

| Active incidents | `incident.affects_service` where `status` in `open`, `resolving` |

| Release test gates | `test_unit_passed`, `test_integration_passed`, `test_regression_passed`, `test_smoke_passed` |

| Release readiness | `release.scorecards.release_readiness_gates.level` |

| Service readiness | `service.test_coverage`, `service.health_status`, `service.scorecards.release_readiness.level` |

---

## 🚨 Hard Gates — Block if Any Are True

Do not proceed until these are resolved:

- Tests are not passing (unit, integration, or e2e)

- No runbook exists for this service

- On-call is not assigned or not aware of this deployment

- Rollback steps are not defined or not tested

- Critical or high-severity security vulnerabilities are unresolved

- Release includes DB migrations that have not been tested in staging

- P1/P2 open incidents exist on impacted services

---

## 📊 Scoring Factors — Evaluate and Score Low / Medium / High

| Factor | Low | Medium | High |

|--------|-----|--------|------|

| Test coverage | > 80% | 60–80% | < 60% |

| Change failure rate | < 5% | 5–15% | > 15% |

| Recent incidents | No P1/P2 in 30 days | P2 in last 30 days | P1 in last 30 days or any open incident |

| Blast radius | 1 service | 2–4 services | 5+ services |

| Change size | At or below baseline | Moderately above baseline | Significantly above baseline |

**Overall risk score:**

- 🔴 **High:** any hard gate triggered, or 2+ scoring factors are high

- 🟠 **Medium:** no hard gates, 1–2 scoring factors are medium or high

- 🟢 **Low:** no hard gates, all scoring factors low or medium

---

## ⚑ Contextual Flags — Note in Summary, Factor into Strategy

These don’t block or score but shift the recommendation:

- Deploy is on a Friday or the day before a public holiday

- Other releases are deploying in the same window

- Service has not been deployed in 60+ days

- Business-critical period is active (end of quarter, major launch, etc.)

---

## 🧠 Rollout Decision Rules

| Condition | Suggested Strategy | Why |

|-----------|-------------------|-----|

| High blast radius OR multi-service + active incidents | Feature Flags | Max reversibility and fine-grained control |

| Prior rollback history OR risky cutover path | Blue-Green | Fast switch and rollback isolation |

| Some uncertainty but healthy baseline | Canary | Progressive exposure with checkpoints |

| All gates green, no active incidents, low blast radius | Full Send | Lowest operational overhead |

**Strategy details:**

- 🟢 **Full Send (rolling):** Simple, no extra infrastructure. Only when all gates pass and blast radius is low.

- 🟡 **Canary:** Start at 5–10% of traffic, monitor error rates and latency, expand incrementally. Limits blast radius without full blue-green cost.

- 🔵 **Blue-Green:** Deploy alongside existing environment, switch traffic after validation. Instant rollback if post-deploy checks fail. Use for high-risk releases.

- 🟣 **Feature Flags:** Decouple the deploy from the release. Use when blast radius is high or incidents are active — kill the feature without a rollback.

> **If the release includes any user-facing feature:** consider a feature flag regardless of risk score.

---

## 📌 Guardrails — Never Skip

- If P1/P2 open incidents exist on impacted services: do not Full Send

- If regression or smoke gates fail: block rollout, or use a highly constrained progressive strategy

- If coverage is below threshold and service scorecard is Basic: require tighter rollout controls

- If data is missing: call it out explicitly and default to the safer rollout option

---

## 🧾 Output Format

🚀 Release Readiness Report

Risk legend: 🟢 Low | 🟠 Medium | 🔴 High

🧩 Services In This Release

| Service | Deployment(s) | Coverage | Health | Scorecard |

|---------|--------------|----------|--------|-----------|

| [name]  | [deployments] | [%]     | [status] | [level] |

💥 Blast Radius

| Services Touched | Products Touched | Customers Touched |

|-----------------|-----------------|-------------------|

| [service list]  | [product list]  | [customer list]   |

🚨 Active Incidents (Open / Resolving)

| Service | Incident | Severity | Status |

|---------|----------|----------|--------|

| [name]  | [id]     | [P1–P3]  | [status] |

(omit this section if no active incidents)

🧪 Quality & Gates

| Check | Result |

|-------|--------|

| Declared release risk | 🟢/🟠/🔴 [level] |

| Release scorecard | [level] |

| Unit tests | ✅/❌ |

| Integration tests | ✅/❌ |

| Regression tests | ✅/❌ |

| Smoke tests | ✅/❌ |

🧭 Rollout Suggestion

[🟢 Full Send / 🟡 Canary / 🔵 Blue-Green / 🟣 Feature Flags]

[One sentence explaining why this strategy fits — reference the specific

incident, scorecard gap, blast radius, or risk factor that drove it.]

---

**Legend:**

Risk: 🟢 Low | 🟠 Medium | 🔴 High

Incident status: 🔴 Open | 🟠 Resolving

Rollout: 🟡 Canary | 🔵 Blue-Green | 🟣 Feature Flags | 🟢 Full Send

Learn more about skills in Port here

Step 3: Build the risk assessment workflow

1. Trigger the workflow manually or when a release is tagged

You can easily choose what triggers this workflow. Could be when a new release entity is created in Port, or when a release tag is applied in GitHub, or when the release manager triggers the workflow manually. It’s up to you.

The release entity you created already has everything attached: the services in scope, deployments, related Jira tickets, etc.

2. Create an agent that gathers context and runs the risk analysis

The agent doesn’t just just run a query and format the output. It reasons iteratively using all the relevant data in your context lake.

It starts by reading the release entity: what services are in scope, what’s linked to it. Then it fetches the scorecard status for each service: test coverage, runbook, on-call, health status. Then it follows the dependency graph, checking each downstream service it finds. If it finds something worth investigating like a degraded service or an open incident, it can look deeper. It follows what it finds.

The agent follows the skill you wrote for it: what to look for, how to assign a weight what it finds, when to flag a hard gate, how to score the overall risk.

At the end of the run, it has everything it needs: which services failed their scorecard, what incidents exist in the blast radius, what the dependency chain looks like, and a risk score. Then it writes the summary according to the format you defined in the skill and fires the Slack message.

3. The release management team gets a Slack message

Then the release channel gets a message like this:

One message that gives the team all the information they need to make a decision. Of course it can be completely customized if you want different information. And you can always open Port for the full context.

4. The release can be blocked based on your rules

When a hard gate is triggered, say during an active incident in the blast radius, tier-1 service degraded, or if rollback steps are untested, the deployment actions are disabled and the release gets blocked.

Only the release manager can unlock it. The override requires a reason, which gets logged to the release entity. That log becomes part of the post-incident record if something goes wrong after an override.

5. Choose to how to deploy via self-service actions (or investigate more)

The deployment buttons in Slack trigger self-service actions in Port (if you want them to). The engineer picks the strategy based on the risk report and their judgment. The action runs the deterministic deployment workflow in whatever backend they use: GitHub Actions, Jenkins, ArgoCD.

But they can also just head back to Port to do some more investigation on their own before committing to a strategy.

After the deploy, the outcome feeds back into Port. The release entity gets updated with the result: clean ship, hotfix required, or incident triggered. That history becomes part of the context lake. The next time a service is assessed, its release history is part of what the agent reads.

Learn more about workflows in Port here

Step 4: Track the ROI

How do you know if it’s working? And how do you know that it’s worth it? (Because someone is definitely going to ask)

I’d start with a simple comparison between the agent and someone from the release management team. Track how often the agent agrees with what your release manager would have decided. When it’s agreeing 80% of the time and up, it’s time to put more trust into the agent.

My suggestion is to try to display some numbers that are easily understandable like meeting hours eliminated or time to deployment decision.

You should also see some engineering numbers change. The hotfix rate should go down, as should MTTR. Release frequency should go up.

The agent saves time and catches bad releases. But without tracking it, you won’t be able to prove it.

Learn more about building dashboards in Port here

I also recently recorded this video walking through how to build this agent.

I hope you enjoyed reading a more practical post about building agentic workflows.

Let me know what I should build next!

Not the models, they were all using the latest ones.

But the skills their agents were using were all over the place. And skills make all the difference.

Some engineers configured their own skills. Others copied from older versions. And some just took some skills they found online.

And we had no visibility into any of it because skills are local config files.

So to solve it we built a skills library. Now every engineer runs one command and gets all the skills delivered to their IDE.

They get required company standards pulled automatically and they can also load optional skills based on what they’re building.

Here’s how we built it:

Step 1: We put all our skills in a library

Skill files are just markdown files and they tell AI agents how your company works. You might have one for security conventions, one for incident protocols, and one for coding standards.

Here’s an example of our triage incident skill:

I’d suggest keeping them in version control and connecting them to the services and teams they’re relevant to.

There are a couple reasons why you’d want to keep them in version control.

Firstly, they change a lot and you want to keep track of those changes.

Second, the destination for the skills is usually an IDE, so having them in git makes it easy for developers to sync them to their IDE.

Lastly, and most importantly for us, having the skills in GitHub (or any version control) lets us auto-discover them into Port where we can connect them to everything else.

When skills are scattered (we call them “shadow skills”), we find it useful to auto-discover them from as many places as possible.

We also get inspiration for skills in other places like PR review comments, ADRs and incident postmortems, onboarding docs, and community libraries like skills.sh.

Once you have your library assembled, it’s time to organize it.

Step 2: Organize into groups and decide what’s required

A library of 200 individual skills isn’t really useful to anyone. Nobody wants to browse 200 files and find the few relevant ones.

To solve that, we group them by use case: engineering standards, frontend, backend, data, infrastructure, etc.

That way, when an engineer gets set up, they pick and import an entire group, not individual files.

Skills should also be either required or optional.

Required skills are non-negotiable. Every engineer and agent gets them automatically, no opt-out:

• Security: monitors for prompt injection, flags attempts to exfiltrate sensitive data, blocks unauthorized external API calls

• Coding conventions: how your team handles errors, names directories, structures TypeScript, writes tests

• Rule governance: where skill files live, how they get named, how changes get versioned

Optional skills are role-specific and load only when relevant:

• React components: component structure, RSC vs. client component rules, Web Vitals priorities.

• Incident triage: walks through affected services, recent deployments, root cause categories, stakeholder update format.

Step 3: Engineers get the skills automatically

Engineers connect to the library using their existing permissions, and the CLI pulls the required groups automatically, then prompts them to choose which optional groups they want: frontend, data and ML, whatever matches their work.

Skills land directly in .cursor/ or whichever IDE config folder they use.

Two things make this work really well for us:

First, updates. Every engineer in Cursor has an automation that listens for changes to skills in Port and pulls new required skills and updates existing ones.

Second, we make it easy for developers to contribute back. They can go to Port and star skills they like and submit new skills directly from their IDE. Then they go through review by the platform team, get merged, and get redistributed to the whole team, basically the same loop as an open source contribution. The library gets smarter over time without the platform team having to own every skill themselves.

Step 4: The feedback loop

We also have a meta-skill: a skill whose job is to watch for repeated corrections. When you fix the agent on the same thing twice in a session, it surfaces: “You’ve corrected me on this twice. Want me to write a skill for it?” If you say yes, it kicks off the skill-creator automatically, pre-loaded with what it just observed.

The loop becomes self-closing. The agent notices its own gaps, proposes the fix, and routes it into the same review and distribution flow. The platform team still reviews everything, but they’re not waiting on engineers to manually notice and submit. The library improves in the background.

Step 5: Track if it’s working

Before the skills library, we had no idea what skill any agent on the team was running. Now we have a dashboard that tells us who’s set it up, which groups they pulled, and when they last synced. If someone is running on a six-month-old version of a skill, or never ran skill init at all, we know.

We also track skill health: how old each skill is, when it was last updated, who owns it. Skills go stale just like documentation does. Anything that hasn’t been touched in 90 days gets flagged.

The contribution pipeline is basically a PR queue for the library: Skills submitted, in review, merged, rejected.

The metric I find most interesting is agent quality signals. We track which review comments keep appearing on AI-generated PRs. If the same comment shows up more than a few times across different engineers, it automatically contributes to a skill or creates a new one.

And we track coverage by team: which teams have required skills loaded, which optional groups they’ve pulled, or where the gaps are.

Why this matters: agent sprawl

Your team probably already has an agent sprawl problem but you may not know it yet: hundreds if not thousands of unofficial different skills, agents, MCP servers being created and used.

A skills library is part of the fix. And once you have one, someone on your platform team can actually answer the question: what does every agent in this org know right now? At most companies, nobody can answer that.

Build it yourself

It’s just a few hours of work for one platform engineer, and once it’s running, the whole team benefits automatically.

So, how can you build this?

Everything you need is right here in this guide

👉🏼 Manage skills with Port

And to help guide you, you can follow along here:

Reach out or leave a comment if you need any help!

I distinctly remember working on and seeing posts on Linkedin mapping out industries. It was fun debating who should be in which category and where an industry was heading.

Now, I’m in the unique position of seeing the industry from a completely different perspective.

When we started Port, I would have said that we were in the DevOps or platform engineering industry. In a sense we still are, but we’re now helping platform engineers build the layer above it: agentic engineering.

So that’s the industry I wanted to map out.

In our mission of moving engineering from manual to autonomous, we’re seeing companies introduce agents to every part of the SDLC - from planning all the way to operating.

With that huge influx of agents comes new needs. Those agents need infrastructure. It’s not the same infrastructure as DevOps though. It’s agentic engineering infrastructure that is specifically designed for agents like context, guardrails, and orchestration.

For that reason you’ll see in the map I’ve separated everything into agents and agentic engineering infrastructure.

While researching, I came to some other conclusions about the market:

1. Coding agents get most of the attention but the market for AI that operates across the entire SDLC is actually much bigger than the market for coding agents. The internet goes wild every time a new coding model comes out which makes sense. But harnessing that model to make it actually operate and be useful beyond coding is where most of the work lies. That’s why I think the market for the harness will be bigger than the models.

2. AI won’t replace the platform stack. It adds a new layer on top. Agents + agent infrastructure sit above the existing DevOps/cloud/observability foundation. Agentic engineering is a mix of deterministic (DevOps layer) and non-deterministic (Agentic) flows.

3. It looks like there aren’t any deploy agents on the market which makes sense. Deployment itself should not be left to agents. But I think there’s room in the market for risk assessment agents that decide which deployment strategy to take.

4. “Agent Experience” replaces “Developer Experience” as the design target. When choosing which tools or infrastructure to work with, the question is no longer “which ones do our developers like?”. Instead, when you plan your stack, you will ask your agents to find tools for you. They will choose tools that work best for them. If a tool doesn’t have a strong MCP server, an agent will find an alternative that does. UI is not nearly as important as data/context.

You might ask: why is Port in so many categories? That’s because platform engineers and platform engineering will be critical to building out the agentic engineering infrastructure blocks. It may not be fully clear to everyone yet, but based on what our customers tell us every day, controlling agentic chaos and agent sprawl will 100% be the platform team’s responsibility.

As always, I'd love to hear what you think. Feel free to reply and let me know which agents and infrastructure you’d choose when building your stack.

The best way to accomplish this was through self-service in a platform you controlled. When a developer needed a new service, they clicked a button and got a repo, a CI pipeline, monitoring, and a Dockerfile. You could let them do this because you built in engineering standards. That combination of self-service and standards was a win-win for you and your developers. They got velocity, you got control.

With AI, developers became builders of agentic workflows. By building agents, they can automate large parts of their jobs. They’re building SRE agents, release agents, skills, etc. They already got permission to use all these AI tools, so they started to stitch stuff together.

What you get is lack of governance & visibility into agents that are taking destructive operations without human oversight.

We call this agent sprawl.

So what does that mean for you as a platform engineer?

How should you think about the relationship between you and developers?

As platform engineers in an AI world, of course you want developers to use AI more. But every time adoption of technology goes up, so does the demand for everything you’re responsible for like costs and governance.

There’s also something uncomfortable about being bypassed by the people you were supposed to enable. You take pride in making developers more capable with your help, but now they’re building agents themselves.

AI has given developers the power to build anything that will automate their work like an SRE agent, deployment agent, or even a basic catalog that helps their agents work.

To make their agents powerful, they want to feed these agents with real actions & access to real data. To do that, they still have to go through you. But even without the actions and context lake you provide, they can still create things that are powerful enough to be dangerous.

Let’s say an engineering team builds themselves an SRE agent without going through you. We can even go a step further and say that it works really well in its task of triaging the issue, saving the team hours per incident.

But what does the agent probably not do?

It probably doesn’t leave an audit trail.

It probably isn’t being respectful of PII.

It probably doesn’t use appropriate credentials to get data.

And the list goes on.

What the team definitely didn’t do is build the agent with the standards their company expects from the start.

And why would they? They have a real, immediate need and AI makes it easy to solve. They’re also feeling the pressure of producing more results by engineering leadership.

Developers shouldn’t have to wait. (And they’re not going to)

With this new tool and the new urgency, they’re just doing their jobs in a new way, primarily doing two things:

1. They’re using MCP to query data and get insights that help them or their agents work.

2. They’re building agents like SRE agents that are getting more capable, taking on more responsibilities across the SDLC.

Developers (and agents) getting the data they need can be solved in a scalable way by providing a rich context lake.

But when all engineers start building agents, how do you facilitate that in a scalable way?

The product management side of platform engineering

Given the new reality we’re all living in, platform teams have three options.

You can do nothing and let developers build agents however they want. The agent sprawl is already here anyway, so why fight it? The problem is you end up with no visibility, no standards, and no way to intervene when something goes wrong (and it will).

You can mandate that all AI development flows through the platform team. You can probably expect a revolt if you choose this route because the tools are already in developers’ hands. Cursor runs locally. Claude Code runs locally. This is a good way to slow down engineering, not speed it up.

Finally, and I think this is the only option that will actually work, you can make the platform compelling enough that developers actively choose to build agents with it.

So how do you make it more compelling?

This is what I like to call the product management side of platform engineering. You first need to understand what developers are actually trying to do, then remove the hardest parts of it.

I’ll let you do the work in your org of talking to developers and finding out what they find difficult when building agents.

But I’ll also give you a look into what I see as some of the common denominators because developers building agents hit the same walls every time. Scattered data, wild integration connections, lack of approval gates (or too many), or too much access to critical systems, to name a few.

I see those issues repeat themselves across all agents at the “kicking the tires” stage. But, if you succeed in removing some of that friction or at least minimizing it, you’ll have a much higher chance that developers will create agents that don’t end up hitting those walls.

There’s also something else here. Developers who build something useful want others to benefit from it, just like in open source. If there’s a place to share the incident triage skill their team built, they’ll put it there. If there’s a registry where others can find it and extend it, it gets better over time. The platform becomes the place where good work accumulates.

That’s what makes a platform compelling. When developers feel they are using an experience that was designed for them (and their agents).

How do you create a compelling agent-building experience for your engineering team?

Now that your developers are agent builders, one of your jobs is to make that experience as seamless and enjoyable as possible. They’re coming to make something new. That’s a creator experience, and it requires a different kind of thinking. This is DevEx 2.0.

Here’s what I think matters most when you’re designing for builders:

1. They need to know what’s available to them before they start. A developer sitting down to build a release management agent shouldn’t have to ask around to find out what data they can access, which integrations exist, or what actions are possible. That discovery has to be seamless.

2. They need early wins. The fastest way to get a developer invested in your platform is to get them to something working quickly. Starter templates, example workflows, pre-built skills are all great places for them to start forking. A developer who gets something running in an hour is far more likely to build the real thing on your platform than one who spent the first day figuring out how to authenticate.

3. They need to see what others have built. This is the open source side of things that developers love. Developers who build something useful want to share it, and developers starting from scratch want to see what already exists. A skill registry or agent catalog will make them feel that the platform is alive. When a developer can browse what their colleagues built and extend it, the platform stops feeling like infrastructure and starts feeling like a community.

Let’s expand on helping your team discover what’s available before they start building agents.

What building blocks should you give them that their agents will need?

I wrote previously about the “Hidden tech debt of agentic engineering” where I expanded on the 7 blocks of tech debt every demo agent creates

A reliable context lake. Agents need to know things in real time and that information needs to be accurate: who owns this service, what changed in the last deploy, what this service depends on, who’s on call right now.

Without this, a team building their own agent spends more time on data plumbing than on the actual logic and gets access to data they wouldn’t be able to access without it. The platform team pulls it into one place, keeps it fresh, and exposes it through an API or MCP that any agent can query.

Pre-cleared integrations. Agents need to talk to systems: GitHub, PagerDuty, Datadog, Jira, your cloud provider. Getting access to each of those is a process. Developers building agents on their own may go through that process independently. By centralizing it, the platform team clears the integrations once and every agent gets them.

A menu of approved, governed actions. Agents don’t just read. They do things: trigger rollbacks, open PRs, restart pods, close incidents. Teams building agents on their own either avoid actions entirely (so the agent just tells you what to do) or wire them up without guardrails. The platform team defines which actions are available as tools, so the question is never “can this agent do this?” but “did we explicitly allow it to?”

Deterministic policies. An agent that suggests a production rollback is fine. An agent that triggers one at 2am without waking anyone up is not. You can’t handle that with a prompt or skill like “Please ask before doing anything in production”. Policies must be encoded as gates: staging auto-approves, production requires human sign-off outside business hours. They either pass or they don’t and they need to be deterministic.

An audit trail. When something goes wrong at 3am, you need to know what the agent did, what data it accessed, what triggered it, and what the outcome was. Teams building their own agents almost never add logging upfront. It feels like overhead at first. The platform team needs to build audit logging into the workflow engine so it’s automatic. Every action is on the record whether anyone thought to ask for it or not. The added benefit of the audit trail is that it also serves as the decision trace that will help agents improve over time.

A review process. Taking an agent from demo to production requires someone from platform to check permissions, test edge cases, and define what production-ready actually means. The platform team must own that lifecycle: from experiment, to reviewed, to trusted, to fully autonomous.

An approval or human-in-the-loop layer. At some point every agent needs to stop and ask a human. Teams building their own approval logic get this wrong in different ways. Some hardcode a Slack message, some skip it entirely. None of them share a model, so when agents need to work across teams, the approval logic breaks at the seam. The platform provides one: these actions auto-approve, these require a human, this is who gets notified. A team that builds on it doesn’t have to think about any of that.

None of these are complicated on their own. But when every engineer is building their own agents, the problem compounds quickly.

The surface area grew

The ground is shifting and your old job didn’t go away. The service catalog still needs maintaining. The golden paths still need updating. The CI/CD integrations still break.

And now on top of all of that, there are agents running in the org that you didn’t build, accessing systems you provisioned, doing things you can’t fully see. The surface area of what you’re responsible for grew, but nobody took anything off your plate to make room.

Just because you don’t have a monopoly anymore doesn’t mean you’re not relevant. You’re actually about to become relevant than ever before. Because developers aren’t waiting and the number of agents is exploding. And the window where the number of agents in your org is still small enough to get ahead.

We talked about a win-win situation for platform engineers and developers. I believe both teams can still achieve a win-win while using AI.

Port is built for platform engineers stepping into this new role who want to build a win-win experience. It gives your team the foundation to give developers the building blocks they need in their new builder role: governed data, pre-cleared integrations, approved actions, approval gates, and more.

Every engineer is a builder now. It’s time for platform to build for them while staying in control.

If you’re new here, I’m Zohar, CEO of Port. This newsletter is where I share what I’m seeing in engineering, AI, and how companies are actually running AI in their SDLC.

Bessemer published a deep dive into how Shopify runs AI across 3,000+ engineers, and it’s one of the more detailed playbooks any company has shared publicly. So I went through it, pulled out the sections that matter most, and wrote my reaction to each + a practical recommendation.

The problem I’m seeing most in the industry is agent sprawl. It looks like Shopify has avoided it.

Here’s how it plays out the bad way: Dev teams each pick their own AI tools, wire up their own integrations, and build their own agents. Suddenly you have thousands of siloed agents and a lot of them do the same thing. You don’t have visibility into what exists across the org, what actions agents can take, or what data they can access. It gets messy. Fast.

Shopify avoided this by building the infrastructure first:

• An LLM proxy routing everything through a single gateway.

• Standardized MCP servers connecting to company data.

• Usage tracking by team + Alerts at $250/day.

Once they had the right infrastructure in place they were able to tell engineers: pick whatever tool you want on top. That’s only possible because the layer underneath is centralized. Everyone else just plugs in without worrying about getting things wrong.

Most companies do the opposite. They start with the tool. Then they might negotiate an enterprise license for Cursor or Copilot, roll it out, and hope for the best. No centralized tracking, no shared integration layer, no visibility into what’s happening across teams. When they need to switch tools or add a new one, they’re starting from scratch every time.

I think of it like going shopping. The platform team sets up the store: the data, the actions, the workflow engine. Then developers walk in and add things to their cart to build what they need. They don’t wait or file a request. They pick what they need and build an agentic workflow. But this only works if the infrastructure is already there. If it’s not, developers run ahead anyway, each team builds their own thing, and the platform team is stuck catching up.

Top-down mandates don’t work for AI adoption and Shopify identified that too.

They tried the opposite: make leadership look effortless using AI, and let that pull people in. Weekly demos instead of KPIs because, as Farhan says, otherwise engineers would just game a metric.

Another clever strategy they are using is a prompt/skill library. When someone figures out a workflow they can share it with others.

Ultimately, you can’t chase developers into adopting something and this is especially true in the AI era. The best platform teams I talk to aren’t building agents themselves. They’re building the infrastructure so their developers can build agents safely. They clear the path: pre-built integrations, a workflow engine, guardrails already in place. Then they get out of the way. The platform team becomes a facilitator, not a gatekeeper.

Tobi’s memo pushed it even further: prove AI can’t do the job before requesting headcount. You can read the memo here:

The 20% productivity number seems like the story here but it’s not. The real story is that Farhan and Shopify are focused on solutions, not more code. Customers don’t care how much code you (or AI) writes. All they care about is if you solved their problem or not.

That’s what AI is enabling: testing 10 approaches instead of 2. Shopify is telling you the unit of productivity has changed. It used to be lines of code shipped. Now it’s problems solved. And when code is cheap, you explore more broadly, which means many, many more things running in your environment, more services, more agents, more surface area to manage.

What Shopify is admitting here is something most companies won’t say out loud: even with a dedicated infrastructure team and unlimited budget, they got the tool-to-persona mapping wrong and had no visibility into spend until it was already a problem. Cursor for non-engineers was the wrong UX at the wrong price. Individual engineers hit tens of thousands in tokens per week before anyone noticed. They bolted on alerts at $250/day after the fact. The message is clear: if you don’t have centralized tracking from day one, you will discover your AI costs the hard way.

I talk to enterprise architects who have no idea what they’re spending on AI tokens across the org. No visibility, no throttling, no control. And these are the proactive ones, the companies trying to get governance in place before the chaos starts. Shopify had every advantage and still needed to course-correct. Most companies are running this experiment with no safety net at all.

Farhan focuses on PR reviews as the bottleneck here. Actually, I think the problem is bigger than that. The number of things engineering needs to review or approve is exploding. Agents now write code, deploy to prod, roll back versions, remediate incidents.

If you put a human in the loop at every stage, you exhaust your engineers and fail to adopt AI at all.

The way I think about it: humans shouldn’t be in the loop. They should be out of it.

When should they be included? Well, it depends on the risk. A PR comment? Nothing bad can happen. Let the agent run, review later. An auto-remediation that restarts a service? Higher risk, but if it’s a governed action with guardrails already in place, let the agent do it and notify the team after. A prod deployment to a tier-1 service? That’s a blocking review. A human should own that decision.

The tricky part is that risk is dynamic. Deploying during a workday is very different from deploying at 2am. Same operation, different context. You can’t hardcode the rules. You need to trust AI to execute and report back, trace every decision, and measure quality over time. You’ll see where it’s reliable and where it still needs a human.

Farhan says “if you don’t figure this out in 2026, you’ll be behind”.

I agree completely. Except I think it’s coming sooner.

And then what happens when an agent works well? The article is oddly silent about this.

How do you take a great agent from demo to production?

• How do they work together?

• Which data can they access?

• Do they need humans in the loop?

• The questions go on…

The platform layer is described well. The agent lifecycle is missing.

I’d be interested to learn more about how Shopify will handle this.

—

Shopify built all of this from scratch with 3,000 engineers and a near-unlimited budget. The pattern they arrived at, having a centralized platform underneath with tools on top, is what every engineering org will need. When that platform exists, developers start building their own agents on top of it and you stay in control.

In 2015, Google published “Hidden Technical Debt in Machine Learning Systems.” That paper turned on the light for machine learning engineers and named all the problems they were experiencing. The diagram they shared also became iconic: a tiny box labeled “ML Code” surrounded by massive infrastructure blocks.

We’re seeing the same pattern for agents. Agents are a small part of the picture, and we want to name all the infrastructure around them.

Agentic engineering systems are especially good at piling up technical debt. They carry all the maintenance problems of traditional software, plus an additional set of agent-specific issues. New agents are being created daily by nearly every employee. Soon you’ll have far more agents than employees.

We define an agent as any process with dynamic decision-making capabilities that can autonomously determine tool usage and execution paths through reasoning and reflection. Decision-making, reasoning, and reflection require all the supporting infrastructure.

Building an agent is easy. But in production, the agent code is the smallest part of the system. Everything around it is where the actual complexity lives.

Over the past few months, through conversations with engineering leaders and our own experience, we’ve mapped out seven infrastructure blocks that surround the agent. Each one is a category of work that no one plans for when building their demo.

Some of these blocks will look familiar if you’ve done traditional engineering: observability, integrations, and governance. Others are unique to agents, such as human-in-the-loop, evals for non-deterministic systems, and the agent registry.

Let’s walk through each one.

1. Integrations

Agents need to reach your actual systems: CI/CD, cloud providers, incident tools, observability platforms, code repos, secret managers, and more.

Without centralized integrations, every team wires up its own connections to agents.

Picture an engineering org with 200 engineers on 30 teams, each with multiple agents. Each engineer generates their own GitLab PAT for coding agents, Snowflake credentials for data agents, Kubernetes service accounts for deployment agents, and monitoring tokens for incident agents.

That’s hundreds of integration points, each one configured individually, debugged individually, and expiring on its own schedule.

When every developer wires their own credentials, each agent sees different data depending on whose token it uses. One developer’s GitLab PAT has access to all repos. Others are scoped to their team. Same agent type, but each has a completely different view of the org.

Or, what happens when GitLab ships a breaking change to their API? Every team that wired its own connection independently debugs the same issue (or submits a ticket to the platform team). Three teams figure it out on Monday. Two more by Wednesday. One team doesn’t notice for a week because their agent only runs during incidents.

What also matters is what comes through those integrations. When three teams connect to the same data source through different paths, their agents can get different answers to the same question. If one team’s integration pulls a 30-day history of deployments, and another team’s integration shows everything from the last 3 years, their outputs will differ.

Right now, MCP is how most teams connect agents to their tools. But let’s not confuse MCPs with integrations. MCP gives agents a standard way to call a tool. It doesn’t manage the credentials for that call, the scope of the data that comes back, or what happens when the API on the other end changes.

Hidden tech debt in integrations looks like:

• An integration auth token expires on a Friday night, and an incident agent silently stops working. Nobody notices until Monday.

• Five teams each maintain their own GitLab connection with different permissions and scopes, unaware that the others exist.

• When an integration updates its API, and every team debugs its connection separately.

2. Context lake

Agents are only as good as the context they can reference and use. They need two kinds of context.

Runtime context: How do you deliver accurate context to agents during their execution?

The runtime context is the live data that agents need for a specific execution, such as information about services, who owns them, and what was deployed recently. It’s the same kind of data humans use when coding or resolving incidents, but more accessible to agents.

Think of a coding agent picking up a ticket to add a retry mechanism to a service. It needs to know: what language and framework the service uses, how retries are handled in other services in this org, who owns the downstream service it calls, and whether there’s been a recent config change to the timeout settings.

Some teams manage their runtime context in markdown files: agents.md, .cursorrules, skills files.

Markdown files are fine for static instructions, such as how to format commits or which linter to use. But the runtime context changes constantly. Service ownership shifts. Dependencies get added. Config values update. Deployments happen every hour. An agent running on a .md file that says “checkout-service is owned by Team Payments” doesn’t know that ownership transferred to Team Commerce last Tuesday. The file was accurate when it was written. By the time the agent reads it, it might not be.

Decision traces: How do you help agents learn from their own past work or the work of other agents?

Decision traces are a history of what has been done before (by humans or agents), why it was decided, and what happened afterwards. Without that history to reference, every agent run starts from zero. Think of an agent opening a PR to fix a flaky test. It doesn’t know that a different agent tried the same fix last week, the PR was rejected because it broke a downstream contract, and the team decided to deprecate the test entirely. So it reopens the same PR. Without decision traces, agents repeat mistakes that humans (or agents) have already resolved.

An agent that resolved 50 incidents has seen patterns a new agent hasn’t, like which fixes worked, which ones caused regressions, and which services are fragile after deploys. Without traces, that knowledge disappears after every run. When multiple agents operate on the same systems, they can’t see each other’s history.

LLM providers are starting to address this with `memory.md` files that can be shared across teams. But the debt still shows up when you have dozens of agents operating. You need to find a way to reliably serve that memory (or just the right parts of it) to specific agents.

Hidden tech debt in context lake looks like:

• Stale, fragmented context that no one owns

• Agents running on agents.md when company standards live in a wiki

• Agents not learning how and why other agents solved a problem, or about the mistakes they made

3. Agent registry

Gaining visibility into what agents exist

The org chart is changing. Instead of just people, you now have 5–10 times that number of agents. They’re being created daily by all your human employees. They’re running without guardrails, they have access to critical infrastructure, and they’re making decisions. They’re also spread across tools like Claude Code, Cursor, n8n, zapier, Notion, AWS, GCP, and more.

The typical pattern goes like this: An engineer builds a triage agent, and their team starts using it to help with incidents. Another team builds its own version because it didn’t know the first one existed. A third team builds something similar but wired to different tools with different permissions.

In a company with 20 or 30 engineering teams, you’ll quickly reach agents with overlapping responsibilities, conflicting behaviors, and invisible dependencies. Before agents can be shared between teams, you need to know they exist.

Delivering instructions to agents

Once you have visibility into your agents, they need the equivalent of an employee handbook: standards, skills, and instructions on how they’re expected to operate.

Today, engineers create skill files for their agents independently. The issue is that when they are scattered across repos without a centralized view, teams end up creating duplicate or inaccurate skills. They often contradict the platform-distributed context. The platform team has far more insight into what to write in skill files than individual teams do.

So how do you get consistent but personalized coding rules, commands, skills, and hooks to the right agents?

This information may even require multiple levels:

• company-wide standards that apply everywhere (secure coding, commit conventions)

• repo-specific instructions (“in this repo, events are created this way”)

• Or team-level rules for subsets of engineers

You’ll need to find a way to reliably deliver this information to thousands of agents, ensuring the right instructions reach the right agents.

Agent creation

Let’s say you’ve gotten visibility on most existing agents and delivered instructions to them. The next question is: How can you control the process of creating new agents without slowing your team down?

That responsibility now falls on platform teams.

Like services in a software catalog, agents should have standardized properties and should be connected to other entities in your company, like other agents, teams, services, deployments, etc.

Without a template, you get the same sprawl problem you just solved. An engineer spins up an agent with no owner, no lifecycle state, and no connection to the service it operates on. It works for them. Nobody else knows it exists. Six months later, someone finds it running in production with expired tokens and no way to contact whoever built it.

“An engineer spins up an agent with no owner, no lifecycle state, and no connection to the service it operates on. It works for them. Nobody else knows it exists.”

Agent creation should follow a standardized template. This doesn’t mean that human employees should be slowed down when they create agents. The opposite is true. By having a standardized process for creating agents, you will help them reach high-quality work faster than if they were created individually.

Agent creation should be allowed from the engineers’ workstations. If they work in Cursor and need to spin up an agent, they should be able to do it from there, not elsewhere. Which means your job, as platform engineers, is to make sure agents created anywhere follow your creation process.

A template doesn’t restrict what the agent does. It makes sure every agent is born with the basics: an owner, a description, the tools it uses, the services it touches, and a lifecycle state. That’s what makes it governable from day one, rather than something you have to chase down later.

Hidden tech debt in the agent registry looks like:

• Invisible agents

• Teams are creating duplicate agents

• Outdated context

• When the CISO asks for an agent audit, but there’s no list to start from

• No clear way of promoting an agent to production

• No versioning, rollbacks, or staging environments

4. Measurement

How do you know if your agents are working? Well, it depends on who’s asking.

An SRE wants to know what the agent did.

An ML engineer or product manager wants to know if it’s getting better or worse.

A VP of engineering wants to know if it’s worth the money.

An end user wants to know that the agent is learning based on their feedback.

So while each person wants to measure something in agents, they all want different types of measurement.

1. How do you know what your agents are doing?

This is observability.

Events, traces, and logs show what actions the agent took, what data it had access to, and if it is still running correctly. Engineering teams understand observability from traditional systems, but with agents, the surface area is wider.

Say you have an agent that autonomously solves Jira tickets. When it receives a Jira ticket, it reads the service catalog to find the relevant repo, pulls recent commits from a GitHub integration, generates a fix with a coding agent, opens a PR back in GitHub, and requests review from the service owner that it found in the software catalog. If the fix broke something, which step went wrong? Did it pull the wrong repo? Misread the ownership? Generate bad code?

If you can’t trace individual links in that entire chain, good luck debugging it.

2. How do you know if your agents are getting better or worse when prompts, skills, tools, and models change?

This is evals

In standard software engineering, you can write a unit test that expects an exact string. In agentic engineering, when every response is different, you need a different approach.

Evals answer a simple question: after you change something (like a prompt or model), is the agent still good?

Without a way to track this, changes go out untested, and output quality degrades silently. Like swapping Sonnet for Opus, and your PR review agent starts approving things it used to flag.

3. How do you know if your agents are actually working for your business?

This is business impact

Every earnings call this year will include the question: “What is AI doing for your business?”.

Most engineering leaders can’t answer it yet, and ultimately, they are the ones responsible for measuring agent cost and ROI.

Tracking spend is the easier part of this equation. You can track token usage, API calls, and compute costs per agent, per team.

ROI is harder to measure. How many tickets did agents resolve? How much engineering time did they save? Did it actually reduce MTTR or just move work around? These numbers are harder to collect and harder to trust. If you can only show the cost side without showing a clear ROI, that’s a bad conversation to be having.

4. How do you give agents feedback on their work?

These are feedback loops

When an agent generates a PR, resolves a ticket, or writes an RCA, did the human who reviewed it accept the output or correct it? This is generally managed with a thumbs-up or thumbs-down, but sometimes it’s the human’s response itself that counts as feedback (something like “No, try again, but do change X instead”).

They’re critical for improving the agent and more important than evals. Agents in the demo phase either don’t collect these signals or don’t act on them.

Hidden tech debt in measurement looks like:

• Not knowing if agent performance is improving or declining over time, and compared to what

• Not being able to measure what happens when a prompt or model changes

• Leadership is asking for ROI, but you don’t have a clear answer

• Failure to collect feedback from humans using agents

5. Human-in-the-loop

There’s a spectrum between fully manual and fully autonomous engineering. On one end, a human does everything. On the other end, an agent acts without asking. Most useful agents live somewhere in the middle, and where exactly they sit depends on the action, the environment, and the risk.

Human-in-the-loop is one of the mechanisms that lets you move agents closer to autonomous safely. It lets you define checkpoints: this action needs approval, this one doesn’t, and this one depends on the environment. The agent runs, but a human confirms the high-stakes decisions before they execute.

For example, a deployment agent might run freely in staging but require approval in production. It can be fully autonomous during business hours, but we need a human in the loop at 3 a.m. The rules are conditional and vary by agent, action, environment, and team.

When you have one agent in a demo, you can hard-code the approval checkpoints as an if statement that pings Slack before a deploy. With 100 agents across 20 teams, hard-coded approval logic doesn’t scale. Every team implements its own version. One team’s agent rolls back production without asking. Another team’s agent requires three approvals to do the same thing. Nobody defined this centrally, so nobody knows which agents can act on their own and which can’t.

And then there’s the orchestration of the approvals themselves. Who gets notified? Through what channel? What’s the timeout if nobody responds? What happens if the approver is on vacation? If approval comes through Slack for one agent, via email for another, and via a custom UI for a third, you now have three approval systems to maintain. The logic around approvals becomes major tech debt, separate from the agents themselves.

If we zoom out, human-in-the-loop is also about visibility into what’s happening inside all the agents working for us. As engineers move into a more managerial role (of agents), they will need a control plane to see what’s in motion, initiate agent work, identify which agents need attention, and take action if needed.

This matters because human-in-the-loop is how you apply change at scale (and companies today either change or die). To succeed in their new job, engineers need to see the agents working, just as they used to see their code working. A team that can watch an agent handle its first ten deployments (see every step, review every decision, and intervene when needed) will trust that agent. A team that can’t, simply won’t.

Hidden tech debt in human-in-the-loop looks like:

• Hard-coded approval code that can’t be changed from one place

• Some agents run without approvals, others have too many

• Multiple approval systems by email, Slack, and custom UI that don’t work with each other

• No shared workspace where teams can see what their agents are doing and intervene if necessary

6. Governance

When a human engineer needs access to a production database, there’s a process. They submit a request, someone approves it, and the access is scoped and logged. It might take an hour or a day, but there’s a record of who has access to what and an audit log of who did what.

When an engineer creates an agent locally, it usually runs with whatever credentials its creator wired in: their API tokens, their service account, their cloud permissions. Most likely, nobody reviewed the scope.

Governance rules for agents need to be specific:

• “Roll back a service, but only if there’s a high-severity incident open.”

• “Deploy to production requires manual approval, always, no matter what agent triggered it.”

• “An RCA report that pulls data from external systems should only be visible to the owners of that service.”

These are the kinds of rules that platform teams need to define in a single centralized place and apply across all agents.

The other side of governance is enforcement. Say you discover a vulnerability in one of your internal APIs and need to block all agents from calling it immediately. Can you? One security engineer should be able to disable a tool in one place and have it automatically disabled across all agents. At most companies, that capability doesn’t exist yet.

You can’t always get access permissions airtight from the start. Things can go wrong, and if they do, you need to know what happened: which agent took the action, what data it accessed, what credentials it used, and who triggered it. Most agent setups don’t produce that audit trail (certainly not if they’re created locally). An agent running locally inherits its creator’s credentials, so every action appears to be the engineer’s personal work. If three agents share a service account, you can’t tell which one made the call. If an agent chains through two other agents before modifying a production config, the audit log might show only the final write, not the reasoning, context, or the chain of decisions that led to it.

Another aspect of governance is cost governance. Agents tend to keep working despite the costs they are racking up. When an engineer creates an agent locally, they probably won’t think about setting a cost limit.

An agent stuck in a retry loop or reasoning in circles will keep burning tokens for hours until someone manually kills it or until the monthly invoice shows the damage. Most teams can tell you their total LLM spend, but almost none can break it down by agent, by team, or by use case. Teams should also be able to easily see where their costs stand. Because when leadership asks what agents cost to run, engineering will need an answer.

Hidden tech debt in governance looks like:

• An agent that shouldn’t run in production, accessing production data

• An RCA agent is publishing sensitive service data to a shared channel

• An agent publishing PII in a public forum

• No way to disable a tool across all agents from one place

• No audit trail of what an agent did or why

7. Orchestration

Most agentic workflows aren’t purely agents. They’re a mix of agents, tools, and people. The debt isn’t in the individual steps. It’s in what happens between them: routing, failure handling, and ownership.

What breaks between the nodes

Take an incident response workflow like the one above. An alert fires. A triage agent picks it up, investigates, and determines the root cause is a deployment issue. It hands off to a deployment agent that rolls back the change. A verification agent checks that the fix worked.

Now imagine the triage agent got it wrong. The real cause was a database timeout, not a bad deploy. The deployment gets rolled back unnecessarily. The database issue persists. The alert keeps firing. Eventually, a human steps in and starts debugging from scratch, but now they’re also cleaning up the rollback that shouldn’t have happened. The failure wasn’t silent. The workflow confidently took a wrong action, and nobody could tell where the bad decision was made.

That’s what orchestration debt looks like in practice. Not necessarily workflows that stop, but workflows that do the wrong thing and are hard to trace.

And every new issue type you add later, like security incidents, config drift, or dependency failures, makes routing harder to test and explain. None of those changes is hard on its own. But when nobody owns the decision of how they connect, they get wired together differently every time.

What’s different from traditional workflow orchestration

Workflow orchestration isn’t new. Teams have been chaining steps together for years in CI/CD pipelines, Airflow, and Step Functions. So why is agent orchestration a different problem?

Traditional workflows are deterministic. Step A produces a known output, step B consumes it. You can test every path because you know every path. Agent workflows introduce non-determinism into chains that previously had none. When you replace a runbook with an agent that reasons about the problem, every downstream step becomes unpredictable. You can’t test every path because you don’t know every path.

There’s also no contract between agents. Services have APIs with schemas and versioned endpoints. Two agents handing off to each other have prompts and natural language. The “interface” is fuzzy. A model update or prompt change in one agent can shift its output just enough to break the next agent in the chain.

A deployment pipeline, for example, should be completely deterministic. The trigger fires, the system knows the severity, deploys to staging, a human approves, it goes to production, and a verification agent checks health. The steps are predefined, the order is fixed, and the risk is high enough that you want it that way.

An incident response workflow is inherently non-deterministic. The root cause could be a bad deploy, a database issue, a misconfiguration, or something nobody’s seen before. The agent has to investigate and decide what happens next.

Most engineering teams need both kinds of workflows. The debt is that there are no shared rules about when to use which. One team sends every triage result straight to a coding agent. Another requires human review before any automated fix. Both work in isolation until an incident crosses a team boundary and the two approaches collide.

Who owns the workflow?

Even if every individual agent has an owner, that’s not enough. The orchestration itself needs an owner.

When an agent modifies a service, and something breaks, who’s responsible? Is it the agent’s owner or the service owner? When a workflow spans three teams, which team owns the outcome? When a step fails mid-chain, is it the failing agent’s responsibility to retry, or the workflow’s responsibility to reroute?

These aren’t theoretical questions. They’re the questions that come up at 2 a.m. when an agent-driven workflow takes a wrong turn, and three teams are in a war room trying to figure out whose agent did what. Individual step failure is easy to debug. Figuring out which decision, three handoffs ago, sent the workflow down the wrong path — that requires tracing that most orgs haven’t built.

“…when an agent-driven workflow takes a wrong turn, and three teams are in a war room trying to figure out whose agent did what.”

Hidden tech debt in orchestration looks like:

• An agent fails mid-workflow, and nobody finds out until the downstream effect surfaces

• No way to trace a decision across agents back to the original trigger

• A workflow spans three teams, but no team owns the outcome

• A model or prompt change in one agent silently breaks the next agent in the chain

When the debt hits

This hidden tech debt gets painful at specific trigger points.

At the exploration stage, there’s no debt. One engineer, one agent, it works. When a team starts using agents for real work, though, integrations and context are the first things that break. An agent accesses customer data that it shouldn’t see because nobody scoped the credentials. An agent guesses a service owner because it didn’t have that context.

When multiple teams run agents independently, the debt piles up faster. Agent registry, measurement, and human-in-the-loop all surface at once. At this stage, about 50% of a team’s capacity would go to building the surrounding infrastructure.

At production scale (agents embedded across most of your engineering org), governance and orchestration become the priority. Some companies see it coming. One architect told me, “We know from experience that chaos will exist. We will try to avoid it from day one.” Others learn the hard way: a VP of platform engineering saw teams independently building the same agents and had to retrofit governance once the sprawl was already in place. Both will end up building the same infrastructure. One will pay for it twice.

This happened before with microservices

This moment feels similar to the evolution of microservices. Each team chose its own technology, did its own infrastructure, and eventually someone had to come in and create standards. There is another platform engineering moment happening now with agents.

Platform engineering used to be a velocity initiative: self-service, reducing the ticket load, and scaffolding new services. With agents, it’s still about velocity, but the platform team is playing catch-up. Engineers aren’t waiting. They’re able to spin up new agents in Cursor or Claude Code whenever they need one. The platform team’s first job is to identify which agents already exist and get them under control. Only then can they do what they’ve always done: make it faster, safer, and easier for everyone else to create and use them.

One DevEx team described their new role to me: “We won’t be the ones designing and creating the agents. What we’re responsible for is making sure devs know how to use them, can interact appropriately, and helping more teams create their own agents.”

What to do about it

Start with visibility. Audit your GitHub org for AI-related workflows and actions. Check how many active API tokens your teams have on Claude, OpenAI, or Bedrock. Look at your workflow tools for anything with an AI node. The goal isn’t a perfect inventory. It’s the first count.

The harder question is agreeing on what counts as an agent. Is a GitHub Actions automation an agent? Is it a Claude Code scheduled task? Is it an n8n workflow with an AI node? You’ll need a working definition before you can catalog them. It doesn’t have to be perfect, but your org needs to agree on one.

There’s also the question of centralized vs democratized? Should the platform team build everything, and devs consume it? Or should the platform team provide guardrails while teams build their own? Both models exist. It probably depends on how much control your culture will tolerate.

You can build this infrastructure now, or you can build it after an agent leaks customer data, burns $300 in tokens overnight, or silently rolls back a production service nobody asked it to touch. You’ll build it either way. The only question is whether you build it before the pain or after.

But Backstage is dead. Not because it was bad. But because the world moved on.

Respect

Backstage did three things nobody else pulled off in 2020.

It’s flexible. You could mold it into whatever portal you wanted. Software catalog, scaffolder, TechDocs, custom plugins. The building blocks were there, and you decided what to build. For platform teams that wanted full control, nothing else came close.

It carries Spotify’s name. When a company running one of the most sophisticated microservice architectures on the planet says “this is what we built to tame our own complexity,” people listened. When the message comes from a practitioner and not a vendor selling a vision, engineers listen even more.

It built a real community. The CNCF incubation, 270+ adopters, 230+ plugins, thousands of contributors. Platform engineering went from a niche discipline to a Gartner-recognized category, and Backstage was at the center of that conversation.

None of what I’m about to say erases it.

What made it painful

Here’s what actually happened when teams tried to run Backstage in production.

You need to be a React developer.

Backstage is a React framework. But platform engineers write Terraform, Go, Python, Kubernetes manifests. Gartner flagged this explicitly, correctly stating that platform engineering leaders are backend people, and asking them to maintain a React frontend with abstractions like createPlugin, createRoutableExtension, and the @backstage/core-plugin-api package was a different job entirely.

One r/DevOps user nailed it: “The idea of backstage is super cool, but for me, as a DevOps engineer, the fact that I need to write a lot of React code instead of GitHub workflows and Terraform files made me leave the project.” (source)

You can’t bring your own data model.

Backstage ships five entity kinds: Component, API, System, Resource, Domain. If your org models data pipelines, ML models, feature flags, or infrastructure tenants, you’re either jamming them into Component with custom annotations or forking catalog internals to add a new kind, and maintaining that fork across every upgrade.

DevOps.com found that tuning the data model is so hard that “many adopters abandon hope and work with the default data model.” (source) That’s not a minor UX complaint. If your catalog doesn’t reflect reality, nobody trusts it, and if nobody trusts it, nobody opens it.

Plugin data stayed siloed.

This one’s technical but it’s the root of a lot of the frustration. Each Backstage plugin runs its own frontend and backend context with its own API clients and its own database tables. The Kubernetes plugin queries your clusters. The CI/CD plugin pulls from Jenkins. The PagerDuty plugin calls PagerDuty. None of it flows into a shared data layer.

So you can’t ask “which services owned by my team had incidents last week that correlated with deployments?”, because the incident data, the deployment data, and the ownership data live in three separate plugin boundaries with no schema connecting them.

The cost was brutal.

6-12 months to stand up a usable instance. Two to five full-time engineers for years, per Gartner. True cost of ownership around $150K per 20 developers.

Another r/DevOps member: “At scale, Backstage is not an ‘other duties as assigned’ sort of tool to own. It will require dedicated resources as it grows. That’s usually the biggest thing people overlook.” (source)

And adoption reflected all of it.

A ShiftMag case study: “Despite zero TypeScript and MERN experience we pulled off deploying Backstage in our platform, with much better user experience. Only 5% of our engineers use it.” (source)

Spotify reported 99% internal adoption. Their own VP of Engineering acknowledged external orgs average around 10%.

AI changed what you should expect from an IDP

OK, so Backstage is painful to get up and running (and adopted).

But pain alone doesn’t kill technology.

What kills it is when the world changes and the pain no longer buys you anything useful.

That’s what happened once AI was introduced to our life.

In 2020, building a portal UI was genuinely hard. Backstage gave you a head start on the UI.

In 2026, AI builds that UI in minutes (you don’t need Backstage for it).

It’s not about AI making Backstage easier. It makes Backstage unnecessary.

The UI isn’t where the value is anymore.

Think about where engineering is going.

We’re moving from manual to autonomous engineering.

Coding assistants were the first wave. What’s already here is agents that pick up Jira tickets, find the relevant repo in your catalog, pull recent commits, generate a fix, open a PR, and request review from the service owner, end to end.

But here’s what nobody talks about: the agent itself is a tiny piece of the system. The real infrastructure is everything around it.

Agents need a context lake, structured, real-time data about your engineering org served through a single endpoint, because an agent running on a stale .cursorrules file doesn’t know that service ownership transferred to a different team last Tuesday. They need to have access only to the data they are allowed. Not only for security and compliance reasons, you also want to minimize their context window to avoid hallucinations.

Engineering needs an agent registry, because when three teams independently build triage agents wired to different tools with different permissions, you don’t have agentic engineering, you have agent sprawl.

Agents need guardrails and human-in-the-loop checkpoints that are defined centrally, not hard-coded as if-statements in each agent’s repo.

Agents need governance, scoped credentials, audit trails, cost controls, because an agent stuck in a retry loop will burn tokens for hours until someone manually kills it or the monthly invoice lands.

Agents and Humans need orchestration, because most real workflows are a mix of agents, tools, and people. Coordinating these moving parts require both deterministic and non-deterministic workflows.

None of that is the agent. All of it is the platform underneath the agent. The infrastructure.

That’s what makes the IDP more important than ever.

It becomes the system of record and system of action for the whole engineering org, the layer that gives agents context, enforces policy, sets controls, and keeps humans in control.

But isn’t Backstage responding?

Yes. And what they’re shipping is actually smart.

v1.40 brought the Actions Registry, plugins can register typed, schema-validated actions with permission enforcement. @backstage/plugin-mcp-actions-backend exposes those as MCP tools, so agents in Cursor or Claude can discover and invoke them. The New Frontend System went “adoption-ready” in late 2025. A new design system is replacing Material UI. At KubeCon Atlanta, the maintainers titled their talk “AI-Native Backstage.”

They clearly see where things are heading. So what’s the problem?

Every one of these improvements sits on the same foundation: React frontend, plugin-based backend, YAML entities in a relational database. The Actions Registry exposes discrete operations, “register a component,” “trigger a scaffolder template.” An agent can call catalog:register-component, but it still can’t reason about your dependency graph, your production-readiness posture, or your org’s standards. It runs commands. It doesn’t understand context.

The MCP integration is “highly experimental” as of early 2026. The initial auth model used static tokens, no per-user identity. v1.43’s Dynamic Client Registration is still maturing.

But the real issue isn’t maturity. It’s architecture.

In Backstage, the UI sits at the center and everything else serves it. An agentic engineering platform puts the context lake at the center, flexible data model, relationship graph, policy enforcement, a human-to-agent collaboration experience, a skills and tools catalog, with the UI as one consumer alongside agent-native interfaces. Getting from one architecture to the other isn’t a version bump. It means decoupling the catalog from the rendering layer, building a real-time API, implementing MCP at the context level (not just action level), and treating agent identity as a first-class citizen.

That’s a re-architecture. Not a feature release. Backstage is adding the right capabilities at the edges. The center hasn’t moved.

What should Backstage do to stay relevant?

Three things.

Stop being a React framework. The world doesn’t need another way to render a service catalog page. AI builds UI faster than any plugin ecosystem can.

Rebuild the catalog as a context lake. The entity model and metadata graph are Backstage’s most valuable asset. Expose them through a real-time API that agents and humans both consume, with guardrails and audit trails built in. Make MCP a first-class interface, not an experimental plugin.

Evolve from portal to platform. The category has moved. What started as internal developer portals is becoming agentic engineering platforms, systems that orchestrate the entire SDLC, power autonomous ticket resolution and self-healing incidents, surface engineering intelligence, and support human-to-agent collaboration natively. Backstage serves humans through a UI. The next generation serves agents and humans through a shared infrastructure layer.

This isn’t a roadmap item. It’s a reinvention. And the window is closing.

The platform engineering market didn’t outgrow the idea of Backstage. It outgrew the architecture.

What Backstage pioneered, a single pane of glass for your engineering org, matters more now than it did in 2020. But the pane of glass needs to face two directions: toward the humans who make decisions, and toward the agents that execute them.

Backstage earned its place in the history of platform engineering. Whether it earns a place in its future depends on what it’s willing to let go of.

Port acts as the system of record and system of action for our customer’s engineering teams, for both humans and agents.

As a system of record, we know about their services, ownership data, deployment history, team structures, and more.

As a system of action, we know what actions engineers are triggering, what workflows are being run, and more.

Now that Port has an AI layer, engineers and agents are interacting with Port in a new way. Instead of looking at data in a dashboard, they ask for it in natural language. Instead of running an action manually, they ask AI to run it for them.

In the past 10 months, platform and software engineers at large engineering organizations have prompted it tens of thousands of times. It’s connected to their most critical systems and they ask complex operational questions and trigger meaningful actions.

So we decided to study exactly how they are using it.

What is AI in Port?

If you’re already familiar with Port’s AI capabilities, feel free to skip this section.

If not, here’s a short history of it:

• Port launched Context Lake, a way for AI agents to query your organizational knowledge and take action with the proper governance in place

• We then launched AI agent widgets, embeddable chat widgets that have the context of the portal

• After that, we released our remote MCP server, a general AI assistant, and a Slack app

• We also launched a few specialized AI features like integration helpers (to help bring your data into Port) and automatic data discovery (to help enrich your software catalog)

Now you’re up to speed on what AI capabilities Port has. (But by the time you read this, we probably have more)

What we wanted to know

We had two questions going in:

1. What are engineers trying to do with Port AI?

2. Do teams change how they use Port AI over time?

We classified each query by what the engineer was trying to do. Were they trying to understand their environment, manage infrastructure, fix a security issue, or something else entirely? Then we looked at how those patterns shifted over time.

Here are our findings:

Finding 1: 33% of prompts ask about scattered engineering knowledge

Questions like “Who owns the payment service?” and “Show me the list of my repositories” and “How do I deploy to production?”

In our research, we classified a prompt as context gathering when it asked for information that should already be accessible in Port: service ownership, team structures, deployment processes, documentation locations.

Let’s explore why this use case is so strong through an engineer’s first few months at a company:

Week 1 - Hello world

You’re given access to a bunch of tools at various stages of the SDLC you may or may not have used before. You’re assigned to a team and need to figure out what they are responsible for. You have an endless list of questions, but you don’t want to interrupt your teammates too much.

“What repos does my team own?”

“What services are in my team’s domain?”

“Where do I start?”

Weeks 2–4 - Hello unfamiliar deployment process

By now, you’ve gotten your first ticket. It’s probably a tiny feature or small bug fix. But it’s on a service you’ve never touched before. So now you have a whole new set of questions like “What does this service depend on?” “What’s the deploy process?”

The answers exist, but they’re spread across Confluence, Notion, Slack threads, and someone’s head. You might even get different answers from different teammates.

Month 2 - Hello real work

Congrats! You just got your first big ticket. But it touches a service your team doesn’t own. Now you’ve started to expand your scope and touch other people’s code, which means gathering brand new context. If you had 30 questions in week 3, now you have 300.

“Why was this built this way?”

“Is there anything I should know about before I change this?”

The truth is, this part never ends. Engineers are constantly gathering context, just in different parts of the codebase or SDLC. This is why we haven’t seen queries of this kind decline over time, even in the same company.

Every one of these questions is answerable by AI, as long as the context is there. And if you have an IDP, it already is.

Finding 2: User trust in AI-feature accuracy is built gradually

We looked at how queries evolved over time within individual organizations, and the same pattern appeared everywhere. Teams never started with complex questions. They start with the simplest thing they can think of and get more and more complex. Like this:

Month 1: “Show me my repositories.”

Month 2: “Who owns the auth service?”

Month 6: “What is the tech team ID for the Network Access team?”

Month 10: “How many deploys were performed last month on service X to prod?”

The AI could of course answer the month-10 question on day one (assuming the context is there). So it’s a matter of trust. The first question is a test. The engineer already knows the answer, or can easily verify it. If the AI gets it right, the next question is slightly harder. If that’s right too, they start relying on it for things they can’t easily check themselves.

We saw this in how teams accessed AI, too. Early on, most usage came through a chat widget. Someone types a question and reads the answer. A few months in, we saw more questions asked through the Slack app or Cursor via MCP. A few months after that, the most active teams were calling the AI API directly, embedding AI into their own internal tools and workflows.

Teams that tried to skip the chat phase and start with autonomous agents tended to get stuck. The teams that let engineers build trust at their own pace got to automation faster than the ones that tried to skip ahead.

Finding 3: Platform teams that continuously improve their software catalog see faster engineer adoption

We encourage our customers to take a product management mindset while they build in Port. After all, they're building a product for their engineers. Taking a product mindset means they're in a continuous feedback loop:

1. Ask engineers where they get stuck

2. Analyze their input

3. Decide which problems to solve

4. Build solutions for them

5. Deliver them to engineers

6. Go back to step 1

So we asked: Do teams that take this product mindset get better results from their AI?

I want to tell you about one customer that has more experience than anyone in this and the results they got.

At the beginning of their journey building a developer portal, they built a quick MVP, and told engineers they could start using it. Not surprisingly, most engineers didn’t use it. Their engineers had a certain muscle memory for their existing tools and couldn’t make the switch.

So the platform team building Port started over. They physically sat down next to the engineers and just watched them work. They realized that what their engineers wanted most was a lightning-quick way to see exactly what they needed from where they were already working.

So they rebuilt, adding more data sources, refining the data model, and creating more relevant self-service actions.

And as the catalog got richer, something clicked for engineers. They started using Port AI substantially more because the answers they got were substantially better.

In his words: “Agents magnify the best practices you have or the lack of practices you have.”

Best practices only come from iteration.

So that’s what we saw in our most successful teams. They didn’t build once and walk away. They ask what engineers need, build it, and then ask for feedback. And repeat.

Finding 4: Our leading customers are building agentic workflows

Most engineers interact with AI through a chat interface. Type a question, get an answer. That accounts for the majority of AI usage.But 12% of queries are for something else. These queries happen during workflows where the AI query happens as a result of some trigger (not a question from a person).

Here are some examples of what I mean:

Security remediation. One organization connected their security scanner to an AI agent. The scanner finds a vulnerability, then creates a Jira ticket. That ticket triggers the agent, which pulls the relevant code, analyzes the vulnerability, generates a fix, and opens a pull request. The security team used to write the fixes - now they review PRs.

Ticket triage. Another organization built an agent that receives every incoming Jira ticket, reads the description, classifies it by type and severity, assigns it to the right team, and sets priority. 36 automated runs in December alone. Before this, a human did that routing manually for every ticket. Now a human only gets involved when the ticket is already classified, prioritized, and sitting in the right team’s queue.

Catalog management. A customer built an event-driven agent that enriches ownership information on Port entities automatically when deployments, incidents, or new services are created. 72% of their total AI usage was machine-initiated.

Generally, we see two types of steps in workflows: steps that result in a text-based response (queries) and steps that result in an action being taken (actions).

Most steps today (73%) are query-based. An event fires, the AI gathers context, and posts a summary somewhere useful (RCAs, bug triage, service ownership). The remaining 27% take action - creating PRs, assigning tickets, or updating records.

I think we’re about to see this numbers flip. The vast majority of AI usage will be for taking action rather than creating a block of text.

What we took from this

Six months from now, the teams building agentic workflows won’t be the exception. Every team we studied started with simple questions and answers. But some iterated continuously on their portal and slowly trusted AI to do more than just answer questions. When they did, they unlocked the real shift: moving from manual engineering to autonomous engineering.

To see in more detail how leading teams are making this transition, have a look at our solution guides. They break down the exact patterns, architectures, and workflows behind becoming agentic in Port.

We were a new SaaS company asking engineers to trust us. And we had to earn it.

So we decided to run like an open source company without being one.

We open sourced our integrations framework (Ocean) so users could extend the product themselves. We put a live demo on the site with no gate because we trusted it to hold up. We built a Slack community instead of controlling the conversation. We launched a free tier so people could try the product without talking to anyone first.

And we put our roadmap on Canny, where anyone can see what we’re building, suggest features, upvote what matters to them, and tell us when we’re wrong.

We believe that when we build this product, we should build it with the people who use it.

What it does for our customers

Today, our public roadmap has some impressive stats.

• 1,250+ feature ideas

• 10,450 votes

• 1,726 comments

• 43,200 users

In a given month, we get about 70-90 new feature ideas, 400-500 votes, and 80-100 comments.

Numbers aside, we’ve found some incredible feature ideas in the roadmap. They’re not necessarily ideas we didn’t know about, but we didn’t realize how much our users wanted them until the upvote count started to rise.

For example, we just shipped our most requested feature “View as”. It completely changes the way our customers build in Port by letting builders see Port exactly how their users see it. It had over 150 votes.

Other requests like custom icons (123 votes) and dark mode (73 votes) seemed like small changes but it turned out people really cared about them and were thrilled when they were released.

Some customers even write requests that read like PRDs, with a target persona, the problem, and a proposed solution. The public nature of the roadmap pushes our customers to think a bit harder about how they express what they need, which helps us.

Instead of a request like ‘add SCIM,’ they explain their access control model and why it matters. That context is important for prioritization.

What it does for us

Publishing our plans creates accountability. When something goes on the roadmap, the timer starts. Our customers, the community, and yes, our competitors, they’re all watching. We find that pressure useful.

Since we use Port heavily ourselves, our own engineers often request features right alongside our customers.

It also forced us to get better at communicating timelines. When customers can see everything, you can’t be vague about what’s next or when it’s coming. High votes don’t guarantee a feature will ship, but they push us to explain why they won’t right now.

The community also surfaces things we may miss. We look at every item, even ones that seem minor. But if a lot of people vote for it, we pay more attention and start asking why. We know community requests can’t be the entire roadmap, but they’re a significant part of it.

Are we worried about copycats?

If other companies in our space see our plans and ship something similar, that’s fine. It moves the whole category forward. Our Director of Product, Dudi, puts it this way: “You can copy features but you can’t copy culture.”

Because Port is a platform, everything connects. For example, custom widgets are built on top of dashboards, which are built on top of the catalog, which ties into automations, scorecards, and everything else. So our IP isn’t a list of features. It’s how everything connects and the approach we took to solve the problem. If a competitor finds itself building their roadmap based on what we publish, their whole strategy is reactive.

We believe that when vendors hide the roadmap, the customer pays the price. They negotiate for information instead of building with the product team. We’d rather everyone in this space be more open about what they’re building. It’s better for the people who use the products.

We’d do it again

People ask us all the time why we opened up the roadmap. By now, it’s just how we work. It takes effort to manage and it requires honest communication about priorities. You also have to be willing to be held to what you say you’ll build.

But it got us to the product we have today which we’re very proud of.

You can see everything we’re working on at our public roadmap. Upvote something, suggest a feature, tell us we’re wrong. That’s what it’s there for.

HubSpot’s CEO told analysts that 97% of their committed code last year was done with AI assist.

Google’s CFO said about 50% of their code is written by AI agents.

Spotify’s co-CEO said their best developers haven’t written a single line of code since December.

These are all from earnings calls in the last four months.

I got curious about this, so I went through 63 companies’ recent earnings transcripts.

21 mentioned AI for internal engineering. And the pattern was the same every time:

These companies all ship software. They all track how often they deploy, how fast they resolve incidents, how their engineering org performs. And they can all tell you what percentage of their code is AI-generated. What they can’t tell you is whether the first number changed the second one.

Across all 63 companies, I found zero mentions of deployment frequency tied to AI.

Zero on MTTR.

Zero on lead time, change failure rate, or incident reduction connected to AI tooling.

The metrics DORA has tracked for a decade as the standard for engineering performance. They exist. The AI number exists. On earnings calls, the two just never appear in the same sentence.

Airbnb’s Brian Chesky said it himself on the call. He noted that getting to 100% AI tool adoption would be a “vanity metric.” The real question, he said, is whether the organization adapts fast enough to use those tools well.

I think he’s right. But even Airbnb reported the adoption number without connecting it to any engineering outcome.

Individual output is up. Organizational output isn’t.

Faros AI looked at telemetry from 10,000 developers across 1,255 engineering teams. Not surveys, actual data from source control, CI/CD, and task trackers. Individual output was up 21%. PRs merged up 98%. But PR review times grew 91%. PR sizes grew 154%.

Net organizational delivery velocity: flat.

Engineers are producing more code. Organizations aren’t shipping faster. The bottleneck moved from writing code to reviewing, testing, and integrating it.

The companies that could connect AI to outcomes had one thing in common

Some companies did give outcome numbers on earnings calls. Pfizer: $600 million saved through Golden Batch AI in manufacturing. Amazon: Q Developer migrated 30,000 Java applications, saving $260 million a year, with 79% of auto-generated code shipped without human changes.

Those companies could draw the line between AI and outcomes because they were already tracking the specific metric AI was going to change. Pfizer had years of batch data (yield, cycle time, cost-per-unit) for the exact process Golden Batch would optimize. Amazon had a clear before (50 developer-days per Java migration) and a clear after (a few hours).

For engineering, this is genuinely harder. Manufacturing batches are comparable by design: same inputs, same process, same expected output. Engineering work isn’t. Every feature has a different scope. Every sprint has different complexity. If your deployment frequency went up 30% after rolling out AI tools, it could be the AI. It could also be that you shipped a simpler roadmap that quarter, or hired three seniors, or finally fixed the CI pipeline that was slowing everyone down. Isolating what AI actually changed from everything else that changed at the same time is a real problem, and most engineering orgs haven’t solved it.

That’s why the claims that do exist tend to fall apart. IBM claimed 45% productivity gains from Project Bob across 20,000 developers. When you look at their own published research, the data was self-reported. Not what people shipped. What they said it felt like.

Spotify said their best engineers stopped writing code. That tells you how the workflow changed. It doesn’t tell you whether Spotify ships more reliably or resolves incidents faster.

Don’t just report AI adoption. Connect it to something.

The code percentages are going to keep climbing. They’ll reach 80, 90% at most companies within a year or two. At that point the number alone stops being interesting, the same way “we use cloud” did a decade ago.

The company that connects the AI number to an engineering outcome — “50% of our code is AI-generated, and our deployment frequency increased 30% over the same period” — will stand out from the nine companies that can only report the first half.

For engineering broadly, that connection still hasn’t been made on any earnings call we looked at. The engineering org that makes it first will be the one everyone else gets compared to.

What you can check this week

If your CEO gets asked about engineering AI on the next earnings call, could your team connect the AI adoption number to an engineering outcome?

Three things worth looking at:

Do you have a before number for the workflows AI is supposed to improve? If you deployed AI coding tools six months ago and didn’t snapshot your deployment frequency, lead time, or change failure rate at that point, it’s going to be hard to show what changed. If you haven’t captured that baseline yet and still can, do it now.

Is your dashboard connecting AI to engineering outcomes, or just tracking AI usage? If the headline metric is Copilot adoption rate or tokens generated, that’s the same kind of number nine other companies already reported this quarter. The numbers that held up on earnings calls were resolution time, cost per unit, hours saved on a named workflow. Worth checking whether yours ties any of those to AI.

Can you name one workflow with a before and after? Say you rolled out AI coding tools in October. Your average PR cycle time before was 4 days. Now it’s 2.5. That’s one workflow, one before, one after. If you can draw that line for even one engineering process, you’ve made a connection most companies in this dataset haven’t.

The 2025 DORA report found that AI amplifies whatever is already there. Teams with strong engineering platforms and observability get real gains. Teams without them get more instability. The foundation matters more than the tooling.

That’s what we’re building at Port. A platform that gives engineering teams the catalog, the baselines, and the visibility to connect AI adoption to what actually changed.

That’s what one of our customers told us recently and he’s a platform engineer at a Fortune 500 finance company. And the practice that trips up the most teams is how they define a “service” in their software catalog.

Is it a repo? A Kubernetes deployment? A business capability like “payments” or “checkout”? A Lambda function?

It may not seem like an issue, but once a company grows past a few teams, that ambiguity creates friction you can feel but can’t easily point to - like an incident where you don’t know who to page.

So if you don’t have an agreed-upon definition, it’s probably one of the most impactful things you can align on.

Here’s how we think about it:

How we define “service” at Port

The definition we’ve agreed on internally: a service is a deployable unit owned by a single team that exposes a stable interface to other systems.

Owned by a single team is clear, but what are deployable units and stable interfaces?

Deployable unit means that your team can change it, release it, and roll it back without requiring other teams to deploy their services at the same time.

Stable interface means that the way other systems interact with the service (API, events, etc.) changes slowly and backward-compatibly so consumers don’t break when the service evolves.

Quick litmus test: is it a service?

If you can answer yes to all of these, it’s probably a service:

• Can you deploy it independently, without coordinating with another team?

• If it goes down at 2am, is there one clear owner to page?

• Can you roll it back without rolling back something else?

• Does it have its own CI/CD pipeline (or a distinct stage in a shared one)?

And a few signs it’s not a service:

• A shared library that other teams import (that’s a dependency, not a service)

• A repo that contains three things deployed on different schedules (that’s three services, not one)

• A Helm chart that bundles unrelated containers (that’s packaging, not a service)

There’s no true universal answer here and some companies might have multiple definitions of service internally (which is fine). The goal is getting to agreement in your engineering department so that your catalog, your on-call, and your scorecards all reference the same thing.

Best practices: what a service should look like in your catalog

Once you've agreed on the definition, the next question is: what are the most important properties that a service should have, and what should it connect to?

• Owning team: just one team (so you know who to turn to when something needs fixing)

• Lifecycle stage: production, staging, deprecated, experimental (so humans and agents know what’s active and what to leave alone)

• Tier/criticality: P0 revenue-critical vs. P3 internal tooling (drives SLO targets and incident response)

• Language/framework: useful for scorecards and security scanning

• Deploy target: where it runs (EKS cluster, Lambda, Cloud Run, etc.)

• Repo: link to source (but remember: the repo is not the service)

• Communication channel: link to Slack or Teams channel (so you or an agent knows where to page the owning team)

On its own, a service isn't worth that much. It needs to be connected to other things to be truly valuable.

• Depends on: what other services does this call at runtime?

• Consumed by: what other services rely on this service?

• Owned by: Both a property and a connection to the owning team

The connections matter more than the properties. Without dependencies, you can't calculate blast radius. Without "Owned by" linked to a team with on-call data, incident routing breaks.

Defining a service matters more now than ever

Picture a P1 incident at 2am. A latency spike is hitting checkout. With a clean service definition, your on-call engineer opens the catalog, sees checkout-service, sees it depends on payment-gateway and cart-service, sees the owner is Team Payments, and starts debugging. The blast radius to other services is clear. The escalation path to senior engineers is clear. Time to action would be just minutes.

How to get started

1. Test if your services are services. Run each service through the litmus tests above. If you can deploy it independently and it’s owned by one team, it’s a service. If not, it may be a dependency, a sub-component, or multiple services bundled together.

2. Audit them against reality. Pull up a few recent incidents and compare what broke against what your catalog says. If the incident damaged three things the catalog calls one service, your definitions might be off.

3. Build the schema. Decide which core properties and relations are important to connect to your services. We recommend starting with owner, lifecycle, tier/criticality, and dependencies.

4. Make it the contract, company-wide. Once the definition and schema are agreed on, treat them like infrastructure. New services must be created from the template. Existing ones get migrated. If there are exceptions, they should get documented, not ignored.

You don't need a definition that covers every edge case. You need a shared one that's precise enough for both humans and agents to rely on.

We went through the Slack threads afterward and mapped out what happened. Then we got together as a team and asked: what would this have looked like if an agent handled the incident end-to-end?

This article is that exercise. The real incident, what went wrong in how we responded, and what we’re building to make sure it goes differently next time.

The incident

On a Friday evening, a customer created 1.7 million automation runs in 90 minutes. Kafka offset lag spiked across multiple services.

PagerDuty fired alerts for three of our teams: Actions, Spotlight, and Builder. Three on-call engineers acknowledged in three separate Slack channels without knowing about each other.

The Actions team found the offset lag first. Their on-call engineer restarted the service worker through a self-service action in Port. It didn’t help. So he then dug into our automation run counts, narrowed the spike to three orgs, traced them all back to one customer. After that, he tried to scale out, but the config override action didn’t exist in Port for this service. So he opened a manual PR, waited for DevOps approval, and hit a failing Argo diff that needed a third person.

Another team called Builder also got a PagerDuty alert at the same time, but started talking about it in a different Slack thread. Their engineer also restarted workers. Also checked New Relic. Also checked Kafka UI. And got the same false hope: “lag seems to decrease as restart rolls out.” It didn’t.

43 minutes in, someone from a 3rd team Spotlight, while dealing with the same incident on their own, finally asked: “do you think our alert is related?” That was the first time anyone connected the dots. Three teams had been investigating the same root cause in parallel, trying the same restart that didn’t work, wasting the same ten minutes twice.

By the time the threads merged, 77 minutes had passed. The remediation plan was two words: scale out. Getting there took over an hour of human coordination across three Slack threads, New Relic, Kafka UI, PagerDuty, GitHub, and Argo.

But we’re working on something big to resolve incidents in record time.

Our first step: A triage agent

We’re building toward an agent that handles the full incident lifecycle. But the first piece is already live: an agent that triages the incident in Port and posts a message to on-call engineers in Slack within minutes.

Two things make it work:

The Context Lake
The Port Context Lake gives the agent everything it needs to know about how our SDLC behaves. It holds everything about our micro-services, deployment history, CI/CD jobs, ownership, dependencies, vulnerabilities, team structure, and much more. When the agent finds a failing service, it already knows who owns it, what recently deployed, what other services depend on it, and which customers are affected because everything in Port is connected.

During the Kafka incident, an engineer spent time tracing three org IDs back to one customer. The context lake already has those relationships. The agent would have just read them instead of piecing them together between different tools.

Equipped with those two things, here's what the triage agent does:

Why Port

The agent does the investigation. But when it's time to act, like restarting a service, scaling out workers, or rolling back a deployment, a human approves it through Port. Here's why that matters.

• how much time the agent saved per incident

• how often engineers accepted the recommendation vs. overrode them

• the sentiment from engineers on how the agent performed

Put these on a dashboard and then you'll know exactly what autonomous incident resolution is worth and where it still needs work.

What’s next

The triage agent is just the first step. We're building Port to heal incidents autonomously. So let's revisit the Kafka incident and see how it would have gone differently.

The incident management future we see

Incident management is moving from reactive to autonomous. We already know agents can do it. The question is how you get there without losing control.

The companies moving fastest are building agents that pursue multiple hypotheses in parallel, correlate across systems, and suggest remediations. They’re participants in the on-call rotation that work across your code, infra, and telemetry.

We think the foundation for that is context. An agent that doesn’t know your architecture, your ownership model, your dependencies, and your playbooks can just be a faster way to get the wrong answer. The context lake and the skill file are what turn a generic agent into one that investigates the way your team does.

If you want to build something similar to what we described, we’re publishing the skill file, the workflow definition, and a guide for how we do incident management at Port.