How Modal uses Devin to investigate 80% of its incidents before an engineer opens the thread

Modal builds the distributed systems, scheduling, and infrastructure that run other companies' AI workloads. That complexity made Modal a later adopter of AI coding tools than most companies its size, because the tools on the market were built for web application code. Modal engineers also need to understand exactly how the platform behaves: when a distributed system fails, someone has to work out the races and edge cases behind it.

Modal started using Devin day to day about six months ago, once it was good enough for systems work. Devin now works across engineering, support, and go-to-market.

Triaging every alert before an engineer

Modal's first real use case was incident response. When an alert lands in Slack, Devin picks it up, judges whether it looks high severity, and investigates before anyone opens the thread.

Because Devin runs in the cloud, Modal's alerting notifies it directly instead of waiting for an engineer to start a session. Devin queries Datadog and other monitoring platforms over MCP, pulls metrics across systems, and forms hypotheses against the codebase.

On roughly 80% of incidents, Devin produces something useful: sometimes the exact cause, other times hypotheses the on-call engineer refines with follow-up questions. The full session log sits in the Slack thread, so engineers can see every step and push back.

Cutting hours of context gathering from support

Matthew Sweeney, Modal's Head of Support Engineering, delegates investigations of customer problems to Devin.

"The combination of Datadog MCP + code tracing + user steering is very effective for understanding what's happened, what should have happened, and if there's an issue. It saves me multiple hours of context gathering every day."

— Matthew Sweeney, Head of Support Engineering

Hardening CI to keep the merge queue moving

Flaky tests and environmental failures were slowing Modal's CI pipelines. Modal merges through a queue, so a single flaky failure can stall every change behind it.

Tom Elliott, a member of technical staff at Modal, set up a daily Devin session that reviews CI failures and opens draft PRs to harden the tests behind them. Devin reproduces the failure on its own machine, changes the test, reruns the suite, and opens a PR. Modal's target is keeping queue lag under 20 minutes, and success against that target has risen from 80% to 95%.

Keeping customer-facing docs up to date

Modal's documentation contains hundreds of runnable code examples, and they are an important way developers learn Modal: someone can copy one and have it working immediately. Modal continuously reruns every example to catch breaks as its SDK and upstream libraries change. Fixing those breaks used to land on DevRel and cost the team roughly half a day a week.

A broken example now starts a Devin session automatically. Devin runs the example on its own machine, sees the error, fixes it, confirms the example works, and opens a PR.

Finding niche bugs in review

The more code a team generates, the more review becomes the constraint. Devin Review is the only AI reviewer on Modal's pull requests. Modal previously ran another AI review bot alongside it and turned it off because engineers found Devin's reviews more useful. Devin is particularly good at niche bugs, including ones that surface several comments deep into a pull request discussion.

Answering technical questions and shipping go-to-market code

Chris Prinz, a GTM engineer at Modal, first used Devin to answer technical questions from new sales opportunities using Modal's own code as context. Solutions engineering does the same, particularly for new hires who are still learning the codebase.

Prinz and the rest of the marketing team also ship with Devin, using it to update Modal's landing pages and in-app upsell workflows so go-to-market changes clear without waiting on engineering. Marketing updates Modal's conference website the same way. GTM engineering has also connected Devin to Modal's CRM over MCP for operational questions, such as who should be invited to a dinner next month.

Automating more of the on-call path

Modal is pushing further into automated SRE work. Today Devin investigates an alert and people read the thread to see whether the investigation holds up. The plan is to route the noisiest alerts through Devin first, letting it identify false positives and page a human only when needed. More automation for go-to-market and support is next.