How The Citation Group Measures Engineering ROI with Devin

About The Citation Group

The Citation Group is the leading provider of compliance, risk management, and certification technology and services, supporting over 125,000 small and medium-sized businesses across the UK, Canada, Australia, and New Zealand.
Backed by private equity and expanded through acquisition, the company faced the challenge of scaling engineering output while managing legacy systems, distributed teams, and accumulated tech debt.

The Challenge: Establishing Trust in AI-Generated Code

Citation’s engineering leaders needed a reliable way to evaluate whether Devin was contributing production-quality work. Traditional metrics, such as sprint velocity, varied too much to isolate Devin’s impact. With distributed teams and partner developers, the concern was that AI-generated changes might not meet the same standards as human engineers, potentially adding technical debt instead of reducing it.

To validate quality, the team initially onboarded Devin to a set of projects and measured quality at the pull-request level. In the first three months, 271 Devin PRs were merged (at an 80% success rate) after senior engineer review.

“By tracking Devin’s pull requests directly, we finally had a clean, production-level signal of impact.” — Anthony Wray, AI Engineering Lead

Structuring Work for Devin

Citation first onboarded Devin through a series of hackathons run by internal and partner teams. Each team designated a lead to document results and share practices. In these early projects, unstructured prompts often produced inconsistent results, but tasks scoped in Jira with clear requirements — and supplemented by documentation pulled directly into Devin sessions — generated PRs that passed review.

Three practices proved critical:

• Jira ticket scoping to define tasks with clear acceptance criteria.
• Devin search-to-session to pull documentation and architectural context directly into a Devin coding session.
• Markdown specification files to provide structured inputs and reduce ambiguity.

“Once we paired Devin with structured specs, the results became consistent. We built a rinse-and-repeat pattern that engineers could trust,” Wray explained.

Use Cases in Practice

Once Devin’s workflow was established, Citation applied it across a range of projects:

• Legacy modernization: A compliance tool originally estimated as a three-month migration (from .NET Framework and AngularJS to .NET Core and React 18) reached a working prototype in two weeks. Devin decomposed the monolith into a clean architecture and delivered vertical slices end to end, from user interface through database entries, with static analysis and automated tests driving coverage above 90%.
• Backlog throughput: Medium-priority tasks such as dependency upgrades and bug fixes that previously slipped sprint to sprint were consistently completed. In one corrective action feature, Devin contributed 147 merged pull requests, about 367 story points of work. This output was comparable to a multi-sprint epic delivered in weeks rather than months.
• Debugging by non-engineers: Business Analysts used Devin to explain unexpected system behavior by querying the codebase directly. In one case, a BA spotted a subtle prefix mismatch, submitted a PR, and had it approved by a senior engineer, preventing a customer-facing bug.
• Customer support: Devin automated root-cause analysis of help desk tickets, reducing the number of issues escalated to engineering.
• Rapid prototyping: Engineers used Devin to build working proofs of concept in a few days. These included extending existing tools and testing new features that would normally wait weeks while teams focused on core delivery.

Scaling Adoption

Devin is now part of daily engineering practice at Citation. Engineers run more than 180 sessions each week, with over 1,200 sessions recorded in GitHub since rollout. What started as using Devin in scoped hackathon projects has scaled into steady, ongoing use in production across the engineering organization, with adoption also extending to business analysts and support teams.

“Devin isn’t just another tool. To get value, you have to change your ways of working. Once we did that, it started delivering results we wouldn’t have reached otherwise.” — Anthony Wray, AI Engineering Lead

Looking Ahead

Citation plans to expand Devin’s role in platform modernization and legacy upgrades over the next twelve months. The team is also working on deeper process integration, including automated Jira scoping and standardized specification files to make AI-driven development more predictable.

For Anthony Wray, the significance goes beyond throughput.

“The real value is that it forces us to rethink process. You can’t just slap AI on the old way of working. You have to redesign for what’s possible now.” — Anthony Wray, AI Engineering Lead

The Citation Group at a Glance