You Don't Need a New Methodology—You Need Better Decision Latency

Why does my project keep failing despite using Agile?

You've read the manifestos, you've tried Scrum, you've even attempted Kanban, and yet your projects still blow up. You're not alone. The Standish Group CHAOS Report, tracking over 50,000 projects, finds only 35% succeed—46% are challenged, and 19% fail outright. And here's the kicker: the CHAOS 2020 data shows Agile projects succeed at 42%, while waterfall projects succeed at just 13%. So Agile is better, but it's not a silver bullet. The truth is, the methodology you pick matters far less than how fast you can make decisions. That's the real research method you need to master.

The thesis: decision latency beats methodology

Let's cut to the chase. The biggest success factor isn't whether you do Scrum or Kanban or XP—it's decision latency. The CHAOS 2020 report identified decision latency as the top factor: 75% of projects with good decision latency succeed, versus only 21% with bad latency. That's a massive gap, bigger than the Agile-vs-waterfall gap (42% vs. 13%). So if you want to improve your project outcomes, stop obsessing over which framework to adopt and start shortening the time between identifying a problem and making a call.

But don't just take my word for it. The 17th State of Agile Report from Digital.ai (2024) found that 71% of organizations use Agile, yet only 11% are 'very satisfied' with it. Why? Because they're doing Agile rituals without the underlying speed. They hold daily stand-ups, but decisions still get stuck in committees. They do sprints, but the product owner is a committee, which violates the Scrum Guide's rule that the Product Owner is one person, not a committee. The framework isn't the bottleneck—your decision-making process is.

What the evidence actually says: Agile, Scrum, and Kanban in practice

Let's dig into the options. Waterfall, as defined by Atlassian, is a sequential, linear process with phases: requirements, design, implementation, testing, deployment, maintenance. Each phase must finish before the next begins. Sounds tidy, but it's a trap. Winston Royce's 1970 paper, which is universally cited as the source of waterfall, actually argued against the one-way sequential model—he said doing everything in a single sequence is unrealistic and recommended building a prototype first. So even the 'father' of waterfall didn't endorse it.

Agile, on the other hand, is an iterative, team-based approach that emphasizes flexibility and customer collaboration (Atlassian). It's not a single prescriptive process but a group of methodologies focused on tight feedback cycles. Scrum, the most popular flavor, organizes work into sprints of 2 to 4 weeks, with roles like Product Owner, Scrum Master, and Developers, and a team of typically 10 or fewer people (Scrum Guide 2020). Kanban visualizes workflow on a board and limits work-in-progress to expose bottlenecks (Atlassian).

Here's a quick comparison to help you decide:

But even within Agile, you have to pick practices wisely. User stories should be INVEST—Independent, Negotiable, Valuable, Estimable, Small, Testable (Agile Alliance). If a story is too big, split it; no individual task should exceed 16 hours (Atlassian). And for estimation, use story points based on complexity, not hours (Atlassian). These practices help, but they're secondary to decision speed.

Addressing the counterargument: 'But we need structure'

'Hold on,' you might say, 'my team doesn't have good decision-making culture. We need a framework to impose discipline.' That's a fair point, but it's actually a symptom of the problem, not a solution. The Scrum Guide (2020) says the framework is 'purposefully incomplete'—it only defines the parts required to implement Scrum theory, leaving room for teams to adapt. If you use Scrum as a crutch, you'll end up like the 83% of organizations that say they practice Agile but aren't high-Agility (Standish Group). The 17th State of Agile found that 34% of respondents either create their own framework or don't follow a mandated one—so flexibility is already common.

Sure, structure helps, but the best structure is one that forces fast decisions. For example, Scrum's Daily Scrum is a 15-minute event for developers to inspect progress and adapt (Scrum Guide). If you use that time to report status instead of making decisions, you're wasting it. Similarly, the Sprint Retrospective is meant to plan ways to increase quality and effectiveness—if you skip it or let it drag, you lose the feedback loop.

Another common objection: 'We need to scale Agile to the enterprise.' But scaling frameworks like SAFe, LeSS, or Scrum@Scale won't fix slow decisions. In fact, the 17th State of Agile found that organizational resistance to change and insufficient leadership understanding are the top barriers to scaling Agile (Digital.ai). That's not a methodology problem—it's a decision-making problem. You could adopt the most rigorous framework in the world, but if leaders can't make timely choices, you'll still fail.

What to do instead: Focus on decision latency

So here's my blunt advice. Stop switching methodologies. Pick one that fits your context—Scrum for feature work, Kanban for support, or a hybrid if that's what you need (Atlassian acknowledges hybrid approaches are common). Then pour your energy into shortening decision latency.

• Identify where decisions get stuck: approval chains, unclear ownership, or waiting for data. The most common culprit is the Product Owner being a committee—make it one person (Scrum Guide).
• Set a timebox for decisions: e.g., 'any blocking decision must be made within 24 hours.' That's not in the fact base, but it's a practice that aligns with the principle of adaptation.
• Use metrics like cycle time to spot bottlenecks: shorter cycle times indicate higher throughput (Atlassian). If your cycle time is high, it's a sign of decision latency.

Here's a concrete example: Imagine a team using Scrum with two-week sprints. They have a Product Owner who is a committee of three, so every backlog item requires a meeting to approve. That meeting takes three days. Meanwhile, the team is blocked. If you switch to a single Product Owner, you cut that latency to zero. That single change could move your project from the 21% success zone to the 75% zone—just by making a decision faster.

And don't forget the human side. The Agile Manifesto values 'individuals and interactions over processes and tools.' That means empowering people to make decisions without waiting for permission. The 12 principles say, 'The best architectures, requirements, and designs emerge from self-organizing teams' (Agile Alliance). Self-organizing teams are only possible if they can make decisions autonomously.

Bottom line

Your single best move is to audit your decision latency and fix the bottlenecks. Methodology matters, but not as much as you think. Agile beats waterfall 42% to 13%, but good decision latency beats bad latency 75% to 21%. So stop tweaking your Scrum board and start making faster calls. That's the research method that will actually save your projects.

Sources

• Standish Group CHAOS Report - https://www.standishgroup.com/
• Agile Manifesto - https://agilemanifesto.org/
• Scrum Guide 2020 - https://scrumguides.org/scrum-guide.html
• Atlassian (Agile) - https://www.atlassian.com/agile
• Digital.ai (17th State of Agile) - https://digital.ai/press-releases/17th-state-of-agile-report-71-use-agile-in-their-sdlc-small-organizations-report-strong-business-benefits-medium-and-larger-sized-companies-continue-to-experience-barriers-in-successfully-scaling-a/