Every time I ask for feedback, I catch myself waiting. Waiting until I have a clearer picture. Until the thinking is more developed. Until the idea is polished enough to be worth sharing.

A recurring idea has emerged from these conversations: we’re running strategy at the speed of bug fixes now. The things that used to take months (shaping a direction, socialising it, building alignment) are now happening in days. Sometimes hours. The AI transition has compressed the planning horizon in ways we haven’t fully reckoned with. And waiting until thinking is polished before sharing it doesn’t just slow you down, it makes the thinking obsolete by the time it circulates.

So I’ve been trying something different. Sharing the half-formed things. Asking for feedback before highly confident in the answers. Running small experiments to see what breaks before committing to a direction.

It’s scary. Not terrifying, more like the uncomfortable but clarifying kind of scary. And the more I’ve sat with it, the more I’ve come to believe that the transparency piece isn’t just a personal preference or a culture aspiration. It’s a structural necessity for the AI transition. It really feels like with AI as a multiplier, the safety necessary here is also a multiplier.

The Foundation

Back in the late 00s, Google ran a study that became known as Project Aristotle. They set out to understand what made teams effective. They studied group norms, team composition, individual personalities, everything they could measure. The finding that emerged, again and again, was psychological safety: the shared belief that the team is safe for interpersonal risk-taking. Speaking up with a half-formed idea. Admitting you don’t know. Challenging a decision without fear of being punished for it.

Safety wasn’t just one factor among many. It was the multiplier. Teams with high psychological safety consistently outperformed teams with lower safety, even when controlling for everything else.

AI is a multiplier too. We’ve talked a lot about AI as a force multiplier for individual productivity: 2x, 5x, whatever the numbers are this quarter. But the multiplier framing cuts both ways.

When psychological safety is low and you introduce AI, you don’t get “low safety + AI = normal outcomes.” You get compounding failure modes. AI amplifies the dynamics that are already there. In an environment where people don’t speak up, where mistakes are punished, where challenging ideas feels risky, AI makes those environments faster and worse. The surface area for errors increases. The velocity of bad decisions accelerates. The institutional memory of what went wrong gets overwritten before anyone can learn from it.

When psychological safety is high and you introduce AI, you get the inverse. People share the half-formed things. Experiments generate evidence. Feedback loops tighten. AI accelerates the good dynamics instead of the bad ones.

This is the metaphor worth sitting with: AI × Psychological Safety. Or something like that. The math isn’t the point. The point is that the base matters more now, because whatever’s there gets amplified. Invest in safety, and AI works for you. Don’t, and it works against you.

The Information Architecture Problem

If thinking in the open is the answer, what’s the system for it?

Here’s where it gets practical. Open thinking without structure is noise. You get half-baked ideas scattered across Slack threads that nobody can find later. You get the same conversations happening in seventeen different places. You get the information equivalent of a codebase with no architecture, technically possible to navigate, but expensive and error-prone.

The challenge is audience sizing: what’s the right audience for thinking at different stages?

Early-stage thinking, the genuinely half-formed stuff, needs a small, trusted circle. Not because it’s secret, but because the signal-to-noise ratio is low. You’re exploring, not presenting. The feedback you need at this stage is “does this direction have legs?” not “here’s a detailed critique of your assumptions.”

Mid-stage thinking, where you’ve run a small experiment and have evidence, opens up. You can bring in a broader set of peers. The signal is stronger because you’ve generated some. This is where the thinking starts to become useful to others, not just yourself.

Late-stage thinking, validated directions, things you’re ready to act on, belongs in the open, broadly. This is where org-wide transparency pays off. When people can see the evidence behind decisions, they can contribute context, flag risks, and align their own work.

The failure mode at most organizations is skipping stages. Putting early-stage thinking in broad forums, where the lack of evidence makes it look undercooked. Or keeping mid-stage thinking in small circles when the experiment’s results could inform other teams. The result is either noise or missed opportunity.

The other failure mode is feedback loops that are too slow. Thinking in the open only works if the people you’re sharing with can engage fast enough to matter. If it takes three weeks to get a response, the landscape has shifted. This is where the speed problem I mentioned earlier bites hardest. The AI transition does not just compress the time to produce ideas, it compresses the time to validate them. Organizations that can give fast, high-quality feedback on open thinking will outlearn those that can’t.

Conway’s Law Has Something to Say

We talk about Conway’s Law mostly in terms of architecture: you ship the org structure into your systems. Teams that don’t communicate produce systems that don’t integrate. The teams that do communicate produce coherent architectures.

But Conway’s Law applies to information flow too. The communication structures of an organization shape what information can flow through it. Team boundaries, ownership models, reporting lines, these are all filters. They determine what gets shared, with whom, at what latency, with what fidelity.

If you want thinking to happen in the open, the org architecture has to support it. Not just tolerate it, actively enable it. That means:

• Boundaries that allow cross-pollination: Teams that only communicate through official channels have information that moves at the speed of process, not conversation.

• Ownership that doesn’t mean isolation: “Owning” a service or domain shouldn’t mean “you’re the only one who can have opinions about it.”

• Leadership that models it: If the senior folks polish before presenting, they’re signaling that early-stage thinking isn’t welcome. The culture follows the modeling.

This is the architectural work that can’t be skipped. You can talk about psychological safety all day. You can encourage thinking in the open. But if the structure around it constrains the flow, the culture won’t overcome it. You have to design for it.

The Multiplier Math

Let’s pull the threads together.

Outcomes = Psychological Safety × Information Architecture x AI Multiplier

When the base factors are strong, the exponent works for you. AI amplifies good judgment, tight feedback loops, and evidence-based iteration. When the base factors are weak, AI amplifies silence, slow feedback, and expensive mistakes.

The implication isn’t just “build a safe culture.” It’s “the investment in safety is now more valuable than it was before, because the multiplier is higher.” This isn’t soft stuff to optimize when you have time. This is infrastructure for the AI transition.

The same is true for information architecture. The cost of bad information flow (fragmented knowledge, slow feedback, duplicated conversations) is amplified when AI is in the loop. AI tools work better when they’re working with coherent, accessible information. They’re worse when they’re working with tribal knowledge, scattered context, and undocumented decisions.

The Ask

This is a moment for leaders to build the base factors, not just adopt the multiplier.

Safety without structure is vibes. Structure without safety is bureaucracy. You need both.

And you need both now. The organizations that figure out how to do thinking in the open (transparent, experimental, fast-feedback) will outlearn those that don’t. AI makes that possible in a way it wasn’t before. The cost of producing and sharing ideas has dropped. The question is whether the org architecture can absorb them at the same rate.

We have a chance to build organizations where AI compounds good work. Where the multiplier works for us instead of against us. Where the speed of strategy matches the speed of bug fixes, and we still get it right.

The scary part (thinking out loud, sharing the half-formed things, asking for feedback before you’ve earned the right to have answers) is also the way in.

We can do hard things. Let’s not skip the hard thing because it’s hard.

What does thinking in the open look like in your organization? What are the structures that enable it, or constrain it?

Related reading:

• Project Aristotle and psychological safety - Google’s research on what makes teams effective

• An Elegant Puzzle by Will Larson - on org design for enablement

• Conway’s Law - on the relationship between org structure and system design

I recently gave a talk framing the AI transition as the industrialization of software engineering. The parallel held—the audience nodded along as I connected AI-assisted engineering to the cloud transition as an earlier shift in the economics of production.

But something nagged at me afterward. The cloud transition changed where and how we ran software, but it didn’t change the fundamental nature of the thing being produced. It was a change in the plumbing, not the water.

Then, last week, I found myself in yet another conversation with a platform lead about “optimizing code review.” They were looking for the right combination of assignment logic, reminder bots, and “reviewer fatigue” metrics to make their process “work really well.”

As we talked, I realized they were trying to win an unwinnable race. They were trying to apply human-rate friction to machine-rate volume.

That’s when it clicked. The cloud wasn’t the right parallel. The data wave was.

Cloud, mobile, and big data arrived together and broke our existing data infrastructure. The “4 Vs of Data” (Volume, Velocity, Variety, Veracity) gave us a language for why. We aren’t in the first industrialization of software engineering. We’re in the second. The data wave was the first.

And just like before, the axis scales of what we produce are shifting, and that shift is what breaks the infrastructure.

The “Solved” Fallacy

You’ve probably seen the headlines: “Coding is solved.” Boris Cherny, head of Claude Code, recently said exactly that. The token sellers love this framing because it sells, well, tokens. It’s the Hadoop moment all over again.

In 2012, people claimed data was “solved” because we had Hadoop. But Hadoop only solved the obvious first problem: storing and processing at scale. It didn’t solve data quality, lineage, or governance. In fact, Hadoop behaved more like a problem scaler than a creator. These issues existed before, but the presence of those problems was scaled out with Hadoop to a point where they could no longer be ignored.

AI producing code is the Hadoop equivalent. It’s the enabling condition, not the solution. The forces that AI-rate code production creates are the actual problem. And if we want to survive this wave without drowning in a swamp of “good enough” boilerplate, we need to understand the 4 Vs of Code.

The Four Forces

The same forces that broke data infrastructure are now breaking code infrastructure.

1. Volume: The Infinite Monkeys. We’re moving from a world where code was a scarce resource written by expensive humans to one where it’s a commodity. Engineers using AI effectively are reporting 2-5x productivity uplifts. That’s not 2-5x more typing; it’s 2-5x more functioning software moving through our systems. More branches, more PRs, more surface area to understand and maintain.

2. Velocity: Continuous Generation. The gap between idea and running code is shrinking. We’re moving past Continuous Integration into Continuous Generation. Cycle times are compressing, and the rhythm of engineering work is accelerating past the point where a human can “keep up.”

3. Variety: Polyglot Sprawl and Cheap Retries. When you can produce code quickly in unfamiliar territory, you stop routing around your skill gaps. This leads to “variety”—engineers working across more languages and frameworks than they’d ever attempt solo. It also means we can generate multiple implementations or “tries” at a problem for nearly the same negligible cost, leading to a proliferation of approaches to the same issue. It sounds like a superpower, but it stresses ownership models. Who owns the Python script inside the Go service that was generated by someone who doesn’t actually know Python? Or the three alternative implementations an agent generated while trying to find the “best” one?

4. Veracity: The Hallucination of Correctness. Generated code has a trust problem. AI produces plausible output with variable correctness. It looks right, it’s well-structured, and it might even pass initial tests. But it can be subtly broken or insecure in ways that are hard for a human to spot at scale.

The Squeeze: Volume × Velocity

The Vs individually are manageable. It’s the interactions where the wheels fall off.

Take Volume × Velocity. This is the “unwinnable race” I was discussing with that platform lead. Every organization running AI-assisted engineering at scale has the same complaint: the review queue.

More code is arriving faster than humans can read it. Our current response is to ask humans to read faster or to “be more diligent.” The arguments that this approach is already breaking are getting louder. This is the “bigger-Postgres-instance” response. It works for a while, but it doesn’t scale.

When an agent produces a complete feature implementation in minutes, requiring a human to read every line before merge isn’t rigor; it’s a scaling constraint wearing the costume of rigor. We don’t read every line of compiler output or generated protobuf code because we have verification infrastructure that makes line-by-line review unnecessary.

Agent-produced code doesn’t have that infrastructure yet.

The “Data Lake” Moment for Code

This is the “Data Lake” moment for our craft.

The data lake didn’t emerge because someone thought it would be fun to store everything in one place. It emerged because volume and variety made “schema-first ingestion” (the data equivalent of code review) untenable.

The infrastructure inverted: store first, schema on read.

Code is heading for a similar inversion: merge first, verify continuously.

Our platforms—Git, GitHub, our CI pipelines—were built on the assumptions of the data warehouse: human-rate production, sequential process, and specialist operators. Those assumptions are currently under a level of pressure they weren’t designed to handle.

The Infrastructure Demand

We’re in the “patching” phase right now. We’re adding better reminder bots and longer review times. But the real shift is going to require new categories of infrastructure:

• Verification Infrastructure: Continuous, automated, and probabilistic. We need systems that can verify the veracity of code without requiring a human to “look at the diff.”

• Context Infrastructure: Making our platforms legible to AI. If your platform confuses humans, it really confuses AI. We need machine-readable context (API docs, schema registries) that turns inference into fact-based operation.

• Absorption Infrastructure: Systems that can handle higher code volume without proportional human review cost. Smarter merge systems and automated quality gates are no longer “nice to haves.”

Stop Typing, Start Stewarding

The ground is shifting. It’s uncomfortable, it’s noisy, and it feels like we’re losing our bearings. But we’ve navigated a wave like this before.

We need to stop treating code as a precious artifact and start treating it like the industrial output it’s becoming. This means shifting our focus from writing to reading, from owning to stewarding, and from gatekeeping to verifying.

If the monkeys are going to be this fast, we’d better be great stewards.

The question isn’t whether the infrastructure will be re-engineered. The question is whether we’ll build ahead of the curve or get dragged past it.

For platform and infrastructure engineering, this is going to be a very interesting few years.

I've been thinking a lot about the story of how we solved the problem of longitude at sea. It was a massive navigational challenge, that added to the existing practice of sailing, that was ultimately solved by a seemingly small piece of technology: an accurate clock. That story reminds us that truly transformational shifts in how humans operate, like being able to sail across the oceans, can come down to leaps in innovation.

The Problem with Extremes and Our Anxiety

A lot of the conversation around AI engineering tools sits at the extremes, but the most likely outcome is, as always, somewhere in the middle. By flapping around about best- and worst-case scenarios, we're missing a lot of what's important.

I've been in conversations where the same people express accidentally contradictory concerns. On one hand, they argue that AI assistants don't really speed us up because coding was never the bottleneck. They claim the tools aren't good enough anyway, so it doesn't really matter. Then, in the same breath, they worry about how new graduates will learn to code if AI is so good that they never have to do it themselves.

These viewpoints show that we're a bit at sea. They highlight the normal anxieties that come with major change. The fear of job displacement for experienced engineers—if the tool isn't good enough, it's a dismissive stance that says, "It won't affect me." Also fear of the unknown for the next generation—what happens to the junior talent pipeline if the fundamental skills change? Both are valid human reactions, but they can't both be true about the technology.

A Moral Panic About Thinking

It's also useful to remember that humans have a history of moral panic whenever a new technology affects how we think. Think about the introduction of books. Many people were concerned that because narrators/authors no longer had to construct arguments or repeat long narratives from their heads, and so the thinking went, books would negatively impact their ability to reason and think.

Do these tools change the way our brains work? Yes. But that was also true of books, and of the internet, and of search engines. Is it for the worse? I don't know, maybe. But in the long run, these shifts have raised the ceiling of what we can achieve.

This happens because the individuals and organizations that learn effectively are the ones that win. We've been too focused on execution and efficiency as the sole drivers of success lately. But organizations that figure out how to help their people learn, pick up new skills, and rewire how they work will be able to get products to market more quickly. Fundamentally, that's what drives success. We've known this for a while—it's a core tenet of research like The Fifth Discipline.

Learning from Our Past, Navigating Our Future

We can also look to our own profession for examples. Think about the introduction of compiled languages over directly writing assembly code. The criticism then was nearly identical to what we hear today: "The compiler will never be able to write this as well as I can. It's never going to be as efficient as I can make it. It's never going to be as correct as I can make it."

It's safe to say that history has declared a clear winner. We now have a towering stack of languages that sit on top of assembly. AI tools that write code are just another natural evolution of this trend toward higher levels of abstraction.

So, how do we navigate this new era? How do we, as experienced practitioners, bring the next wave of engineers along with us? How do we help our organizations adapt?

The principles are the same. We still need to pair, mentor, and give people projects that help them grow. But the mechanics are different. New engineers will be swinging AI tools more natively, more instinctively, and earlier in the process. We will have to learn along with them, and that's the uncomfortable part. But we've been through this before.

This disruption is large and it's certainly unmooring. But because we've been through this before, we do know how to navigate it.

It's time to go build our clock.

Share

Subscribe now

Surely at this point everyone has heard the story where Replit deleted a production database, overriding its rules and instructions along the way to do so, and another instance with the Gemini CLI deleting a load of code followed right behind. These are just examples of the coverage that because AI makes mistakes its dangerous and shouldn’t be used. Definitely pay close attention to failures, and then dig in and lean back on our engineering chops, as really the AI tooling is incidental in these examples.

To a decent degree, we are just falling for our human vulnerability of being distracted by significant negative events, we’re working against evolution on this, and so I get the reaction. What I’m really really looking for is our rational sides to take over and work the problem.

There are two things that stand out to me as lessons to take forward. Solid engineering practices remain incredibly important, and what is really going to matter is the rate of failures being introduced into our systems.

Guardrails and incident management practices are an obvious set of things key to reducing and managing the risk and impact of bad things happening. Note I said reduce, not eliminate. This is has always been core to this school of thought, zero risk if even achievable is likely cost prohibitive, and so we approach this from managing the risk and what guardrails we need in place to do that.

Human engineers have been happily killing production systems for as long as we’ve had them (so so many examples come to mind, both personal and legend), it would have been a really weird response if we just stopped writing code as the only way to prevent those issues. Over time we built up incident response skills, and then incident management processes that fed into making guardrail improvements, this is how we reduce impact and risk.

Share

The part that came later was basically the codifying of the DORA 4 metrics as a core view of tracking the throughput and quality of the changes we make. There are certainly other metric sets you can build as your feedback loop, but you definitely should have something.

What you really must know is are you shifting either the rate, or your recovery time from, failures. This is true whether or not you are in the midst of an AI push. We acknowledge that some early work has shown that increases in throughput and change size (from using AI tools) has negatively impacted change failure rates. Thats actually just a predictable outcome from the model of delivery here, but it might be ok. What you really want to think about is if the shift in your risk profile is ok for the increased value you are getting from shipping more, thats a tradeoff decision for you and your team.

Finally, as a human engineer do not blame the AI tool, and don’t tolerate anyone else blaming either. They are tools, you should learn and know how to wield them risk and all. But it’s more important than that. If we as engineers will remain involved (and so employed) then the tools are our delegate, this is how we should be approaching this in all ways and this is a concept with a long legal basis, treat the choices as such.

So if your delegate goes off the rails and does a bunch of destructive things, you have delegated poorly whether you understood that or not, you still blew that play. That is your signal to dig in, learn and get some new guardrails in place.

Subscribe now

First up, the actual act of writing code may not be the bottleneck, but that doesn’t mean that it isn’t a bottleneck. Bottlenecks are simply places in your production process where work to be done piles up, so hands up if you have an empty backlog. Bueller? Bueller?

Now I think there is a fair take that the act of implementing for an individual engineer isn’t uniformly faster yet. Thats a combination of tooling, reskilling and cultural issues at play, so sure, we aren’t transformationally faster (yet) at this phase; but there definitely are teams that are racing up this curve.

What we also have to remember is that even if the phase doesn’t get way faster, the AI native development can also change the nature of how change is offered. Spinning up alternative implementations at the same time carries close to the same engineering time as one implementation when using agentic tools. Instead of big alignment discussions based on opinions or theories up front, we can just try things and then select winners.

If your team can’t decide and align between two similarly scoped libraries for your project? Implement a proof of concept with them both to decide using AI tools. The tests and selections will be reusable across both, so additional cost is small, but comes with much more information for selection, and in all likelihood faster in time.

Which brings us to the implicit core of the point the pieces are making. Generally speaking we have implementation and coordination activities as engineers, so what we’re really saying is it doesn’t matter if implementation speeds up, as coordination is the larger factor in determining speed, which has certainly been true. The miss here is that as the costs of implementation shift, and especially shift the optional costs, the coordination points will both move and reduce as a cost factor in the process of shipping change.

We’re seeing this in organizations where the product team vibe code as a discovery and learning step, and even if engineering is no faster afterwards, the overall value stream gets much faster as we get better information input in the first pass, so fewer total passes before something is shipped as done.

These shifting costs also have effects within a dev team, so many more things move under the bar of ‘I can just knock that out with little effort right now’. There is also this wonderful realization that instead of assembling 4 people for an hour meeting to dig into design choices for a refactoring, you can just spin up your agents, and offer 2 or 3 pull requests as options. Generally its harder to shout down working code, and so even large shifts are much much faster to get through.

These organizational rebuilds are sure to be messy during the transition to AI native engineering, but these are things that we’ve seen before, and of course bottlenecks will appear in different places as processes reform. I’ve certainly seen enough to be very bullish on the impact this will let me, my team and my org deliver.

This is a very similar set of forces I think to Kent Beck’s ideas in an article about how slow deploys cause meetings. Lets phrase it at the positive end; AI native development reduces meetings, less time doing coordination means more time on implementation activities, and really I would have thought engineers at large would have been all about that shift in our day to day experience.

It is extremely clear that we are in the midst of a disruption wave crashing around us, we know from history that in the midst of that prediction is a fools game, and yet that isn’t stopping the hype machine from creating noise mostly about the extremes.

At one extreme is the ‘within 2 years we won’t even need engineers’, which has already been part of the discourse for a couple of years. Yes folks, next year will be the year of the linux desktop. Of course this is being used to drive layoffs and fears in some businesses, that is both bad and predictable. While those cuts hurt they are a long way off of the amount that would indicate a true replacement level belief among company executives.

The other extreme is the skeptic pole, something like ‘all this vibe coded crap is gonna burn everything down’. There is an absolute ton, several books worth really, to say about making things work as products and companies scale, but vibe coding really just looks to empower folks at very early idea experimentation. Any reasonable developer experience and product focused engineer I think should celebrate this, and then offer to roll up the sleeves to make the things work for real.

When taken in the large, across the entire industry which produces software (which is everywhere, so thats kind of everyone), we will experience the landing of this wave in multitudes. The extremes will be true to some extent, but the majority experiences I think will drop again into two clusters, and that really comes down to one choice.

do you want to amplify bad habits, or rebuild better habits and processes and amplify those

Fundamentally the idea is that AI based tooling lowers your effort (toil, cognitive load, etc) required for similar lift, this implies it’s a force multiplier on your habits; your choice is do you want to amplify bad habits, or rebuild better habits and processes and amplify those. Funnily enough DevOps at it’s core has always been about this question, and yet gave birth to a bunch of products and dogma that promised to be one stop utopia, returning to the core mental processes of this will separate success from failure.

This choice does apply, as always, to individual engineers and to engineering orgs in the large, the micro and the macro view. The early numbers are not super encouraging, DORA’s most recent state of DevOps report and a special impact of AI report suggest that we’ve taken these tools and were just slinging more code, which given our flow models has had the predictable result of harming effectiveness and outcomes.

To leaders I say get serious about refactoring demands, incentives and the support you offer folks to rebuild their workflows, we are a long way from maturity and stability in the marketplace here, but waiting it out is a bad strategy; you should have been doing this anyway. Because waiting is bad, also get comfortable with a ton of experimentation with fuzzy ROI projections, looking for high probability long term outcome choices is hard enough to be impossible in the current climate.

For my fellow engineers, for years we’ve complained about technical debt, so much so that the term has really lost meaning. The core of that has always been roughly, yes, we know better and that we need to write tests and documentation (irony being AI tooling works best when these do exist), but we just don’t have the time. Well, now those things are easier to do, much easier really, so it’s definitely the time to start doing it better.

Part of the reason for the negative reaction to vibe coding is because that is pretty close in truth to the at large truth about software engineering. We’re already drowning in code that is hard to wrestle with, the idea of massively increasing that problem is rightly terrifying.

While the AI wave (or hurricane maybe) is large, fast and incredibly uncertain, our way through is learning by experimentation, as it always has been.

Leave a comment

We dig into the consensus scale, and talk about theories and stories of bringing people along with change.

Mentioned during the conversation:

1. Why You Need to Take Risks as an Engineering Leader - Neha Batra on Dev Interrupted podcast

2. Supercommunicators - Charles Duhigg

During the conversation, some materials came up and those are linked below;

1. An Elegant Puzzle: Systems of Engineering Management - Will Larson

   1. And the same authors blog

2. Choose Boring Technology - Dan McKinley

3. Thinking In Systems: A Primer - Donella H. Meadows

4. The Death and Life of Great American Cities - Jane Jacobs

5. Paved Paradise: How Parking Explains The World - Henry Grabar

For this first episode of the Fixme podcast, I’m joined by Will Howell to chat about monorepos, especially having multiple monorepos…

Will is currently an Architect at Okta, continuing his career long push to make engineering platforms that just do the right thing TM.

Sitting down with your coffee in the morning, your heart sinks as you see a new block on your calendar for the day; ‘Developer Velocity Strategy’. The invite list is a discouraging mix of VPs, senior directors and architects adding up to a solid 20 invitees, the description says little past ‘lets discuss how we’re finally going to fix our velocity issues’, but hey at least there’s a doc linked as a pre-read!

But then the doc is only two thirds of one page, and seems to really just be the statement “Driving down our PR size will dramatically increase our velocity”, a cursory explanation as to why small changes are a desirable thing, and a couple of links to some recent engineering performance benchmarking reports coming out of a couple of startups selling engineering org analytics tooling, clicking through you see familiar looking metrics and breakdowns.

At the appointed hour you join the zoom, wait out the 7 minutes of laggards for the meeting to startup; 6 of you are on the zoom, 7 in the room and who knows where the others who accepted are. The meeting starts over the sound of someone hitting their laptop so hard you wonder if they are digging for oil, you can just make out the word for word read of the pre-read, and then the floor is opened up for discussion.

You raise your zoom hand to wait your turn, hoping the room will notice. After 10 minutes they haven’t, so you speak up, “what evidence do we have that smaller PRs will increase velocity, and what are we going to do to drive down PR size?”. The response wanders around the credentials of the report source, why these metrics are interesting, why this one metric out of the dozen or so in the report is meaningful, and then tails off, failing to answer either question, so you rephrase slightly ‘what tactical efforts are we considering to push developers to make smaller changes?’. And thats when it happens.

”Oh, we don’t need to drive anything, that which gets measured gets improved”

You watch, dumbfounded, as 3/4 of the attendees nod, some smiling. Silence is now the way forward, groupthink has accepted the magic ability of counting to change an outcome. There is another half hour of, well, waffle, but everyone seems very happy that we’ll be printing gold any minute now and the meeting disbands.

The Diagnosis

What the meeting has missed is that these reports typically classify teams based on measures, its basically clustering on characteristics that typically at most implies correlation. Causal links between performance level and a practice is sometimes there, but in a prescribed direction, and normally in the direction of the ability of the team to provide business value (revenue or some other real world thing). The internal document though implicitly assumes a causal link between a characteristic and being high performing, and is then saying that by adopting a practice, we will drive to that performance. This is on its own extremely unlikely; not mathematically guaranteed impossible, but unlikely for sure.

If we had more smaller changes, its likely we would be delivering more things by count, but by value? Maybe, maybe not, its honestly tough to know. Smaller changes seem to be more likely on a team with existing effective delivery (lead time, change failure rate etc), so does small changes improve delivery, or does good delivery enable small changes? In all likelihood, unknown.

But then there is the widely accepted platitude.

These broad blanket short statements have a comforting simpleness to them; they sound nice, we want to believe them.

However, it would take you no time at all to look around the world and find absolutely horrifying examples of things we know the measure or count of, and that only continue to get worse.

In short, we’ve confused correlation and causation, and we’ve dropped into accepting simple sounding things for truths, and we’re trying to remove context to make a problem more personal and tractable.

Progress Despite Surroundings

These scenarios are pretty common, and pretty demotivating all around. That said we normally do have the ability in some size of domain to work on change more systematically, and if you’re employed by a business there is an (at least) implicit responsibility to try to improve the status quo.

As always, telling folks that they are wrong and/or have misunderstood really fundamental things is not a play with a high success rate, and the skepticism that comes through in the framing of the questions pretty much takes us right up to that door.

In the room, try to get folks to agree to some targeted experiments, nudge towards a larger context of the metrics not just one piece. Often if we frame this as ‘whats two or three things that would be good candidates to measure, work on, and see how things shift?’ we can get to a more fruitful discussion, and also plausibly to there are no good candidates and maybe dropping the whole idea.

We can also work with things we own make sure we track the full context. Not only can we do this, we should be. Knowing and working on our effectiveness and the collective tooling and processes that contributes that is ultimately our own responsibility.

At the end of the day, we just want to convert the simplistic statement into ‘if we theorize about a characteristic we’d like to alter, then measure the state, try some changes and measure again, then we can improve’, also known as basically the scientific method, or software engineering as we also sometimes like to call it.

However, if you still end stuck in the simplistic circular arguments, and struggle to iterate on even your own sphere, it might be time to start looking for an escape hatch.

The idea of this space (a newsletter and podcast to begin) is to work through those tales, find the mistakes at the bottom, hopefully find a better path, and have a good laugh while we do.

Even with a deep faith in the ability of tech to change whats possible and drive positive change, we must acknowledge that our thought processes and motives matter a great deal in how our products reach and affect the world.

In short, how matters.

We’ll dig for the logical fallacies and other very very human mistakes that individuals and organizations make, we’ll look at how to survive them, and together we’ll bring better products and change into the world as a result.

Subscribe now