On Zeno’s Migration and Tech Leadership
Jean Yang, founder of Akita Software, has been talking on Twitter lately about the chasm between the technical “guidance” from large tech firms like Netflix and Amazon and what your average, run-of-the-mill tech firm does in the wild. I’ve started calling this latter group, a much larger cohort than the former, “small tech” loosely defined as 10-50ish engineering types. This idea of Zeno’s Migration, a riff on Zeno’s Paradoxes for those who slept through their classics courses, is full of promise like the Dichotomy paradox that says because a journey is a set infinite steps, it can’t even begin much less end which is especially apropos of many small tech companies cloud migration plan.
The idea here is that in order to complete a migration, you have to travel halfway an infinite number of times. The constant refrain from technical leadership is that next quarter we’re going to be “X” farther along the journey on the migration de jour. This migration often involves shiny, jangly technology choices like “rewrite the front end in React” or “migrate everything to the cloud”. Unfortunately, this constant refrain ignores hard strategic realities on the ground where a small tech firm has serious limitations in what it can accomplish. In an ideal world, free of harsh realities, these constraints would be manageable. Alas, there are no ideal worlds.
I think the problem arises because the pervasive definitional concept in our industry is “The Project”. When everything is a project, it sets up incentive conflicts in priority determination. In an organization that is highly typical in the industry where “the business” and “technology” are considered different entities, the migration is always a project defined and defended by “technology” which puts it at odds with other projects defined by “the business”. Even at organizations that nominally follow a more product based organizational structure, the key concept is projects involving kickoffs and planning and story writing and infinite other steps. As with most systems, the focus at this level is too granular and misses the forest for the trees.
A more holistic solution is to step back at least one level in the system to “The Team” and assess the situation there. The product team becomes the atomic unit of measure, not the project. This requires a shift in thinking and language but when done correctly, Zeno’s Migration melts away in the same way Zeno’s Paradoxes do because we shift from a discrete function with a start and an end to an infinite series (the outcomes and measurements of the team on a continual improvement cycle ala DevOps philosophy). In this model, a cloud migration just becomes part of the continual improvement process.
This still raises the question of when to do such things. With limited resources, even a well organized product team needs guardrails on what to focus on. This is where organizational technical leadership can provide value with written visions and strategies. In tandem with these strategies, it’s critical to have metrics measured in outcomes that are regularly reviewed and publicized to the broader organization. These strategies should be cyclical in nature, e.g. they do not have an end but define measurements like “we will migrate X services per quarter or half”. Without these strategies defined and monitored for outcomes, even well organized product teams in the typical technology business will fail to do this migration work because it’s too easy for it to fall on the floor.
A corollary to Zeno’s Migration that I’ve seen and heard a lot is the lift and shift strategy of cloud migration. An organization, hearing that “the cloud is the future” or whatever, moves (or more often pays some consulting firm to move) their entire operation to the cloud with the idea and promise that this allows the architecture to then move to a more cloud native form over time without having to migrate where the services live.
Of course, this achieves none of the benefits of a cloud native architecture and moves all of the existing maladaptive processes along with it. But returning to “The Project” as atomic unit, this is easy to reason about, define and measure. I think there is also often an unstated concept here that technology leadership maintains related to cost and incentives. Over time, as subsidies and cost breaks expire, running an org in the cloud just like you did on prem is likely to be much more expensive. The idea is that at that point, it would be easy to explain to the business how a rewrite or rearchitecture will save money which is always the easiest metric to measure.
There are two things going on here, one part sociological and one part technical (which is convenient since these are all sociotechnical systems). The sociological part involves humans tendency to look at achievement as An End. When we look at greatness, we think of a finished thing and not as something that happens to just be quite far along the continuum of that particular thing. The second issue is that in order to deal with the continuum of technical growth and improvement, we need to have a way to move successfully from vision to tactics which is what Richard Rumelt calls Strategy. We must define the problem, the constraint and actions required to solve the problem. This helps identify tradeoffs and things we cannot do in order to achieve the vision. This is where good technical leadership shines brightest and is often missing from existing tech growth plans.
In the end, I think technical growth is contextual. A team of 30 engineers can’t read a paper from Netflix and apply it to their situation. Leadership must analyze the existing context for opportunity and then define strategies for that opportunity that define the problem to be solved and the actions that will be taken (or not taken which is a critical part of strategy) to succeed. Then the strategy must be publicized and monitored for progress along its continuum. As growth occurs, the strategy and context should be revisited for modifications as necessary. This is all hard management work that in my experience is difficult to develop and nurture. There are systems involved here with first order incentives that are easier to both talk about and create that work against the harder second order incentives to dig into the history and context of a given situation and create solutions specifically for it. Until technical leadership as an industry is better at thinking about these systems, we’ll continue to read shiny new ideas from large tech organizations and try to pound that round object into the square hole we are dealing with.