Product Evolution by lean selection

Back to products now 🙂

In spite of the odd sounding title of this post, I will start by admitting the contents aren’t novel. I am providing a different vehicle to communicate the key tenets of Lean Product Management from the first principles. I love the vehicle used here (evolution) and my intention in writing the blog is more to enjoy the analogy. I intend to map my learnings in both the fields of Biology and Product Management in retrospect.  A few key tenets on Lean Product Management are given below.

  1. Love the problem not the solution
  2. Always focus on de-risking the initiative against the available parameters
  3. Make small incremental changes to your product and test them
  4. Set yourself to quickly respond to feedback

One of the key aspects I try to understand when I am looking at a product is how did this product evolve to the current state? What could the next increment? In that vein, I find the current view of the evolution of a product is a complete antithesis of the word evolution. Therefore, I want to synthesise my understanding of the biological evolution by natural selection and share my perspective on this topic. To me the understanding product evolution is critical to create an environment where the product can sustain and thrive. The understanding also gives us the framework to identify the causalities that led to the success and failure of an initiative.

The proposition

A product is an organism

I consider a product as an organism. A single-celled product is one which solves one problem and cannot be decomposed to multiple components or sub-products. A multi-celled, complex product, however, can be decomposed into a lot of sub-products each solving independent problems. I have detailed my views on products and their composition in a previous blog.

Why this analogy comparing products and organisms?

The broad goal for organisms and products is survival. An organism evolves to maintain a balance in the ecosystem. When the ecosystem isn’t balanced, then the entire system perishes. An organism has to protect itself from its competitors or predators. However, those predators and competitors are needed for the species to maintain the balance. An organism adapts to small and incremental changes to the ecosystem. However, when there is a drastic change fundamentally destroying the primal needs of the species, then the species go extinct. An organism constantly alters itself to test how it performs in the ecosystem. If that change works then it continues and if it doesn’t then it fails. A product operates in an analogous way. It forms out of a need and lives in tune with the environment.

In summary, both these entities are comparable for the below reasons.

  1. Live within the ecosystem
  2. Save themselves from predators/competitors but also need them for maintaining balance
  3. Go extinct as the ecosystem dramatically changes their causal factors
  4. Go through continuous incremental changes to test their sustainability

Why the evolution analogy?

Evolution by natural selection is defined as

Organisms best adapted to their environment tend to survive and transmit their genetic characters in increasing numbers to succeeding generations while those less adapted tend to be eliminated.

If I replace organisms with products that perfectly explains the situation on the product space. Also, evolution has some basic traits which make it an amazing vehicle to ride on for narrating the concepts around product management. Evolution creates an organism through random small changes which have survived the test of time.

  1. Evolution is non-linear
  2. Evolution has no foresight
  3. Macroevolution is caused by microevolution over time
  4. Evolution is not keen on creating a solution but solving for maintaining equilibrium
  5. Evolution happens in small increments and short feedback cycles.

Product Evolution – the prevailing opinion

When the term Product Evolution is used in organisations these days, it communicates one of the below two meaning.

  1. A plan of how the product will change over time
  2. A roadmap of features and a plan which also communicates the strategy, evaluation and approach

Both these meanings have a fundamental issue. They are both predictive exercises. While evolution is an approach towards sustenance, these two are exercises to figure an approach to get to an outcome which is assumed to give success. This not only assume the problem but also the solution and when it will be needed. Product Teams have to instead set outcomes as boundaries and let the process of evolution guide them.

Since we are in the topic of evolution, I am amused by the irony caused by our prefrontal cortex. The prefrontal cortex is responsible for planning, complex cognitive behaviour and decision making amongst others. I feel humans tend to overestimate the capability of the prefrontal cortex by assuming it can be used to predict the future. No individual has ever been able to do it. We can all fit our recollections with the findings to create a pattern. This is also the reason why planning is different from prediction.

Product Evolution by Lean Selection

My Assumptions: The base intuitions of this approach

I am making two basic assumptions here in order to proceed with this approach.

  • People have to reach an outcome. The product is a tool to achieve that outcome.
  • The tools adapt to the changes in the environment over time. If they don’t they perish.

First, I would like to illustrate these intuitions of mins with an example. People want to have the streets lit during the nights so that they can see clearly and commute. Initially, everyone carried their source of light. Then, authority got centralised. The government or community had oil lamps for lighting the streets. As technology evolved, people had fluorescent lamps powered by electricity. Now, we have solar powered independent units which power LED bulbs. The outcome is always lit street and it has always been met. The tool to meet the outcome has changed with new inventions entering the environment. If a company made oil lamps and never changed the tool with times to help customers get the same outcome, they would have been out of business when the fluorescent lamps started coming up.

Defining Product Evolution

According to me, Product Evolution is a result of incremental changes to the product. If these changes are guided by the principles of careful selection to sustain in the ecosystem then the product continues to live. The primary attribute to achieve this sustenance is the admission that we cannot have the foresight to predict the tool in the long run.

The best equation to evolution by natural selection is the Price equation which was defined by George Robert Price. The Price equation to get the change in traits across generations is given below.

Z =1/n ∑i  iZiΔ𝑧 = Z’ – Z

Δ𝑧=1/𝑤(𝑐𝑜𝑣(𝑤𝑖,𝑧𝑖)+𝐸(𝑤𝑖Δ𝑧𝑖))

where z stands for a trait, and w stands for the fitness level needed for that trait and i is the index of the subpopulation. Z’ presents the collective traits of the next generation. Cov is the covariance function and E is the expectations function. The expectation function presents factors other than the direct lineage which affects the traits.

The same equation holds good for a product too. If I have to simplify this for a product, then I will end up with

P’ = P + Δp

Δp = ∑i  OiWhere O is a product trait to reach an outcome. The product team will ensure that Δp continues to improve the covariance function so that the traits improve the fitness level of the product.

Though there are plenty of valid criticism of the Price equation, I feel it is the best approximation we have got to mathematically represent the Evolution by Natural Selection. A change in a trait is influence by how fit the trait is with the population and the expectations of the trait in the population.

In the product context, a product increment is a collection of traits each intending to cause an outcome. A product increment which doesn’t increase the product’s fitness quotient in the population will have to be removed. The smaller the quantum of change the shorter the feedback cycle and smaller the impact both positive and negative.

Lean Selection – A way to achieve this outcome

As mentioned before a product can be single-celled or multi-celled. Consider each cell as an individual component or sub-product. Each cell can therefore individually evolve. Let us take the example of the eye. The evolution of the eye happened in parallel with the evolution of the brain or any other organ. Each organ has a specific goal and goes through incremental changes. The collective communication system between the organs called the neural network also evolves with time. The product evolution has a similar approach. The biological evolution is also fundamentally lean. By lean I mean it is risk-averse, economical and controlled. For example, the entire population doesn’t go through the same change, only a tiny fraction does. If that change helps the population then it propagates. I call it the ‘Lean Selection Experiment’.

The core principles of lean Selection experiment are

  1. Identify a need
  2. Make minor changes to one component
  3. Test it with a small population
  4. Get feedback
  5. Decide whether to drop those changes or continue

A look at Lean Selection Experiment: Eye as a product

I want to first explain the evolution of the human eye. Charles Darwin once said, till this day the eye makes me shudder. Each step in evolution happens as an experiment. The only difference between Lean Selection and Natural Selection is that Natural Selection is a random process while LeanExperiment is a more controlled process. Natural Selection doesn’t actually have an understanding of the problem space. In Lean Selection, we can artificially do that by performing generative research and running tests.

Eye: Experiment #1

  1. Need: Need to know when the is a movement in my environment to protect myself
  2. Solution: Develop a  sheet of light-sensitive cells to detect the difference between light and dark.
  3. Test: A small number of species develop a light-sensitive sheet
  4. Feedback: Can detect a change in environment
  5. Decision: Continue with the change

Eye: Experiment #2

  1. Need: Want to know the direction from a predator is coming
  2. Solution: Bend the sheet into an indentation
  3. Test: A small number of species get a light-sensitive indentation
  4.  Feedback: Can detect the direction of the predator based on the shadow
  5. Decision: Continue with the change

Eye: Experiment #3

  1. Need: Need to know the nature of the predator
  2. Solution: Bend the sheet into a deeper cup
  3. Test: A small number of species get a light-sensitive cup
  4.  Feedback: Can detect the rough shape of the predator
  5. Decision: Continue with the change

Eye: Experiment #4a

  1. Need: Need to know a more precise nature of the predator
  2. Solution: Bend the sheet into a pinhole camera kind of eye (Mollusc called Nautilus)
  3. Test: A small number of species get a pin-holed camera
  4.  Feedback: Can detect the shape of the predator
  5. Decision: Continue with the change

Eye: Experiment #4b

  1. Need: Need to protect the eye
  2. Solution: Have a transparent sheet to protect
  3. Test: A small number of species with a transparent sheet protecting the eye
  4.  Feedback: Eye is protected
  5. Decision: Continue with the change

Eye: Experiment #5

  1. Need: Need to a clearer image
  2. Solution: Expand the sheet to make it into a lens
  3. Test: A small number of species with an eye lens
  4.  Feedback: The image is clearer
  5. Decision: Continue with the change

Eye: Experiment #6

  1. Need: Need to a clearer image
  2. Solution: Form a coating to form a reflecting tissue
  3. Test: A small number of species with a reflector eye – eg. a Scallop
  4.  Feedback: The image is clearer
  5. Decision: Continue with the change

These experiments are not linear which means Experiment 5 is not after Experiment 6. Also, one could stop after a specific outcome is met. Further, this evolution of the eye happens in parallel to the other components of the organism. When we run multiple such tests we end up with multiple solutions for the same problem.

I suggest reading this text to get a complete understanding of the evolution of the eye.

MVP in different contexts

Since its conception, the word minimum viable products has metamorphosised into multiple acronyms.

  1. Minimum Viable Experience (MVE)
  2. Minimum Compliant Product(MCP)
  3. Minimum Reliable Product(MRP)
  4. Minimum Delightable Product (MDP)
  5. Minimum Testable Product (MTP)

I am not sure what was the intention of each of these but for me, an MVP is a minimum set of outcomes which gives me enough feedback to take a call on whether to proceed with the next experiment or otherwise. In some instance, it could just be a single outcome and in others a set of outcomes.

I sometimes get into a discussion on how to compete with other players using MVP. If one intends to compete with Google search engine or Amazon eCommerce or eBay auctions, then their starting position is incorrect. One can never achieve get MVP this way. The starting position could be to check if there is a market for secure searching or a market for searching academic journals.

Here is a design for an experiment

Search Academic Journal: Experiment #1

  1. Need: Need to search peer-reviewed academic journals on a specific subject
  2. Solution: Manually create a collection of academic journals. Create a mechanism to search these journals
  3. Test: Share it with the University students who study the subject and measure the usage
  4.  Feedback: Talk to the students to learn more.
  5. Decision: Continue if the students come back repeatedly.

As you might see in the above context, I am storing the documents. I am not crawling any server to find these documents. I have manually stored and indexed these documents. This is a way to come up with an MVP without intending to directly compete.

What is required for this setup to succeed

There are 3 critical components for this approach to succeed.

  1. Infrastructure
  2. Team
  3. Culture

Infrastructure

Teams need the right infrastructure to run these experiments. The infrastructure should allow teams to test a small sample of the population, roll back, scale the changes without much effort. If infrastructure becomes a bottleneck then the experiments tend to get bloated thereby eroding the principle.

Team

The team needs to have dedicated capabilities to perform the following functions.

  1. Research customers
  2. Identify the experiment
  3. Design the experiment
  4. Develop the solution
  5. Measure the success
  6. Manage the finance
  7. Coordinate with other teams

Culture

The team culture is another key component for the success of this. There are 7 principles

  1. Love the problem not the solution.
  2. Make small incremental changes to your product and test them
  3. Plan the experiment and not the solution
  4. Set yourself to respond to feedback both within the team and from the customers
  5. Respond just in time to avoid waste
  6. Do not fill your time with solutions (product managers with more stories, designers with solutions and developers with lines of code)
  7. Collaborate with other members instead of competing with them

Conclusion

In conclusion, I feel enabling a product to evolve through the lean selection criteria is the best way to create a healthy sustainable product. The lean selection criteria drive product teams to explore the changes in the business environment, constantly engage with customers, run small experiments and adapt to the feedback. This pushes the team to constantly look at outcomes rather than solutions.

Decompose products to avoid scaling teams

As a proponent of the lean approach to product development, I have lost count of the number of times I have been asked, “…. but, how do you scale this approach?”. Since completely unrelated people have asked me this question, I have to assume sincerity of the question. I am also beginning to wonder that the existing frameworks to scale (like SAFe or LeSS) haven ‘t worked. However, nothing suggests that scaling lean approach shouldn’t work. Toyota, one of the most quoted companies for lean production has over 300k employees and manufactures of 18 million units a year.

Where are we in the field of software product development getting it wrong? I want to address this without sounding like I had a moment of epiphany. I am going to avoid the issue instead of solving it. Like Kent Beck usually says, “if an idea is good and it turns out to be true someone smart would have done it. If the idea is stupid then you have a chance no one else is dumb enough to try it.” Many people much smarter than me have attempted to solve the scaling issue, so I am just going to eliminate the roots of the issue instead of solving it.

In a nutshell, I want to flush out decompose to avoid scaling instead of scaling for growth. The issue with scaling is it not only assumes the problem of organic growth but assumes the solution to reach there is by scaling team(s).

Let me start by putting forward a couple of questions.

  • What does scaling mean?
  • How is scaling solved in the traditional software development models? Is it solved at all?

These questions are important as they helped me identify that when people mean scaling, they mean one or more of the below items.

  1. I need to develop this product faster.
  2. I need to ensure the teams are aligned with the broad vision.
  3. I need to ensure that there is coordination within the team and across teams

So, scaling is not an issue. The real issue is why, when and how.

My learning

I want to summarise my learning on this topic through the below points.

  1. When in doubt about how to structure a team go back to the first principles on a product.
  2. Do not fall in love with the tool
  3. Identify if what you are doing should be part of your product or another one which is dependent on yours.
  4. Assume you will not be in the product team for long. Make it easy for the other person who might pick it up from where you left.

First Principles

Definition of a product

Kotler, P., Armstrong, G., Brown, L., and Adam, S define a product as “A product is anything that can be offered to a market that might satisfy a want or need.”.  Mike Cohn defines a couple of key constructs of a product.¹

  • Products provide benefits to a market
    • A product can exist within another product. A pen, for example, may have replaceable ink cartridges. The pen is a product. But so are the ink cartridges within the pen.
  • Products can be defined recursively
    • When we identify subproducts within a larger product, we need to be careful that each subproduct provides benefits to a market.

Attributes of a product

Martin Fowler came to my rescue to give the fundamental principles in his ‘goto conference’ speech on Microservices.

Screen Shot 2018-10-02 at 5.07.36 pm

The key attributes of a product are the same as the one Martin Fowler used to describe his microservice.

  • Componentization: A component is a unit of software that is independently replaceable and upgradeable.
  • Organised around business capability: The microservice approach to division is different, splitting up into services organized around business capability
  • Products not projects: a team should own a product over its full lifetime. The product mentality ties in with the linkage to business capabilities. Rather than looking at the software as a set of functionality to be completed, there is an on-going relationship where the question is how can software assist its users to enhance the business capability.
  • Decentralised governance: One of the consequences of centralised governance is the tendency to standardise on single technology platforms. Experience shows that this approach is constricting – not every problem is a nail and not every solution a hammer.

So, after looking at the definition of a product and the characteristics of a microservice and with the approach of Lean Startup, I came up with the below understanding of a product.

The answer to ‘Why?’

The answer to why scaling became an issue is because the products were not decomposed properly. The above principles can be decomposed to the below snippet.

A solution can be classified as a product if

  • It provides benefit to the market (Problem – Solution Fit)
  • It is independently replaceable and upgradable

Even if

  • It exists within another product
  • If it is part of the same engagement/project

And should

  • Be organised around business capability
  • Provide decentralized governance

If the product is not decomposed, it will result in

  • Loss of context
  • Team feeling directionless
  • Erosion of practices
  • Releases feel bloated
  • Tools become roadblocks
  • Practices don’t feel like they can scale

The answer to when?

The next question to address is when to begin decomposing so that we avoid the issue of scaling. I want to use the nomenclature of Lean Startup to drive this point. The first point is when you reach the problem – solution fit.

Decision Point#1: Problem – Solution Fit

  • Criteria
      • How many problem – solution sets are there?
      • Does solution can constitute a separate product?
  • Next Steps
    • Prioritise one problem – solution fit
    • If both the problems have to be started and can be started, then start them as two different teams
    • If both the problems have to be started and cannot be started because of dependencies, then identify the right pre-conditions for starting the second product stream

Decision Point#2: After each round of research

  • Criteria
      • Have we validated a new problem altogether?
      • Do we need to solve this problem now?
  • Next Steps
    • Kickstart another stream of work
    • Highlight the reasons to start this as a separate stream

Decision Point#3: Product – Market Fit

  • Criteria
      • Have we validated the product – market fit through our MVP?
      • Do we need to scale to more problems?
  • Next Steps
    • Kickstart another stream of work
    • Run them as separate teams if possible

The answer to ‘How?’

The high-level approach to decomposing a product to form multiple sub-products involves three key components

  1. Timeframe or Duration of the activity
  2. Structure of the backlog
  3. Structure of the team

Here is a simple flow which can be used to determine how to decompose a product.

Screen Shot 2018-10-02 at 7.43.45 pm

The idea is to determine the answer to the key questions.

  1. Do I create a product?
  2. Do I spin off a separate team?
  3. What should that team look like?
  4. How much coordination is required?

When the product is decomposed this way, it results in multiple small teams. The teams should balance team autonomy and intrateam collaboration. The governance for this will be the minimum needed to keep the balance going.

References

1 https://www.mountaingoatsoftware.com/blog/what-is-a-product

“Are you deprioritising design?”

As a product manager have you faced the question, “Why are you deprioritising design?” from your designers? If yes, then it is a symptom of a broken process. You are probably facing at least one of the below issues.

  1. Your product delivery value stream is not lean. There is a disconnect between the design and the execution.
  2. Your MVP is not delightful. This means your product is looking to be functional without being usable and delightful.

For the purpose of this blog, I want to look at the first issue as the second one is easier to deal once we handle the first.

The concepts of lean, customer-centric design and team collaboration have existed for decades now. However, I still see companies struggle to put them to practice. So, most of what I say is going to be well known from a theoretical standpoint. The theory suffers to give people tools to identify when practices are corroding the intent. I will take an example to drive my point here.

The ‘every product’ story

Eric and Evan are the founders of a startup which develops products to educate people in the field of psychology. Their latest in that list is an online quiz product. Jim is the product manager of the product and Joan is the product designer. Both Jim and Joan are avid practitioners of lean product management and customer-centric design. Who is going to argue against both those concepts? At least, Eric and Evan are not going to do that.

Through their research, Jim and Joan have validated the need for the product and tested the solution through their product. Eric and Evan were very happy with the designs they saw. The mockups had 4 screens.

  1. Signup
  2. Select a test
  3. Take the test
  4. View results and post them

The designs had fancy breadcrumbs which will help the customers navigate. Now, Joan’s role in the team was reduced to support the engineers with the designs needed. Jim, on the other hand, decides that the MVP is just a set of questions and answers on the same page. This will give him learning on framing of the questions and structuring. It eliminates all the features on the screens which don’t address this problem. It also merges the steps 3 and 4 given above. The team follows the best practices of agile software development to deliver a good working software.

Joan looks at the output at is not impressed. She feels her work has not been respected. It opens up a lot of disgruntlement. She goes to Jim and asks the million dollar question “why have you deprioritised design?”.

What happens next is called damage control in corporate terms. Eric and Evan are involved in discussions with Jim and Joan. After discussions, they will invariably reach one of the below conclusions.

  1. Joan will be asked to stay quiet and respect the role of PM.  Jim takes the call.
  2. Eric and Evan (as sponsors) will take a call balancing Jim and Joan wishes. They reach a settlement between Jim and Joan.
  3. Jim will be told that Joan has the full control of the design. From now onwards, Joan will be the gateway for design. The design will go through her review before it goes to the customer.

I have been Jim, interacted with Jims and Joans and have also seen the legacy of Jim and Joan in a company. Waterfall design using the customer-centric design techniques and agile delivery seems to be the latest amongst the smart-stupid mistakes in the product development space.

Does any of the above three points actually solve the issue? No, almost never.
Is it the fault of either Jim or Joan? No, it is a collective failure. From the individual vantage point, they might look right.

So, why did this happen and how to address this?

Mistakes made

Mistake #1: Why did we design those screens?

The first mistake made is to design those four screens. They solved multiple outcomes.

  1. The customer can take a quiz to see the result
  2. The customer can choose a quiz to take
  3. The customer can signup so that they can get the result over email
  4. The customer can post the result

Which is the most important point outcome to solve? If Jim’s answer to this is the first, then Joan and Jim should have collaborated to solve for just the first one. Once Joan has solved for all four and shared it, it is hard to take something out.

Mistake #2: Why did we move the designer to a support role?

The designer is an integral part of the team. By moving them to of the core product team, their role is reduced to support. This leaves them in the capacity of either a powerless contributor or a powerful gatekeeper. Either of these two is not collaborative.

Mistake #3: Why is the team not aligned on MVP?

The team is not aligned on the MVP. Assuming the first two mistakes have already happened, it is paramount to have got an alignment on the MVP and the resultant impact on the design. This means the designs reflect what needs to be done for MVP, the MVP is shared with the sponsor and validated with the customer.

Mistake #4: Why were the symptoms addressed instead of the cause?

None of the three proposed solutions has actually addressed the core issue in the process. Eric and Evan should have reflected on how they reached this stage and fix the cause instead of addressing the symptom.

Learnings

Here are my learnings from this approach.

Learning #1: Solve for one problem at a time

Identify the outcomes which have been validated, then prioritise the outcome which has to be solved. The prioritisation, though a product management function is best done as a team. The designer can solve for the prioritised outcome.

Learning #2: Easy to add features than remove

Even while solving for one problem, it is important to think if the features are important for achieving the customer outcome in question. It is always easy to add features later than removing something which has been done.

Learning #3: Design to a team is just as vital as development

The design is how the product helps achieve the customer outcome. So, the designer becomes a vital part of the team. It is always strange that even the team claiming to be extremely agile, always end up having designer outside or even outsourced.

Learning #4: There is no alternative to team alignment

Teams have to be aligned with the goal and the outcomes they are trying to achieve. If the three critical functions (product management, design and development) are not aligned, then the decisions are going to drive discontent and inefficiency.