Building up : Product and performance metrics

Code does not evolve in a vacuum. It does not appear from nowhere, it does not just run on the developers computer and it does not just end in the code repository.

Any change to a code base comes from a set of inputs and motivations, it's shaped by existing code, and influence how the product behaves in front of a user, or many.

A wider perspective on it's made

That's why it's important for software engineers to get a wider perspective about their work. It's important to talk with people working on the product from different angles and perspectives even if those people don't write code, if they just use it.

Understanding how it's used, how a bug is experienced, how a feature is missing from the product will greatly improve how the fix is done, how the feature is added and how the product is perceived and used.

Talking directly with customer support agents about issues encountered by users will help figure out solutions that actually make sense and help them. Staying isolated from the end user usually result in products that are difficult to use and avoided.

There is a whole lot of steps to take within the organization to smooth things up and get everyone a wider perspective on what, why and how to improve the product in a way that matters to the end users.

In the software engineering side, it can involve getting less or non technical people involved when thinking, designing and preparing changes to the product. It's important to discuss not only how those changes will look but also how those changes will impact the users and how success will be measured.

A wider perspective on how it runs

In the same manner, when the product has evolutions coming out, those evolution don't just end their life in the code repository. They go live to users. There starts the next feedback loop on how those changes are actually behaving as we planned they would, in the real world.

The feedback loop should be about how the problem we were trying to solve is being solved better or not by those last changes. If the team has spent some time assessing how to measure that before hand then it's easier to assess what the next steps will be.

In a practical way it's important for software engineers to get a perspective on how their work impacts the users with concrete numbers.

Product metrics (number of posts, number of views, ...) are important to know the usage of the product. Performance metrics (response time, error rate, ...) are important to know how the product is actually fulfilling its promise of availability and features.

Both those sets of metrics should be involved in the feedback loop too. They will both help shape and schedule the next evolutions to be done by the team.
Product metrics will help decide what to do, performance metrics will decide how to do them too.

Data and what to do with it

It's true that data can used to say whatever we want. Yet, storing, analyzing and visualizing data can give great insight in how the product is doing.

If the product data is often very specific to each product and company, there are more general guidelines when it comes to performance.

Performance metrics, especially the key 3 indicators (response time, error rate and the number of request per unit of time (minutes or seconds)) are key to give the engineering team an understanding about how their work is directly impacting the product experience.

Just like how it's good for the team to monitor and devise estimations about how product metrics evolve with changes in the product, it's good to monitor and devise how the performance metrics evolve with changes in the product.
The comparison of expectation vs reality will give plenty of feedback to decide what to do next.

More on this

Google SRE book has a great chapter on "Service Level Objectives" which are key when considering performance : https://landing.google.com/sre/sre-book/chapters/service-level-objectives/ . It's a great read for technical people, but it also can help less technical people get a grasp on the topic.

Intercom has a nice post covering product metrics : https://landing.google.com/sre/sre-book/chapters/service-level-objectives/ . It will help tech and non tech getting a grasp on what it is about and what to do with them.

Fancy talking more on this topic ? Contact me to discuss how I can help your team establish or review existing product and performance metrics in your projects.

Need help ?

We specialise in helping small and medium teams transform the way they build, manage and maintain their Internet based services.

With more than 10 years of experience in running Ruby based web and network applications, and 6 years running products servicing from 2000 to 100000 users daily we bring skills and insights to your teams.

Wether you have a small team looking for insights to quickly get into the right gear to support a massive usage growth, or a medium sized one trying to tackle growth pains between software engineers and infrastructure : we can help.

We are based in France, EU and especially happy to respond to customers from Denmark, Estonia, Finland, France, Italy, Netherlands, Norway, Spain, and Sweden.

We can provide training, general consulting on infrastructure and design, and software engineering remotely or in house depending on location and length of contract.

Contact us to talk about what we can do : sales@imfiny.com.