In today’s world everyone is surrounded with data in all shapes which flow in numerous directions. We both produce and consume it, often without being aware of that process. Social media, news outlets, shops and virtually every other system in production exists to ingest, transform and act on this virtual information. Why is it then that the development process is still often not guided by available statistics and instead is based upon stakeholders and engineers sticking to a ‘methodology’ I call guess-driven design? Let’s take a closer look at what data should be made available to development teams and how to get full advantage from it.
Before we dive deep into the world of information, we should know what we are looking for. Typically, dev teams want to at least know if deployed software operates correctly and marketing team wants to know actual and projected revenue streams but this should not be an end of our quest for better decision making. We can have data points on not only the system itself but also on users' interactions with software, their engagement and happiness. We can, and we should gather statistics on the team itself to accurately measure its performance and be vigilant to observe project detriments and blockers. In the end, we can group information which is useful during the development process in 3 categories: data on the system, its customers and team. All of them are crucial to transform a team into a highly-productive, well oiled machine able to not only produce value according to agreed plans but also to react to changing circumstances in a fast manner.
The first category is systems metrics, a set of practises commonly known as logging and monitoring. Fortunately, this type of data usage is quickly becoming widespread as more and more companies learn this simple truth: it is much better to catch issues as early as possible, which, in an ideal world would be pre-deployment. No matter how good an automatic test set we have or how scrupulous our testers are, bugs may go through the pipeline unnoticed. In that case, a robust monitoring system might just save us a lot of stress and time by alerting the team that something is not going as planned, often before the issue will be perceived by the customer group and marked as an incident. This lets us rollback changes fast and start working on a patch immediately. Another case can be made that by monitoring resource usage by the system we can plan ahead many scaling or maintenance operations long before load on the system becomes too big, achieving seamless and less stressful transmission time for all engaged in the process.
Next in line is customer data. This particular category of information is also well known nowadays and for good reasons. Gathering such statistics as customer satisfaction and engagement, traffic or in-site heatmaps of interactions allow our team not only to discover desired features or potential revenue streams but also help immensely in an instance when it’s time to retire some functionalities whose maintenance costs may be higher than actual value. This is what data-driven design means - develop what the audience wants, do not second guess their needs.
The third group of invaluable information comes from within the development team. Data on the development process itself is the least recognized and there is enormous room to improve in this regard. Knowing the system and its customers won’t help much when the team is unorganized and has no idea how to measure its own performance and satisfaction (think about this: you know what to do, but have no idea how to arrive at the desired outcome). I am not talking here about commonly known and harmful KPIs such as lines of code written each day, but about concrete, actionable data such as metrics stated by DORA Research Program. Statistics like daily deployments, change failure rate or median lead time for change can tell the entire story on how our team is performing and is scientifically proven to be a big factor in happiness and burnout rate of software engineers. Tracking such information and setting appropriate goals can drive the development team to become better, more efficient and happier while reducing stress, dropout and burnout at the same time.
But this is still not the end. While carefully designed metrics can tell us a story of how effective the team is, it will not reveal most of the small details that influence this big picture. Consider all activities happening during the typical development process… do you know how much time is lost on inconsequential interactions? How many times is the task reassigned? How long sprint planning takes? Why do your daily standups drag on or how much time is spent at meetings waiting for each other? Probably you have no idea. Coding may be difficult in itself but when you think about the full picture of the software development you will quickly find that it’s much harder and actual coding may be a much smaller part of total effort than any of us would like. In the world of omnipresent data, this is the ultimate border to be breached - knowing what in the developer’s day-to-day routine is taking his/her precious time and how to improve that, relieving the team and giving back time for activities that move the product forward.
It is worth noting that all of this data gathering and analyzing is a tedious and slow process and choosing what and how to track is often dependent on a large number of variables. This also changes dynamically throughout the project's lifetime. If we track too many metrics our signal to noise ratio will be high, conversely if we track not enough of them they might be useless or misguide us. It would be really beneficial if we could easily automate all of that, up to the point where every and only important information is available at hand, delivered to the team each day, letting them focus on making proper decisions instead of drowning in the ocean of charts, graphs and logs.
Of course, these are just rough guidelines of what to look into and as always, should be taken with a grain of salt - in the end, it is the dev team that eventually knows best what applies to their situation. Just remember that as with anything else in IT - whatever you do, make sure it is automated.
I develop apps for the sheer need of creation, and I love to share my thoughts and give back to the community. I have experience working with super small teams and enormous ones as well. No matter the size, the best way to contribute to a project is to improve the effectiveness of its processes.
.png)
.png)
