DATA ANALYSIS
Quantitative data about users and usage is part of design research. But valuable data insights usually can't be found in the standard numbers such as exit rates or download times. It requires careful analysis of trends and context. That's why setting up a plan for how to measure success is key.
Effort and expertise
We had a Data Analyst in the UX team at gemeente Amsterdam, and we made sure he joined us for every online form that we set out to improve. Together we created a plan for how to get measure progress while we started to change parts of the funnel. This was no easy task the first time we did this but we got better at it each time. It taught me that data analysis can be very useful, but only with the help of data experts, and only if you put serious time and effort into it. It also added this question to the mix: How can we design solutions in such a way that it helps to measure success?
Metrics can be chaos
When you start to improve an existing product or service, looking at Web statistics and other quantitative numbers about usage and users can seem a good way to start. But without context and without knowledge about data analysis it's like looking at chaos, and misinterpretation of the data quickly occurs. To find useful and trustworthy metrics you really need to know what you're doing, or get an expert involved.
Automatic dashboards
Once you've managed to understand metrics and context, dashboards can be a nice to have for monthly and quarterly reporting A dashboard can provide a quick overview of important KPI's. At gemeente Amsterdam we wanted a dashboard for each online form with average monthly customer feedback ratings and the amount of transactions online versus offline. Unfortunately, we weren't able at the time to automatically generate the data for this, so this nice to have was never implemented.
