 |
Brent DykesDirector of Industry ConsultingAdobe Systems |
You may have heard the terms “reporting” and “analysis” used as though they were interchangeable. While both of these areas of Web analytics draw upon the same collected Web data, reporting and analysis are very different in terms of their purpose, tasks, outputs, delivery, and value. Without a clear distinction of the differences, an organization may sell itself short in one area (typically analysis) and not achieve the full benefits of its Web analytics investment.
Most companies have analytics solutions in place to derive greater value for their organizations. In other words, the ultimate goal for reporting and analysis is to increase sales and reduce costs (i.e., add value). Both reporting and analysis play roles in influencing and driving the actions, which lead to greater value in organizations.
For the purposes of this blog post, I’m not going delve deeply into what happens before or after the reporting and analysis stages, but I do recognize that both areas are critical and challenging steps in the overall data-driven decision-making process. It’s important to remember that reporting and analysis only have the opportunity of being valuable if they are acted upon.
Purpose
Before covering the differing roles of reporting and analysis, let’s start with some high-level definitions of these two key areas of analytics.
>> Reporting: The process of organizing data into informational summaries in order to monitor how different areas of a business are performing.
>> Analysis: The process of exploring data and reports in order to extract meaningful insights, which can be used to better understand and improve business performance.
Reporting translates raw data into information. Analysis transforms data and information into insights. Reporting helps companies monitor their online businesses and be alerted to when data falls outside of expected ranges. Good reporting should raise questions about the business from its end users. The goal of analysis is to answer questions by interpreting the data at a deeper level and providing actionable recommendations. Through the process of performing analysis you may raise additional questions, but the goal is to identify answers, or at least potential answers, that can be tested. In summary, reporting shows you what is happening while analysis focuses on explaining why it is happening and what you can do about it.
Tasks
Companies sometimes confuse “analytics” with “analysis.” For example, a firm might be focused on the general area of analytics (strategy, implementation, reporting, etc.), but not necessarily on the specific aspect of analysis. It’s almost like some organizations run out of gas after the initial set-up-related activities and don’t make it to the analysis stage. In addition, sometimes the lines between reporting and analysis can blur -- what feels like analysis is really just another flavor of reporting.
One way to distinguish whether your organization is emphasizing reporting or analysis is by identifying the primary tasks that are being performed by your analytics team. If most of the team’s time is spent on activities such as building, configuring, consolidating, organizing, formatting, and summarizing, then that’s reporting. Analysis focuses on different tasks, such as questioning, examining, interpreting, comparing, and confirming. (I’ve left out testing as I view optimization efforts as part of the action stage.) Reporting and analysis tasks can be intertwined, but your analytics team should still evaluate where it is spending the majority of its time. In most cases, I’ve seen analytics teams spending most of their time on reporting tasks.
Outputs
On the surface, reporting and analysis deliverables can look similar: lots of charts, graphs, trend lines, tables, stats, etc. However, there are some subtle differences. One of the main differences between reporting and analysis is the overall approach. Reporting follows a push approach, where reports are pushed to users who are then expected to extract meaningful insights and take appropriate actions for themselves (i.e., self-serve). I’ve identified three main types of reporting:
1. Canned reports: These are the out-of-the-box and custom reports you can access within the analytics tool or that also can be delivered on a recurring basis to a group of end users. Canned reports are fairly static with fixed metrics and dimensions. In general, some canned reports are more valuable than others, and a report’s value might depend on how relevant it is to an individual’s role (e.g., SEO specialist vs. Web producer).
2. Dashboards: These custom-made reports combine different KPIs and reports to provide a comprehensive, high-level view of business performance for specific audiences. Dashboards might include data from various data sources and are also usually fairly static.
3. Alerts: These conditional reports are triggered when data falls outside of expected ranges or some other predefined criteria is met. Once people are notified of what happened, they can take appropriate action as necessary.
In contrast, analysis follows a pull approach, where particular data is pulled by an analyst in order to answer specific business questions. A basic, informal analysis can occur whenever someone simply performs some kind of mental assessment of a report and makes a decision to act or not act based on the data. In the case of analysis with actual deliverables, there are two main types:
1. Ad hoc responses: Analysts receive requests to answer a variety of business questions, which might be spurred by questions raised by the reporting. Typically, these urgent requests are time-sensitive and demand a quick turnaround. The analytics team might have to juggle multiple requests at the same time. As a result, the analyses cannot go as deep or wide as the analysts may like, and the deliverable is a short and concise report, which may or may not include any specific recommendations.
2. Analysis presentations: Some business questions are more complex in nature and require more time to perform a comprehensive, deep-dive analysis. These analysis projects result in a more formal deliverable, which includes two key sections: key findings and recommendations. The key findings section highlights the most meaningful and actionable insights gleaned from the analyses performed. The recommendations section provides guidance on what actions to take based on the analysis findings.
When you compare the two sets of reporting and analysis deliverables, the different purposes (information vs. insights) reveal the true colors of the outputs. Reporting pushes information to the organization, and analysis pulls insights from the reports and data. There may be other hybrid outputs, such as annotated dashboards (analysis sprinkles on a reporting donut), which appear to span the two areas. You should be able to determine whether a deliverable is primarily focused on reporting or analysis by its purpose (information/insights) and approach (push/pull).
Next: Why context is king.
