Key takeaways from this article:
- Due to the trend of protecting users online, we have less and less data about them. But this does not necessarily mean a shortage of user data to work with.
- You always need to be aware of what you have measured, what factors affect the measurement, and what data you are comparing the measured numbers against.
- The most important thing is correctly setting up measurement of everything that matters for a given project.
Why we are getting less and less data
In recent years, several changes have taken place that offer greater protection of user data — but at the same time make life harder for analysts, marketers, project managers, developers and others.
The most significant changes and tools that restrict data collection include:
- the GDPR regulation, in force since 2018, which governs what data about users you can collect and process, and in what circumstances,
- cookie banners, through which you have been required since January 2022 to ask users for consent to store preference, analytics or marketing cookies and other data,
- shortening of stored cookie lifespans used for measurement, which all browsers are doing to varying degrees,
- the Schrems II ruling from 2020, which restricts the transfer of personal data from the EU to the US (based on the so-called Privacy Shield),
- ad blockers that block advertising and sometimes entire data measurement.
On top of all this, add the natural loss and error rate in measurement, which occurs for example due to measurement code loading failures or poor connections.
How to deal with it. And is it actually a problem?
The above limitations do not mean things are wrong or that you cannot measure anything. You just cannot expect to see 100% of user data in Google Analytics or any other tool. That was never the case even before the introduction of new cookie banners and regulations.
A lower consent rate (the ratio of users who consent to cookie storage versus those who view the page) does not necessarily mean unreliable data that you cannot trust. In most cases, users whose consent you do not have behave on the website in the same way as those whose consent you do have.
"If you measure 70% of users, you can reasonably assume that the remaining 30% do not behave significantly differently."
With some exceptions. For example, after a one-off larger investment in acquisition advertising and new marketing channels, you may be bringing visitors to your site who behave differently from your typical users. You must take this into account when evaluating results.
The overall number of website users also matters. Tools like Google Analytics evaluate traffic sources, user behaviour and goal completion. For websites with thousands of visits and hundreds of completed goals (e.g. transactions or form submissions), you will logically have more data, and losing some of it will not affect accurate evaluation. Conversely, for smaller websites where goals are completed just a handful of times per month, this can be a complication and the resulting distortion is greater.
"If your measurement is set up correctly, even 50% of data can be perfectly adequate for evaluating the efficiency of online activities on larger projects."
How to use analytics tools correctly
The general rule is to use analytics tools for evaluating trends over time and the behaviour of different segments on the website, rather than for specific absolute numbers.
You will gain more by comparing ratios — such as conversion rate — than by comparing absolute numbers. You can also track rising and falling trends. Ratio metrics and trends will show whether things are working correctly and alert you to potential problems.
"Say out loud what each analytics tool is actually for. Then only ask it for that."
We also recommend comparing data from analytics tools with internal data — for example from your CRM. If there are significant discrepancies in transactions between Google Analytics and your CRM, or if year-on-year data in GA shows a decline while CRM shows growth, it may help to distribute actual revenue across all channels according to their conversion rate. However, such an approach is sensitive to external factors, such as one-off marketing activities.
What to watch out for when comparing data
When making year-on-year comparisons, also consider whether any regulation came into effect that affects measurement (such as the cookie banner). Be equally careful if you switch analytics tools. Differences in data may be caused by:
- different metric definitions,
- different data collection logic,
- or the technical workings of the measurement codes themselves.
Currently, for example, the transition to Google Analytics 4 and the end of Universal Analytics measurement is being addressed (for free Google Analytics, this takes effect on 1 July 2023). We recommend deploying GA4 as soon as possible.
Both tools use such different measurement logic that the resulting data will be very different. Key differences to bear in mind include:
- A session has the same 30-minute duration in both tools, but in GA4 it is not interrupted at midnight or on arrival with a new UTM parameter.
- At the time of writing (late 2022), GA4 also has relatively limited filter functionality.
- Attribution in GA4 is data-driven, meaning the tool distributes traffic and transactions based on data. This should allocate traffic and conversions across sources better and more accurately than the original rule-based model.
- GA4 redistributes data from direct traffic across other channels retroactively for up to 7 days, which UA does not do.
- In both tools, conversion sources are attributed according to different models.
For even better data evaluation, it is ideal to have GA4 connected to BigQuery and evaluate data outside the interface.
Also be aware of differences between GA and marketing platforms such as Google Ads, Facebook or Sklik. Data will frequently diverge, and with cookie banners deployed the discrepancies will grow further. This is not an error — it is a difference in attribution models and attribution windows. For example, in Google Ads a conversion is attributed to the date of the click, while in GA it is attributed to the date the conversion occurred. We therefore recommend comparing data using the Conversions and Conversion Value metrics (by conversion time).
How to ensure sufficient data
More often than regulations restricting data storage and processing, the problem lies in measurement configuration. To obtain as much data as possible and interpret it correctly, focus on the technical correctness of measurement and on measuring everything you actually need for evaluation — for example the entire user conversion journey, not just part of it.
There are also options for measurement without consent, namely:
- Consent mode, defined by Google, which allows certain tools (Google Analytics, Google Ads, Sklik) to measure some data even without consent (a small percentage of data, mostly not visible in Google Analytics anyway). One issue is that data from Google Analytics is still sent to the US (including user IDs). Some countries therefore prohibit or restrict this option (allowing only anonymised data to be sent).
- Cookieless measurement — measurement without cookies or user data. This means you get an overview of page views and events on the website, such as transactions. However, you cannot link these to any user data or the sources of their visit.
