Data Discrepancies

Trying to make sense of the amount of data you collect in Kissmetrics can be daunting, especially if you notice a difference between our data and data from your databases or another analytics platform. This page discusses the following:

  • Reasons we might not capture 100% of the data
  • How to determine where the root of the discrepancy may come from

Best Effort Accuracy

Kissmetrics, like all analytics vendors, makes its best effort to achieve 100% accuracy. However, despite all of our best efforts, there are many things that are not within our control. These include:

  • Slow connections
  • Browser software that is outdated or has limited features (mobile browsers)
  • Privacy-protection software
  • Browser settings or modifications
  • Spiders and bots that can throw off counts
  • When one person uses multiple computers or browsers
  • When multiple people use one computer or browser
  • Firewalls
  • DNS issues
  • Hardware failures

Entire whitepapers are dedicated to these challenges. But in spite of the challenges, we still continue to try to innovate and get closer and closer to the goal of delivering 100% of your data.

What Can I Do?

There are some events, especially Revenue-related events, where it would be very important to ensure that we capture every instance. For these events, it’s effective to use our server-side libraries or other integrations, if possible. Though this means it can take more time to implement, you get two main benefits:

  1. You have a definitive source of data (your own database) on which to ground your Kissmetrics data.
  2. Our server-side libraries include mechanisms for retrying the sending of data, if for whatever reason our tracking servers did not receive everything on the first attempt.

You can refer to this article for combining data from our JavaScript Library with our other data libraries.

Steps For Investigation

If you suspect that Kissmetrics’ reports disagrees with the data you were expecting, these are the types of investigative questions our support team would use to approach the situation.

1. Let’s look at the people affected.

For almost all of our tools, you can segment all the way down to the individual person level, by segmenting by Customer ID. At this point, our support team looks for some sample people who are affected by this data discrepancy to try to find patterns in what was recorded, what wasn’t recorded.

Additionally, our People Search lets you narrow in on sample people affected.

Of the people you’re reporting on, are there any patterns you can see?

2. Which events or properties are involved?

First off, are we looking at the right event or property? If you weren’t the one who originally implemented all of the events, you might discover some similarly named events in your account. It’ll help to sync up with the person who implemented to get on the same page.

Does this event represent the action I’m interested in?

3. Let’s check if the events are properly implemented.

When you step through the expected flow, does your activity trigger the events?

  • Perhaps the easiest way to verify your events work is to use our Live tool. If you leave Live open, you can step through.
  • For our more advanced users, you can even look for network activity to our API. You’ll see this in the form of HTTP requests formatted according to our API specifications. This is the format that you’ll see in the logs generated by our server-side libaries.
  • If you’ve set up the events through the Event Library, here are some Event Library tutorials that may help indicate whether you have set up the rule properly or not.
  • Again, for the events our JavaScript handles on its own, we explain how those events occur. Same goes for any direct integration we’ve done.

Do we consistently receive an event when someone does this action?

4. Which reporting tool is being used?

Kissmetrics provides a number of specialized reporting tools, each with their own options to present the data in their own way.

  • If in doubt, we try to explain how our tools work in our tools section.
  • One other page to note is our property options page, which addresses the different ways you can group people into segments using the properties we’ve recorded.

Do you understand what our reporting tools present to you?

5. Ask us!

If you’d like some help with the above steps, please contact our support team! We’re happy to help.

What Other Analytics Professionals Say About Data Discrepancies

Ensure your analytics strategy is to reduce data inaccuracy as much as possible. Don’t focus on getting 100% accuracy. That does not exist.

I believe that.

There are no “right numbers”. I am a part of a generation that believed that and we always tried to reach that goal (I grew up in the world of data warehouses and business intelligence and ERP and CRM systems). But painfully I have learned that you can either focus on that, or you can use the data you got. Analytics data gives you 900% more information than you have through traditional channels. It is only 90% “right”, but the missing 10% is outweighed by the fact that you can now make decisions that are so much better informed.

It will take a while for the current crop of business leaders to “get it” – and sadly many many web analytics practitioners / consultants / vendors to get it. We need to realize there is more money to be made not peddling our services that make things “accurate” but rather peddling our ability to find raw awesome insights (whatever the tool the company has).

–excerpt taken from Avinash Kaushik’s (Author of Web Analytics 2.0, Web Analytics: An Hour A Day, and Digital Marketing Evangelist for Google Analytics) blog post on data reconciliation

Kissmetrics Status

If there are any known site-wide issues, we will post a notice to status.kissmetrics.com. This message will also appear at the top of the website when you log into your account.

Is anything on this page unclear? Suggest edits on Github!