Data Collection: Tools and Technologies to Track Data From Primary Sources

👋 Hey there, welcome to the Data-led Almost Weekly newsletter! In this issue, I am sharing tools and technologies to get data collection right.

This is a continuation of the previous issue so if you didn’t get a chance to read through it or if you’re new here, I highly recommend checking out issue #2 first.

Implementing a data tracking project successfully requires a set of people, processes, and tools. People and processes were covered in the last issue, so in this one, let’s look at tracking tools and technologies.

It’s good to keep in mind that the terms tools and technologies are not used interchangeably – tools are specific products that fall under one or more technologies or categories.  

Tracking tools fall under two main categories – Customer Data Platform or CDP (that does a lot more than tracking) and Customer Data Infrastructure or CDI that is purpose-built for tracking.

Disclaimer: One of my goals with Data-led Academy is to remove bias from tooling recommendations. The tools mentioned here are not necessarily the best tools in the categories they operate in, nor are they the only ones – they are either market leaders or are relatively new and gaining traction in the data communities.

A CDP is an all-in-one solution that not only takes care of data collection from primary or first-party sources (website, web app, and mobile apps), but also has the capability to ingest data from secondary or third-party sources (external tools used for sales, marketing, advertising, etc.) 

More importantly, a CDP’s core capability is identity resolution which makes it possible to create user segments or audiences by combining customer data from multiple sources and syncing those segments to external tools. [1]

Below I have only mentioned horizontal CDPs that are industry-agnostic but there are many vertical CDPs that cater to the needs of specific industries such as SaaS or ecommerce and they are definitely worth exploring if you’re looking to invest in one. 

Segment is by far the most popular CDP vendor on the market (which led to its acquisition by Twilio for $3.2B) but what’s interesting and not very well-known is that Segment offers multiple products, and one of them, Personas, is their CDP offering which is sold as an add-on to their core product, Connections.

Most people refer to Connections when they talk about Segment and it has long been the go-to tracking solution for companies of all sizes. So even if you don’t need CDP capabilities, Segment Connections can take care of your tracking needs.

Segment also offers a data governance tool called Protocols which is also sold as an add-on to Connections. 

mParticle is the most popular Customer Data Platform after Segment. However, unlike Segment, mParticle only offers a bundled solution that takes care of tracking, governance as well as identity resolution and audience building.

mParticle is a strong Segment alternative used by some of the largest tech companies today. However, since mParticle hasn’t unbundled its offerings, the cost can be prohibitive for small and medium-sized businesses.

CDI is not yet a well-defined category and is used in various contexts today. However, tracking is a core component of Customer Data Infrastructure and therefore, it makes sense to categorize purpose-built tracking tools as CDI. If you have better ideas, I’m all ears!

It’s helpful to keep in mind that all the CDI solutions mentioned here are, one way or the other, alternatives to Segment Connections. But unlike Connections, these tools don’t ingest data from secondary or external data sources.

So, in alphabetical order, here are various CDI solutions worth exploring:

Avo enables you to create a smart tracking plan using an easy-to-use interface and then generates code for your developers to implement tracking based on the events and properties defined. This gives product managers more control over the taxonomy and eliminates the scope for errors during implementation.

Freshpaint offers a hybrid solution wherein besides tracking data via code (which is always recommended), you can set up auto-tracking that gathers data without code (also known as implicit tracking). The hybrid approach brings more flexibility to teams with limited engineering resources. [2]

Iteratively is an alternative to Avo and they both have a similar approach to tracking. They both offer limited integrations but integrate with Segment as a destination, allowing you to further send data to all the destinations supported by Segment.

This might seem odd as Iteratively and Avo are also alternatives to Segment Connections but it makes sense to use them in conjunction with Connections until they expand their destinations library. 

MetaRouter is a relatively new solution that is focused on server-side tracking and offers private cloud and on-premise installations which are ideal for larger companies with stricter norms for data privacy, security and compliance. [3]

Rudderstack also offers multiple products but their core product is exactly like Segment Connections. However, besides offering a cloud version, RudderStack is open-source allowing you to deploy it on your own servers if you have the resources to maintain an open-source product. 

Snowplow calls itself a behavioural data collection platform that is also open source and also offers a cloud version. Snowplow’s approach is different in that it only syncs data to data warehouses and doesn’t support any other cloud destinations.

Also, implementing Snowplow requires expertise in its proprietary technology and is therefore only suitable if your company has a dedicated data team.

I know this is a lot to digest but also hope that it helps you understand the data collection landscape!

This issue covered tools and technologies to track data from primary or first-party sources – your website, web app, or mobile apps.

The next issue will be focused on collecting data from secondary or third-party sources – external tools used to engage or transact with customers.

Okay enough said! Do send in your questions by replying to this email or via Twitter.

Thanks for reading! 🙏

[1] What exactly is a Customer Data Platform?

[2] How is implicit tracking different from explicit tracking?

[3] How to decide whether to track data client-side or server-side?