Countly - Measurement Platforms

Measurement Platforms Full Review

Countly

www.count.ly

Type: Customer Journey Attribution
Platforms: Mobile, Web (Desktop)
Core Offering:
- On premises attribution solution, white label, supporting 1^st and 3^rd party plug ins

Countly was founded in 2013 alongside the leading attribution SDKs (appsflyer, adjust, kochava, branch) but their approach was to offer Advertisers with on premises attribution – allowing advertisers to install the software on their own servers and have the ability to develop further features and manage ad network integrations themselves.

When Facebook (currently the most important media channel for mobile advertisers) created the “MMP program” – Count.ly was not part of this, hence, any advertiser using Count.ly must still use a 2^nd attribution SDK to track attributions on Facebook if they don’t wish to integrate the facebook SDK.

Countly made a strange bet that advertisers will prefer having their own attribution stack and technology rather than licensing any of the solutions offered in the market.

During 2013, there were indeed more attribution providers – including attribution offered by ad networks (like Fiksu, TradeMob, Millennial Media, Jumptap, etc.), free attribution offered by the large platforms (Google, Facebook), and attribution offered by licensed services (appsflyer, adjust, kochava, branch)

Count.ly seems to have made the wrong bet, as while Appsflyer grew to over 1,000 employees, Count.ly over the past 9 years only employes 40 people

During a time where mobile was exploding – no one needed nor wanted to waste time in setting up their own attribution stack and maintaining integrations with over 3,000 ad networks.

With count.ly – setting up a postback seems to be an exercise in patience and persistency.

Table of Contents

Integration / Development Needed

Well….Count.ly is an on-premises solution, hence, you need to install it on premises.

There are plenty of online guides about what server specs you’ll need ($$$!) and a ton of instructions on how to setup the application on your servers.

There’s no mentioning if Count.ly supports multiple servers – so if you are an advertiser that plans to run advertising at high scale – note that you’ll need to probably do additional development yourself to make the software scalable.

There are various steps like:

Choosing the hardware you’ll need (count.ly recommends your own server farm)
Downloading the git files or running an installation via Digital Ocean
Installation via Docker
Configuring the DNS
Configuring an email delivery
Configuring server monitoring
Securing countly

Those are just the “basics”

There are also guides for:

Upgrading countly server
Deployment scenarios
Docker and Kubernetes
High availability deployment
Backing up
Configuration
Configuration of HTTPS and SSL
Installing let’s encrypt for HTTPS
Securing MongoDB
Cross device reporting

And all of this is before you even get to touch the SDK integration to track mobile traffic and conversions.

The good news is that Countly offers SDK for multiple platforms:

iOS •Android SDK (Java)
Web SDK (Javascript)
Node.js SDK (Javascript)
PhoneGap & Cordova SDK (Javascript)
React Native SDK (Javascript)
Flutter SDK (Dart)
Windows Phone & Windows SDK (C#)
C++ SDKs (C++)
Unity SDK (C#)

The bad news is that this is how much documentation your engineering team will need to review to get one sdk initiated:

Our engineering teams estimated the integration process at 5 man-years.

We decided that only 4 of us will go ahead and install countly, and the rest of us will share access in order to evaluate the platform for the sake of review.

Onboarding

Once you’ve completed the exhausting integration process, you should be ready to get started.

For this, Countly offers plenty of online user guides to explain how to use the dashboard

The dashboard experience is beautifully made. It’s relatively intuitive to understand and use, we also like the default theme used for the graphs and figures.

Dashboard like the “time of day” are very nice to look at and make sense to review:

One of the most obvious things you want to use a measurement platform for is to launch campaigns with media partners, ad networks and ad platforms, providing postbacks or at least some sort of performance notifications to the platforms you’re working with, so that they can also have an indication of how many conversions were attributed to them.

Most MMPs call this “integrations” or “Partnerships”.

With Count.ly – this is not straight forward. Countly was designed to provide analytics and attribution to the advertiser / developer – but the whole topic of providing postbacks seems to have been forgotten.

To work with any ad network / ad platform you will need to setup a tracking postback per campaign!!!!

We searched for the most important ad networks and ad platforms and there were no ”set configuration” for those.

So if you want to setup postacks, you’ll need to do this per campaign, per ad network and continue doing this every time you setup a new activity

Support

Countly scored the lowest out of all measurement platforms when it comes to support.

You can only speak to a human when you are interested in licensing the software – but once you do – don’t expect to hear from a person ever again.

Countly will direct you (via email) to their online support and community – which indeed is very detailed, but can be a nightmare to your engineers and yourself.

If countly completely crashes – that’s on your team to figure out. (this happened several times to us).

If countly crashes your application once you submitted it to the app store – good luck.

If you cannot get your postbacks to work for Facebook / Google / IronSource / Applovin / Vungle / Unity , etc….you have absolutely no chance to get any help other than submit a thread in the community and wait for an answer.

FYI – The community is not very active

Useful insights

The idea behind countly was similar to that of Appnexus – allowing anyone to develop 3^rd party plug ins. In reality, there are only 22 plug ins available for countly and all of them were developed by countly.

Unfortunatly, none of the plug ins and the platfrom itself does not provide any insights.

The core of the platform is reporting, analytics, attribution.

Accuracy

The Countly measurement platform software is completely exposed. Given that there is no oversight and cross advertiser learning, every advertiser is there for themselves.

Since you’re suppose to install counly yourself, we’ve also seen strange discrepancies that no one could explain to us.

None of us managed to get countly’s reporting to match our own app store numbers. Not by a long-shot.

The differences were between 70% and 2,300% !

Secure

Countly got the lowest score in security. Your engineering team will need to work exceptionally hard just to get SSL and HTTPS to work for the platform.

Anyone can easily spike up the traffic by generating false clicks and false conversions.

There is no server side validation and no checksum between server and SDK making the platform software penetrable to any person seeking to do harm.

The countly community is full of unanswered questions about fraudulent activities, millions of clicks coming from the same IP, installs that are happening without impressions or clicks counted and conversions showing up from nowhere.

Countly is the gift to any fraudulent publisher.

Privacy compliant

As long as you are compliant with privacy regulations, countly will follow your steps.

Countly as a software does not process anything on their own servers – so it’s completely up to you to manage compliancy with any privacy regulations, whether if it’s: GDPR, CCPA, COPPA, Apple ATT, Apple ITP, Google sandbox.

Unbiased vs. Biased

Countly is very much an unbiased solution.

It’s not easy to integrate nor operate – but there is no bias in the pricing model nor the platform itself as far as we’ve seen.

Pricing Model

Countly offers a few versions of the platform – an open source community based free version, and an enterprise package.

The enterprise software pricing is a monthly licensing package cost, with no additional costs for volume or spend.

We appreciated the true software pricing model.

But – It’s unfair to compare countly with all of the other measurement platforms we reviewed, as to install, setup and operate countly, you will need a fully dedicated engineering team!

Price

Countly’s pricing was below $5,000 for the licensed package, but most of us needed at least 4 full time engineers to get things going, and an additional engineer on full time to maintain the platform. Overall, Countly ended up costing close to $20,000 per month and the outcome was completely inoperable given the discrepancy and lack of security.

Review Summary:

Countly is an on premises software which means you need to install it on your own servers. What’s very cool about it is that it allows any advertiser to construct their own attribution model, including developing on-top customized plug ins. As an on-premises solution, countly is a pretty cheap solution – but you’ll need a full team of developers to set things up and integrate your data, as well as at least one full time developer to maintain the platform. Every additional network you’ll want to work with will force you to more development work. The mobile app solution requires an SDK , collecting user level data. The solution is not promising for a post IDFA world.

We’re actually not sure why bother with on-premises attribution solutions. Attribution is a commodity these days. It was a huge stretch to get going with countly, when there were such prominent solutions available.

Pros & Cons:

On-premises solution is a unique offering
Requires heavy tech resources to setup and use
Support was pretty awful. There was no documentation available, and we had to go through many (!!!) calls to get things going.
Countly doesn’t handle SKadNetwork postbacks. You’ll need to use another solution for iOS

Measurement Platforms Review for Countly:

Support	★★★★☆☆☆☆☆	Onboarding	★★☆☆☆☆☆☆☆
No Dev Needed	★★☆☆☆☆☆☆☆	Privacy Compliant	★★★★★★☆☆☆
Accuracy	★★★★☆☆☆☆☆	Pricing Model	★★★★★★★★☆
Useful Insights	★★☆☆☆☆☆☆☆	Price	★★★★★★★☆☆
Unbiased	★★★★★★★★☆	Secure	★☆☆☆☆☆☆☆☆