Measurement Platforms Full Review

MetricWorks

http://www.metric.works

MetricWorks was founded in 2016 as a different company (“Target Circle”).

Target Circle was an affiliate marketing company, that pivoted their activities in 2019 as MetricWorks.

The promise was very big – Marketing Automation + LTV Prediction + Top Down Incrementality measurement. Sounds like the best measurement platform – but no. far from it. MetricWorks makes promises you cannot cash in.

MetricWorks has 10 employees. The company is HQ in Norway, but only 1 person is based there – rest are scattered in remote locations.

Integration / Development Needed

Lets just say that our tech teams were not so happy about integrating metricworks. For the sake of the integration, metricworks wanted us to provide them with 12 months of historical data.

The integration itself was very in transparent. At no point in time (even after getting “onboarded” have we ever understood what was done with our data.

After providing the unreasonable amount of data to MetricWorks – the “analysis” ended up taking over 4 weeks for MetricWorks to provide us with a set of “tests” we needed to do.

Tests included things like: “Stop Facebook for 10 days” which none of us appreciated.

The amount of data requested for integration is extremely large:

• Date
• Channel
• Campaign
• Source
• App
• Spend
• Country
• Install data
• Installs
• Session D0
• Session D1
• Retained users D0
• Retained users D1
• Revenue amount D0
• Revenue amount D1
• Revenue event D0
• Revenue event D1
• Unique payers D0
• Unique payers D1

Try pulling this amount of data for 12 months. We did. We failed. Multiple times.

After multiple failed attempts, MetricWorks told us that it’s fine if we only send historical data for 1 month. That sounded dodgy and unprofessional.

Onboarding

After the excruciating integration process, you would think that you are ready to get a login to a platform to see “top down incrementality insights” but all you get are meetings where MetricWorks will tell you what you should do in order to “increase incrementality based on econometrics”.

MetricWorks offer technology but provides just a consulting service.

We became extremely worried when MetricWorks asked us to create an account to what seems like a white label of an affiliation platform to allow them to run marketing automation to conduct the “experiments”.

What we noticed immediately in the platform was that all “channels” offered are hardcore affiliates – typically the ones you want to stay away from!

MetricWorks misses out, oh, only the top 50 ad networks used by most advertisers:

AppLovin, Unity Ads, Vungle, Snapchat, DV360, Bing, Liftoff, TikTOk, Apple Search, Tapjoy, Twitter, AdColony, Fyber, Digital Turbine, Mintegral, Outbrain, Taboola, Aarki, Jampp, Remerge, Adikteev, Kayzen and the list goes on and on…

Support

MetricWorks doesn’t provide any support documentation and no support platform.

There is no support policy – so what you should expect is at the grace of MetricWorks.

As most of the employees are only sales people – whenever you have a question about the “algorithm” or econometrics – you’ll mostly get answers that don’t answer your questions.

For the price Metricworks charges – it would have been nice if they hired a full time support team to provide reasonable answers.

And while MetricWorks is a Norwegian company – there is no-one to support you in European day time. You’ll need to wait until late evening to get any response from anyone.

Useful insights

The idea behind MetricWorks is to create cloned campaigns as a “synthetic control” and later Pause All Campaigns in a Major country for 5 days, measuring the difference.

And this was suppose to help us understand incrementality or the performance of our campaigns how exactly ?!?!?!

Nothing in what MetricWorks told us made any sense, and none of the people we spoke with were able to explain their “insights” to us.

The insights felt extremely superficial.

Basically what we were proposed continuously was to pause certain channels in order to understand if those channels created lift or not – but what happened, was the pausing a major channel was out of the question for all of us (i.e. no Advertiser will pause Google or Facebook for 5 days, unless they want to commit career suicide).

MetricWorks recommended that we would enable marketing automation so that their core system (i.e. the affiliate management system) will automatically turn off affiliates.

Well – any performance marketing still working with affiliate marketing in 2021, knowing that 90% of affiliate marketing generates fraudulent conversions, is probably not so worried about their marketing being “incremental” to begin with.

Metricworks “solution” seems like a way to sell their affiliate marketing platform.

We did want to show what metricworks used in their onboarding deck to help us understand how the platform works.

If you understand what this means – we’ll award you $1,000,000 in MetricWorks credit 🤪

Accuracy

MetricWorks provided wild “insights” after processing or analyzing our data.

Here are examples of one of our budget split and “insights” we received from MetricWorks, as well as the outcome of us accepting those tests

Budget split and performance before MetricWorks ”insights” , and what the insights were:

So, we went ahead and accepted the recommendation – and look what happened…

So – while we indeed spent less budget – our organics dropped, and performance for every channel also went down. The ”insights” we got from MetricWorks was easily challenged and did not contribute much to our marketing activity.

Secure

The insights we received from MetricWorks did not provide us with a level of trust we were expecting, and the fact that the person we spoke with kept insisting that “there is no connection between the affiliate marketing company and MetricWorks” thought we were fools. A quick google search taught us everything we needed to know.

MetricWorks gave us insights to spend more budget with a channel that we intentionally picked as we knew It was producing fraud results. The solution was based on the data already received from attribution solutions, and the fact that this specific “channel” had over 1 million clicks to generate only 100 conversions did not at all seem suspicious to MetricWorks.

More will be explained when we go into their pricing model and price…

Privacy compliant

This is the only point of review where MetricWorks scored well. During data integration (which took over a month) MetricWorks indeed did not request any user identifier data.

They were ok working with raw level conversion data that did not include IDFA, AAID or Cookies.

However, for the solution to work, MetricWorks did request sensitive data that included:

• Session D0
• Session D1
• Retained users D0
• Retained users D1
• Revenue amount D0
• Revenue amount D1
• Revenue event D0
• Revenue event D1
• Unique payers D0
• Unique payers D1

(Taken from MetricWorks integration documentation).

Some of us were very worried that this information could leak into the wrong hands, as MetricWorks is connected to an affiliate program – our biggest fear was affiliates would know about our user patterns and replicate this across networks, thus making us believe that a channel is producing good results, where in reality – it is producing fraud.

Those of us that work with ad networks that accept affiliate type publishers did see a strange surge in fake performance.

Unbiased vs. Biased

There was a strange Bias with MetricWorks. As a supposed software solution – none of us were given access to a software, but we only got recommendations during ”work meetings” where MetricWorks placed some random “experiments” for us (like stopping the activity in a certain country for 3-10 days).

The pricing model that is based on media spend showed that MetricWorks intention is not to help us optimize our spend while cutting out waste – but au contraire(!) – most of the insights made little to no sense, pushing our spend towards what MetricWorks deemed as “great sources” to spend with, eager to introduce us with random affiliates.

Pricing Model

MetricWorks pricing model is based on a percentage of your media spend. This was not what we expected from a solution that claims to be a technology solution for measurement of incrementality and marketing automation.

The price starts at 2% and goes down to 0.50% at a spend of $1,000,000 and above.

For some of us, this pricing model was not possible to even consider – as some of us had spends of over $100M with the “lowest” one having only a budget of $24M / year.

MetricWorks pricing is not suitable for any large spender, and the solution itself seems to be more suitable for small advertisers who have yet to understand the lack of value in affiliate marketing

MetricWorks does offer a discount based on two levers:

• Your willingness to provide a public case study
• Commitment and pre-payment (based on expected media spend) for one year, or 6 months

Within 14 days of contract end, you cannot opt out of the contract – and given that the integration took a LOT longer than 14 days (from MetricWorks side) – beware of this and we strongly recommend that you negotiate the ability to opt out of the contract within 14, 30, 45, 60 days, if you even consider signing up with MetricWorks.

Price

MetricWorks was the most expensive solution we tested (averaging at $48,100 per month!) and the least valuable solution we reviewed.

There’s nothing in the solution that accounts for it to be worth almost $50,000 per month, especially given that you have very little support, integration takes over a month, the insights are useless and the output and value is not helpful.

When it comes to measurement platforms – MetricWorks was a no from us.

Review Summary:

MetricWorks is a subsidiary of an affiliate platform called target circle. The company claims to offer incrementality testing, ltv prediction, and marketing automation. What we’ve seen is that the company offer an analysis of incrementality based on econometrics data, requiring at least three months of data or a full year of data. MetricWorks method of measurement incrementality was asking us to stop all advertising activities in a country in order to get a report that suggested which of the channels was incremental and which was not (?)

The claim was that metricworks has full marketing automation to facilitate the blackout tests – but we have never seen any dashboard in action. Only reports.

Pricing was a complete shock. The average price we paid averaged at $48,100 – the most expensive solution we tested. We were offered a 50% discount in retrospect if we would stay on as a customer.

Pros & Cons:

• Pricing model is based on media costs, and is very expensive for the value received
• There is no platform. The analytics is done by a team and insights are provided as a report
• MetricWorks wanted an insane amount of historical data
• Integration and onboarding took over 6 weeks
• The solution requires stopping marketing activities on major marketing channels (we were asked to stop campaigns on Google, Facebook, AppLovin, Snap…)

Measurement Platforms Review for MetricWorks:

Support	★★☆☆☆☆☆☆☆	Onboarding	★★★☆☆☆☆☆☆
No Dev Needed	★★★★★☆☆☆☆	Privacy Compliant	★★★★★★★★☆
Accuracy	★★★★☆☆☆☆☆	Pricing Model	★★☆☆☆☆☆☆☆
Useful Insights	★★★★☆☆☆☆☆	Price	★☆☆☆☆☆☆☆☆
Unbiased	★★★☆☆☆☆☆☆	Secure	★★★★★★☆☆☆