Best of Price Power Podcast from the Last 6 Months | Price Power Podcast Ep. 17

Speaker 9: My name is Jacob Rushfinn,
founder and CEO of Botsi and the

host of Price Power, the podcast for
subscription apps to grow their revenue.

So typically, I'm having a
conversation with world-class

growth or app leaders, but this week
we're doing something different.

This is a best of episode.

We haven't done this before, so
curious to know what you think.

We've got 12 guests, 16 clips, and one
hour of the most tactical advice from

the past six months of the podcast.

You're gonna hear from Alice Muir
Korczakova, Daphne Tideman, Katerina

Gams-Regler, Michael Perizek, Barbara
Galiza, Ashley Black, Shumel Lace,

Marcus Burke, Lucas Moskin, Gabe Kwakye,
Xavier Baez, and Anthony Scarpaci.

Let's go

Jacob: one concept that that's super
interesting to understand better uh to

kind of drive retention early engagement
monetization I is this strategic friction

Alice Muir: Mm-hmm.

Jacob: How how do you think about that
And maybe you can give some examples of

how this worked and people can use it

Alice Muir: Yeah, so I think
the, the best example that I

have of that is MyFitnessPal.

So MyFitnessPal basically, um, I think
most, I mean, it's a household name now

I think many people know MyFitnessPal.

Um, but for those that don't, it's
basically an app that helps you to

track, uh, calories basically and other
kind of, uh, macro your macros and

other health and nutrition information.

And it's targeted at people that wanna
get in shape or they wanna lose weight,

or they have a goal around that.

Um, and what MyFitnessPal
originally did was they had a lot

of stuff on the free tier, um,
including barcode scanning, right?

Um, so essentially you could manually
input your calorie information

or your nutrition information.

Um, and you could also for, for speeds
and, and, um, efficiency, you could also

use a barcode scanner to do that as well.

And then behi, what they had behind
the paywall was more premium kind

of information about like recipes
or going deeper into like macros

and like this type of stuff.

Right.

Um, and like I would say that there wasn't
a lot of strategic friction there because

essentially a user that wanted to lose
weight, who was focused on their, their,

you know, ca the, the simple calories
in ca calories out, they could pretty

much do everything with MyFitnessPal
and they didn't have to subscribe.

Right.

And then a few years later, as
MyFitnessPal grew, they kind of cottoned

onto this and they realized, right.

Okay.

Actually one of our most
valuable features, which is

the speed and efficiency part.

Um, is free.

So let's put that behind a paywall.

And I think I understood their logic.

I think it was very smart what they did.

I think they did it far too late
because customers, um, or users were

already expecting to have that for free.

Right.

And, and it had been
that way for, for years.

And so, like, um, I, I worked with
MyFitnessPal back when I worked at

Feature, and, and I always think about
that, um, example, when I think about

strategic friction or I'm trying to
explain it, and it's exactly as you

said, it's, it's helping users get
like 80% of their problem done, but

you're leaving like the last 20%.

If they really want to complete their
problem, um, or they really want to like

do something well, and it's out like
something that's outcome driven, they

need to upgrade to be able to do that.

So if we go back to the
MyFitnessPal example, you know.

What there with strategic friction,
a user could, you know, still input

their calorie information or their
nutritional information, but if they

wanted that speed and efficiency part,
they actually needed to be a subscriber

to do that so they could do it in a
way that wasn't painfully frustrating.

Jacob: this connects to how
you're talking about, um, or

differential between time to first

Daphne Tideman: Yeah.

Jacob: versus time to core value as

Daphne Tideman: Yeah.

Jacob: of activation.

Can you tell me more about
that, that framework?

Daphne Tideman: Yeah, so that's really
inspired by the kind of like SaaS

product led, uh, uh, growth space.

Uh, I think we Bush talks about a
similar concept also in his book.

Um, but I'm not sure if he's the one
who originally came up with it 'cause

it's been cited in different places.

But I think it's an interesting
framework, uh, just 'cause I don't

want to claim for something that
I haven't embedded in anyway.

Um, the time to first value.

That's kind of that initial, like what
we were talking about earlier, that mini

like little aha moments where Houston
might not have gotten the full value,

but they see that app could help them.

It's like, this is for me, this
is something that could help me.

It's like, you know what Noom
does with a personalized plan and

reassuring you throughout the,
uh, weight loss app just for.

Not familiar.

Um, they give you a personalized plan.

They reassure you, uh, throughout it.

They ask you questions
and teach you things.

And those are little
moments of perceived value.

And they could be action based, but they
could also just be learning something.

And that's, we want that time to first
value really to happen in a first session,

and ideally happened quite quickly that
we built enough trust to keep them going.

Um.

And then the time to core
value, that does take longer.

I think for most apps, like it probably
does take a few days to get there.

Um, and remember I'm working with
wellness apps, so the time to first

value is, uh, to core value is probably
slightly longer than some other apps.

Um, for some apps it could be on day
one already, but it's like, usually

it's that user is doing something
repetitively and feeling that benefit.

So actually, you know.

Have committed to doing, you know,
a meditation or two already and

are feeling, starting to notice a
difference and they're really building,

starting to build a habit around it.

Um, so like, for example, for blinkers.

The first value could be, oh, you see that
you could actually read a book and feel

convinced, like, Hey, I can actually read
a summary and I can make time for this.

It's really easy to use.

Fits into my lifestyle.

There's a book that I wanna read, and then
you're starting to get excited about it.

But like the time to call a value might
be either finishing that first book

or having a free day streak and what
that core value is, or how many times

that is, is a little bit dependent on
what actually predicts that retention.

Jacob: Do you think, um, breaking
those different moments up

allows teams to be more focused?

Daphne Tideman: I think it makes it
easier to understand that you don't

have to have a big thing in the
first moment, and it also makes it

easier to actually measure the timing
of these things and see like, hey.

If the time to core value is too long,
we probably need to get them first to the

first value a bit shorter and quicker.

Because usually that's then what's taking
too long that they're not seeing the first

value and we need to shorten that down.

And I think it also helps 'em like
kind of break it up into stages versus

seeing it as like one flat metric.

'cause I think that's the
other challenge is like.

It isn't just one metric.

Often in activation there's often
like a first step or second step.

And that's what I mean with meant early
on with that 2D versus like three day

thinking of like realizing like it
isn't just this, like hey, we need to

do one little thing in their retain.

No, it's usually a few different things
and getting them into, you know, a setup

where they are actually gonna use it.

So like.

For example, for a, uh, time
blocking app, the, uh, first value

might be actually seeing like, Hey,
I've managed to block everything.

Get myself set up.

This is exciting.

Like, I can choose exactly
what I've been struggling with.

I feel like it will actually help me.

But then the core value might be
having like a first full session,

or it might be having two sessions.

So I think having those different
moments helps you understand where you're

losing people and what the issue is.

Jacob: I've often heard, uh, and
kind of used this myself in terms

of like a setup moment, uh, aha
moment or activation and then

Daphne Tideman: Yeah,

Jacob: moment,

Daphne Tideman: yeah, yeah.

That's another phrase used often or.

Jacob: And so we talked a lot about
like what, um, to think about activation

metrics, how to influence this, but,
but how do you actually test whether,

um, an activation metric that's real or
vanity or actually influencing anything

Daphne Tideman: Yeah, so this is
where having good data tracking

is, is kind of the prerequisite.

I think that's one of the biggest
challenges I see with this is that

if we are not tracking the right
things it, and we can't backtrack

it, then it's gonna take a while
before we can get this confidence.

But what I'm looking for is
like, okay, activated users are

retaining significantly better.

When we test that across different like
cohorts and channels, we understand that

like, hey, this is consistent across most
of the cohorts, most of the channels.

It isn't just like one, like high
because sometimes you have it, for

example, that, um, users who come
through like a strong organic source,

like for an influencer that they trust,
they might naturally kind of retain,

uh, better or do a certain action
because they've been told to do that.

But it's not predictive of.

All users.

Um, and then I'll look at like,
okay, does this metric actually

also improve the downstream metrics?

So what we also talked about, um, where
create, if they're activated and using,

but if they're not paying, um, that's
probably not going to be like helpful.

So I think those are kind
of the things that I.

We'll look for and then, um, without
going on a user interview around change

again, um, I would say they're also very
powerful and kind of like sense checking.

Is that also in the user's mind?

What matters to them?

Is that actually what
they perceive as value?

Because it might be.

I think there's a difference
between what you need someone to

do to get value out of your app
versus what someone sees as value.

And both are important, but.

I would consider like a good
activation metric to be also

reflective of them getting value,
not just you getting them set up.

Jacob: how important is it
to think about your pricing?

Uh, how big a lever is pricing.

Yeah, we'll start there.

I have a few.

Ekaterina Gamsriegler: Mm-hmm.

Definitely a big lever.

Very powerful, massive.

So, um, however, um, I.

Most of the apps I've personally worked
with, and I would say been exposed

to, like throughout my career, usually
start monetizing very cautiously.

Uh, like so, um, they're more
likely to start with prices that

are, that are lower, um, than what
they can, uh, potentially charge.

And I don't see it as a bad sin.

I, I see it as a nice.

Way to start experimenting was like
how many users you can convert without,

um, pissing them off basically.

Because very early you usually have, um,
very small, very loyal, um, set of users,

uh, to whom you're providing value, right?

Uh, so, um, that's why I think overall,
um, in the industry it's much more common

to see the prices go up rather than, um.

Some subscription app lowering the prices.

Uh, we see it and we, uh, maybe to
some extent also get conditioned by

examples from, um, Netflix or Spotify,
which are, uh, content apps which

deliver very similar amount of value.

Throughout the user experience.

And, uh, this is a completely, um,
separate category from, um, for

example, self-improvement, um, apps,
so like personal improvement apps.

Um, for this, I would say, yeah, it's not
very common to lower the prices over time.

Uh, and, uh, however personally
I see it, um, have seen it a

few times in the past as a very,
uh, valuable and working tactic.

Um, it definitely hits your lifetime
value, like that's for sure.

You need to be prepared for this.

You need to, um, be ready for this trade
off and you need to find the balance

between the two, between your conversion
to purchase rate and getting enough paying

customers and, um, the lifetime value.

Uh, so, um, in my head I see roughly.

Uh, four um, cases when
lowering the price.

Makes sense.

Like the first one is when you are
charging something that is way off,

um, the mental category into which
you are falling into the customer's

head, because each of us has that.

Um, there are products
that are a nice to have.

There are products that are, would,
would, let's say painkillers, we

are willing to pay different prices
for, um, different, uh, for them.

Uh, a second reason would be that if
you never localized prices before, and

in some countries, they also simply
don't match, uh, the mentality and the

acceptable price ranges, um, or the
payment methods that are used there.

Um, also lower prices, uh, might sometimes
simply aligned with your company mission.

So if you are about making
something accessible or, um.

Um, maybe monetize and through
donations, like I think this is,

uh, also fair, um, to go this route.

Um.

And another, uh, another reason can be
is when again, there is an established

category in the consumer's mind, and
all your competitors have a certain

feature parity, um, and, uh, deliver
the experience that the users expect.

And you don't have that.

You simply know that if free
critical things are missing for that.

So, um, all of these
that you, um, maybe can.

Um, get signals for, um, during user
interviews or like, um, exploring the

market during the competition research,
um, together with very low conversion

rate, um, to trial and purchase would
signal to me that, um, decrease in

the price might be a good test to run.

So, um.

If your users are well aware of your
offer, um, and yet, uh, nobody's buying,

um, of course, like sometimes the way you
communicate value can be the same to fix.

So let's say, um, you tell.

A certain story about your product
in the onboarding and on the paywall.

So sometimes you can try to fix the story,
uh, before you actually fix the number.

Uh, but sometimes you just have to
fix the number and, um, then end up

with, um, having a solid stream of
paying customers from whom you can

get more and more data and insights
to figure out where to go next.

Jacob: Yeah, I, I think that's
great advice because everybody

just says, raise your prices.

You're not charging enough.

Raise your prices.

You'll make more money.

But, but in reality, it's,
it's not that simple.

something I was really curious about
is, you know, you talked about looking

at both short term revenue and it's
like 13 month revenue or 13 month LTV

prediction when evaluating price test.

How do you blend these signals
when deciding whether to actually

ship a variant from an experiment?

Michal: Yeah.

Yeah.

So for kind of all price test, we figure
out that we just can't focus on new

revenue as we usually did for some other
tests like design, payroll, design test.

we've seen that the prices, particularly
when we've increased prices, what we've

seen then is renewal rates for like
monthly plans and some signals for lower

renewal rates for the yearly plans.

So we wanted to make sure that
by essentially shipping a new.

A set of prices, uh, for what
we wanted to make sure that we

are not harming the, the, yeah.

Long-term revenue or like Yeah, like
one year, one year two, et cetera.

So we are thinking like
how to, to evaluate it.

Early enough, so we don't need to wait.

Uh, yeah, like a few months or
a year when we do a price test,

it won't be really convenient.

Uh, so we figured out that what we
could do is to, instead of like a

prediction for, we call it like.

Bound 13, which is essentially, yeah, kind
of you, you have like the full revenue or

for the full one year, and then you have a
sort of like a glimpse of, or like a peak

on what's gonna be the year two revenue.

So, so what we've done do
we take the new revenue for

essentially what we get in day one.

in the first days.

And then we essentially have a sum
with new revenue that one end, the

13 month new, uh, revenue projection.

And then what we get is essentially
like a, yeah, sort of like a

long term revenue indication.

And that's how we've then sort
of decide on, on the variant

on whether we essentially ship
it or not, or roll out or not.

The, the testing variant.

Jacob: Do you have this, um, calculation
like automated set up in like a data

warehouse or a dashboard, or is this
manual you're doing this each time?

Michal: It was, it was, it was sort
of like a report template, uh, which

we then reduced for the prices we
pulled the data usually like manually

from amplitude or revenue kit, uh,
for the renewal day, renewal rates.

And yeah, the calculation for, yeah, those
13 month revenue projection was sort of.

The first sort of input, which was super
key, was a proxy to renewal rate, what

we used as a proxy was like seven day
cancellation rate because we find out

that in data that actually like lots of,
I think two to 30% of the cancellation

like subscription cancellation happened.

In the early days in, let's say,
yeah, the first seven or 10 days.

So in the first seven or 10 days,
you already kind of know what the

re renewal rate might be for Yeah.

Certain cohort.

we measure that seven day
cancellation rate for, yeah, for

each variant, each price point, and.

We then use it, uh, to predict what
the renewal rate will be for the

year one, for yearly plans or for
month one for the monthly plans.

And, uh, we essentially take, yeah, we,
we know that let's say the seven day

installation rate for, for the testing
variant is 20% higher than the baseline.

So we know that likely they,
the renewal rate for month one.

It could be also like 20% higher,
20% And essentially using that,

yeah, that inside, then we can
calculate all the other renewal rates.

And then with that we can actually, yeah,
for the predict what's gonna be the total

revenue for the monthly subscription,
let's say for the whole month.

So for the 12 month, uh uh, next.

Very similarly for, for
yearly subscription.

So we, it's important to, if you have
enough data, of course, for monthly

and yearly plans, uh, to calculate,
basically calculate it separately.

So you have like seven day
cancellation rate for the monthly

plans and the seven day cancellation
rate for the yearly plans.

Then if you had that, and if you get
enough confidence in those, in those

data, then you are able actually
to, to quite confidently predict

what's gonna be the renewal rate.

And then can again, use that sort of.

Simple calculation to calculate,
uh, yeah, the predicted renewable

rate or predicted renewal revenue
for that particular variant.

Jacob: Yeah, that makes a lot of sense.

And you found, you know, it, it
makes sense for annual where, um.

The, uh, uh, you know, there, there's
this one, one term, you found that for

monthly as well, that even just for the
monthly first period, that the monthly

renewal patterns follow the same kind
of curve, and that even in those first

seven days of monthly cancellations
based on your pre-existing data,

Michal: Yeah.

Jacob: you can model out what
that's kind of, uh, uh, the curve

will be in, in terms of renewals.

Michal: we, what we, what we did
for the monthly, we essentially used

that, uh, the seven day consideration
to calculate a month, one, so.

Renewable.

And then for month two, month three, or
month, month three, month four, month

five, we essentially kept the curve
in terms of percentage, like let's say

month, month three was this month two.

So essentially the curve is same,
but the starting point is different

because yeah, early, early in

Jacob: Hmm.

Michal: the subscription, uh, there were
less people or more people canceling.

Uh, and then the curve
sort of stayed the same.

But since in module, we've got the
majority of the subscription, uh,

in the yearly plans, essentially,
like what was like super essential

for us was the year plan.

So we got it a bit easier because
yeah, as you said, it's just

just essentially one number.

It's just one renewal rate.

But yeah.

But

Jacob: Right, right.

And so.

Michal: we applied for monthly.

Jacob: Right.

And so potentially if you have a large
number of monthly users, you could maybe

get a little more complicated, uh, if
it's, but it's, it's not the priority for

you where the majority of the accuracy
comes from projecting annual plans.

Correct.

Michal: Yeah, exactly.

Jacob: what you're more
concerned about, have you?

Michal: sorry.

Jacob: Yeah, I was just gonna ask if
the, um, you found that even with price

increases or decreases in the annual
plan, the cancellation pattern still

stays the same and so that those seven
day cancellation rates remain accurate.

It's just like you say, um, you know, a
multiplier, a little up or a little down,

but that's still that, you know, the, the.

Ratios remain the same.

Michal: Hmm.

Yeah, we've, we've checked after,
let's say six or nine or 12

months after we roll out the new.

Essentially the, yeah, the
higher price or lower price.

Then we check the renewal rate
and compared with what the

model or the prediction said.

luckily for us, yeah, we could see that
there was, uh, the, the pattern, it

wasn't, yeah, it was maybe like exactly
the same, but we've seen that even,

we've predicted that the renewal rate
is gonna go, uh, down by, let's say 20%.

Yeah.

It kind of did really go down by.

Roughly it was like something
between like five to 25%.

So we've seen that

and uh, regarding like whether like
still the seven day cancellation rate

kind of stayed reliable or relevant and
we can actually even like, essentially

even lower rank or higher, but I
think it's still fees, like pretty.

Big chunk of cancellations
happening in the first, yeah,

seven to 10 days, actually.

So we still see that, which is
Yeah, which is still like a, at

least like a good signal for that.

We can still use that
sort of proxy to Yeah.

To translate, uh, or to use it for, yeah.

Predictions of renewal rates
in the Yeah, in the future.

Jacob: Yeah, so, so in terms of that, uh,
using cancellation rate as that predictor

for, uh, new apps, maybe launching that
don't have a full year's worth of annual

renewal data, do you have any like easy
multiplier to use for them if you receive.

X number of cancellations
in the first seven days.

Multiply that times four, and you'll
probably get a good renewal rate.

You know, of course it's not,

Michal: Yeah.

Yeah.

Jacob: every app has slightly different
renewal rates, and it's not perfect, but

just for, for the people to get a range.

Michal: Well, I think, uh,
for like new apps, I think.

What's good is to look
for some benchmarks.

There's like a couple of
benchmarks, uh, available.

Uh, yeah, I think it's easier to,
quite easy to search, uh, and look

for your app or your vertical,
uh, and look and uh, retention.

So I, the yearly attention is
probably be somewhere between,

could be like 20 to 30% or 15 to 25.

Really, I think depends here
on the market, on the vertical,

on the various, various factor.

But, uh, if you don't have yet any
history in your renewal rates, I think

the, yeah, the benchmarks are probably
the best resource, uh, for that.

And then, yeah, you can use
that sort of as a prediction of

what's gonna, your apps will be.

Speaker 7: Today we're
talking with Barbara Gza.

I first came across Barbara
from a written content and post

on LinkedIn a year or two ago.

She has 15 years of marketing
growth analytics experience.

She's worked with brands including
Microsoft WeTransfer, Molly,

V-I-O-D-B-T Labs, Dentsu and More.

Now she's the founder of Fix My Tracking,
a service for marketers, dealing with

broken or untrustworthy conversions.

Barbara: Yeah, like the problem usually
with, you know, with subscription

apps is that usually, you know, the,
the conversion to when it actually,

you know, becomes revenue tends to
happen outside lookback windows.

So, you know, for example.

Meta the, I think the maximum, if
I'm not mistaken, look back window

that you can have, there is seven
days post click and it's, it's rare.

The situations where you
have a revenue event for a

subscription product that occurs.

those seven days, because usually
there is a period of a trial and

a trial can be, you know, yeah.

Seven days.

14 days.

It can be a longer period there too.

So it's important to have.

And the most important reason, you
know, to have this value is so you

are making a distinction to the app
platform on what's valuable using

and what's a not so valuable user.

So usually, you know what the
most advanced, you know, app

companies are doing is that they're
passing, um, an offline event.

So like a server to server event
that's, you know, estimates, calculates.

How likely it is for the user
to convert, or what do they

predict their, their value to be?

Jacob: And that is a kind of predictive
revenue value or so they're, and

they're passing that back to the
ad network in the kind of revenue

parameter as a, uh, and is that done?

We have

30% of our trials convert.

And is that a simple estimation

of that, or, or is it more complicated?

Barbara: basically, you know, like 30%
of your trials is not going to tell

you which trials are develop ones.

So that's what you need to get to.

You need to get to a place where, you
know, you're trying to identify out of

these, you know, a hundred people that
started the trials, which 30 ones are

the most likely to go on to, to convert.

So that's kind of the predictive
value that you're looking at.

And that is done, you know.

different ways, but you know, like
a common way that you know, um.

Companies do is like looking at
the features that they've used, you

know, um, in their first 24 hours or
in the first 48 hours of app usage.

You know, certain features for
products tend to indicate that

the user is more likely to pay.

Perhaps there's other signals that.

Are relevant to your business.

They also take it in something like,
you know, they've already looked

into the plans, like you can see that
they visited the page that has plans,

they've shown an interest to subscribe.

That is even for B2B companies, visits to
the pricing page is a very common variable

that is used to build predictive models.

Jacob: Got it.

Yeah.

Yeah.

And it's, I I think it's,
it's not simple, right?

It's not a, just a,

a simple thing to build
these predictive models.

I, I know, I'm sure some teams
do it internally, there's

a lot more companies
springing up to offer this

churn, I know is one.

I think,

uh, there, there's lots of others out
there that, uh, uh, solve this problem.

But it's,

Barbara: Yeah,

Jacob: not a simple one.

It's probably a later
term strategy, right.

It's probably not.

If you're a newer app, is this
something you should think about

or is this more optimizing, you
know, scaling large campaigns?

Barbara: well, I think.

Like, I think for most businesses they
can probably postpone having some type

of predictive value because maybe they
have, you know, like other signals,

you know, that they can take into
consideration to assess how user is worth.

But with mobile apps, with
subscription apps in specific, because

the revenue action is occurring.

Almost always outside of the look back
window, you need to have something in

place that tells the ad platform, this
is a good user or this is a bad user.

Like for the first iteration, you're not
going to have, uh, a predicted LTV that,

you know, like matches two years from
signup, obviously like that we're talking

about like very, very highly developed.

Algorithms, but at the very least,
you need to be like, okay, this

user, you know, back to the app
within, you know, two hours.

Like, you know, they've, you
know, they've clicked through the

features like you need to have.

Some kind of signal that you're using and
that can be as simple as, you know, like

we're gonna take old people that create
an account and then we're gonna, you know,

okay, either they're worth zero, either
they're worth 10, either they're worth 20.

You know, it can be something
even as simple as that, but you

need to have some kind of vari.

That tells you how much
this user is worth.

Otherwise, what's going to happen is
you're going to be stuck with running cac.

So you're gonna be stuck with, you
know, um, a bid based model where

you're paying like per install.

And then what the platform is
going to do is going to optimize

for the cheapest installs.

And the cheapest installs usually is, you
know, like if you've run pay campaigns

for apps, it's like 17 year olds.

You know, like it's usually, you
know, the cheapest install is usually

not the install that converts into
a paying customer, so you cannot

really postpone for too long
if you want to do pay media, at

Jacob: Yeah.

Barbara: at scale.

Jacob: Today on the Price Power Podcast,
we're talking with Ashley Black.

Ashley is a Google app
campaign's rockstar.

She was head of app ad sales at Google
where she shaped Google's go-to market iOS

roadmap and worked with thousands of apps.

She's now the founder of Candid
Consulting, where she's bringing all

this knowledge to the world to deliver
high performing marketing strategies.

something else I, I saw you writing about,
uh, recently was that if you're, if you're

not getting, good enough signal with
maybe gist, uh, trial conversion events,

Ashley: Yeah.

Jacob: you should look deeper at,
uh, retention events to further

optimize those campaigns and, and
getting those stronger signals.

I thought that was like, made,
made so much sense and I think, um.

Um, you know, I, I work a lot in kinda
the retention monetization world,

so, you know, I, I do similar kind of
analysis and correlation analysis to

see like, you know, what, uh, early,
um, early engagement events correlate

with long-term conversion retention.

And so, you know, when is that, or I
guess, talk me through like how you

think about doing that and, and then
when that's a good strategy for apps.

Ashley: I think honestly, you should
be thinking about it all the time.

Um, I think too often people are like,
sort of have this mindset on like,

okay, well what's that one event that
we're going to track and measure?

And, um, you have to kind of think
about it a little bit more dynamically.

And so, you know, the example that
you're referencing or what I had

talked about is like, I had, um.

Had one client who was optimizing
for just registration complete.

So they just wanted to see who was going
through their whole registration flow.

And if they completed that whole
flow, then like that's what they

wanted to optimize for, but.

Ultimately, like those users weren't
sticking around for that long.

Like the retention on those users wasn't
great, and retention was a big play for

them because for this particular app, they
really cared about monthly active users.

That was sort of their, their main,
um, KPI that they were looking at.

So we did a bunch of analysis
to try and find, well, what was

another event that was happening
with a great frequency, right?

Because like that's one thing like,
yeah, sure everybody would love to

optimize for like something super
deep funnel, but then you can't

optimize based off of five users.

So like we need something that has volume
and for them they're a messaging app.

What they ended up, what we ended
up finding is that, oh, actually

it's not so much the registration
complete, but if they actually start

a conversation with somebody, so
they start messaging somebody, that

is when we see the stickiness, the
retention improve a lot, and again.

It at volume, right?

There were other things that
showed greater stickiness, but

like the number of users was too
low, so we couldn't use those.

So that's how we ultimately ended up
deciding to pivot for them and say,

we're gonna shift away from just this
registration complete filling out a bunch

of, you know, questionnaire questions,
um, and, and move toward them actually

starting to take some action in the app.

Um, and so I think if you can think
through, you know, what are those early.

Early additional, um,
actions that a user takes.

I think, you know, you and I are
familiar with one particular app,

right, that we've, um, talked about
and, and worked on, and one thing that

we're looking at is like, okay, what
if we stop just optimizing for a start

trial and instead like the, the action
that they take immediately after.

So did they watch, like,
let's call it the tutorial?

Did they complete the tutorial after the
start trial or did they, you know, um,

I don't wanna give it away, but like,
you know, did they do something else

immediately following something that
probably happens within like the next

five minutes after starting the trial?

And is there enough volume in
that for us to be able to then.

Optimize for that action
instead of just start trial.

So, you know, it's gonna vary
depending on your, um, your

app and like how it functions.

But thinking beyond just, you know,
what is sort of required of the user.

Jacob: Yeah, this reminds me of
like activation analysis to figure

out, you know, general, like what
is our activation moment in the

app and we're trying to figure out
what leads to long-term retention.

And so, you know, I, I do that.

By, you know, we we're trying to
figure out the right path for a user.

How, what is this activation
moment that we can kind of guide

users to through product changes
or growth changes or experiments.

And it sounds, you know, pretty similar
to kind of that process of looking

at a number of events and trying
to figure out what correlates with

long-term retention or, or conversion.

Um, and so is that like.

generally how you'd recommend
someone trying to go find this event?

Just looking at kind

Ashley: Yeah,

Jacob: between a bunch of events
and seeing what, uh, uh, ha

has, you know, potential for
like a predictive uh, uh, value.

Ashley: yeah, for sure.

Um, hopefully, you know, you have a
team or you're capable of doing it

yourself to be able to understand.

And I, and I think honestly it.

A lot of times it's, um, it's
probably a little bit obvious if,

if there's like a clear flow, right?

Like, let's use the messaging app.

It's like, okay, did they, if they started
a conversation, it means that they had

somebody else to talk to on the app.

And so that right there shows that
there's like more stickiness, right?

Like they're, they're, they're joining
or they're on the app so that they

can communicate with somebody else
as opposed to, let's just say like.

Watching a video in it, like that's not
as meaningful probably to you because

it's, that's not the intent of the app.

Um, they're not in that particular app.

They're not like serving
ads in videos or anything.

And so there's not as much
value in a user like that.

Um, so I don't know.

To me, I think that they, it kind of
stands out a little bit, but yeah,

you gotta do some of the analytics
to figure it out in this case.

Um, you know, like it did take us a
while to sort of like, understand.

You know what?

There were so many other events that,
like I said, showed high retention, but

we needed something that happened within
24 hours, really, really quick, so that

we could still send that signal back
to Google in a short period of time.

And like I said, there was still
a high volume of users so that

we were getting enough volume.

Jacob: a super popular topic,
which you are very well suited to

speak about is Signal engineering.

Uh, and I know day 30 is built around this
idea and, and I was reading something you

wrote, I really liked your description
of, um, choosing conversion events

that better represent user value than
what ad platforms can see on their own.

Um, can you tell me a little bit more
about that, how you landed on that framing

of, of signal engineering and for day 30.

Shumel Lais: Yeah.

So I think, um, the term sig signal
engineering, I, I don't think I can take

credit for coming up with that term.

I think that was, um, Thomas Petit
and, and Eric, they, they've been

kind of posting on the topic.

Um, but the concept of.

Coming up with a predictive signal.

Um, that's kind of what we started
looking into, uh, before that

term kind of became popular.

But in, in essence, it's,
it's the same thing.

so what we, uh, are aware, we can
see these ad platforms like meta,

they're becoming very algorithmic.

Um, so it's very.

Uh, they've kind of taken away a lot
of the controls that marketeers have,

whereas historically, you may be managing
your bids and budgets and targeting

and, and they're really pushing you
into this world of just like, give us

your goal and we'll try and find the
best users to, to reach that goal.

So the topic of signal
engineering is really like, how

do we feed these ad platforms?

The best goal, um, especially in this iOS
world where, uh, there's a limitation on

how far visibility the ad platforms have.

So if, if I take meta as an example.

They're typically, uh, limited to
seven days of, of visibility, terms

of that goal event that you can send.

Um, but what marketers typically care
about, or, or, or apps, subscription

apps in particular is, um, usually
something a lot further down funnel.

So typically, uh, revenue or,
or, or return on ad spend.

And a lot of that doesn't materialize
or outside of this window.

that's really where this topic
of signal  engineering comes in

of how can we, um, find something
that's better, correlates better.

To that longer term, uh, value
that you're after, that the ad

platforms otherwise couldn't see.

So that's really kind of how
we landed, uh, on that topic.

And, and some of it came from, um,
historically from, from, from some past

campaigns going back a few years ago, um,
where I teamed up with a data scientist.

And we ended up, uh, solving
for this problem where there was

just a delay in a user behavior.

Um, and that's got when I got
introduced into the world of

machine learning and data science
and how that could overcome that.

Um, so that was kind of
where the idea stemmed from.

But I think more recently, um, it is been
kind of coin signal engineering, which I

think is, is a pretty good name for it.

Jacob: Yeah.

Got it.

That makes sense.

And so maybe you can, you can
dumb it down for me a bit.

Like, what's, like, can you like a, a
simple example of like what's actually

happening here and how, like, you know,
some app, what an app is actually doing.

Shumel Lais: So I think, um, if, if we
park kind of machine learning and, and

data science for a second and just think
at the most of basic level, um, signal

engineering is really just the, the.

Process of selecting, uh, an event
or creating, uh, a conversion event.

so you always start with what's the goal.

So let's say our goal is a
paying subscriber of an app.

Um, let's assume there's a 30 day file.

That's what we're trying
to bridge the gap to.

So, uh, the status quo, what a
lot of apps would go after is

optimizing to a trial start event.

because for anyone to become this
paying subscriber, they would

go through this trial first.

So that's kind of the starting point.

then signal engineering is,
can we find something better?

So at the most basic level, you may
find other behaviors that correlate.

With someone actually
going on to subscribe.

So if you, uh, let's say a language
learning app, it could be the number

of lessons that you completed.

So could we do, you know,
start trial plus, um, someone

who's completed five lessons?

That would be like a very basic example
of how you could, uh, build, you know, one

output or, or a signal to the ad platform.

So instead of just going for
trials, go for people who do

a trial and a behavior as well

Jacob: i've also heard, um, you
know, people talking about qualified

trials where trials where the
user didn't cancel immediately

in the first hour or something.

Shumel Lais: Yeah.

And that, and that's another
really good example as well.

Um, and that works when
it's, you're almost like just

purely trial focused as well.

Um, and then there's the question
of how long do you wanna wait?

give for those trials to cancel.

Um, and certain apps you'll see, like
usually you find it with slightly younger

demographic, um, focused, targeted apps
where they're just very familiar with

this ability to cancel immediately.

So you see a big portion of
those cancellations happening.

Straight away.

So that's where it works quite well.

And then you've got some apps that are
maybe have like a slightly longer user

journey, um, and slightly, uh, more mature
users who actually want to test the app.

So you may not get as many of those
cancellations early, but that's

also a very good example of signal
engineering at the most basic level.

Jacob: Do you think like this concept
of signal engineering is become.

Because of the trial, because trial's
essentially a low quality signal that,

because if you are, if people are buying
direct and just purchasing, that's,

that's the highest quality signal.

Right.

And so is it all kind of like this,
trying to get a proxy around this and,

and you know, is it, is this mostly
driven from that subscription model,

but then also like platform limitations?

Is that, how is that, do you think
that's kind of the, the genesis.

Shumel Lais: I, I think it's, um,
definitely driven by the, this kind of

structural issue with attribution and,
and, and the platform's visibility.

Um, and I don't think it's just
limited to subscription apps.

If anything.

Actually, I think this has been around
a lot longer in the gaming space.

Um, and in gaming there's a lot
more variability  📍 📍 📍 📍

Hey, Humel, I'm super excited to
have you here today on the podcast.

Thanks for having me.

. So it's not just has someone subscribed
just, or no, it's a case of how much value

do I think they're gonna, uh, be worth?

Or, you know, how many in-app tokens
they're gonna, they're gonna spend.

So there's a lot more to predict.

So for subscriptions, trial
is definitely an aspect.

Um, and I definitely agree, like we also
see people who have a mix of trial, a

non trial or initial purchases as well.

but typically those initial
purchases are shorter.

Uh, so they're typically like
weekly or monthly subscriptions.

And even in that instance, um.

You know, they, they're offering
those because they know over, you

know, three months or four months,
those users are gonna renew.

And so again, there's still an opportunity
for signal engineering there because

there's still this gap of visibility.

So yes, meta could see that initial
purchase and that will probably

be stronger than a start trial,
uh, for users who, who do that.

but there's still a difference between
a user who renews only, you know,

once or doesn't renew a tool versus
one that's still retained in, you

know, six weeks or 12 weeks time.

Jacob: Yeah.

So regardless if it's just trials,
you know, because you're right.

Yeah.

A lot of times if you have a, we.

Weekly plan.

You know, you may not offer a trial,
but you're still trying to, you know,

understand and predict what is the
true LTV of this user based on their

attributes, based on what we see, are
they gonna retain what's their real value?

And then it's also like, um, the, uh,
you know, combination of different

purchase types, upsells, uh, and then
sending the right signal for all of

those where it's like, if you're just
sending on a trial star, do you combine

that with your other purchase signal?

How do you differentiate all that?

And so it gets you
pretty complicated quick.

Shumel Lais: So I think visibility
is like one aspect of it.

Um, so you could argue, you know,
if you've got an app with a three

day trial, um, only, and, and then
they convert and it's only an annual

plan, do you need signal engineering?

You, you know, there's an argument to
say, does it fit within the seven days?

So that's where you have these other
aspects of signal engineering, which is.

It kind of plays to the specific
requirements of the ad platform.

Jacob: Hmm.

Shumel Lais: Uh, so if I take meta
as an example, there's another

variable which is signal volume.

So not just signal accuracy of
predicting feature value, but

also is it getting enough volume.

So sometimes you might find actually
you have visibility into purchases,

um, or there's a really short trial.

But the people who actually
convert are already 20% of

the people who start a trial.

So all of a sudden you lose a lot
of volume and platforms like meta

still need that signal volume.

So you're kind of balancing these
two aspects, which I'd say the,

the key aspects is the volume and
then the, the precision side of it.

Jacob: when does, you know, ML
based signal engineering make sense?

Shumel Lais: Yeah.

So I'd say, um, one of the requirements
of signal engineering, or it's like if

I, if I double down a bit on meta, is
where majority of people's spend is going.

there's, this isn't like a, a definitive
rule, but a general rule that we've

seen is like, you wanna get at least
10 conversions per campaign per day.

that is what I would set
as a, as a starting point.

If your current trial starts aren't
getting that, then you definitely don't

wanna be doing signal engineering because
you're not hitting that threshold.

And although 10 sounds like not a lot,
um, sometimes depending on the, the

markets that you're targeting, that
cost per trial can be quite high.

It can be from, you know, $20 or the up to
a hundred dollars, for example, depending

if, especially if you're targeting the us.

So if you multiply that by 10, all
of a sudden, you know, the, the

spend starts increasing quite a lot.

that's where I would say spend
is quite a big aspect of signal

engineering to, and that answers
should we go down funnel or not.

So I think that's just the
gen, the general principle.

whether we should get into ML based, um.

Uh, signal engineering is one,
is answering the first part.

The next is the, the
kind of data you have.

Um, and do you have enough as well?

So what we've typically found is it's
quite rare that, um, people have enough

behavioral data tagged in their MMP.

Um, they usually have to go into
something like a mix panel and.

The ML models are only
as good as your data is.

So, you know, again, if you are
like a language learning app, do

you have all the lessons tagged up?

Do you have different
tracks, um, tagged up?

Like all of this can become inputs.

so yeah, so I'd say once that's
how I kind of look at data.

It's less to do with like number of users.

It's more, um, the number of conversions
you're starting with to the ad platform.

Jacob: Why do you need it an MMP?

Um, uh, are you still
using the kind of MM.

E post backs to the
network to to optimize.

Shumel Lais: The MMP is more
from a measurement perspective,

Jacob: Okay.

Shumel Lais: so the MMP is really
more, um, one is yes, the MP data

can feed in, uh, into the ML model.

So some people have enough
behavioral data in the MP.

The benefit of the MMP for signal
engineering is purely measurement.

So, so what we would do whenever we do
signal engineering is we don't wanna

just blindly run it, we wanna AB test it.

So we would have a campaign optimizing
to start trial, and then we'd have

a campaign optimizing to, let's say,
this predictive subscriber event.

by having an m and p, that's how you
can truly see like your day 30 roas,

um, for the start trial campaign
versus a predictive signal campaign.

without an MP, you wouldn't
be able to answer that.

Jacob: Got it.

Got it.

So, so it's not necessarily for the signal
engineering side, but for the measure

measurement, understanding of kind of
a cohorted, you know, roas for example.

Shumel Lais: And, and where the MMP
can also help is it can also act as

like the gateway for the signal itself.

there's again, a couple of options.

Um, for anyone doing signal engineering,
you can either send these signals

or server side events directly to
meta using the conversion API, you

could send it to your MMP and then
they would forward it to meta.

The benefit of it going through the MP
is you could, that one point can also

forward it to TikTok, it could forward
it to app loving liftoff and all the,

all these other ad platforms as well.

Um, so, but both work in
principle, but that's just another

benefit of going through an MMP.

Jacob: And then I think that like,
uh, uh, you know, idea about the

creative is the targeting, right?

I think that's another message you have
and, and some multiple levels, right?

The creative is the targeting
through, what ratio is it,

what type of creative is it?

But then also what's the actual
message on the creative 'cause That'll

inherently get shown to different people,
different people it'll resonate with.

And then you can understand how those,
um, creative messages convert based

on user goals or other questions
that you asked during onboarding.

And then, uh, figuring out how to layer
it all together is the, the tricky part.

Marcus Burke: That's
where it gets interesting.

Yeah.

And yeah, kind of that media type
format piece is something people tend

to forget, like they think of well.

When I call out women or men in my hook,
then it's gonna target women versus men.

But then, yeah, really if, if
you never make aesthetic, then

your likelihood of showing up on
Facebook feed is a whole lot lower.

So kind of the media type has a
big, big role in this just because

different content types are kind of
native to these different placements.

So really.

As you're kicking off a new count,
like always test very broadly, like

look into carousels, video carousels,
statics, uh, nine to 16, one to one.

Um, I'm just testing.

I'm, uh, preparing a test with playable
for, for a client of mine, which are

actually only work on Facebook feed.

From what I know, I never tried
them before with Facebook.

Um, so yeah, there again that if
that, um, ad is only gonna show up in

Facebook feed, then that could be super
interesting because that's where kind

of a high quality older audience lives.

And if you never made a playable before,
then you would never see kind of a

hundred percent Facebook feed distribution
or any ad, because even a static also

goes on Instagram feed and others.

F there's always gonna be kind of some
wastage somewhere where kind of due

to the broad targeting, your ad ends
up in places where you shouldn't be.

And then you can always kind of try
to optimize and figure out like, how,

how do I push it in the kind of best,
uh, the most optimal distribution.

Jacob: And so, you know, what I'm hearing
is to really kind of scale campaigns, you

need to figure out these efficiencies.

You need to understand where the
opportunities are to kind of get hit

your, hit your goals, and to be able
to scale there where you're, if you're

just hitting broad on everything.

You know, you're good at things are gonna
get expensive, uh, uh, faster, um, and

you're not gonna really understand why,

Marcus Burke: Yeah, pretty much like
in the end you are, you are, you're

gonna hit a ceiling quite quickly just
because meta, if you throw everything

into one campaign, one ad set, then meta
just optimizes for the kind of lowest.

Cost per event, and especially for
a subscription app that's usually

not related to business value.

You're driving cheap trials.

A cheap trial isn't worth
anything until it converts.

So the algorithm just shouldn't taking,
shouldn't be taking all the decisions.

You need to kind of infer that
additional knowledge around what's

happening after someone starts a.

Jacob: Yes.

And so passing the right signal, right
data, the right trials so meta can go

find more of those trials that you say
are important, not what it infers are, are

Marcus Burke: I mean that's, that's
then already kind of signal engineering.

You can of course, try to pass
it as an actual signal into the

algorithm so it gets smarter.

Even if you don't do that, then
you have that kind of knowledge.

So use that knowledge to build your
campaigns in a way that it spends on

the audiences that are most relevant.

That's kind of the two sides there.

Um, yeah, signal engineering also
very hot, hot term these days.

Um, did a webinar with Thomas
with Revenue CAD the other day.

Um, like the closer you can move
your, um, conversion signal to an

actual business outcome, the better.

Of course, meta gets at buying
the right audiences and not just

the cheapest ones in the end.

Um, but yeah, that's
definitely advanced territory.

Like smaller advertisers tend to
be starting on like a star trial

or maybe a subscription purchase if
they kind of have also like direct

purchases, um, on their paywall.

Um, and then over time maybe you move into
something like a qualified trial where

you figure out, well, let's kind of not
send signal for some of these people.

Or like only kind of infer
some additional data points.

Like, I don't know, OS is usually an
interesting one where like older, uh, like

newer devices and os is tend to convert
better, um, age, as we already said.

So you can get really smart over
time and there's like companies like.

Day 30 journey, um, those signal
engineering providers that help

you basically look across all of
your data points and like find out,

well, if we cut these 40%, how much
higher would ultra conversion be?

Um, and that definitely gets very
interesting as you scale, because.

As that AI is taking over more and more.

So really creative and signal are the
two main things you can be working on.

There is some work to be done with
account architecture as we just said,

but then other than that, it's not like
you're gonna set up 50 ad sets, some

targeting male and female in different age
groups as you used to back in the days.

Um, so definitely Signal is one of the
levers that you can still be pulling.

the algorithm knows very little
about what makes your business

profitable in the end, because you're
optimizing for that cello event.

Um, while if you're moving closer and
closer to, to business value, then you

have a smarter algorithm, meaning you
can have more call it consolidation just

because it can actually see that well.

A cheap cost per registration from
Instagram reels isn't worth anything.

Um, because people don't
convert to that, um, uh, to that

subscription or to that purchase.

Meaning it will theoretically over
time start, um, deprioritizing that

placement because it sees it's not
driving an efficient kind of cost

per purchase for your business.

Um, but of course there is.

Like in the end, I, why I say like,
don't over consolidate is that whole

part that we discussed before, that
in the end different creatives go

onto different placements and meta is.

To this date pretty bad at
kind of diversifying their and

distributing spend more evenly.

Oftentimes it will pretty quickly
hone in on like a creative winner

saying, well, this is the very best
ad and I'm gonna spend 80% plus of my

budget in an ad set on that very ad.

And of course.

Everyone talks about kind of
creative diversity and like

creating an account that of course
is sustainable and free of risk.

But if meta always pushes all your
spend into one creative, then that

is pretty risky in the end because
if that creative stops working,

then performance is gonna die.

So it's always your job to kind of.

Push away from that a bit.

And that's why I kind of tend
to tell my clients, well, don't

put all your eggs in one basket.

Make sure we have a few ad sets with
that are different, um, delivering

to different kind of placement mix,
its audiences and if one dies, then

we're still good with the others.

And of course that architecture in the
end should then be driven by how much

does meta actually know about my business?

This is very, are they very smart?

Am I optimizing for value because
I'm kind of sending back my purchases

with values attached to them?

I maybe even set a raw as target,
um, something that is not super

common for subscription apps,
but for example, in gaming.

Um, there, of course you have to
worry much less about them buying

cheap traffic that's not converting.

While if your event is very upper
funnel, then you need to do the job

of structuring campaigns in a way
that they go after the audiences

that you know have highest value.

I hope that makes sense.

Jacob: really interesting.

Yeah.

Yeah, it does.

It does.

Um, and so for, for pretty much any
app with a seven day trial, like.

Until you get a lot of data and
a lot of understanding of kind

of what users are valuable, it's
kind of impossible to get to that

revenue, or real value metric, right?

Sending that back.

So you're always stuck with this
trial that's kind of incomplete.

And so that goes back to what we were
talking about before of trying to

guide the algorithm to the right users.

Do you.

Think it makes sense for more subscription
apps to test not having a trial or

test certain plans with no trial to get
some of some more revenue data earlier.

Marcus Burke: Um, I've seen apps doing
very, very well, um, with not offering

a trial and optimizing for that.

Um, but that was in the AI vertical
and I think kind of AI apps, like there

is a very high willingness to pay.

Like we had purchase rates beyond 10% even
without offering a trial, which is crazy.

Um, so.

In the end, I think it's a nice
kind of, um, challenge to think

about, like, do we actually.

Do we actually need the trial?

Do we need the trial on every product?

What kind of signal would it give us
if there was direct purchases involved?

Just because kind of a, someone buying
something on day one and actually giving

you money is a very, very good signal
for the algorithm and someone just

starting a trial isn't, and I think it
makes sense to at least challenge that

idea, that kind of everything always
needs to be kind of trial, um, on all

the products in seven days just because.

Yeah, you can't really optimize
for, for like, um, a, a purchase.

Then it's also outside the
kind of, uh, the attribution

window of seven days post click.

And also with a EM, which is
their, their fingerprinting.

In the end, the accuracy deteriorates
quite badly, um, over time just

because signal isn't available anymore.

So you need to optimize for
something that happens on day one.

Preferably in the first session.

And with a trial, as we said before, we
can enrich it a bit and try to figure out,

well, which are my most, uh, relevant and
high quality trials, and only send those.

Um, but of course it's never
gonna be a true purchase.

And having that in the mix somehow
can be very helpful, I think.

Um, yeah, and like it.

In the end, like a lot of headaches were
gone when you, when we just optimized

for, um, optimized for pure purchases,
it was clear that meta only receives kind

of who is, uh, in the end valuable to
us and, um, that made sure we're never

wasting any money on, uh, low quality
audiences, which was great to see.

Speaker 3: Today on the Price Power
podcast, we're talking with Lucas Moskin.

I was pretty excited to talk to
Lucas because he is likely one of

the most knowledgeable people in
the world on how mobile attribution

actually works under the hood.

He was a senior acquisition
manager at Monkey Taps.

He's been an advisor to large
companies like Life 360 and is

now the founder of App Stack.

Jacob: It seems like it's positive
for the overall app ecosystem where

there maybe was too much confidence
in your attribution previously,

that what was unfounded where
you thought that you knew exactly

or come from, came from, or kind of
the performance of this channel where

maybe now kind of the veil has been
uncovered of maybe your attribution

wasn't so accurate before and, and now
that actually aggregated is, is better.

Um, or, or maybe not.

I don't know.

What do you, what do you think?

Lucas Moscon: I don't think it's better
or I think it's like it just forces

you to see all the layers It forces you
to see the blended deterministic the

probabilistic Like again it's just it's
all about I think that for example when

I talk with apps And they're like oh I
want to see if my campaign is on iOS have

a return on ad spend of 100 on day seven
Or they even want to know if the ad has

a Rose Day seven of 100 I tell them like
dude that's not how it works today Uh I

wish that's the case and basically what
I try to help them is to build To try

to abstract their business from robots
metrics and focus on ROI which ensures

like more realistic numbers So for example
I've been advising really big uh health

and fitness application and basically
their main source of truth and where

they make most of their decisions an ROI
chart where they just basically fetch

ad spend from the ad networks revenue
coming from the store consoles or their

subscription platform and also divide
that by install date so they know okay

these cohort of users it's profitable
the ad network level or even at the

company level day in 60 You know so by
focusing on that level of aggregation

it's super safe for them to double
spend or double down current strategy

Jacob: Yeah, that makes sense.

Where don't get too caught up in like
every single ad set or even every single

channel as long as you're spending.

Profitably and, and your business
is growing and you're making

money at, at a whole, like, that
should be successful, right?

Like that's, that's what, how
you should measure yourself.

Lucas Moscon: And also believe that
for example if you are if you have the

mindset to extend The time in which you
get the money back I think that's also

an interesting growth lever because most
people will be optimizing towards the

other way and most of them will realize
that it's almost impossible and it's

when you want to scale the return out
spend it's even harder more ad spend the

the lowest is the robot for companies
that are really taking the risk and

willing to go a little bit further and
say okay I'm getting my money back on

day 60 happens if I push these to 180
days They will be outspending their

competitors and they they will be by
doing this they will be able to spend

more money and grow more Still you
you need to have cash The cashflow is

important to do this strategy But I also
believe that this approach or this might

to go beyond of instead of decreasing
increase the target for your return spend

Jacob: Do you think that increases
risks for advertisers where it

takes longer for you to find out?

Um, if, if something is profitable,
what if something changes in your

product, changes in your conversion?

Um, does it, uh, can you project
that early on and, and, and, or, or

is that an increased risk because it
takes 90 days or 120 days to find out?

Lucas Moscon: I think that it'll depend
on the confidence level that you have

on your proxies for the early days So
if you historically you see that on

day seven if you have a rise below 0.5

it means that you will never get your
money back And that has the case for

the past two years Then maybe that's
a strong proxy that this cohort is not

going to perform as expected But I agree
with you It's it's of course it's more

risky but because it's more risky it
means there's a reward on the other side

Jacob: you were talking about
how blended CPA is irrelevant.

I, I was thought it was so interesting.

I think that we all have these
metrics that we rely off of, and,

and the, the changing ad, , world
sometimes means that, you know, these

metrics aren't relevant anymore.

And I'm not an AD acquisition,
uh, you know, growth acquisition

expert, and others have told me
that, you know, looking at blended.

RO roas return on ad spend is the
only true way to go to understand

what's actually going on.

So I was curious to understand the
difference in your mind, how is blended

CPA different from blended roas and,
and was curious to dig in more, uh,

to what you're talking about there?

Marcus Burke: Sure.

Um, so I mean, first of all, for
context, I, the, the presentation.

Was named something like Meta Ads AI
aligning your kind of full funnel.

So in the end, um, I looked into
kind of what are the levers.

I, as a full funnel growth marketer
can still be pulling while there's a

lot of stuff that is taking over by
ai, uh, sooner, sooner than later.

And so.

When I talk about blended CPAI talk about
like what's reported by meta in the end.

Um, I totally agree with everyone you talk
to, that you always want to look at your

overall blended business numbers because
if your ad platform tells you everything

looks fine, but you are losing money
on the back end, then nothing is fine.

Um, but when looking at your CPA
metrics or cost per trial, cost per

subscription, whatever you are feeding
that ad platform, I mostly see people

kind of running their creative tests and
optimizing, optimizing their campaigns

just based on the kind of cost per
result that is reported in Ads Manager.

But then meta is.

Such a vast platform, like you
can be running ads on something

like 20 plus placements.

Um, with each of these having totally
different user behavior, totally

different audiences behind them.

And therefore, if you see a 10 bucks cost
per trial, let's say, um, in your cost per

results, and that's from Instagram reels.

Then that's gonna be a very different
story compared to if it's 10 bucks

cost per trial coming from something
like a Facebook feed, just because

that's used by different audience.

So don't just look at what's kind of
coming from your campaign in total.

Look at the breakdowns.

Figure out kind of that traffic
composition to understand, am I driving

valuable traffic or just cheap traffic?

And I mean, oftentimes people ask me like,
what's a good cost per trial on meta?

And it's like, well, that.

Depends on like your down funnel, the
kind of your pricing, and in the end,

that traffic composition, like I've seen
apps scaling profitably with a hundred

dollars, uh, cost per trial, and I've seen
apps burning money at $10 cost per trial.

Um, so in the end, planet CPA
doesn't matter for that reason.

Jacob: Can you set, um, different
CPA goals per placement or are you

saying post the fact, do your analysis
and try to break down by placement?

Marcus Burke: So in the end, you
have a few tools available there

to kind of steer that composition.

One is creative.

In the end, creative equals targeting.

So depending on the ad you're using that
is gonna address a different audience

and a different kind of placement.

For example, if you run a static ad, it's
much more likely to run on Facebook feed.

If you run a 90 16 short form video, then.

More likely to run in Instagram
reels just by kind of where

that native media type belongs.

And, um, then the audience target
targeting comes, comes on top.

If you speak to like a pain point for
elderly people, then that's likely

to show up on Facebook just because
that's where that audience hangs out.

Um, while it's, it's if it's a
young, um, a younger issue, then.

That's usually gonna go on
Instagram and potentially reels.

Um, so based on the creative type,
you can already steer targeting.

And then what I do is basically group
my creatives by where I'm expecting

them to deliver and update that over
time as I actually see where it goes.

So if I find, well, I have this kind
of short form video, it always goes

on Instagram reels, then I have an.

Set that only contains these types of, um,
creatives while I have another ad set that

only contains the types of creatives that
go onto Facebook feed and like the more

mature high value, um, uh, placements.

And over time that grows quite complex.

Like at times like then I have like
not just one business as usual campaign

with one ad set doing all the work, but
I have like maybe five, 10 different

ad sets that all contain kind of
different messaging angles, media types

that I know in the end will result
in a different down funnel behavior.

Um, while each of them is just
targeted broadly, like they all

have the same setting, but the
creative does the targeting.

The other, um, tool that was
recently released actually, or that

recently was, uh, released in Ads
Manager UI at least is value Rules.

Um, and that's a way to actually
encode different, um, value

on an audience placement base
even when you're running broad.

So you can say, well.

My, uh, money should be spent on all
placements, and you decide where,

uh, based on the creative, but I'm
already encoding in the algorithm

that, for example, Instagram reels
is 30% less valuable because I

know it never performs as well.

Or you're saying, well, my 45 plus
age group male, um, should always be

bit up by 40% because I know that's
kind of my, um, most interesting ICP.

Speaker 2: Today on the Price Power
Podcast, we're talking with Gabe Kochi.

Gabe has been a leader in the
mobile marketing world for years.

He founded an agency called Inci.

Pia.

He co-authored the Advanced App Store
Optimization Handbook, worked with

massive brands, including Walmart,
Coinbase, Duolingo, and more,

Gabe: I was telling 'em all about you
know, what, what I learned at Lingo and

before, but to your, to your question,
um, I was able to recognize which

pieces were there that were important
and what to try and how, where to

have confidence and what to look for
because I had run and scaled, um.

Campaigns with Coinbase, uh, with,
with, um, Walmart, with, um, people

fund, just like all these other,
um, companies back in the day.

And so I knew, for instance, that
influencer content, um, not just UGC,

but like really people that are good
behind the camera or the phone and that

have a way of like attracting attention
and getting people to like, like, and,

you know, engage in content that works.

And that we needed to find it.

It's really about, especially
in paid social creative hits,

a lot of the creatives you
try, it's just not gonna work.

And it's only the small percentage of
creatives, like everything is the Pato

principle everywhere all the time.

So too with creatives and, and scaling ua.

We, when I came in, I tested,
you know, some creative that

they had in the account already.

We tried making some, like
some static ads, some graphic

design ads, motion graphics.

I tried making a few videos myself.

Um, we had our, some of our teachers
who had videos and those were

like one of the earliest hits.

Um, one of our teachers who
was doing the, the colors, um,

in, uh, American sign Language.

Um, but then I saw that the.

Cr, the, the influencer videos
actually wasn't fully organic, or it

wasn't, most of it wasn't organic.

We did contracts with influencers,
I saw that a couple of them had

really good, um, virality scale.

And so that was that early signal,
like, okay, there's something here,

and what if we chuck that into
advertising and we see how that goes.

And, you know, we tried a few of them,
some of them didn't work, and then

we kept trying and I kept looking
for those hits, the things that, like

a day.

see that meta is starting to scale
this up and it loves it and you're

getting great, um, CPIs great, cts great
cost per trials and you can see that?

like this is really strong performance.

so then I just started opening up the
budgets and really expanding out the

audience targeting and really like getting
more surface area for these creative,

this strong creative to get out there.

And on the side we started up a
pipeline where we started to, to

work with more and more creators
and pull in more influencers.

And so constant testing,
um, was, was important.

But um, Yeah, it was realizing
that creative hits what drive paid

social, um, that there were some.

Promising creatives in the, in the mix
overall that hadn't been deployed from,

uh, Instagram organic to, um, Instagram
advertising and to just when it started

to work, have the, the guts to, uh, 10 x
the budget and the co-founders as well.

Like, I checked in with them,
you know, props to them.

They're like, I'm like, okay, you
know, this is the time to press.

The, budget, when it's working, go all in.

So can I, can I raise the budgets?

You know, 10 x in like a short
period of time, do we have the

cash flows to support this?

And they said, yep, we're good.

So, so then go in.

And that's, another thing like.

It's temporary.

Creative hits are temporary and
you get this excellent performance

period, but then it always at some
point saturates and comes down.

So you have to learn how to like
navigate these periods where you have it.

Press in, press your advantage, and when
you don't have it and you get into a

drought, pull back, realize that you've
just gotta work to get another hit.

And you're go, you're
gonna feel like a failure.

You're gonna feel like, ah.

You know, shit like why isn't it working?

Nothing's working.

Our performance is so bad.

that's the dynamics of modern paid social.

You just gotta get a strong
enough hit, um, that it works.

Xavier: we always recommend our clients
to start, I mean, unless they're not

targeting the US market, but it's very
rare, um, to, to focus, focus on the US

market first, get some winners, and then
once you have that you can, you can,

um, you can expand to other countries.

Jacob: And, and so really you need the
new creative because you're putting

in a different language where it's,
it's, you need the French person for

the French market to feel natural.

Xavier: Yes and

Jacob: Okay.

Xavier: Um, why?

Yes and no, because actually
that's actually counterintuitive.

we, I mean, if you have a top
performing creatives in, um, in

English, in US English, it can actually.

B, a top performing for
non-US, non-English speaking

countries, French and Germany.

I mean, it

Jacob: Yeah.

Xavier: it doesn't work as
much as in the US but it works.

So the, so you can use actually
the original ad for better result.

What we recommend is getting also
this same creatives with local

credit, with French credit if
you're targeting the French market.

But my, uh, my advice would be do both

Jacob: Yeah.

Xavier: test.

English speaking ads, but also use
local creators or use AI to get, um,

to get, um, creative in local language.

Jacob: Do you think of the, let's say,
uh, English, English UGC in the French

market and French UGC in the French market
as like different placements where you

have, you know, audience network and you
have Facebook feed, where potentially the

different creators reach different people,
and so you have like different reach?

Or is it just that sometimes
it works, sometimes it doesn't.

Xavier: Yeah, no, it's a good question.

Especially with meta.

They, they, you know, they
launched, uh, under meta recently,

and, um, the diversity of key.

So, for example, if you launch an a
campaign, um, so let's say you, you

launch in 20 different creatives.

If you get 20 different creators by,
let's say, two or three creators,

or maybe you are only one creator,
meta or even TikTok, they will

consider that it's almost the same ad.

Okay?

So they will not reach,
reach out to new audiences.

it's.

super important to have
a diversity of creators.

The more diverse your creators,
uh, base is the more likely

you reach out to new customers.

So, so it's very, very important to
have many, many different creators and

this is how we can help because we.

Usually provide one creative for one
creator, and we have like 50 K active

creator including 20 5K in the us.

So you can get many different,
uh, many different faces, uh, to

promote your app super seamlessly.

Also, with ai, now you can define
any personnel you would like.

That makes it even easier to, um,
to, um, to multiply the incarnation

that will, uh, talk about your app.

Jacob: Got it.

Got it.

That makes sense.

That makes sense.

Uh, so, so we start with 20 to 40, how
many do you usually end up with, uh,

you know, based on that performance?

What do you, what do you usually see?

Xavier: So just first a number.

You, you need to know
that it's completely okay.

Uh, like, and, and an ad that
doesn't perform is pretty normal.

So usually the success rate is
around maybe 5%, something like that.

So, so.

Basically, as I told you earlier, the more
you scale, the more creative you need.

Plus you will need the ad
fatigue, so you will end up with

hundreds and hundreds of ads.

I mean, our number one clients now,
uh, today, they, we are delivering

more than 1000 creatives per month

Jacob: Wow.

Xavier: you an idea.

Jacob: Yeah.

What, what, like, you don't
have to give the exact, but like

what spend level do you need to
support that level of creatives?

Xavier: Uh, yeah, on their side, are, um,
they are, they are, um, seven figures per

Jacob: Right, right.

Xavier: in terms of media spend.

Um, so they're, they're big spender
on, uh, both TikTok and meta ads.

but, uh, but even with, uh, five or
six figures, you need, uh, minimum

20 to 50 creatives per month.

Jacob: Yeah.

Yeah.

Just because of the
creative fatigue is so fast.

Uh, yeah.

Yeah.

And not all of it performs.

Xavier: Yeah, basically a game of numbers,

Speaker: today on the Price Power Podcast
we're talking with Anthony Scarce.

Anthony was a global VP of Growth at
Acorn's, senior director at NerdWallet.

He's also worked in growth roles
at Betterment and Blue Apron.

He's now a growth consultant and
advisor for consumer subscription brands

through his own consultancy, t Nomatic.

And today we're talking referral programs.

Anthony Scarpaci: so when I think
about referral programs, there

are quite a few key elements
of what makes them successful.

When you mentioned Dropbox and
PayPal, the first thing I would

share there is them being some of
the earliest brands to do this space.

Uh, the, the world of this incentive
based structure was generally pretty

new, so, uh, it drove a lot of
interest and enthusiasm from the base.

Uh, and so the testing of these.

Two-sided offers of either
free storage or, um, additional

incentives worked really well
for them to start to drive scale.

And in the case of Dropbox, there's also
something really endemically, um, sort of

natural within the product experience that
makes sense for, uh, that, that growth

loop from an organic perspective as well.

Um, for brands that are, uh.

Not necessarily, um, ones that
have a sharing component, uh,

naturally baked into their product.

Uh, and they're starting to think
about referral programs beyond

natural word of mouth after they
have really strong product market fit

and have hit a good level of scale.

I like to think about it in,
uh, what I like to call the, the

right framework, um, which is just
something I, based on what I've seen

and then what I've led at Acorns.

Um, and so we could break that
down of the different components.

'cause I think it would be
helpful to think about all

the various, uh, facets of it.

Um, the first one being r is relevance.

So one area that a lot of brands
struggle is they think, well, they

just need to do cash incentives.

But if cash is not aligned to
the, the product delivery or what

the customer's relationship to
the brand is, it can be jarring.

It could not be motivating.

So you need something that is both
relevant for the referring individual as

well as the customer on the other side.

Um, the next one is, uh, okay.

How do you think about that incentive
structure and beyond the relevancy,

you need to think about something
that is gonna get somebody out of

bed, if you will, to start to refer.

'cause one of the biggest challenges that
I see in a lot of referral programs is

businesses have, uh, this small evergreen
program that sits in the background.

Um, it's a small incentive.

Not necessarily the most compelling.

It's not, uh, well promoted and so it
doesn't get a lot of participation.

Um, so really thinking about what
are gonna be the incentives that

you use to get people excited
and motivated, but also are gonna

further entrench both your existing
customer and your referring customer,

customer deeply into the product.

Um, so I've done, you know, a lot of
testing at brands like Acorns where

we were testing, you know, certain
types of incentives, uh, and then

identifying, well those things.

May not match, uh, as well to the
long-term customer behavior that we want.

So we continue to experiment with
ultimately what are the incentives

that are gonna lead to a satisfied
customer on both sides, who's in

love with the product, and further
becomes, you know, an authentic

advocate for the brand going forward.

Um, so that's, I, um, g is guardrails.

So while that's great, while you have a
lot of these compelling incentives, you

need to make sure that you're putting
the right mechanisms in place to prevent

fraud, gaming, uh, all of those factors.

Um, a good example of where this went
wrong, earlier in my career, I was at,

uh, uh, blue Apron when it was pre IPO,
um, seemed like a rocket ship at the time.

And within that meal delivery kit
space, everybody was effectively

doing referral programs where there
would be a free box that you could

send to a, a friend or family member.

No strings attached.

No questions asked.

So one, there would naturally be some,
uh, some gaming from individuals just

by changing locations or individuals
who live at the same address.

But then also for the friend side, there
was no mechanism to acquire them to build

that habit or give them a chance to build
that habit paired with the incentive.

And what you see now is, um, in the
remaining leaders in that space, like

HelloFresh, uh, any sort of referral
programs require that there is, you

know, a, a series of deliveries for the
customer to really try the product and

make sure, um, you know, and really give
it a chance to see if it's, you know, a

fit for their lifestyle and, and for them
to fall in love with it, which leads to

more sustainable, you know, profitable
customer acquisition through that channel.

So I think that's one area that
some brands also get it wrong,

not thinking about guardrails and
the, the right requirements of the

program when getting it started.

Um, the next one, uh, h
So this one is really key.

And this one I think, you know,
broadly from a marketing perspective,

there's always areas that are missed.

Uh, this is about being human-centric
and by being human-centric, I

think there's a few layers there.

There is, uh, transparency, um, and
authenticity, but there's also the

consistency of the customer journey
and how you present the program to

those customers, uh, within your app
experience, through email, even through

paid or organic social amplification.

Um, when I look at some referral
programs of existing clients that I

have now, um, or just other brands.

Sometimes I see, well, maybe they
have the right incentives in place,

but they're not getting them in front
of the customer, uh, in a really

prominent way or at the right time.

Um, so that's a key area to think about.

Well, okay, let's take this,
the journey, one step at a time

for the referring customer.

How do they learn about the program?

As we talked about before, make
sure those incentives are compelling

enough for them to be interested
in, in activating and participating.

How do you reduce all of the frictions
to make it as easy as possible for their

them to share the referral opportunity
with their friends and their network?

And then how do you make sure every single
touch point along the way continues to

reinforce, um, what are the, what are
both sides getting out of it, reminding

them they're in the right place, providing
the education on the product to that

new customer along the way, and making
sure that as they go through the signup

process, they know I'm in the right place.

I'm still, I still have this
incentive following me so

that I'm gonna get rewarded.

I know my friend will get rewarded.

Um, and, and close that full loop,
um, to make sure that it feels

consistent and congruent, uh, from
an experience perspective versus.

Piecemealing all these components
that don't feel natural or

make sense to a customer.

Um, and that's one area that, that
people sort of fall off as well.

Um, and then the, the last one of
t is really sort of two pieces.

It's timing and tracking.

So timing, we talked a little bit
upfront about like when is the right

time for you to have a referral program?

And part of it is you have to be
post product market fit, you have

to have customers who love you.

Um, you have to already have
hit some of those milestones.

And then you also have to have a
meaningful scale of your customer base

to make the program do anything sort of
really meaningful for you because you

have to be careful about cannibalization.

Organic word of mouth.

So the moment that you start to
incentivize in any meaningful way, uh,

sharing, you're, you're naturally gonna
have some level of cannibalization of

those who have come in before, uh, who
would, sorry, who would refer otherwise.

So you really wanna make sure
that you cut that piece down.

Um, or, or you can, sorry, I should
say you really wanna make sure

that you consider that before uh,
you start to incentivize things.

'cause it will change your blended.

Customer acquisition costs and
economics for the business.

Um, and then lastly on tracking,
this is underlying fundamentals.

You need to make sure you have a
system so that you can track, uh,

successfully all the referrals that are
happening from your existing customers,

um, so that you maintain a really
positive experience for both of them.

And ideally, uh, you get to a place
that you can create some transparency

and visibility for them to know the
status of their referrals and when

they're getting paid for successful
referrals, all of those pieces.

So, um, there's a lot there.

But yeah, to reinforce sort of the
framework is this right framework, right?

Relevancy, incentives, guardrails,
uh, human centricity, um,

and then timing and tracking.

And I think by making sure you consider
and walk through all of those steps,

a brand can understand, is referral
the right thing for us right now?

Do we have the capacity to start
to test into this and, and where

it might be the biggest gap so we
can refine the program over time.

Speaker: Thanks for listening.

Hope you enjoyed.

Please go to price power podcast.com

to see all the episodes.

Go to Spotify and YouTube and
give us a subscribe and follow

so you don't miss any episodes.

Alright, talk to you next time.

All.