This is a story of the Gumshoe test campaign. This case study covers inputs for our methodology, test campaign setup and launch details, some interesting learnings, outcomes of the completed test campaign, "phase 2" which was about doubling down to get more registrations, and our final recommendations and remarks.
In case you're new here, we offer turnkey testing to measure demand and clarify the target market for new business ideas, products or campaigns. We help savvy founders, heads of innovation and agents of change.
Gumshoe is proposed to be "a fun tracking device for monitoring your child’s whereabouts". The creator is Mohammed Billoo, a mechanical engineer and family man.
The Gumshoe test campaign is a success story for Validation.Run, as it became the #2 best performing campaign we’ve ever run. Mohammed elected to run a second round of the campaign to generate even more registrations. We're all thrilled with the results and Mohammed is now continuing development with the knowledge that his idea has measured interest.
Context On Campaign VRC023
We ran the original classic campaign from late-April to early-May. It's a 3-week turnaround which includes 2 weeks to run the actual test part of the campaign, bookended with setup at the beginning and reporting at the end. Our entire methodology is standardized to maintain the efficacy of our data.
We've done a lot of work to streamline the whole process as well, to make it as efficient as possible. Gumshoe was our 24th campaign and our immediate growth goal is to run 100 campaigns. We're already able to see a normal distribution in the data we've collected, which matches other benchmark data that we pull from trusted industry resources. So when we say your idea is exhibiting demand above the benchmark, you know that's real.
Inputs To Our Methodology
Start With What You Know
Mohammed's experience with hardware products, especially around embedded systems, coupled with his perspective as a parent has clearly led him to this idea. We think it's good practice to start in a niche you know well. As a member of the target market, you're already at an advantage having experienced the pain points that you're seeking to solve.
No Assumptions
It's also good practice to check your assumptions and biases around what ideas will work. There are many complex variables that go into business success: like market conditions, competitors and alternatives to name a few. The point of our process is to measure what we can, to gain at least a bit of control in this situation. We're using a more scientific approach to get a measure of what real people think, instead of just guessing.
The #1 Question
The first question we ask of all our clients is "what's the #1 thing they're looking to learn from your target market?" This helps us understand where the client is at in terms of expectations, and also what they're literally seeking to learn in the process. It's a great place to start the conversation.
Mohammed had 3 design features he was considering for the final product and he wanted to get an idea of how to prioritize these:
Easy for Parents to Affix
Stylish Enough For Kids
So Secure It Won’t Be Removed
We set out to answer this specific question in the process of running the campaign.
See here for a “Whiteboard Session” explaining our methodology in more detail.
Setting Up & Running Our Classic Test Campaign
It Begins And Ends With Positioning
The next step is a discovery phase where we set out to define the positioning statement. We find this is the most crucial piece of our test campaigns and any marketing initiative for that matter. It's where you define what the thing is, who it's for and why they would be interested. If you don't have this locked down, you risk being off-target with your marketing efforts. Aside from measuring general demand through our process, we are specifically looking to validate positioning for the new idea. The goal is to have data backing up your assertions on who your targeting and why this is a fit for them.
How We Test
Once we have the positioning statement nailed down we get to work on creative assets and the ad campaign itself. The summary of how we run tests is that we use targeted ads (Google Search) to drive traffic to a landing page with the new offer. We then look at the engagement data through this mini funnel, including registrations on the page, and compare against benchmarks to derive our measure of demand. If a campaign does very well, as it did with Gumshoe, we'll see immediate registrations and high engagement data.
Gumshoe example Google Search Ad
Gumshoe landing page with offer and CTA
After we've built everything and received client sign-off, we go live. The test runs for a minimum 2 weeks because that's the amount of time it takes Google Ads to complete its learning phase. It's bad practice to make any changes to an ad campaign during this time frame, as it will interrupt Google in learning how best to serve your ads.
Immediate Signups, Good Signals
Gumshoe flew out of the gates with a CTR (Click Through Rate) above benchmark and its first registration on day 1. At this stage we were cautiously optimistic.
Fast forward to day 6 and now we're cruising at around 2x the CTR and 3/4 of the CPC (Cost Per Click) benchmarks. We're also getting some definition on the importance of the 3 design features Mohammed had asked about. Ok, now we're getting excited!
Always Interesting Learnings
Every now and then, we get some real oddball insights through this process. Another reason to check one's assumptions. Due to the nature of the ads we had built, we were seeing searches being made for "child tracking device hidden" and "subcutaneous GPS for kids". Not at all what we were intending or even looking for, but still insightful.
Perhaps this is more proof of my theory that the most innovative products sit at the intersection of impressive and creepy. It's bleeding edge stuff!
Outcomes
Success
The original test campaign closed out a success. We clocked Curiosity (CTR) at 108% above the benchmark, Cost (CPC) at 22% below the benchmark, and Demand (CVR - conversion rate) at 21% above benchmark. All good signs!
Performance gauges from the report summary page.
Mohammed also ended up with 31 registrants. These are all real people interested in learning more about Gumshoe.
The signup form we use to capture these expressions of interest has an open text field that people can add to if they chose. It provides qualitative data direct from would-be consumers and forms another signal that we use.
Customer development has already begun and can continue as Mohammed engages with his new audience through product development updates and requests for feedback.
Confirmed Positioning
Perhaps most importantly, we got clear data validating the specific messaging used to offer up this new product idea. Not only do we now know how to describe this new offering and for who it's for, but we also now have a scale of how important the launch features are to these parents:
“Easy for Parents to Affix” at 593 impressions
“Stylish Enough For Kids” at 412 impressions
“So Secure It Won’t Be Removed” at 288 impressions
It seems simple in hindsight, but it's powerful to prove through experimentation that "GPS tracker for kids" is the way to go.
Doubling Down with Phase 2
New Goal Of 100 Registrations
Since the campaign was such a success and we were generating leads at 60% below benchmark CPA (Cost Per Acquisition), Mohammed elected to run a second phase to build up his new audience even more.
We didn't make any substantive changes to the campaign. We only made a handful of keyword changes to dial it in even more. And we added a checkbox on the signup form for people to claim a free prototype if they wanted. A great idea from Mohammed, boosting the feedback loop to support his product development.
We ran the campaign for 4 more weeks and ended up with 115 total registrations!
Conclusion & What’s Next
We achieved what we set out to do in helping Mohammed measure demand for Gumshoe and clarify his target market. We also generated a list of real people asking for his offering! He now has a clear advantage regarding his distribution strategy.
Mohammed can build in confidence. He has a direct line of communication to the parents, the people he's building this for. He can take their feedback and put it right into product development, accelerating delivery and satisfaction for everyone.
But the deeper benefit is avoiding becoming another statistic: "No market need" is one of the top reasons startups fail. You can spend a fraction of traditional business costs to get the right data, in weeks, not months. And then use this to make an informed decision on how to proceed with your next business opportunity.
With our methodology at your disposal, there is no excuse for wasting time and resources on a hunch. Let us know when you’re ready to run with it.