A/B Testing in Marketing

Listen, if you say “fail fast” but spend forty minutes in a post-mortem grilling the team on why a test underperformed — you just taught everyone in that room that failure is dangerous. If you say “take risks” but give the bigger raise to the person who hit their number safely rather than the one who tried something ambitious and came up short — you’ve told your team exactly what you actually value. You can’t talk your way to an experimentation culture. Ninety-five percent of the signal comes not from what you say, but from those three decisions. Reward. Promote. Fire.

The companies that grow the fastest scale their experimentation programs. These are the 3 keys: 1. Trustworthy experiments 2. Institutional memory 3. Data culture Let me explain each. — PILLAR 1: TRUSTWORTHY EXPERIMENTS Three challenges block trust. Here’s how to solve them: Challenge 1: Outlier Customers One enterprise client can skew data like 200 average users. Results warp. You build for the 1%, not the majority. Solution: Use stratified sampling. Balance test groups by customer size. Turn outliers into insights, not noise. Challenge 2: Novelty Effects Week 1 shows amazing results. By Week 6, you're back to baseline. This classic trap wastes months on temporary wins. Solution: Track metrics over weeks, not days. Create holdout groups to measure true impact. Don't celebrate until you see sustained value. Challenge 3: Consistency Issues Different teams get contradictory results. Trust collapses. Progress stalls. Solution: Standardize methodology across teams. Create unified playbooks. — PILLAR 2: INSTITUTIONAL MEMORY Most companies run experiments but fail to build lasting knowledge. Here are the 3 elements you need: Element 1: Batting Average View Track your success rate (industry average: 33%). Measure your average lift (typically 8%). Focus on high-probability experiments instead of random testing. Element 2: Frictionless Documentation Documentation fails when it's manual work. Automate capturing rationale, setup, and results. When documentation is automatic, it actually happens. Element 3: Cross-Team Learning Growth, marketing, product—each runs valuable experiments. Insights often die in silos. Build shared repositories. New hires gain years of wisdom instantly. — PILLAR 3: DATA CULTURE Even perfect experiments fail without the right cultural foundation. These 3 elements create that foundation: Element 1: Standardized Definitions Create a metrics dictionary everyone follows: Revenue = Monthly recurring revenue only Engagement = Sessions >2 min with 3+ page views When everyone measures the same way, results become comparable. Element 2: Truth Over Gaming Value right actions over being right. Create safe spaces for negative results. Element 3: Statistical Literacy Help teams understand error margins. Separate signal from noise. No advanced degrees required. Just enough knowledge to make good decisions. — LEARN MORE In my deepdive (free, no paywall thanks to Statsig): https://lnkd.in/etAGf7Nu — THE BOTTOM LINE The cost of not building this system? Testing the same ideas repeatedly. Forgetting what you've learned. Seeing competition pull ahead. What pillar do you need to focus on?

I watched a client go from taking weeks to launch experiments... to literally a few hours. Here's how I helped them 👇 When I started working with this client, their experimentation program was slow. Every idea had to work its way through a maze of approvals, backlogs, and coordination across teams. By the time something launched, the window of opportunity had already closed. I wanted to help them move faster. MUCH faster. So I started introducing the things that set world-class experimentation cultures apart from everyone else. What, pray tell, might those things be? Well, I love explaining with a particular and very true story that goes around at a former employer: there was a copywriter who had an idea on her bike ride to work, came into the office (this is pre-pandemic, people), made the change in her CMS (which was connected to the experimentation platform), and launched it as a global experiment running before her first coffee. That’s the kind of speed I wanted my client to experience, and we made it happen. How? Sure, we tightened up their tech and data flows so experiments could run smoothly. But this was the minor point in all honesty. The real shifts came from bringing together a cross-functional team who had the skills to deliver autonomously, getting leadership backing for that team to take risks, and setting a clear and focussed goal for the team to rally behind. We removed unnecessary “approvals,” facilitated the essential conversations, created focus, and rewarded pace without compromising rigour (improving it, actually). The team became empowered to make their own decisions and built a culture that normalised risk-taking. The result was night and day. Just weeks earlier, ideas took months to get through approvals, builds, and launches. All before even monitoring, reporting, and decision making (if any) took place. Now? The team could go from an ideation session to launching quick wins in literally hours. And, I can tell you first hand, when a team experiences this shift from moving like a snail to sitting up front of a rocket ship, that acceleration brings creativity, confidence, and energy. The kind that spreads across teams and compounds. It’s infectious. Oh, and did I mention that they started getting more runs on the board too? 😉 So recap, what makes these cultures different? And what did we instil to help this team accelerate that fast? ✅ Leaders who empower and trust their teams. ✅ Teams with genuine ownership and motivation to create impact. ✅ A culture that celebrates learning, not just winning, and treats failure as fuel for improvement. That’s what lets someone move from an idea on a bike ride → to a global experiment before their first coffee. If you want to innovate and experiment at lightning speed, don’t just look at your tech. Look at your culture. Could your org handle that kind of speed? 👇

What happens if you layer a free trial on top of a freemium experience? We recently did this for a $11M ARR SaaS and are seeing promising results… Context: One of our clients offers both freemium and free trials. Some CTAs on their site lead to freemium (no credit card, limited features). Other CTAs lead to opt-out trials (credit card required, 14 days to test everything). Although the CTAs looked identical to users, they led to completely random experiences. Freemium users who wanted to upgrade could only pay immediately. But opt-out trial users got 14 days to test paid features before getting charged. So we introduced trials for freemium users. Now when they want to test a premium feature, they can start an opt-out trial - enter credit card, test for 14 days, get charged after. Result: freemium-to-paid conversion rates improved significantly. Why did it work? Freemium users were already bought into the product. They'd used it and found value. Giving them a trial removed the friction of committing to paid features they'd never tested. This is one of those simple changes that feels obvious in hindsight but most companies still aren't doing it.

If you’re still testing creatives the same way you did in 2022… you’re already losing. 2025 is a different animal. More platforms. More formats. More distractions. And algorithms that reward speed of iteration over “perfect” production. Here’s the truth no one likes to admit: Your creative testing process is probably the single biggest choke point in your growth. 1. Volume > Perfection I get it. You want that one polished, cinematic ad that “crushes it” for six months. But the reality? That’s a lottery ticket. The brands winning in 2025 aren’t looking for a unicorn. They’re running dozens of creative variations every week, different hooks, angles, and formats, because they know frequency of testing beats guessing right. The question is no longer: “Will this ad work?” It’s: “How fast can we find the winners and kill the losers?” 2. Hooks Are the New Battlefield: Scroll speed is at an all-time high. If your hook doesn’t land in 1.5 seconds, you’re invisible. I break hooks into three buckets when testing: Problem-first: Agitate the exact pain point your audience feels. Pattern interrupt: Visual or copy that makes them stop mid-scroll. Social proof: Start with customer words, not brand words. In testing, the hook is the variable I isolate first, because it’s the highest-leverage change you can make for a faster read on performance. 3. Test for the Funnel Stage, Not Just the Platform: Too many marketers test “TikTok creatives” or “Meta creatives” without thinking about where in the customer journey they’re being shown. Your mid-funnel retargeting ad doesn’t need to be a 30-second cinematic story. It might just need a quick, benefit-driven product demo with a direct CTA. Conversely, cold traffic needs more narrative and education. Test creative by awareness stage, not just ad placement. 4. Creative → Landing Page Consistency: One of the biggest creative testing fails I see? Finding a winning ad… and sending traffic to a generic, unaligned landing page. In 2025, ad-to-landing-page congruence is non-negotiable. Your hero image, headline, and CTA on the LP should feel like the next chapter of the ad’s story, not a reset. We test landing page variants in parallel with ads, because the best creative in the world can’t fix a conversion drop-off. 5. Data Loops, Not Guess Loops: If your creative team isn’t meeting weekly with your media buyers, you’re leaving money on the table. We run tight data loops: Media buyers bring performance data. Creative team uses that data to inform the next round of variations. No egos. No “pet” creatives. If it’s not performing, it dies. In 2025, creative strategy is just as much a numbers game as it is an art.

We once tested a blue CTA button against a green one. The result? No meaningful difference. Then we tested a different offer behind the button and saw a 47% lift in conversions. This reinforced something we often see with clients: many teams are focused on optimizing surface-level elements, when the real opportunity lies deeper—in the offer itself. If you're running A/B tests and seeing minimal impact, it may be time to ask: Have your design changes had a noticeable impact? If not, it might be worth looking at how the offer itself is connecting with your audience. Small design tweaks can help. But, sometimes, it's the offer behind them that deserves the real focus. Here are five offer variables we’ve tested that tend to drive meaningful change: → Framing → Commitment Level → Personalization → Outcome-Oriented CTAs → Format Bonus tip: It’s often worth testing 𝘳𝘦𝘮𝘰𝘷𝘪𝘯𝘨 offers or simplifying choices. Reducing noise can improve clarity and performance. If your conversion rate has plateaued, don’t just test new button copy. Step back and ask: is the offer itself aligned with what your audience actually wants at that moment? We’ve seen teams get far better results by focusing less on polish, and more on relevance, clarity, and value. Would love to hear from others. What’s one offer test you’ve run that led to unexpected results? --- Follow Michael Cleary 🏳️🌈 for more tips like this. ♻️ Share with a marketer testing design instead of value.

Meta ads have changed AGAIN. And the advice most marketers are still following? Completely outdated. Here’s the so-called “expert wisdom” I still hear every day: ❌ “Audience testing doesn’t matter” ❌ “Launch 50 creatives every week” ❌ “Just keep tweaking the same ad” ❌ “Horizontal scaling is the way to grow” ❌ “Your account structure is everything” ❌ “Track CPM obsessively” ❌ “Always sell benefits” Sounds familiar? The truth: this playbook is backdated. But then what does work in 2025? After working across accounts with ₹50Cr+ in ad spend, here’s what’s actually driving results right now: ✅ Mobile-first video creatives win. Reels + Stories with thumb-stopping hooks in the first 3 seconds crush static ads. Meta’s algorithm thrives when you feed it creative variety. ✅ First-party data is gold. Syncing customer lists (buyers, subscribers, high-LTV users) → building value-based lookalikes = 25–40% lower CAC. ✅ Advantage+ & AI-led optimization. Advantage+ Shopping + dynamic creative testing consistently outperform manual setups, with 20–30% stronger ROAS. ✅ UGC > Studio-polished ads. Raw, authentic content-unboxings, testimonials, everyday use-gets up to 3–4× higher engagement than polished creative. If you’re still stuck on old “best practices,” you’ll burn budgets fast. But if you lean into these new levers, Meta ads still scale like crazy in 2025. 👉 What’s one Meta ads “rule” you think is completely dead in 2025? Drop it in the comments - I’ll share my take on it. #PerformanceMarketing #MetaAds #PaidSocial #EcommerceGrowth #DigitalMarketing #MarketingStrategy

This is not a new marketing idea… In fact, it’s one of the oldest tricks in the beauty industry and it’s making a comeback. Decades ago, Avon and Oriflame made scratch-and-sniff perfume samples an everyday experience. They tucked fragrance strips into catalogues and magazines, allowing customers to smell new launches while sitting at home, no stores, no queues, no sales pressure. Luxury brands followed suit. Calvin Klein, Dior, Chanel, Lancôme, Gucci, YSL - all used scented pages in Vogue, Elle, and Harper’s Bazaar. You’d flip the page, rub your wrist, and instantly decide if that perfume was “you.” The magic? You experienced the product at your own pace, in your own environment, without the pressure of a salesperson hovering nearby. Now in 2025… it’s trending again. Rare Beauty just took the same concept and reimagined it for Gen Z and Millennials with their “Snatch & Sniff” billboards in New York. You’re walking down the street in SoHo, you spot the billboard, you peel a strip, smell it… and maybe, you film it for TikTok. It’s the same strategy Avon and Chanel used in print, but recontextualised for the urban, content-driven, on-the-go world we live in now. Why this works brilliantly today: Self-directed experience → No sales pitch. You control the moment. Frictionless product trial → No counter, no tester bottle, no store visit. Sensory engagement → People don’t just see your brand; they smell and interact with it. Built for virality → Every peel-off could be a TikTok or Instagram story. Memorable → You remember where you were when you first smelled it. The real marketing lesson: - You don’t always need to invent something brand new. - You need to find timeless strategies that worked… and adapt them to a new context, platform, and audience. From coffee tables in the ‘80s to NYC streets in 2025, the principle is the same: Let the product sell itself. Would you stop for a snatch & sniff billboard? Or would you walk right past it? 👇 ----- Repost ♻️ to help your network ----- Follow Priya Kodali for more like this #MarketingStrategy #LuxuryMarketing #ExperientialMarketing #BrandActivation #RareBeauty #FragranceMarketing #OutdoorAdvertising #BrandEvolution

The Perfect Ad Creative Framework 8-Figure Brands Are Using in 2025... Our proprietary methodology - tested across dozens of D2C brands with millions in spend→ The 6Layer Creative Methodology: 1. Start With Audience (The Who) Beyond demographics, understand: • Market awareness level (Unaware → Problem Aware → Solution Aware → Product Aware → Most Aware) • Market sophistication (solutions they've tried before) • Psychographics, behaviors and funnel stage Best sources: Customer reviews, forums, direct conversations with customers. → Example: A skincare brand targets differently based on how many acne solutions the customer has tried before - awareness and sophistication determine messaging. 2. Define Your Core Offer (The What) Position your product based on their awareness: • Match their sophistication level • Focus on transformation, not features • Bridge the gap between current and desired state → For unaware customers: Educational bundles that introduce your solution → For sophisticated customers: Advanced products with unique differentiation 3. Determine Your Emotional Angle (The Why) There are 10 core buying emotions that drive most purchases - identify which 2-3 matter most for your specific audience, here are a few: • Feeling Security   • Feeling Belonging • Self-Actualizing Pair the desired emotion with your offer to naturally create an angle that shows your audience why it matters to them. The emotional angle connects your offer to deeper motivations - it's why someone truly buys. → Example: Premium cookware sells pride of mastery and joy of providing, not just pots. 4. Develop Your Creative Concept (How the Angle is Communicated) This is where you wrap your emotional angle in a compelling package that: → Creates pattern interrupt without confusing the audience → Delivers your emotional angle with clarity → Makes your offer feel inevitable Your concept must captivate while maintaining clarity. → Example: Dollar Shave Club's "Our Blades Are F***ing Great" concept. This triggers the male target's desired emotion of achievement and pride, granting them the opportunity to realize it through a purchase of the razor. 5. Tap Into Cultural Moments (The When) Connect your message to what's happening in your audience's world: • Seasonality relevant to your product • Social narratives your audience cares about • Cultural conversations they're already having This amplifies relevance by making your message feel timely and important. It's also a great way to reuse old offers and angles—just pair it with a new moment. 6. Production Quality & Style (The Wrapping Paper) Finally, decide on execution quality that reinforces your positioning: • Format: Video, photo, carousel, animation? • Quality level: High-production, UGC, in-house? • Aesthetic: Minimal, bold, authentic, premium? Each layer builds on and strengthens the previous one. When campaigns underperform, analyze which layer disconnected rather than starting over.

Most “creative testing” isn’t testing at all. ❌ What most brands call testing looks like this: - Swap a background color - Change the CTA button - Recut a 15s video into 12s That’s not testing concepts. That’s tweaking. And tweaks won’t scale. Here’s what true creative testing actually looks like: 1️⃣ Test Big Levers, Not Pixels ↳ Messaging angles, hooks, story structures, and offers drive performance, design polish doesn’t. ↳ Each new concept should explore a unique customer insight, not a color palette. 2️⃣ Volume is the Multiplier ↳ One new ad a month isn’t testing, it’s gambling. ↳ High-velocity testing means dozens of new concepts, fast iteration cycles, and letting data pick the winners. 3️⃣ Kill Fast, Scale Winners ↳ Read signals early: thumbstop rates, CTR, cost per add-to-cart. ↳ Don’t wait weeks,ads tell you in 48 hours if they’re worth scaling. 4️⃣ Iteration > Reinvention ↳ Once you have a winning concept, spin 10-20 variants: new hooks, new visuals, different headlines. ↳ Keep what works, evolve what doesn’t. The best brands treat creative testing as R&D for growth. Not a side project. Not a design exercise. A system that fuels scale. Found this useful? Like, follow, and repost ♻️ so others can too! ps. struggling with creative bottlenecks? We can help.