Some key takeaways:
• UX and CRO are two sides of the same coin
• Qualitative research helps explain the “why” behind user behavior
• A/B testing and experimentation are becoming more mainstream across industries
• Continuous learning and adaptability are critical as AI changes the digital landscape
• Treating life like an experiment can help you grow through challenges and change
• The future will still need human oversight, strategy, and critical thinking
Transcript
Marc:
All right, everyone. Welcome to another episode of UX Pathways. I have the honor of being joined by Deborah O’Malley. Deborah, how are you?
Deborah O’Malley:
I’m doing good.
Marc:
Glad to have you here.
Deborah:
I’m so excited to be here.
Marc:
I’ve been excited to have you on because we’ve collaborated for many years across many different areas, and you’re always sharing so much knowledge with the community. I wanted people to better understand your path, your journey, and how you got to where you are today.
Can you tell us about your current role in the industry and describe your journey?
Deborah’s Journey into CRO and Experimentation
Deborah:
Sure. I’ve been in the industry much longer than I’d like to admit because it makes me sound old, but it’s been about two decades now, which is hard to believe.
Today, I’m considered an industry leading CRO and A/B testing expert, but I definitely didn’t start there. It was a difficult climb with a lot of setbacks along the way. The industry was very male dominated when I entered it, and experimentation wasn’t even commonly called experimentation yet. People debated whether it was conversion rate optimization, CRO, or optimization with an “S” or a “Z” depending on where you lived.
The industry has matured a lot since then, and thankfully women are much more represented in leadership positions today.
Interestingly, my story actually started back in fourth grade. We had to do a hypothesis based science experiment, and I became fascinated with what people noticed first visually. I cut out shapes and colors from construction paper, posted them on a board, and asked people what they saw first.
At the time I had no idea, but that was essentially my first eye tracking experiment.
Years later, after completing a bachelor’s degree in communication studies and graphic design, I pursued a master’s degree specializing in eye tracking technology and studying how people viewed advertisements. Everything suddenly came full circle.
Eye Tracking, Psychology, and Message Framing
Deborah:
One of the key things I studied during my master’s research was message framing. Specifically, whether people paid more attention to positive messages or negative messages tied to loss aversion.
What I found was fascinating. People initially paid more attention to negative, loss framed messaging. But when it came to cognitive processing and recall, positive messages actually performed better. Negative messages grabbed attention, but people resisted internalizing them.
That research really shaped how I think about user behavior and persuasion today.
Entering UX and Discovering A/B Testing
Deborah:
After my master’s, I expected to go into eye tracking technology, but the industry moved more toward heat mapping tools instead.
Instead, I joined a local UX firm where I worked on large scale usability studies for the Government of Canada. During that time, one of my colleagues introduced me to a website called WhichTestWon, which I know is where you and I first connected years ago.
The site challenged users to guess which A/B test variation performed better and then revealed the real world results. I absolutely loved it because it blended psychology, design, analytics, and experimentation.
Eventually, WhichTestWon posted a job opening for a content creator. I applied and got the role, which transitioned me fully from UX into CRO and experimentation.
Over the years, I realized UX and CRO are really two sides of the same coin, but they approach problems very differently.
Bridging UX and CRO
Deborah:
Today, I work with Nielsen Norman Group and recently recorded an A/B testing course focused on user experience.
What I’ve noticed is there’s still a huge divide between UX and CRO. UX tends to focus on qualitative research with smaller sample sizes, while CRO focuses heavily on quantitative methods and statistically significant data.
Instead of working together, the disciplines often criticize each other.
What I advocate for is mixed methods research. We shouldn’t be choosing qualitative or quantitative. We should combine them.
Qualitative research helps explain the “why.” Quantitative research tells us the “what” and the “where.” Together, they create a much fuller understanding of user behavior.
The Loss of WhichTestWon and the Birth of GuessTheTest
Deborah:
Unfortunately, WhichTestWon was eventually sold, rebranded, and dissolved. It was heartbreaking because it contained years of case studies and historical experimentation knowledge dating back to the early days of the web.
People kept reaching out asking where they could still find those resources.
That’s what inspired me to create GuessTheTest, which has now been running for about seven years. It contains hundreds of A/B testing case studies, articles, and resources for practitioners.
At the same time, I continue consulting and helping organizations apply experimentation strategies in real world environments.
Defining User Experience
Marc:
That’s an incredible journey. One thing you mentioned that stood out to me was this divide between UX and CRO. How do you define user experience today, and why is that definition important?
Deborah:
It’s important because the divide became deeply embedded very early in the industry.
I came across an old article from Jakob Nielsen around 2005 where he argued that A/B testing was not the answer and that qualitative research was far superior because it explained the “why.”
That mindset contributed to UX practitioners heavily favoring qualitative methods, while CRO professionals emphasized large scale quantitative testing.
Instead of combining approaches, the two disciplines split into separate camps.
Now, because I’ve lived in both worlds, I see the opportunity to bridge them. It’s not about proving one method is better. It’s about integrating them to get a complete picture of user behavior.
Living Life as an Experiment
Marc:
You’ve clearly adapted throughout your career. Is there a particular mindset or philosophy you use when approaching change?
Deborah:
Absolutely. I genuinely believe people should live their lives as experiments.
I deeply internalize the idea of continuous optimization. When you see life through an experimentation mindset, failure becomes less personal. You become more willing to take risks because everything becomes a learning opportunity.
Whether it’s your career, your health, your wellbeing, or your mindset, you’re always iterating and improving.
That experimentation mentality has shaped how I approach both life and work.
AI and the Future of UX
Marc:
There’s obviously a lot happening with AI right now. What trends or technologies do you think people can’t ignore?
Deborah:
The biggest elephant in the room is definitely AI.
Honestly, at least once a day I wonder whether websites will even exist in five years. Will users still browse websites, or will AI agents simply deliver structured information directly to people without traditional interfaces?
If that happens, how do we optimize for it?
Will our users even be humans anymore, or will they increasingly be AI agents interacting with systems on our behalf?
Those are the questions keeping me awake at night.
At the moment, AI still requires a human touch. We still need people to guide, edit, and shape the output. But things are evolving incredibly quickly.
Career Advice for the Future
Marc:
What advice would you give people trying to navigate their careers right now?
Deborah:
I think adaptability is everything.
One area with huge opportunity is structured data, JSON, and understanding how AI systems consume information. Technical professionals who understand how AI interprets websites and content will have enormous opportunities.
For writers and content creators, I still believe there’s a future, but the skill is shifting toward editing and refining AI generated content into something authentically human.
And another massive area is governance, privacy, and AI policy. The amount of data these systems collect is enormous, and organizations are going to need people who understand ethics, privacy, and responsible AI governance.
Closing Thoughts
Marc:
Your journey really highlights how deep and specialized UX careers can become. A/B testing is only one small part of user experience, but you’ve built an entire career around it and helped countless people along the way.
Thank you so much for sharing your story today.
Deborah:
Thank you for having me. I really appreciate the opportunity.
Where to Learn More
Deborah:
People can visit my site, GuessTheTest.com. There are hundreds of free A/B testing case studies, articles, and resources available there.
I’m also active on LinkedIn under Deborah O’Malley, and I’d love to connect with people there as well.
Marc:
Thank you again for joining the podcast.
Deborah:
Thank you, Marc. Appreciate it.
Leave a Reply