Skip to Main Content
HBS Online
  • Courses
    Open Courses Mega Menu
    • Business Essentials
      • Credential of Readiness (CORe)
      • Business Analytics
      • Economics for Managers
      • Financial Accounting
    • Leadership & Management
      • Leadership Principles
      • Management Essentials
      • Negotiation Mastery
      • Organizational Leadership
      • Strategy Execution
      • Power and Influence for Positive Impact
      • Leadership, Ethics, and Corporate Accountability
    • Entrepreneurship & Innovation
      • Entrepreneurship Essentials
      • Disruptive Strategy
      • Negotiation Mastery
      • Design Thinking and Innovation
      • Launching Tech Ventures
    • Strategy
      • Strategy Execution
      • Business Strategy
      • Economics for Managers
      • Disruptive Strategy
      • Global Business
      • Sustainable Business Strategy
    • Finance & Accounting
      • Financial Accounting
      • Leading with Finance
      • Alternative Investments
      • Sustainable Investing
    • Business in Society
      • Sustainable Business Strategy
      • Global Business
      • Sustainable Investing
      • Power and Influence for Positive Impact
      • Leadership, Ethics, and Corporate Accountability
    • All Courses
  • For Organizations
    Open For Organizations Mega Menu
    • Corporate Learning
      Help your employees master essential business concepts, improve effectiveness, and expand leadership capabilities.
    • Academic Solutions
      Integrate HBS Online courses into your curriculum to support programs and create unique educational opportunities.
    • Need Help?
      • Frequently Asked Questions
      • Contact Us
    • Black, Latinx, and underrepresented minority professionals
      Pathways to Business

      Stories designed to inspire future business leaders.

  • Insights
    Open Insights Mega Menu
    • Business Insights Blog
      • Career Development
      • Communication
      • Decision-Making
      • Earning Your MBA
      • Entrepreneurship & Innovation
      • Finance
      • Leadership
      • Management
      • Negotiation
      • Strategy
    • All Topics
    • Sample Business Lessons and E-Books

      Gain new insights and knowledge from leading faculty and industry experts.

    • Free Guide

      Learn how to formulate a successful business strategy.

  • More Info
    Open More Info Mega Menu
    • Learning Experience
      Master real-world business skills with our immersive platform and engaged community.
    • Certificates, Credentials, & Credits
      Learn how completing courses can boost your resume and move your career forward.
    • Learning Tracks
      Take your career to the next level with this specialization.
    • Financing & Policies
      • Employer Reimbursement
      • Payment & Financial Aid
      • Policies
    • Connect
      • Student Stories
      • Community
    • Need Help?
      • Frequently Asked Questions
      • Request Information
      • Support Portal
    • Apply Now
Login
My Courses
Access your courses and engage with your peers
My Account
Manage your account, applications, and payments.
HBS Home
  • About HBS
  • Academic Programs
  • Alumni
  • Faculty & Research
  • Baker Library
  • Giving
  • Harvard Business Review
  • Initiatives
  • News
  • Recruit
  • Map / Directions
HBS Online
  • Courses
  • Business Essentials
  • Leadership & Management
  • Entrepreneurship & Innovation
  • Strategy
  • Finance & Accounting
  • Business in Society
  • For Organizations
  • Insights
  • More Info
  • About
  • Support Portal
  • Media Coverage
  • Founding Donors
  • Leadership Team
  • Careers
  • My Courses
  • My Account
  • Apply Now
  • …→
  • Harvard Business School→
  • HBS Online→
  • Business Insights→

Business Insights

Harvard Business School Online's Business Insights Blog provides the career insights you need to achieve your goals and gain confidence in your business skills.

 
Filter Results Arrow Down Arrow Up

Topics

Topics

  • Accounting
  • Analytics
  • Business Essentials
  • Business in Society
  • Career Development
  • Communication
  • Community
  • ConneXt
  • Decision-Making
  • Earning Your MBA
  • Entrepreneurship & Innovation
  • Finance
  • Leadership
  • Management
  • Marketing
  • Negotiation
  • News & Events
  • Productivity
  • Staff Spotlight
  • Strategy
  • Student Profiles
  • Technology
  • Work-Life Balance

Courses

Courses

  • Alternative Investments
  • Business Analytics
  • Business Strategy
  • CORe
  • Design Thinking and Innovation
  • Disruptive Strategy
  • Economics for Managers
  • Entrepreneurship Essentials
  • Financial Accounting
  • Global Business
  • Launching Tech Ventures
  • Leadership Principles
  • Leadership, Ethics, and Corporate Accountability
  • Leading with Finance
  • Management Essentials
  • Negotiation Mastery
  • Organizational Leadership
  • Power and Influence for Positive Impact
  • Strategy Execution
  • Sustainable Business Strategy
  • Sustainable Investing
Subscribe to the Blog
*
Please complete this required field.
Email must be formatted correctly.
Please complete all required fields.
RSS feed

Filters

Topics

Topics

  • Accounting
  • Analytics
  • Business Essentials
  • Business in Society
  • Career Development
  • Communication
  • Community
  • ConneXt
  • Decision-Making
  • Earning Your MBA
  • Entrepreneurship & Innovation
  • Finance
  • Leadership
  • Management
  • Marketing
  • Negotiation
  • News & Events
  • Productivity
  • Staff Spotlight
  • Strategy
  • Student Profiles
  • Technology
  • Work-Life Balance

Courses

Courses

  • Alternative Investments
  • Business Analytics
  • Business Strategy
  • CORe
  • Design Thinking and Innovation
  • Disruptive Strategy
  • Economics for Managers
  • Entrepreneurship Essentials
  • Financial Accounting
  • Global Business
  • Launching Tech Ventures
  • Leadership Principles
  • Leadership, Ethics, and Corporate Accountability
  • Leading with Finance
  • Management Essentials
  • Negotiation Mastery
  • Organizational Leadership
  • Power and Influence for Positive Impact
  • Strategy Execution
  • Sustainable Business Strategy
  • Sustainable Investing
Subscribe to the Blog
*
Please complete this required field.
Email must be formatted correctly.
Please complete all required fields.
RSS feed

What Is A/B Testing & What Is It Used For?

Graphic showing A-B testing
  • 15 Dec 2016
Anna Vallee Author Staff
tag
  • Analytics
  • Business Analytics
  • Business Essentials
  • CORe

You may have experienced meetings where a lot of ideas are circulated about how to improve an existing product or service. In these meetings, differing opinions can quickly turn into a battle of long-winded defenses. Fortunately, the emergence of A/B testing—once thought to be exclusive to tech firms—has become a viable and cost-effective way for all types of businesses to identify and test value-creating ideas.

Related: The Advantages of Data-Driven Decision Making

What Is an A/B test?

In statistical terms, A/B testing is a method of two-sample hypothesis testing. This means comparing the outcomes of two different choices (A and B) by running a controlled mini-experiment. This method is also sometimes referred to as split testing.

What Is A/B Testing Used For?

A/B testing is often discussed in the context of user experience (UX), conversion rate optimization (CRO), and other marketing and technology-focused applications; however, it can be valuable in other situations as well.

Although the concept of A/B testing was galvanized by Silicon Valley giants, the rationale behind A/B testing isn’t new. The practice borrows from traditional randomized control trials to create smaller, more scalable experiments.

For this reason, professionals also perform A/B testing to gather valuable insights and guide important business decisions, such as determining which product features are most important to consumers.

A/B testing is a popular method of experimentation in the fields of digital marketing and web design. For example, a marketer looking to increase e-commerce sales may run an experiment to determine whether the location of the “buy now” button on the product page impacts a particular product’s number of sales.

In this scenario, version A of the product page might feature the button in the top-right corner of the page while version B places the button in the bottom-right corner. With all other variables held constant, randomly selected users interact with the page. Afterward, the marketer can analyze the results to determine which button location resulted in the greatest percentage of sales.

New call-to-action

An Example of A/B Testing

As a basic example, let’s say you’re an abstract artist. You’re confident in your technique but aren’t sure how the outside world—and, more importantly, art critics—will respond to your new paintings. Assessing the quality of art is a famously challenging process.

To employ A/B testing for this scenario, start by creating two different paintings that are alike. As you paint both pieces, change one small thing—for instance, add a red square to one painting and not the other. Again, this means that everything about the paintings is alike except for this one modification. Once the change is made, display the two paintings in randomly selected art galleries across the country and wait for your art agent, or another unbiased third party, to gather reactions and report back.

After each painting has been placed in a reasonable amount of art galleries, the feedback may reflect that the painting with the red square received significantly more praise, or maybe it didn’t. The hypothetical outcome doesn’t matter. Rather, what matters is that you can be reasonably confident that your change will or will not make the painting better, and you can go on to create better art as a result.

America's Most Wanted by Komar and Melamid
"America's Most Wanted" by Komar and Melamid used a different technique (surveys) to create a painting that catered to the American public's art preferences. Source: Dia Art Foundation.

The randomization aspect of this design is explicitly emphasized because randomization is the gold-standard for eliminating biases. Art is a subjective field and evolves over time. So do the preferences and opinions of customers, clients, or coworkers. A/B testing isn’t a static process, and tests can be repeated or complemented if companies believe that findings may not be valid or applicable anymore.

As a final note, it’s imperative that A/B testing design be rigorous to ensure the validity of results. Furthermore, there may be some decisions where internal opinions are more cost-effective or timely.

Making Data-Driven Business Decisions

Companies like Google, Amazon, and Facebook have all used A/B testing to help create more intuitive web layouts or ad campaigns, and firms in all industries can find value in experimentation. Using data-driven decision-making techniques empowers business leaders to confidently pursue opportunities and address challenges facing their firms. Customers benefit and companies can reap measurable monetary returns by catering to market preferences.

Do you want to learn how to apply fundamental quantitative methods to real business problems? Explore Business Analytics and our other analytics courses to find out how you can use data to inform business decisions.

This post was updated on January 12, 2021. It was originally published on December 15, 2016.

About the Author

Anna Vallee is a former Research and Teaching Assistant for the Business Analytics course at Harvard Business School Online. She received her Ed.M from the Harvard Graduate School of Education in 2015 where she studied experimental and quasi-experimental research design, applied data analysis, and management practices related to non-profit and educational institutions. Prior to joining HBS Online, she was the Manager of Research and Data Analytics at another Boston-based edtech startup. A lifelong learner, she is always looking for a great book to read.
 
All FAQs

Top FAQs

How are HBS Online courses delivered?

+–

We offer self-paced programs (with weekly deadlines) on the HBS Online course platform.

Our platform features short, highly produced videos of HBS faculty and guest business experts, interactive graphs and exercises, cold calls to keep you engaged, and opportunities to contribute to a vibrant online community.

Are HBS Online programs available in languages other than English?

+–

We expect to offer our courses in additional languages in the future but, at this time, HBS Online can only be provided in English.

All course content is delivered in written English. Closed captioning in English is available for all videos. There are no live interactions during the course that requires the learner to speak English. Written English proficiency should suffice.

Do I need to come to campus to participate in HBS Online programs?

+–

No, all of our programs are 100 percent online, and available to participants regardless of their location.

How do I enroll in a course?

+–

All programs require the completion of a brief application. The applications vary slightly from program to program, but all ask for some personal background information. You can apply for and enroll in programs here. If you are new to HBS Online, you will be required to set up an account before starting an application for the program of your choice.

Our easy online application is free, and no special documentation is required. All applicants must be at least 18 years of age, proficient in English, and committed to learning and engaging with fellow participants throughout the program.

After submitting your application, you should receive an email confirmation from HBS Online. If you do not receive this email, please check your junk email folders and double-check your account to make sure the application was successfully submitted.

Updates to your application and enrollment status will be shown on your Dashboard. We confirm enrollment eligibility within one week of your application.

Does Harvard Business School Online offer an online MBA?

+–

No, Harvard Business School Online offers business certificate programs.

What are my payment options?

+–

We accept payments via credit card, wire transfer, Western Union, and (when available) bank loan. Some candidates may qualify for scholarships or financial aid, which will be credited against the Program Fee once eligibility is determined. Please refer to the Payment & Financial Aid page for further information.

We also allow you to split your payment across 2 separate credit card transactions or send a payment link email to another person on your behalf. If splitting your payment into 2 transactions, a minimum payment of $350 is required for the first transaction.

In all cases, net Program Fees must be paid in full (in US Dollars) to complete registration.

What are the policies for refunds and deferrals?

+–

After enrolling in a program, you may request a withdrawal with refund (minus a $100 nonrefundable enrollment fee) up until 24 hours after the start of your program. Please review the Program Policies page for more details on refunds and deferrals. If your employer has contracted with HBS Online for participation in a program, or if you elect to enroll in the undergraduate credit option of the Credential of Readiness (CORe) program, note that policies for these options may differ.

 

Sign up for News & Announcements


  • • Please complete this required field.
  • • Email must be formatted correctly.
  • • Please complete all required fields.

Subject Areas

  • Business Essentials
  • Leadership & Management
  • Entrepreneurship & Innovation
  • Strategy
  • Finance & Accounting
  • Business & Society

Quick Links

  • FAQs
  • Contact Us
  • Request Info
  • Apply Now
  • Support Portal

About

  • About Us
  • Media Coverage
  • Founding Donors
  • Leadership Team
  • Careers @ HBS Online

Legal

  • Legal
  • Policies
Harvard Business School
Copyright © President & Fellows of Harvard College
  • Site Map
  • Trademark Notice
  • Digital Accessibility