In this lesson, we will cover how to setup a subject line or content AB test without a rollout where you will be able to split your contacts in several groups without having the option to choose a winning version. Setting up an AB Test requires a working knowledge of CONNECT and its basic features. Before reading this article, we recommend that you review the following articles:

  • How to Set Up a Message
  • How to Upload a List
  • How to Set Up a Deployment

1.  Creating an AB Test Message


Creating an AB test message




2.  Setting up the Message


Setting up the message


  • The process for setting up your AB Test message is very similar to the regular process for setting up a standard message except that you need to define the Test Type and where you wish to get the HTML of your message from.
  1. Test Type: Subject line This will allow you to define and test various subject lines with the same HTML template. Content This will allow you to define and test various versions of your template with the same subject line.
  2.  Using HTML from: This feature allows you to grab the HTML code from a Template or an existing Standard message and use it for the AB Test message. If you are setting up a Content test, it will appear as the first version of your content.



3.  Subject Line A/B Test Message


Subject Line A/B test message


  • To define your subject line versions, navigate to the Envelope tab where can define up to 10 different Subject line versions.
  • For each subject line, you can take full advantage of the Dynamic subject line feature and add in personalization or logic to test different subject line scenarios. 
  • To add a new version, simply click on the "+" icon. You can also remove versions by clicking on the “-“ icon.



4.  Content A/B Test Message


Content A/B test message


  • To define your content versions, navigate to the HTML tab where you can define up to 10 different Content versions.
  • Content versions require a few extra steps in order to create each version. 
    • To add a content version, click on the “+” icon next to version changer.
    • A dialog will appear that will ask you to either copy the HTML from an existing version or from a Template or existing Standard message.
  • For each version, you will also need to create a text version on the Text tab. 
  • Finally, you have option of define different link names for each version on the Click Tracking tab.



5.  Previewing each Version


Previewing each version


  • The Preview tab provides a great opportunity to view each of the different versions of your AB Test message.
  • By selecting the version in the drop down menu, you will be able to preview each version of the subject lines or content.



6.  Sending Tests of each Version


Sending tests of each version


  • Another way to test the versions of your AB Test message is on the Test tab.
  • You can select which versions you would like to send to each of the recipients you select (including yourself).



7.  Setting up a Deployment


Setting up a deployment


  • Now that you have set up your message, you can start setting up the AB Test Plan. To start, navigate to the Deploy section, click on the AB Testing tab and click on "Create New A/B Test".
  • The first step of the set-up process will look very similar to the set-up of a regular deployment. The difference here is that you need to provide a name for your Test Plan.
  • We have also introduced a new feature where you are able to save and return to an AB Test Plan set-up as long as you define a “Test Name” and click “Create Test Plan”.
  • However, in order to move on to the “Test Plan Builder” step, you must define the message and the target before it will become available.
  • To finish off this step of the set-up, select your AB test message, define your target and choose your option for scheduling.




8.  Setting up your Sending Strategy


Setting up your rollout group


  • Once you are on the Test Plan builder step, you can review the versions from your AB test message and the total size of your target before you split it into test and roll-out groups.
  • To get started, select “No Roll-out” from the Sending Strategy drop down. 
  • This will cause a slider to appear where you can change the splits for your test groups, ultimately affecting how large your roll-out group is (in red).




9.  Defining your Test Groups


No Rollout


  • You can splits test groups based on percentage or count : There is a slider that represents each version, you adjust the percentage and count by simply moving the slider.
  • (High number contacts will have to be split in percentage.) 
  • Reminder Notification Set the time for your notification, once a winner is recommended an email will be send to you with the results.
  • Recommend a winner statistically based on Open rate, Unique click rate or Unique click rate as % of opens Repeat the notification until a winner is chosen Send a notification to other stakeholder.



10.  Deployment Review


Deployment Review


  • The deployment review step will allow you to review everything before you begin the test plan. 
  • Just as a reminder, when you “Start” the test plan, only the version or “Split” groups will be sent to.
  • The Rollout group will not be deployed to until you select a winning version. 
  • Toggle between version to preview the different data points Preview splits in count by version.



11.  AB Test Plan Review


AB Test Plan review


  • After launching the AB Test plan, you will be taken back to the Deploy section. From here, you can : See the status of your AB test Click on the test plan name to view your test plan.