Beware! Slow Down Your A / B Testing
A couple months, ago we decided to implement A/B ad testing for one of our clients for their top four performing ad groups. We would have 2 different ad copies per ad group that were optimized based on conversions. Depending on which of those text ads wasn’t converting in comparison to the other, the lower of the two was replaced. The client would send us new text ads, and sometimes landing pages, to test every two weeks. This would eventually become the flaw of the “top performing” ad groups.
A good two months has gone by since we implemented ad testing. As we expected, the testing hurt their quality score as average cost per click and average position both took a hit. Below is the result of one of the four ad groups that we had implemented for ad testing. The results were quite identical for the other three ad groups.


As you can see from above, at the start of the ad testing implementation, average CPC was at $2.01 and average position was at 7.2. When the first rounds of changes were made, average CPC jumped to $2.03 and Avg. position went to 7.3. As changes were being made to text ads and landing pages every two weeks, you can see that the average CPC and average position were steadily increasing every week. During the last week of text ad testing, average CPC hit a high of $2.37 with an average position of 8.3.
Since we have always been a strong believer that it takes at least two weeks to firmly establish a quality score in Google, these types of results only further solidified those beliefs. Once the client saw the evidence of our analysis, we were able to suggest that text ad and landing page changes be done once a month; this way we can establish a better quality score, lower average CPC, and improve ad positioning. This month I hope to stop the bleeding and get back to the good old days.
June 12, 2008 7:22 am Paid Search








