Python Excercises – HTTP, Cookie and Dictionaries

Few days ago, I created a simple Python script. This serves as my Python exercise as I’ve been studying Python for over a year. This script tests site’s Multi-Variant tests or A/B tests to check if the traffic split is correctly computed.

Backgrounder

Our main site is a few years old and enhancements come and go. Before an enhancement goes live to production, the marketing ladies wants to prove that the new enhancement really works and users engage more to the site.

Developers will create different versions for the site or specific page, 1 for the old version and 1 for the enhanced version. We put some analytics stuff that the marketing ladies will consolidate later on and based on the report, they will know if the new enhancement produces more sales, increases user engagement, or whatever.

On the backend, we set specific percentage when the customer will experience the new enhancement, for example a new home page will be experienced by 10% of all users or a new shiny button will be experienced by 40% of all users. It is set via cookie.

What are we trying to create

Our QA already have a script to verify that the MVT traffic split is correctly distributed based on the percentage given. They are using PHPUnit and Selenium for that purpose. For example, if a shiny button is expected to be experienced by 40% of the whole user traffic but on tests it shows to say 60% or 10%, therefore, the MVT traffic split fails.

I thought it could be done easily via cURL on PHP but I was trying to explore Python hence I created this simple script.

The script

The script simply accepts some configuration as to what is the expected traffic percentage per tests, what cookie name to use and what URL to crawl. I manage to play around with Cookie manipulation, dictionaries and HTTP. See below:

import urllib
import urllib2
import cookielib

class MVTSplitTest:
    expected_split = {}
    split_stat = {}
    max_run = 500
    test_url = None
    traffic_cookie = 'mvt_traffic'
    control_traffic = 'a'

    def __init__(self, url, input_split, max_run=500):
        self.test_url = url
        self.expected_split = input_split
        self.max_run = max_run

    def run_test(self):
        for x in range(self.max_run):
            print 'Calling %s at %d of %d' % (self.test_url, x + 1, self.max_run)
            t = self.fetch_url(self.test_url)
            if t:
                print 'Hits traffic %s with value %s' % t
            else:
                print 'No valid traffic hit'

    def print_result(self):
        print ''
        for key in sorted(self.expected_split.keys()):
            split = self.expected_split[key]
            print 'Traffic for %s: %s' % (key, split['name'])
            if key in self.split_stat:
                print self.split_stat[key]
            else:
                print 'None'

    def fetch_url(self, url):
        cj = cookielib.MozillaCookieJar()
        opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
        urllib2.install_opener(opener)

        urllib2.urlopen(url)
        
        split = self.get_cookie(self.traffic_cookie, cj)
        ret = None

        if split:
            traffic = self.get_split_letter(urllib.unquote(split.value))

            if traffic:
                if traffic == self.control_traffic:
                    ret = (self.get_split_name(traffic), 'Control')
                    if not traffic in self.split_stat:
                        self.split_stat[traffic] = 0

                    self.split_stat[traffic] = self.split_stat[traffic] + 1
                else:
                    mvt_value = self.get_mvt_value(traffic, cj)

                    if mvt_value:
                        ret = (self.get_split_name(traffic), mvt_value)
                        if not traffic in self.split_stat:
                            self.split_stat[traffic] = {}

                        if not mvt_value in self.split_stat[traffic]:
                            self.split_stat[traffic][mvt_value] = 0

                        self.split_stat[traffic][mvt_value] = self.split_stat[traffic][mvt_value] + 1
        return ret

    def get_mvt_value(self, traffic, cookie_jar):
        result = None

        if traffic in self.expected_split:
            mvt = self.expected_split[traffic]

            if 'split' in mvt and 'cookie' in mvt:
                mvt_cookie = self.get_cookie(mvt['cookie'], cookie_jar)

                if mvt_cookie:
                    letter = self.get_split_letter(urllib.unquote(mvt_cookie.value))

                    if letter in mvt['split']:
                        result = letter

        return result

    def get_split_letter(self, str):
        return str.split('|')[0]

    def get_split_name(self, traffic):
        result = None

        if traffic in self.expected_split:
            if 'name' in self.expected_split[traffic]:
                result = self.expected_split[traffic]['name']

        return result

    def get_cookie(self, name, cookie_jar):
        result = None

        for c in cookie_jar:
            if c.name == name:
                result = c
                break
        
        return result


if __name__ == '__main__':
    split_config = {
                    'a': {'name': 'Control'},
                    'b': {'name': 'Search Auto Suggest', 'split': ['a', 'b'], 'cookie': 'mvt_auto_suggest'},
                    'c': {'name': 'Search Tabs V2', 'split': ['a', 'b'], 'cookie': 'mvt_search_tabs'},
                    'd': {'name': 'Vehicle Selector V2', 'split': ['a', 'b'], 'cookie': 'mvt_vehicle_selector'},
                    'e': {'name': 'New Home Page', 'split': ['a', 'b'], 'cookie': 'mvt_new_home'},
                    'f': {'name': 'Checkout Force Login', 'split': ['a', 'b'], 'cookie': 'mvt_force_login'}
                    }

    url = 'http://staging.yoursite.com/'
    max_run = 500

    t = MVTSplitTest(url, split_config, max_run)
    t.run_test()
    t.print_result()

To run the script, simply execute the script. For example the script’s file name is testsplit.py, you run:

python testsplit.py
This entry was posted in Python and tagged , , , , , , . Bookmark the permalink.

Related Posts

Leave a Reply

Your email address will not be published. Required fields are marked *