Improving Marketing Efficiency Using Predictive Analytics for M&S (II): Customer Targeting Using Supervised Learning

MSIN0094 Case Study

Author

Affiliation

Dr. Wei Miao

UCL School of Management

Last week, M&S used unsupervised learning to segment customers based on their spending and income. While segmentation allows M&S to identify customer groups and tailor general marketing strategies, it doesn’t directly tell us which customers M&S should target. In this part of the case study, we take a step further by targeting individual customers using supervised learning models, which help identify who is most likely to respond positively to marketing offers.

Recently, M&S has launched its highly anticipated Beauty Advent Calendar for 2024, a curated selection of beauty and skincare products worth over £300. This limited-edition calendar is available to customers for only £50. With the holiday season approaching, M&S wants to maximize the reach and response of its marketing campaign by promoting the advent calendar offer to the right customers.

M&S decides to use a conventional mailing marketing strategy, where customers receive color-printed leaflets via Royal Mail to their doorsteps. Each mail costs £1.5 to produce and another £0.5 to mail to the customers. If a customer responds to the offer, M&S expects them to spend £35 on full-price clothing, homeware or beauty, and purchase the advent calendar at £50. The COGS for clothing, homeware, and beauty products is 85%. And the COGS for the advent calendar is 90%.

Question 1

Based on the information provided, calculate the following values:

1. cost_per_offer: the cost of sending an marketing offer

2. profit_per_customer: the profit from a customer if a customer responds to the marketing offer

Answer

The cost of sending an offer is the sum of the production cost and mailing cost.

# cost of sending an offer
# production cost + mailing cost

cost_per_offer <- 1.5 + 0.5

The profit from a customer is the revenue from the customer minus the cost of goods sold (COGS). Note that the COGS for clothing, homeware, and beauty products is 85%, and the COGS for the advent calendar is 90%. And the responding customer is expected to spend £35 on full-price clothing, homeware, or beauty products, and purchase the advent calendar at £50.

# profit from a responding customer

COGS_grocery <- 0.85
COGS_advent <- 0.90

profit_per_customer <- 35 * (1 - COGS_grocery) + 50 * (1 - COGS_advent)

1 Blanket marketing

M&S’s starting point is blanket marketing, a simple strategy where marketing offers are sent to all customers in the data set. Although easy to implement, blanket marketing can lead to wasted marketing costs, as not all customers are interested or responsive. To analyze the effectiveness of blanket marketing, we’ll calculate the Return on Investment (ROI) for this strategy and compare it to more targeted approaches.

M&S has used blanket marketing and sent marketing offers to all 2000 customers in the database. Please calculate the total marketing costs below.

Question 2

Please calculate the total marketing costs for sending marketing offers to all 2000 customers in the dataset. The dataset is the same dataset used in the previous case study.

Answer

First, we can load the dataset.

pacman::p_load(dplyr, ranger, rpart, rpart.plot)
data_full <- read.csv("https://www.dropbox.com/scl/fi/2q7ppqtyca0pd3j486osl/data_full.csv?rlkey=gsyk51q27vd1skek4qpn5ikgm&dl=1")

data_full <- data_full %>%
    mutate(Income = ifelse(is.na(Income), mean(Income, na.rm = TRUE), Income))

Next, we already know the cost of sending an offer is cost_per_offer <- 1.5 + 0.5. We can calculate the total marketing costs by multiplying the cost per offer by the number of customers in the dataset.

total_costs_of_mailing_blanket <- cost_per_offer * nrow(data_full)
total_costs_of_mailing_blanket

[1] 4000

Next, we’ll calculate the total profit from blanket marketing by summing up the profits from all responsive customers in the dataset. M&S has sent out the mail to all 2000 customers in the dataset, and recorded their responses. There is a variable Response in the dataset, which indicates whether a customer responded to the marketing offer. We can use this variable to identify responsive customers and calculate the total profit from the marketing campaign.

First, let’s calculate how many customers responded to the marketing offer in the dataset.

Question 3

Based on the Response variable in the dataset, calculate the total number of customers who responded to the marketing offer. And then calculate the total profit from the marketing campaign.

• Tip: you can use data_full$Response to extract the Response variable as a vector in the dataset. Based on this vector, you can calculate the total number of responding customers and the total profit from the marketing campaign.

Answer

• First, we can calculate the total number of customers who responded to the marketing offer in the dataset. We use the sum() function to sum up the Response variable, which is a binary variable indicating whether a customer responded to the marketing offer.

# below is the total number of responding customers in the test set
sum(data_full$Response)

[1] 301

• Second, among the customers who responded, we can calculate the total profit from the marketing campaign. Each responding customer is expected to bring us a profit of profit_per_customer.

# Multiply it with profit_per_customer, we get total profits from the marketing campaign
total_profit_blanket <- sum(data_full$Response) * profit_per_customer

Return on Investment

ROI = (Profits - Initial Investments) / Initial Investments, which means the return rate of an investment activity.

• ROI needs to be a positive number in order for the company to make profits from the investment.

Therefore, the Return on Investment (ROI) on the marketing offer would be

Question 4

Calculate the Return on Investment (ROI) from blanket marketing.

Answer

# net profits = total_profit_blanket - total_costs_of_mailing_blanket
# initial investment = total_costs_of_mailing_blanket
# So ROI is below:

ROI_blanket <- (total_profit_blanket - total_costs_of_mailing_blanket) / total_costs_of_mailing_blanket

ROI_blanket

[1] -0.2286875

A negative ROI from blanket marketing means, the company makes a loss from sending offers to all customers. The reason is that, not all customers are responsive to our marketing offers. It does M&S no good from sending offers to those customers who would not respond anyway.

2 Customer Targeting Using Supervised Learning

We have seen that blanket marketing is not an efficient strategy for M&S. In this section, we will use supervised learning models to target customers who are more likely to respond to the marketing offer. By doing so, we hope that M&S can improve its marketing efficiency and increase the ROI of the marketing campaign.

2.1 Break-Even Analysis: Break-Even Response Rate

In order to break-even, we can calculate the break-even response rate from customers, which is the minimum response rate we need of a customer in order not to lose money from sending the marketing offer.

Only if a customer responds to us with the minimum response rate can we recover the costs of making a marketing offer. The higher the response rate, the more expected profit we can make from the marketing campaign.

If we send offers to customers whose expected response rate is lower than the break-even response rate, we make a loss by expectation.

Question 5

Calculate the break-even response rate from customers. The break-even response rate is the minimum response rate we need from a customer in order not to lose money from sending the marketing offer.

Answer

break_even_response <- cost_per_offer / profit_per_customer

break_even_response

[1] 0.195122

Next, we will use supervised learning models to predict the response rate of individual customers. We will then target customers whose predicted response rate is higher than the break-even response rate.

2.2 Data Analytics Using Supervised Machine Learning

2.2.1 Select meaningful features/predictors from data_full, named data_full_small

We need to select meaningful features/predictors from the dataset to train the supervised learning models. We can use the following features as predictors:

• Since ID is solely a customer identifier, it should be removed from the final data

• Since Dt_Customer is a character string, which cannot be directly used in the model, we should also remove it

# Use select to remove the above two variables
# Tip: a minus sign before the variable name can remove that variable

data_full_smaller <- data_full %>%
    select(-ID) %>%
    select(-Dt_Customer)

2.2.2 Construct a training set and a test set

Tasks: randomly divide data_full into a training set and a test set

# use nrow() to count the number of rows in data_full_smaller
n_rows_data_full <- nrow(data_full_smaller)

# set a seed, so that we can get the same set of results every time we rerun the model
set.seed(8888)

# use sample() to randomly draw row index from data_full
training_index <- sample(
    x = 1:n_rows_data_full, # draw from 1 until 2000
    size = 0.7 * n_rows_data_full, # size is 70% of 2000
    replace = FALSE
) # do not sample with replacement

# think of data_full_smaller as a matrix, we can select rows based on training_index
data_training <- data_full_smaller %>%
    slice(training_index)

# minus sign means deselecting rows
data_test <- data_full_smaller %>%
    slice(-training_index)

2.2.3 Train a decision tree

Tasks: load the rpart and rpart.plot packages. Follow the code examples in the lecture notes and try to train a decision tree on data_training

pacman::p_load(rpart, rpart.plot)

# write your codes below from what we learned in class.

decision_tree <- rpart(
    formula = Response ~ MntWines + MntFruits + MntMeatProducts + MntFishProducts +
        MntSweetProducts + MntGoldProds + NumDealsPurchases + NumWebPurchases + NumCatalogPurchases +
        NumStorePurchases + NumWebVisitsMonth + Complain + Year_Birth + Education +
        Marital_Status + Income + Kidhome + Teenhome + Recency,
    data = data_training,
    method = "anova"
) # because we need probabilities (a continuous variable) later on to decide targeting, here we sepcify "anova" rather than "class"

# visualize the decision tree we have built
rpart.plot(decision_tree)

2.2.4 Train a random forest

Tasks: load the ranger packages. Follow the code examples in the lecture notes and try to train a random forest on data_training

pacman::p_load(ranger)

# write your codes below:

set.seed(888)
randomforest <- ranger(
    formula = Response ~ MntWines + MntFruits + MntMeatProducts + MntFishProducts +
        MntSweetProducts + MntGoldProds + NumDealsPurchases + NumWebPurchases + NumCatalogPurchases +
        NumStorePurchases + NumWebVisitsMonth + Complain + Year_Birth + Education +
        Marital_Status + Income + Kidhome + Teenhome + Recency,
    data = data_training,
    probability = TRUE,
    num.trees = 500
)

# make prediction on the test set, which returns a model prediction object
prediction_from_randomforest <- predict(randomforest, data_test)

# mutate a new column in data_test for the predicted probability from random forest
# the prediction_from_randomforest$predictions gives a matrix
# so we need to the take the second column using [,2]
data_test <- data_test %>%
    mutate(predicted_prob_randomforest = prediction_from_randomforest$predictions[, 2])

2.2.5 Predict response rate from decision tree model

First, we have already trained the decision tree model, named decision_tree, from the training set. We can predict the probability of test set customer responding to our marketing offer, using predict().

# use predict() to make prediction on the test set
# Note that prediction_from_decision_tree is already a vector,
# which we can directly mutate into
prediction_from_decision_tree <- predict(decision_tree, data_test)

# mutate a new column in data_test for the predicted probability
# note that from decision tree, predict() can directly give a vector rather than matrix
# so we no longer need to extract the second column (as we did with random forest)
data_test <- data_test %>%
    mutate(predicted_prob_decisiontree = prediction_from_decision_tree)

Tasks: We should only send marketing offers to consumers whose expected or predicted response rate is larger than the break-even response rate. This is called targeted marketing.

# mutate a new binary indicator for whether to target a customer based on predicted prob from decision tree model
data_test <- data_test %>%
    mutate(is_target_decisiontree = ifelse(predicted_prob_decisiontree > break_even_response, 1, 0))

Finally, we have decided to send marketing offers to selected responsive customers. We can then compute the ROI for targeted marketing as in the blanket marketing case.

# total marketing costs
total_costs_of_mailing_decisiontree <- cost_per_offer * # cost per offer
    sum(data_test$is_target_decisiontree) # the sum of is_target is the total number of customers who receive the offer

# filter out customers who receive the marketing offer
data_test_targeted_customers <- data_test %>%
    filter(is_target_decisiontree == 1)

# total profits from responding customers

total_profit_decisiontree <- sum(data_test_targeted_customers$Response) * # sum up to get how many targeted customers actually responded
    profit_per_customer # the profit per responding customer

# Compute ROI
ROI_decisiontree <- (total_profit_decisiontree - total_costs_of_mailing_decisiontree) / total_costs_of_mailing_decisiontree

ROI_decisiontree

[1] 0.7545045

If M&S uses random forest, an arguably better supervised learning model, to conduct targeted marketing, we can follow a similar logic as above, and compute the ROI from using random forest.

# mutate a new binary indicator for whether to target a customer based on predicted prob from random forest model
data_test <- data_test %>%
    mutate(is_target_randomforest = ifelse(predicted_prob_randomforest > break_even_response, 1, 0))

# total marketing costs
total_costs_of_mailing_randomforest <- cost_per_offer * sum(data_test$is_target_randomforest)

# get the number of responding customers who are targeted: the second method

data_responding_targeted_customers <- data_test %>%
    filter(is_target_randomforest == 1) %>% # who are targeted
    filter(Response == 1) # who responding among targeted customers

# total profits from responding customers
total_profit_randomforest <- nrow(data_responding_targeted_customers) * profit_per_customer

# Compute ROI
ROI_randomforest <- (total_profit_randomforest - total_costs_of_mailing_randomforest) / total_costs_of_mailing_randomforest

ROI_randomforest

[1] 1.113701

Caution

Note that random forest model package in R uses C++ as the backend to speed up the computation. Therefore, if you use a Macbook, you should see the same ROI as above; however, if you use a Windows machine, you may see a slightly different ROI due to the different random number generation algorithms used in C++ on Windows and Mac OS.

On Windows, if you see 1.217836, it is correct.

Predictive analytics model can help the company boost the marketing ROI by allowing M&S to target customers who are more likely to respond to the marketing offers than the break-even response rate. By doing so, M&S saves unnecessary marketing costs on those unresponsive customers and therefore improves its marketing efficiency.