Improving Marketing Efficiency Using Predictive Analytics for M&S (II): Customer Targeting Using Supervised Learning

MSIN0094 Case Study

Author

Affiliation

Dr. Wei Miao

UCL School of Management

Published

October 29, 2025

Last week, M&S used unsupervised learning to segment customers based on their spending and income. While segmentation allows M&S to identify customer groups and tailor general marketing strategies, it doesn’t directly tell us which customers M&S should target. In this part of the case study, we take a step further by targeting individual customers using supervised learning models, which help identify who is most likely to respond positively to marketing offers.

Recently, M&S has launched its highly anticipated Beauty Advent Calendar for 2025, a curated selection of beauty and skincare products worth £330. This limited-edition calendar is available to customers for only £60 when a customer purchases it alongside other products. With the holiday season approaching, M&S wants to maximize the reach and response of its marketing campaign by promoting the advent calendar offer to the right customers.

M&S decides to use a conventional direct mail marketing strategy, where customers receive colour-printed leaflets via Royal Mail to their doorsteps. Each mailer costs £1.5 to produce and another £0.5 to mail to the customers. M&S also plans to include a small sample of the beauty product, which costs M&S £2 to produce. If a customer responds to the offer, M&S expects them to spend £35 on full-price clothing, homeware, or beauty, and purchase the advent calendar at £60. The COGS for clothing, homeware, and beauty products is 85%. The COGS for the advent calendar is 90%.

Based on the information provided, calculate the following values:

cost_per_offer: the cost of sending a marketing offer
profit_per_customer: the profit from a customer if a customer responds to the marketing offer

The cost of sending an offer is the sum of the production cost and mailing cost.

Code

# Calculate the total cost of sending one marketing offer.
# This includes the cost of the mailer (£1.5), postage (£0.5), and a product sample (£2).
cost_per_offer <- 1.5 + 0.5 + 2

The profit from a customer is the revenue from the customer minus the cost of goods sold (COGS). Note that the COGS for clothing, homeware, and beauty products is 85%, and the COGS for the advent calendar is 90%. And the responding customer is expected to spend £35 on full-price clothing, homeware, or beauty products, and purchase the advent calendar at £60.

Code

# Calculate the profit generated from a single customer who responds to the offer.

# Define the Cost of Goods Sold (COGS) for regular products (85%).
COGS_grocery <- 0.85
# Define the COGS for the advent calendar (90%).
COGS_advent <- 0.90

# Calculate the total profit.
# Profit from regular products: £35 spend * (1 - 0.85 profit margin).
# Profit from advent calendar: £60 price * (1 - 0.90 profit margin).
profit_per_customer <- 35 * (1 - COGS_grocery) + 60 * (1 - COGS_advent)

1 Blanket Marketing

M&S’s starting point is blanket marketing, a simple strategy where marketing offers are sent to all customers in the dataset. Although easy to implement, blanket marketing can lead to wasted marketing costs, as not all customers are interested or responsive. To analyse the effectiveness of blanket marketing, we’ll calculate the Return on Investment (ROI) for this strategy and compare it to more targeted approaches.

M&S has used blanket marketing and sent marketing offers to all 2000 customers in the database. Please calculate the total marketing costs below.

Please calculate the total marketing costs for sending marketing offers to all 2000 customers in the dataset. The dataset is the same dataset used in the previous case study.

First, we can load the dataset.

Code

# Load necessary R packages for data manipulation (dplyr), decision trees (rpart, rpart.plot), and random forests (ranger).
pacman::p_load(dplyr, ranger, rpart, rpart.plot)
# Load the customer dataset from a remote CSV file.
data_full <- read.csv("https://www.dropbox.com/scl/fi/2q7ppqtyca0pd3j486osl/data_full.csv?rlkey=gsyk51q27vd1skek4qpn5ikgm&dl=1")

# Preprocess the data.
data_full <- data_full %>%
    # The 'mutate' function adds or updates columns.
    # Here, we're handling missing values in the 'Income' column.
    # 'ifelse' checks if an 'Income' value is NA (is.na).
    # If it is, it's replaced with the mean of all non-missing 'Income' values (mean(Income, na.rm = TRUE)).
    # If it's not NA, the original value is kept.
    mutate(Income = ifelse(is.na(Income), mean(Income, na.rm = TRUE), Income))

Next, we already know the cost of sending an offer is cost_per_offer <- 1.5 + 0.5. We can calculate the total marketing costs by multiplying the cost per offer by the number of customers in the dataset.

Code

# Calculate the total cost of the blanket marketing campaign.
# This is the cost per offer multiplied by the total number of customers in the dataset.
total_costs_of_mailing_blanket <- cost_per_offer * nrow(data_full)
# Display the total cost.
total_costs_of_mailing_blanket

[1] 8000

Next, we’ll calculate the total profit from blanket marketing by summing up the profits from all responsive customers in the dataset. M&S has sent out the mail to all 2000 customers in the dataset, and recorded their responses. There is a variable Response in the dataset, which indicates whether a customer responded to the marketing offer. We can use this variable to identify responsive customers and calculate the total profit from the marketing campaign.

First, let’s calculate how many customers responded to the marketing offer in the dataset.

Based on the Response variable in the dataset, calculate the total number of customers who responded to the marketing offer. Then calculate the total profit from the marketing campaign.

Tip: you can use data_full$Response to extract the Response variable as a vector in the dataset. Based on this vector, you can calculate the total number of responding customers and the total profit from the marketing campaign.

First, we can calculate the total number of customers who responded to the marketing offer in the dataset. We use the sum() function to sum up the Response variable, which is a binary variable indicating whether a customer responded to the marketing offer.

Code

# Calculate the total number of customers who responded to the offer.
# The 'Response' column is binary (1 for response, 0 for no response),
# so summing the column gives the total count of responders.
sum(data_full$Response)

[1] 301

Second, among the customers who responded, we can calculate the total profit from the marketing campaign. Each responding customer is expected to bring us a profit of profit_per_customer.

Code

# Calculate the total profit from the blanket marketing campaign.
# This is the number of responding customers multiplied by the profit generated per responder.
total_profit_blanket <- sum(data_full$Response) * profit_per_customer

Return on Investment

ROI = (Profits - Initial Investments) / Initial Investments, which means the return rate of an investment activity.

ROI needs to be a positive number in order for the company to make profits from the investment.

Therefore, the Return on Investment (ROI) on the marketing offer would be

Calculate the Return on Investment (ROI) from blanket marketing.

Code

# Calculate the Return on Investment (ROI) for the blanket marketing campaign.
# ROI = (Total Profit - Total Cost) / Total Cost.
# It measures the net return relative to the initial investment.
ROI_blanket <- (total_profit_blanket - total_costs_of_mailing_blanket) / total_costs_of_mailing_blanket

# Display the calculated ROI.
ROI_blanket

[1] -0.5767187

A negative ROI from blanket marketing means the company makes a loss from sending offers to all customers. The reason is that not all customers are responsive to our marketing offers. It does M&S no good to send offers to those customers who would not respond anyway.

2 Customer Targeting Using Supervised Learning

We have seen that blanket marketing is not an efficient strategy for M&S. In this section, we will use supervised learning models to target customers who are more likely to respond to the marketing offer. By doing so, we hope that M&S can improve its marketing efficiency and increase the ROI of the marketing campaign.

2.1 Break-Even Analysis: Break-Even Response Rate

In order to break even, we can calculate the break-even response rate, which is the minimum response rate we need from a customer in order to avoid losing money from sending the marketing offer.

Only if a customer’s expected response probability is at least the minimum response rate can we recover the costs of sending a marketing offer. The higher the response probability, the more expected profit we can make from the marketing campaign.

If we send offers to customers whose expected response rate is lower than the break-even response rate, we will lose money on average.

Calculate the break-even response rate from customers. The break-even response rate is the minimum response rate we need from a customer in order not to lose money from sending the marketing offer.

Code

# Calculate the break-even response rate.
# This is the minimum response rate required to cover the cost of sending an offer.
# It's calculated as the cost per offer divided by the profit per responding customer.
break_even_response <- cost_per_offer / profit_per_customer

# Display the break-even response rate.
break_even_response

[1] 0.3555556

Next, we will use supervised learning models to predict the response rate of individual customers. We will then target customers whose predicted response rate is higher than the break-even response rate.

2.2 Data Analytics Using Supervised Machine Learning

2.2.1 Select Meaningful Features/Predictors from `data_full`, Named `data_full_small`

We need to select meaningful features/predictors from the dataset to train the supervised learning models. We can use the following features as predictors:

Since ID is solely a customer identifier, it should be removed from the final data
Since Dt_Customer is a character string, which cannot be directly used in the model, we should also remove it

Code

# Use the 'select' function from dplyr to remove specified columns.
# The minus sign (-) before a column name indicates that it should be dropped.
data_full_smaller <- data_full %>%
    select(-ID) %>% # Remove the ID column.
    select(-Dt_Customer) # Remove the Dt_Customer column.

2.2.2 Construct a Training Set and a Test Set

Tasks: randomly divide data_full into a training set and a test set

Code

# Get the total number of rows in the cleaned dataset.
n_rows_data_full <- nrow(data_full_smaller)

# Set a seed for the random number generator to ensure reproducibility.
# Using the same seed will produce the same random split every time.
set.seed(8888)

# Randomly sample row indices for the training set.
training_index <- sample(
    x = 1:n_rows_data_full, # The range of indices to sample from (1 to total number of rows).
    size = 0.7 * n_rows_data_full, # The size of the sample (70% of the data for training).
    replace = FALSE # Sample without replacement to ensure each row is selected at most once.
)

# Create the training set by selecting rows with the sampled indices.
data_training <- data_full_smaller %>%
    slice(training_index)

# Create the test set by selecting all rows *not* in the training set.
# The minus sign deselects the rows specified by training_index.
data_test <- data_full_smaller %>%
    slice(-training_index)

2.2.3 Train a Decision Tree

Tasks: load the rpart and rpart.plot packages. Follow the code examples in the lecture notes and try to train a decision tree on data_training

Code

# Load the required packages for building and plotting decision trees.
pacman::p_load(rpart, rpart.plot)

# Train a decision tree model using the rpart function.
decision_tree <- rpart(
    # The formula specifies that 'Response' is the target variable,
    # and all other listed variables are predictors.
    formula = Response ~ MntWines + MntFruits + MntMeatProducts + MntFishProducts +
        MntSweetProducts + MntGoldProds + NumDealsPurchases + NumWebPurchases + NumCatalogPurchases +
        NumStorePurchases + NumWebVisitsMonth + Complain + Year_Birth + Education +
        Marital_Status + Income + Kidhome + Teenhome + Recency,
    data = data_training, # The training dataset.
    method = "anova" # Use the "anova" method for regression trees to predict a continuous probability.
)

Code

# Visualize the trained decision tree.
# This plot helps to understand the rules the model has learned.
rpart.plot(decision_tree)

2.2.4 Predict Response Rate from Decision Tree Model

First, we have already trained the decision tree model, named decision_tree, from the training set. We can predict the probability of test set customers responding to our marketing offer, using predict().

Code

# Use the trained decision tree model to predict response probabilities on the test set.
prediction_from_decision_tree <- predict(decision_tree, data_test)

# Add the predicted probabilities as a new column to the test dataset.
# The 'mutate' function creates a new column named 'predicted_prob_decisiontree'.
data_test <- data_test %>%
    mutate(predicted_prob_decisiontree = prediction_from_decision_tree)

Tasks: We should only send marketing offers to customers whose expected or predicted response rate is higher than the break-even response rate. This is called targeted marketing.

Code

# Create a new column 'is_target_decisiontree' to identify which customers to target.
# A customer is targeted (value 1) if their predicted response probability
# is greater than the break-even response rate; otherwise, they are not targeted (value 0).
data_test <- data_test %>%
    mutate(is_target_decisiontree = ifelse(predicted_prob_decisiontree > break_even_response, 1, 0))

Finally, we have decided to send marketing offers to selected responsive customers. We can then compute the ROI for targeted marketing as in the blanket marketing case.

Code

# Calculate the total cost of the targeted marketing campaign using the decision tree model.
# This is the cost per offer multiplied by the number of targeted customers.
total_costs_of_mailing_decisiontree <- cost_per_offer * sum(data_test$is_target_decisiontree)

# Filter the test set to include only the customers who were targeted.
data_test_targeted_customers <- data_test %>%
    filter(is_target_decisiontree == 1)

# Calculate the total profit from the targeted campaign.
# This is the number of targeted customers who actually responded, multiplied by the profit per customer.
total_profit_decisiontree <- sum(data_test_targeted_customers$Response) * profit_per_customer

# Compute the ROI for the decision tree-based targeted marketing.
ROI_decisiontree <- (total_profit_decisiontree - total_costs_of_mailing_decisiontree) / total_costs_of_mailing_decisiontree

# Display the calculated ROI.
ROI_decisiontree

[1] 0.2247984

2.2.5 Train a Random Forest (Next Week)

Tasks: load the ranger packages. Follow the code examples in the lecture notes and try to train a random forest on data_training

Code

# Load the 'ranger' package for building random forest models.
pacman::p_load(ranger)

# Set a seed for reproducibility of the random forest model.
set.seed(888)
# Train a random forest model.
randomforest <- ranger(
    # The formula is the same as for the decision tree.
    formula = Response ~ MntWines + MntFruits + MntMeatProducts + MntFishProducts +
        MntSweetProducts + MntGoldProds + NumDealsPurchases + NumWebPurchases + NumCatalogPurchases +
        NumStorePurchases + NumWebVisitsMonth + Complain + Year_Birth + Education +
        Marital_Status + Income + Kidhome + Teenhome + Recency,
    data = data_training, # Use the training data.
    probability = TRUE, # Set to TRUE to get class probabilities, which we need for targeting.
    num.trees = 500 # Specify the number of trees in the forest.
)

# Use the trained random forest to make predictions on the test set.
prediction_from_randomforest <- predict(randomforest, data_test)

# Add the predicted probabilities to the test dataset.
# The output 'prediction_from_randomforest$predictions' is a matrix with probabilities for each class (0 and 1).
# We need the second column [, 2], which contains the probability of a positive response (Response = 1).
data_test <- data_test %>%
    mutate(predicted_prob_randomforest = prediction_from_randomforest$predictions[, 2])

If M&S uses random forest, an arguably better supervised learning model, to conduct targeted marketing, we can follow a similar logic as above, and compute the ROI from using random forest.

Code

# Create a targeting indicator based on the random forest model's predictions.
# A customer is targeted if their predicted probability is above the break-even rate.
data_test <- data_test %>%
    mutate(is_target_randomforest = ifelse(predicted_prob_randomforest > break_even_response, 1, 0))

# Calculate the total cost of the random forest-based targeted campaign.
total_costs_of_mailing_randomforest <- cost_per_offer * sum(data_test$is_target_randomforest)

# Identify the customers who were targeted and actually responded.
data_responding_targeted_customers <- data_test %>%
    filter(is_target_randomforest == 1) %>% # Filter for targeted customers.
    filter(Response == 1) # Filter for those who responded.

# Calculate the total profit from the campaign.
# This is the number of responding targeted customers multiplied by the profit per customer.
total_profit_randomforest <- nrow(data_responding_targeted_customers) * profit_per_customer

# Compute the ROI for the random forest-based targeted marketing.
ROI_randomforest <- (total_profit_randomforest - total_costs_of_mailing_randomforest) / total_costs_of_mailing_randomforest

# Display the calculated ROI.
ROI_randomforest

[1] 0.7863176

Caution

Note that the random forest model package in R uses C++ as the backend to speed up the computation. Therefore, if you use a Macbook, you should see the same ROI as above; however, if you use a Windows machine, you may see a slightly different ROI due to the different random number generation algorithms used in C++ on Windows and macOS.

Predictive analytics models can help the company boost the marketing ROI by allowing M&S to target customers who are more likely to respond to the marketing offers than the break-even response rate. By doing so, M&S saves unnecessary marketing costs on those unresponsive customers and therefore improves its marketing efficiency.

--- title: "Improving Marketing Efficiency Using Predictive Analytics for M&S (II): Customer Targeting Using Supervised Learning" author: Dr. Wei Miao thanks: "This case was prepared by Wei Miao, UCL School of Management, University College London for the MSIN0094 Marketing Analytics module. This case was developed to provide material for class discussion rather than to illustrate either effective or ineffective handling of a business situation. Names and data may have been disguised or fabricated. Please do not circulate without permission. All copyrights reserved." subtitle: "MSIN0094 Case Study" institute: UCL School of Management date: "`r (first_date+lubridate::dweeks(4))`" date-format: long format: html: default pdf: number-sections: true toc: false number-depth: 2 fontsize: 9pt linestretch: 1.25 knitr: opts_chunk: echo: false include: false warning: false message: false error: true bibliography: references.bib execute: freeze: auto cache: true --- ```{r} #| echo: false #| include: true #| fig-align: 'center' #| out-width: '5cm' knitr::include_graphics("images/Week 5/beauty advent calendar 2025.png") ``` Last week, M&S used unsupervised learning to segment customers based on their spending and income. While segmentation allows M&S to identify customer groups and tailor general marketing strategies, it doesn't directly tell us which customers M&S should target. In this part of the case study, we take a step further by targeting individual customers using supervised learning models, which help identify who is most likely to respond positively to marketing offers. Recently, M&S has launched its highly anticipated [Beauty Advent Calendar for `r lubridate::year(first_date)`](https://www.marksandspencer.com/c/beauty-box), a curated selection of beauty and skincare products worth £330. This limited-edition calendar is available to customers for only £60 when a customer purchases it alongside other products. With the holiday season approaching, M&S wants to maximize the reach and response of its marketing campaign by promoting the advent calendar offer to the right customers. M&S decides to use a conventional direct mail marketing strategy, where customers receive colour-printed leaflets via Royal Mail to their doorsteps. Each mailer costs £1.5 to produce and another £0.5 to mail to the customers. M&S also plans to include a small sample of the beauty product, which costs M&S £2 to produce. If a customer responds to the offer, M&S expects them to spend £35 on full-price clothing, homeware, or beauty, and purchase the advent calendar at £60. The COGS for clothing, homeware, and beauty products is 85%. The COGS for the advent calendar is 90%. ::: {.content-visible when-format='pdf'} ::: {.callout} **Question `r question_index<-question_index+1; question_index-1`** Based on the information provided, calculate the following values: 1. `cost_per_offer`: the cost of sending a marketing offer 2. `profit_per_customer`: the profit from a customer if a customer responds to the marketing offer ::: ::: ::: {.content-visible when-format='html'} ::: {.panel-tabset} ### **Question `r question_index-1`** Based on the information provided, calculate the following values: 1. `cost_per_offer`: the cost of sending a marketing offer 2. `profit_per_customer`: the profit from a customer if a customer responds to the marketing offer ### Answer The cost of sending an offer is the sum of the production cost and mailing cost. ```{r} # Calculate the total cost of sending one marketing offer. # This includes the cost of the mailer (£1.5), postage (£0.5), and a product sample (£2). cost_per_offer <- 1.5 + 0.5 + 2 ``` The profit from a customer is the revenue from the customer minus the cost of goods sold (COGS). Note that the COGS for clothing, homeware, and beauty products is 85%, and the COGS for the advent calendar is 90%. And the responding customer is expected to spend £35 on full-price clothing, homeware, or beauty products, and purchase the advent calendar at £60. ```{r} # Calculate the profit generated from a single customer who responds to the offer. # Define the Cost of Goods Sold (COGS) for regular products (85%). COGS_grocery <- 0.85 # Define the COGS for the advent calendar (90%). COGS_advent <- 0.90 # Calculate the total profit. # Profit from regular products: £35 spend * (1 - 0.85 profit margin). # Profit from advent calendar: £60 price * (1 - 0.90 profit margin). profit_per_customer <- 35 * (1 - COGS_grocery) + 60 * (1 - COGS_advent) ``` ::: ::: # Blanket Marketing M&S's starting point is blanket marketing, a simple strategy where marketing offers are sent to all customers in the dataset. Although easy to implement, blanket marketing can lead to wasted marketing costs, as not all customers are interested or responsive. To analyse the effectiveness of blanket marketing, we'll calculate the Return on Investment (ROI) for this strategy and compare it to more targeted approaches. M&S has used blanket marketing and sent marketing offers to all 2000 customers in the database. Please calculate the total marketing costs below. ::: {.content-visible when-format='pdf'} ::: {.callout} **Question `r question_index<-question_index+1; question_index-1`** Please calculate the total marketing costs for sending marketing offers to all 2000 customers in the dataset. The dataset is the same dataset used in the previous case study. ::: ::: ::: {.content-visible when-format='html'} ::: {.panel-tabset} ### **Question `r question_index-1`** Please calculate the total marketing costs for sending marketing offers to all 2000 customers in the dataset. The dataset is the same dataset used in the previous case study. ### Answer First, we can load the dataset. ```{r} # Load necessary R packages for data manipulation (dplyr), decision trees (rpart, rpart.plot), and random forests (ranger). pacman::p_load(dplyr, ranger, rpart, rpart.plot) # Load the customer dataset from a remote CSV file. data_full <- read.csv("https://www.dropbox.com/scl/fi/2q7ppqtyca0pd3j486osl/data_full.csv?rlkey=gsyk51q27vd1skek4qpn5ikgm&dl=1") # Preprocess the data. data_full <- data_full %>% # The 'mutate' function adds or updates columns. # Here, we're handling missing values in the 'Income' column. # 'ifelse' checks if an 'Income' value is NA (is.na). # If it is, it's replaced with the mean of all non-missing 'Income' values (mean(Income, na.rm = TRUE)). # If it's not NA, the original value is kept. mutate(Income = ifelse(is.na(Income), mean(Income, na.rm = TRUE), Income)) ``` Next, we already know the cost of sending an offer is `cost_per_offer <- 1.5 + 0.5`. We can calculate the total marketing costs by multiplying the cost per offer by the number of customers in the dataset. ```{r} # Calculate the total cost of the blanket marketing campaign. # This is the cost per offer multiplied by the total number of customers in the dataset. total_costs_of_mailing_blanket <- cost_per_offer * nrow(data_full) # Display the total cost. total_costs_of_mailing_blanket ``` ::: ::: Next, we’ll calculate the total profit from blanket marketing by summing up the profits from all responsive customers in the dataset. M&S has sent out the mail to all 2000 customers in the dataset, and recorded their responses. There is a variable `Response` in the dataset, which indicates whether a customer responded to the marketing offer. We can use this variable to identify responsive customers and calculate the total profit from the marketing campaign. First, let's calculate how many customers responded to the marketing offer in the dataset. ::: {.content-visible when-format='pdf'} ::: {.callout} **Question `r question_index<-question_index+1; question_index-1`** Based on the `Response` variable in the dataset, calculate the total number of customers who responded to the marketing offer. Then calculate the total profit from the marketing campaign. - Tip: you can use `data_full$Response` to extract the `Response` variable from the dataset as an R vector. Based on this vector, you can calculate the total number of responding customers and the total profit from the marketing campaign. ::: ::: ::: {.content-visible when-format='html'} ::: {.panel-tabset} ### **Question `r question_index-1`** Based on the `Response` variable in the dataset, calculate the total number of customers who responded to the marketing offer. Then calculate the total profit from the marketing campaign. - Tip: you can use `data_full$Response` to extract the `Response` variable as a vector in the dataset. Based on this vector, you can calculate the total number of responding customers and the total profit from the marketing campaign. ### Answer - First, we can calculate the total number of customers who responded to the marketing offer in the dataset. We use the `sum()` function to sum up the `Response` variable, which is a binary variable indicating whether a customer responded to the marketing offer. ```{r} # Calculate the total number of customers who responded to the offer. # The 'Response' column is binary (1 for response, 0 for no response), # so summing the column gives the total count of responders. sum(data_full$Response) ``` - Second, among the customers who responded, we can calculate the total profit from the marketing campaign. Each responding customer is expected to bring us a profit of `profit_per_customer`. ```{r} # Calculate the total profit from the blanket marketing campaign. # This is the number of responding customers multiplied by the profit generated per responder. total_profit_blanket <- sum(data_full$Response) * profit_per_customer ``` ::: ::: ::: callout-note ## Return on Investment ROI = (Profits - Initial Investments) / Initial Investments, which means the return rate of an investment activity. - ROI needs to be a positive number in order for the company to make profits from the investment. ::: Therefore, the **Return on Investment** **(ROI)** on the marketing offer would be ::: {.content-visible when-format='pdf'} ::: {.callout} **Question `r question_index<-question_index+1; question_index-1`** Calculate the Return on Investment (ROI) from blanket marketing. ::: ::: ::: {.content-visible when-format='html'} ::: {.panel-tabset} ### **Question `r question_index-1`** Calculate the Return on Investment (ROI) from blanket marketing. ### Answer ```{r} # Calculate the Return on Investment (ROI) for the blanket marketing campaign. # ROI = (Total Profit - Total Cost) / Total Cost. # It measures the net return relative to the initial investment. ROI_blanket <- (total_profit_blanket - total_costs_of_mailing_blanket) / total_costs_of_mailing_blanket # Display the calculated ROI. ROI_blanket ``` ::: ::: A negative ROI from blanket marketing means the company makes a loss from sending offers to all customers. The reason is that not all customers are responsive to our marketing offers. It does M&S no good to send offers to those customers who would not respond anyway. # Customer Targeting Using Supervised Learning We have seen that blanket marketing is not an efficient strategy for M&S. In this section, we will use supervised learning models to target customers who are more likely to respond to the marketing offer. By doing so, we hope that M&S can improve its marketing efficiency and increase the ROI of the marketing campaign. ## Break-Even Analysis: Break-Even Response Rate In order to break even, we can calculate the break-even response rate, which is the **minimum response rate** we need from a customer in order to avoid losing money from sending the marketing offer. Only if a customer's expected response probability is at least the minimum response rate can we recover the costs of sending a marketing offer. The higher the response probability, the more expected profit we can make from the marketing campaign. If we send offers to customers whose expected response rate is lower than the break-even response rate, we will lose money on average. ::: {.content-visible when-format='pdf'} ::: {.callout} **Question `r question_index<-question_index+1; question_index-1`** Calculate the break-even response rate from customers. The break-even response rate is the minimum response rate we need from a customer in order not to lose money from sending the marketing offer. ::: ::: ::: {.content-visible when-format='html'} ::: {.panel-tabset} ### **Question `r question_index-1`** Calculate the break-even response rate from customers. The break-even response rate is the minimum response rate we need from a customer in order not to lose money from sending the marketing offer. ### Answer ```{r} # Calculate the break-even response rate. # This is the minimum response rate required to cover the cost of sending an offer. # It's calculated as the cost per offer divided by the profit per responding customer. break_even_response <- cost_per_offer / profit_per_customer # Display the break-even response rate. break_even_response ``` ::: ::: Next, we will use supervised learning models to predict the response rate of individual customers. We will then target customers whose predicted response rate is higher than the break-even response rate. ## Data Analytics Using Supervised Machine Learning ### Select Meaningful Features/Predictors from `data_full`, Named `data_full_small` We need to select meaningful features/predictors from the dataset to train the supervised learning models. We can use the following features as predictors: - Since ID is solely a customer identifier, it should be removed from the final data - Since `Dt_Customer` is a character string, which cannot be directly used in the model, we should also remove it ```{r} # Use the 'select' function from dplyr to remove specified columns. # The minus sign (-) before a column name indicates that it should be dropped. data_full_smaller <- data_full %>% select(-ID) %>% # Remove the ID column. select(-Dt_Customer) # Remove the Dt_Customer column. ``` ### Construct a Training Set and a Test Set *Tasks*: randomly divide `data_full` into a training set and a test set ```{r} # Get the total number of rows in the cleaned dataset. n_rows_data_full <- nrow(data_full_smaller) # Set a seed for the random number generator to ensure reproducibility. # Using the same seed will produce the same random split every time. set.seed(8888) # Randomly sample row indices for the training set. training_index <- sample( x = 1:n_rows_data_full, # The range of indices to sample from (1 to total number of rows). size = 0.7 * n_rows_data_full, # The size of the sample (70% of the data for training). replace = FALSE # Sample without replacement to ensure each row is selected at most once. ) # Create the training set by selecting rows with the sampled indices. data_training <- data_full_smaller %>% slice(training_index) # Create the test set by selecting all rows *not* in the training set. # The minus sign deselects the rows specified by training_index. data_test <- data_full_smaller %>% slice(-training_index) ``` ### Train a Decision Tree *Tasks*: load the `rpart` and `rpart.plot` packages. Follow the code examples in the lecture notes and try to train a decision tree on `data_training` ```{r} # Load the required packages for building and plotting decision trees. pacman::p_load(rpart, rpart.plot) # Train a decision tree model using the rpart function. decision_tree <- rpart( # The formula specifies that 'Response' is the target variable, # and all other listed variables are predictors. formula = Response ~ MntWines + MntFruits + MntMeatProducts + MntFishProducts + MntSweetProducts + MntGoldProds + NumDealsPurchases + NumWebPurchases + NumCatalogPurchases + NumStorePurchases + NumWebVisitsMonth + Complain + Year_Birth + Education + Marital_Status + Income + Kidhome + Teenhome + Recency, data = data_training, # The training dataset. method = "anova" # Use the "anova" method for regression trees to predict a continuous probability. ) ``` ```{r} # Visualize the trained decision tree. # This plot helps to understand the rules the model has learned. rpart.plot(decision_tree) ``` ### Predict Response Rate from Decision Tree Model First, we have already trained the decision tree model, named `decision_tree`, from the training set. We can predict the probability of test set customers responding to our marketing offer, using `predict()`. ```{r} # Use the trained decision tree model to predict response probabilities on the test set. prediction_from_decision_tree <- predict(decision_tree, data_test) # Add the predicted probabilities as a new column to the test dataset. # The 'mutate' function creates a new column named 'predicted_prob_decisiontree'. data_test <- data_test %>% mutate(predicted_prob_decisiontree = prediction_from_decision_tree) ``` *Tasks*: We should only send marketing offers to customers whose expected or predicted response rate is higher than the break-even response rate. This is called targeted marketing. ```{r} # Create a new column 'is_target_decisiontree' to identify which customers to target. # A customer is targeted (value 1) if their predicted response probability # is greater than the break-even response rate; otherwise, they are not targeted (value 0). data_test <- data_test %>% mutate(is_target_decisiontree = ifelse(predicted_prob_decisiontree > break_even_response, 1, 0)) ``` Finally, we have decided to send marketing offers to selected responsive customers. We can then compute the ROI for targeted marketing as in the blanket marketing case. ```{r} # Calculate the total cost of the targeted marketing campaign using the decision tree model. # This is the cost per offer multiplied by the number of targeted customers. total_costs_of_mailing_decisiontree <- cost_per_offer * sum(data_test$is_target_decisiontree) # Filter the test set to include only the customers who were targeted. data_test_targeted_customers <- data_test %>% filter(is_target_decisiontree == 1) # Calculate the total profit from the targeted campaign. # This is the number of targeted customers who actually responded, multiplied by the profit per customer. total_profit_decisiontree <- sum(data_test_targeted_customers$Response) * profit_per_customer # Compute the ROI for the decision tree-based targeted marketing. ROI_decisiontree <- (total_profit_decisiontree - total_costs_of_mailing_decisiontree) / total_costs_of_mailing_decisiontree # Display the calculated ROI. ROI_decisiontree ``` ### Train a Random Forest (Next Week) *Tasks*: load the `ranger` packages. Follow the code examples in the lecture notes and try to train a random forest on `data_training` ```{r} # Load the 'ranger' package for building random forest models. pacman::p_load(ranger) # Set a seed for reproducibility of the random forest model. set.seed(888) # Train a random forest model. randomforest <- ranger( # The formula is the same as for the decision tree. formula = Response ~ MntWines + MntFruits + MntMeatProducts + MntFishProducts + MntSweetProducts + MntGoldProds + NumDealsPurchases + NumWebPurchases + NumCatalogPurchases + NumStorePurchases + NumWebVisitsMonth + Complain + Year_Birth + Education + Marital_Status + Income + Kidhome + Teenhome + Recency, data = data_training, # Use the training data. probability = TRUE, # Set to TRUE to get class probabilities, which we need for targeting. num.trees = 500 # Specify the number of trees in the forest. ) # Use the trained random forest to make predictions on the test set. prediction_from_randomforest <- predict(randomforest, data_test) # Add the predicted probabilities to the test dataset. # The output 'prediction_from_randomforest$predictions' is a matrix with probabilities for each class (0 and 1). # We need the second column [, 2], which contains the probability of a positive response (Response = 1). data_test <- data_test %>% mutate(predicted_prob_randomforest = prediction_from_randomforest$predictions[, 2]) ``` If M&S uses random forest, an arguably better supervised learning model, to conduct targeted marketing, we can follow a similar logic as above, and compute the ROI from using random forest. ```{r} # Create a targeting indicator based on the random forest model's predictions. # A customer is targeted if their predicted probability is above the break-even rate. data_test <- data_test %>% mutate(is_target_randomforest = ifelse(predicted_prob_randomforest > break_even_response, 1, 0)) # Calculate the total cost of the random forest-based targeted campaign. total_costs_of_mailing_randomforest <- cost_per_offer * sum(data_test$is_target_randomforest) # Identify the customers who were targeted and actually responded. data_responding_targeted_customers <- data_test %>% filter(is_target_randomforest == 1) %>% # Filter for targeted customers. filter(Response == 1) # Filter for those who responded. # Calculate the total profit from the campaign. # This is the number of responding targeted customers multiplied by the profit per customer. total_profit_randomforest <- nrow(data_responding_targeted_customers) * profit_per_customer # Compute the ROI for the random forest-based targeted marketing. ROI_randomforest <- (total_profit_randomforest - total_costs_of_mailing_randomforest) / total_costs_of_mailing_randomforest # Display the calculated ROI. ROI_randomforest ``` ::: {.callout-caution} Note that the random forest model package in R uses C++ as the backend to speed up the computation. Therefore, if you use a Macbook, you should see the same ROI as above; however, if you use a Windows machine, you may see a slightly different ROI due to the different random number generation algorithms used in C++ on Windows and macOS. ::: Predictive analytics models can help the company boost the marketing ROI by allowing M&S to target customers who are more likely to respond to the marketing offers than the break-even response rate. By doing so, M&S saves unnecessary marketing costs on those unresponsive customers and therefore improves its marketing efficiency.

1 Blanket Marketing

2 Customer Targeting Using Supervised Learning

2.1 Break-Even Analysis: Break-Even Response Rate

2.2 Data Analytics Using Supervised Machine Learning

2.2.1 Select Meaningful Features/Predictors from data_full, Named data_full_small

2.2.2 Construct a Training Set and a Test Set

2.2.3 Train a Decision Tree

2.2.4 Predict Response Rate from Decision Tree Model

2.2.5 Train a Random Forest (Next Week)

2.2.1 Select Meaningful Features/Predictors from `data_full`, Named `data_full_small`