library(dplyr)
library(knitr)
library(caret)
library(pROC)

Loading Data

set.seed(123)
df <- read.csv("wdbc.data", header = FALSE)

names <- c(
  'ID', 'Diagnosis', 'radius1', 'texture1', 'perimeter1', 'area1', 
  'smoothness1', 'compactness1', 'concavity1', 'concave_points1', 
  'symmetry1', 'fractal_dimension1', 'radius2', 'texture2', 'perimeter2', 
  'area2', 'smoothness2', 'compactness2', 'concavity2', 'concave_points2', 
  'symmetry2', 'fractal_dimension2', 'radius3', 'texture3', 'perimeter3', 
  'area3', 'smoothness3', 'compactness3', 'concavity3', 'concave_points3', 
  'symmetry3', 'fractal_dimension3'
)
colnames(df) <- names

In this step, I’m loading the dataset from my local folder. Since the file doesn’t have a header, I manually added column names.

Basic Check

head(df)

missing<-data.frame(apply(df, 2, function(x) sum(is.na(x))))
colnames(missing) <- "Missing Number"
missing

prop.table(table(df$Diagnosis))

## 
##         B         M 
## 0.6274165 0.3725835

names(df)[!sapply(df, is.numeric)]

## [1] "Diagnosis"

par(mfrow = c(5, 6), mar = c(3,3,2,1))
invisible(mapply(function(x, nm) hist(x, main = nm, xlab = nm), df[3:32], names(df[3:32])))

Here, I’m doing some quick checks, missing values, data types, and class balance for the target column (Diagnosis), just to get an idea of the data quality. I also plotted histograms for all numeric columns (except ID and Diagnosis) to see the overall distribution of each feature.

Convert Diagnosis into Factor

df$Diagnosis <- factor(df$Diagnosis, levels = c("M", "B"))

I converted the target column Diagnosis into a two-level factor, with “M” first and “B” second. This helps later when calculating things like ROC or AUC.

Check for Outliers

normalized <- subset(df, select = -c(ID, Diagnosis))
t <- data.frame(Column=character(), Outliers=integer(), Percentage=numeric(), 
                stringsAsFactors = FALSE)
for (i in colnames(normalized)) {
  m <- mean(df[[i]], na.rm = TRUE)
  s <- sd(df[[i]], na.rm = TRUE)
  z <- abs((df[[i]] - m) / s)
  rows.outliers <- which(z > 3)
  percent <- round((length(rows.outliers)/nrow(df))*100, 1)
  
  t <- rbind(t, data.frame(Column = i,
                           Outliers = length(rows.outliers),
                           Percentage = percent))
}

kable(t, caption = "Outliers", align = 'c')

Outliers
Column	Outliers	Percentage
radius1	5	0.9
texture1	4	0.7
perimeter1	7	1.2
area1	8	1.4
smoothness1	5	0.9
compactness1	9	1.6
concavity1	9	1.6
concave_points1	6	1.1
symmetry1	5	0.9
fractal_dimension1	7	1.2
radius2	7	1.2
texture2	9	1.6
perimeter2	8	1.4
area2	6	1.1
smoothness2	7	1.2
compactness2	12	2.1
concavity2	6	1.1
concave_points2	6	1.1
symmetry2	11	1.9
fractal_dimension2	10	1.8
radius3	6	1.1
texture3	4	0.7
perimeter3	6	1.1
area3	10	1.8
smoothness3	3	0.5
compactness3	10	1.8
concavity3	7	1.2
concave_points3	0	0.0
symmetry3	9	1.6
fractal_dimension3	9	1.6

normalized$Diagnosis <- df$Diagnosis

In this step, I checked for outliers using a z-score method with threshold of z > 3. This helps show how many extreme values each column has and the percentage of outliers overall. I didn’t remove them since they only make up a small portion of the data.

Train-Test Split

sample_size <- floor(0.8 * nrow(normalized))
set.seed(123)
index <- sample(seq_len(nrow(normalized)), size = sample_size)

train <- normalized[index, ]
test  <- normalized[-index, ]

train.x <- subset(train, select = -Diagnosis)
train.y <- train$Diagnosis
test.x  <- subset(test, select = -Diagnosis)
test.y  <- test$Diagnosis

I split the dataset into 80% training and 20% testing, which is a standard ratio for most machine learning tasks. The training set is used to train the model, and the test set is used at the end to evaluate its performance.

Normalization with Z-score

m <- sapply(train.x, mean)
s <- sapply(train.x, sd)
scale_with <- function(X, m, s) as.data.frame(sweep(sweep(X, 2, m, "-"), 2, s, "/"))
train.x_s<-scale_with(train.x, m, s)
test.x_s<-scale_with(test.x, m, s)

Here, I normalized all features using z-score scaling. The mean and standard deviation were calculated only from the training set to avoid data leakage, and then applied to both the training and test sets.

Helper Functions

knn_mode <- function(x) {
  ux <- unique(x)
  index <- match(x, ux)
  counts <- tabulate(index, length(ux))
  ux[which.max(counts)]
}

knn_distance <- function(p, q) {
  sqrt(sum((p - q)^2))
}

knn_all_dists <- function(train, u) {
  n <- nrow(train)
  v <- numeric(n)
  for (i in 1:n) {
    p <- train[i, ]
    v[i] <- knn_distance(p, u)
  }
  return(v)
}

knn_predict <- function(train_x, train_y, u, k) {
  if (is.data.frame(train_y)) train_y <- train_y[[1]]
  d <- knn_all_dists(as.matrix(train_x), as.numeric(u))
  ds <- order(d)[1:k]
  pred <- knn_mode(train_y[ds])
  factor(pred, levels = levels(train_y))
}

knn_predict_all <- function(train_x, train_y, test_x, k = 5) {
  n <- nrow(test_x)
  pred <- factor(rep(NA, n), levels = levels(train_y))
  for (i in 1:n) {
    u <- test_x[i, ]
    pred[i] <- knn_predict(train_x, train_y, u, k)
  }
  return(pred)
}


knn_score <- function(train_x, train_y, test_x, k) {
  if (is.data.frame(train_y)) train_y <- train_y[[1]]
  n <- nrow(test_x)
  scores <- numeric(n)
  pos = 'M'
  for (i in 1:n) {
    u <- test_x[i, ]
    d <- knn_all_dists(as.matrix(train_x), as.numeric(u)) 
    n <- order(d)[1:k]
    scores[i] <- mean(train_y[n] == pos)
  }
  scores
}

knn_acc <- function(test_y, pred, positive = 'M') {
  TP <- sum(test_y == positive & pred == positive)
  TN <- sum(test_y != positive & pred != positive)
  FP <- sum(test_y != positive & pred == positive)
  FN <- sum(test_y == positive & pred != positive)
  accuracy <- ((TP + TN) / length(pred)) * 100
  
  return(list(
    accuracy = accuracy, 
    True_Positive = TP, 
    True_Negative = TN, 
    False_Positive = FP, 
    False_Negative = FN
  ))
}

This section defines all the helper functions needed for k-NN:

knn_mode: finds the most frequent label among neighbors.
knn_distance: calculates Euclidean distance between two points.
knn_all_dists: uses knn_distance to compute the distance between one sample and all points in the training data.
knn_predict: uses those distances to predict one label.
knn_predict_all: loops through all test samples to predict their labels.
knn_score: calculates the fraction of “M” neighbors (used later for ROC/AUC).
knn_acc: computes accuracy and returns confusion matrix metrics — TP, TN, FP, and FN.

K Evaluation

k <- seq(1, 51, by = 2)
acc <- numeric(length(k))

for (i in seq_along(k)) {
  x <- k[i]
  prediction <- knn_predict_all(train_x = train.x_s, train_y = train.y, test_x = test.x_s, k = x)
  result <- knn_acc(test_y = test.y, pred = prediction)
  acc[i] <- result$accuracy
}

acc_k <- data.frame(k = k, accuracy = acc)
plot(acc_k$k, acc_k$accuracy, type = "b", 
     xlab = "k", ylab = "Accuracy", main = "kNN Accuracy vs k")

which.max(acc_k$accuracy)

## [1] 1

In this part, I tested different values of k from 1 to 51, to see how accuracy changes. The model performs best when k = 1, but to make sure, I’ll double-check it using cross-validation to confirm if it’s real or just overfitting.

Cross Validation

runs<-list()

for (i in 1:5) {
  folds <- createFolds(train.y, k = 10)
  cv_res <- data.frame(k=integer(), fold=integer(), Accuracy=double())
  for (f in seq_along(folds)) {
    idx <- folds[[f]]
    cv_train.x <- train.x[-idx, , drop=FALSE]
    cv_train.y <- train.y[-idx]
    cv_test.x <- train.x[idx, , drop=FALSE]
    cv_test.y <- factor(train.y[idx], levels = levels(train.y))
    m <- sapply(cv_train.x, mean)
    s <- sapply(cv_train.x, sd)
    cv_train.x <- scale_with(cv_train.x, m, s)
    cv_test.x <- scale_with(cv_test.x, m, s)
    for (K in k) {
      pred <- knn_predict_all(cv_train.x, cv_train.y, cv_test.x, K)
      acc  <- knn_acc(cv_test.y, pred)
      cv_res <- rbind(cv_res, data.frame(k = K, fold = f, Accuracy = acc$accuracy))
    }
  }
  summary <- aggregate(Accuracy ~ k, data = cv_res, mean)
  runs[[i]]<-summary
}

runs <- do.call(rbind, runs)
result <- runs %>%
  group_by(k) %>%
  summarise(
    Accuracy = mean(Accuracy, na.rm = TRUE),
    .groups = "drop"
  )
result

opt_k <-result$k[which.max(result$Accuracy)]
opt_k

## [1] 3

Here, I ran a 10-fold cross-validation, repeated 5 times, to find a more reliable value for k. Each fold uses a different subset of the training data for validation. Inside every fold, I re-scale the features based only on that fold’s training portion to keep things leak-free. After averaging all folds and runs, I picked the k with the highest mean accuracy as the final one.

ROC & AUC

score <- knn_score(train.x_s, train.y, test.x_s, k=opt_k)
roc <- roc(test.y, score)

## Setting levels: control = M, case = B

## Setting direction: controls > cases

roc

## 
## Call:
## roc.default(response = test.y, predictor = score)
## 
## Data: score in 53 controls (test.y M) > 61 cases (test.y B).
## Area under the curve: 0.979

plot(roc)

auc(roc)

## Area under the curve: 0.979

Now that I have the optimal k, I calculated the ROC curve and AUC score using the test set. The score represents the probability of correctly distinguishing malignant (M) from benign (B) cases. AUC close to 1 means the model performs very well — in this case, 0.979, which is excellent.

Final Prediction

final_pred <- knn_predict_all(train.x_s, train.y, test.x_s, k=opt_k)
knn_acc(test.y, final_pred)

## $accuracy
## [1] 95.61404
## 
## $True_Positive
## [1] 49
## 
## $True_Negative
## [1] 60
## 
## $False_Positive
## [1] 1
## 
## $False_Negative
## [1] 4

Finally, I used the best k from cross-validation to make predictions on the test set. The accuracy came out above 95%, showing that the model can classify malignant and benign cases with high confidence.

kNN classification for cancer

Feng, Chen

Loading Data

In this step, I’m loading the dataset from my local folder. Since the file doesn’t have a header, I manually added column names.

Basic Check

Here, I’m doing some quick checks, missing values, data types, and class balance for the target column (Diagnosis), just to get an idea of the data quality. I also plotted histograms for all numeric columns (except ID and Diagnosis) to see the overall distribution of each feature.

Convert Diagnosis into Factor

I converted the target column Diagnosis into a two-level factor, with “M” first and “B” second. This helps later when calculating things like ROC or AUC.

Check for Outliers

In this step, I checked for outliers using a z-score method with threshold of z > 3. This helps show how many extreme values each column has and the percentage of outliers overall. I didn’t remove them since they only make up a small portion of the data.

Train-Test Split

I split the dataset into 80% training and 20% testing, which is a standard ratio for most machine learning tasks. The training set is used to train the model, and the test set is used at the end to evaluate its performance.

Normalization with Z-score

Here, I normalized all features using z-score scaling. The mean and standard deviation were calculated only from the training set to avoid data leakage, and then applied to both the training and test sets.

Helper Functions

This section defines all the helper functions needed for k-NN:

K Evaluation

In this part, I tested different values of k from 1 to 51, to see how accuracy changes. The model performs best when k = 1, but to make sure, I’ll double-check it using cross-validation to confirm if it’s real or just overfitting.

Cross Validation

ROC & AUC

Now that I have the optimal k, I calculated the ROC curve and AUC score using the test set. The score represents the probability of correctly distinguishing malignant (M) from benign (B) cases. AUC close to 1 means the model performs very well — in this case, 0.979, which is excellent.

Final Prediction

Finally, I used the best k from cross-validation to make predictions on the test set. The accuracy came out above 95%, showing that the model can classify malignant and benign cases with high confidence.