【PyTorch】Kaggle深度学习实战之Skin Cancer MNIST: HAM10000

矫情吗；* 2023-01-21 12:16 4阅读 0赞

### 目录 ###

*  数据集概述
 *   *  Description
     *   *  Overview
         *  Original Data Source
         *  From Authors
 *  初次尝试 (CNN)
 *  利用imblearn库处理不平衡样本
 *  第二次尝试 (CNN)
 *   *  my\_dataset.py
     *  train\_CNN.py
 *  第三次尝试 (ResNet18预训练模型)
 *   *  train\_ResNet18.py

# 数据集概述 #

本次实战练习的数据集来自Kaggle的[Skin Cancer MNIST: HAM10000][Skin Cancer MNIST_ HAM10000]。官方的Description如下：

> ## Description ##
> 
> ### Overview ###
> 
> Another more interesting than digit classification dataset to use to get biology and medicine students more excited about machine learning and image processing.
> 
> ### Original Data Source ###
> 
>  *  Original Challenge: [https://challenge2018.isic-archive.com][https_challenge2018.isic-archive.com]
>  *  [https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/DBW86T][https_dataverse.harvard.edu_dataset.xhtml_persistentId_doi_10.7910_DVN_DBW86T]
> 
> \[1\] Noel Codella, Veronica Rotemberg, Philipp Tschandl, M. Emre Celebi, Stephen Dusza, David Gutman, Brian Helba, Aadi Kalloo, Konstantinos Liopyris, Michael Marchetti, Harald Kittler, Allan Halpern: “Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)”, 2018; [https://arxiv.org/abs/1902.03368][https_arxiv.org_abs_1902.03368]  
> \[2\] Tschandl, P., Rosendahl, C. & Kittler, H. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 5, 180161 doi:10.1038/sdata.2018.161 (2018).
> 
> ### From Authors ###
> 
> Training of neural networks for automated diagnosis of pigmented skin lesions is hampered by the small size and lack of diversity of available dataset of dermatoscopic images. We tackle this problem by releasing the HAM10000 (“Human Against Machine with 10000 training images”) dataset. We collected dermatoscopic images from different populations, acquired and stored by different modalities. The final dataset consists of 10015 dermatoscopic images which can serve as a training set for academic machine learning purposes. Cases include a representative collection of all important diagnostic categories in the realm of pigmented lesions: Actinic keratoses and intraepithelial carcinoma / Bowen’s disease (akiec), basal cell carcinoma (bcc), benign keratosis-like lesions (solar lentigines / seborrheic keratoses and lichen-planus like keratoses, bkl), dermatofibroma (df), melanoma (mel), melanocytic nevi (nv) and vascular lesions (angiomas, angiokeratomas, pyogenic granulomas and hemorrhage, vasc).  
> More than 50% of lesions are confirmed through histopathology (histo), the ground truth for the rest of the cases is either follow-up examination (followup), expert consensus (consensus), or confirmation by in-vivo confocal microscopy (confocal). The dataset includes lesions with multiple images, which can be tracked by the lesionid-column within the HAM10000\_metadata file.  
> The test set is not public, but the evaluation server remains running (see the challenge website). Any publications written using the HAM10000 data should be evaluated on the official test set hosted there, so that methods can be fairly compared.

本次训练我使用的数据集为`hmnist_28_28_RGB.csv`，其中包含10016行2353列（含标题行），其中前2352列为每张图片各通道各像素点的灰度值 (28×28×3)，最后1列为`label`，即该图像的标签。在这份数据集中，`label`的取值为`[0, 1, 2, 3, 4, 5, 6]`。

# 初次尝试 (CNN) #

在观察过数据集之后，我的第一想法是可以把之前训练过的MNIST的模型拿来训练这份数据集，因为MNIST数据集每张图片的大小也是28×28，两份数据集最大的差异在于MNIST的图片是灰度图，而HAM10000的图片是RGB三通道的彩色图（Kaggle也提供了灰度图版本的HAM10000数据集）。另外，MNIST本质上是一个十分类问题，即将图片分类为`[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]`这十个类别之一，而HAM10000本质上是一个七分类问题。  
基于以上想法，我们可以将以前训练MNIST的模型修改一下：将第一个卷积层的`in_channels`参数由`1`修改为`3`，将最后一个`nn.Linear`的参数由`(128, 10)`修改为`(128, 7)`。

> 注：原先训练MNIST的CNN模型如下所示。此前该模型训练MNIST数据集可达到0.99875的泛化精度。
> 
>     CNN(
>       (conv_unit1): Sequential(
>         (0): Conv2d(1, 16, kernel_size=(3, 3), stride=(1, 1))
>         (1): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
>         (2): ReLU(inplace=True)
>       )
>       (conv_unit2): Sequential(
>         (0): Conv2d(16, 32, kernel_size=(3, 3), stride=(1, 1))
>         (1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
>         (2): ReLU(inplace=True)
>         (3): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
>       )
>       (conv_unit3): Sequential(
>         (0): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1))
>         (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
>         (2): ReLU(inplace=True)
>      )
>       (conv_unit4): Sequential(
>         (0): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1))
>         (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
>         (2): ReLU(inplace=True)
>         (3): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
>       )
>       (fc_unit): Sequential(
>         (0): Linear(in_features=2048, out_features=1024, bias=True)
>         (1): ReLU(inplace=True)
>         (2): Dropout(p=0.5, inplace=False)
>         (3): Linear(in_features=1024, out_features=128, bias=True)
>         (4): ReLU(inplace=True)
>         (5): Dropout(p=0.5, inplace=False)
>         (6): Linear(in_features=128, out_features=10, bias=True)
>       )
>     )

修改后的`train.py`如下：

import torch.nn as nn
    
    
    class CNN(nn.Module):
        def __init__(self):
            super(CNN, self).__init__()
            self.conv_unit1 = nn.Sequential(
                nn.Conv2d(in_channels=3, out_channels=16, kernel_size=3),
                nn.BatchNorm2d(16),
                nn.ReLU(inplace=True)
            )
            self.conv_unit2 = nn.Sequential(
                nn.Conv2d(in_channels=16, out_channels=32, kernel_size=3),
                nn.BatchNorm2d(32),
                nn.ReLU(inplace=True),
                nn.MaxPool2d(kernel_size=2, stride=2)
            )
            self.conv_unit3 = nn.Sequential(
                nn.Conv2d(in_channels=32, out_channels=64, kernel_size=3),
                nn.BatchNorm2d(64),
                nn.ReLU(inplace=True)
            )
            self.conv_unit4 = nn.Sequential(
                nn.Conv2d(in_channels=64, out_channels=128, kernel_size=3),
                nn.BatchNorm2d(128),
                nn.ReLU(inplace=True),
                nn.MaxPool2d(kernel_size=2, stride=2)
            )
            self.fc_unit = nn.Sequential(
                nn.Linear(128 * 4 * 4, 1024),
                nn.ReLU(inplace=True),
                nn.Dropout(p=0.5),
                nn.Linear(1024, 128),
                nn.ReLU(inplace=True),
                nn.Dropout(p=0.5),
                nn.Linear(128, 7)
            )
    
        def forward(self, x):
            x = self.conv_unit1(x)
            x = self.conv_unit2(x)
            x = self.conv_unit3(x)
            x = self.conv_unit4(x)
            x = x.view(x.size(0), -1)
            x = self.fc_unit(x)
            return x

但是在实际训练中，这个模型并没有取得非常好的训练结果，而是在训练开始后不久就出现了严重的过拟合现象——训练集精度达到了0.95以上而测试集精度却还不到0.75。  
我重新对数据集进行观察，发现了之前被我忽略的一个问题。在HAM10000数据集中，每一类样本的数量并不相同，其中标签为`4`的样本数量最多，标签为`3`和`5`的样本数量最少。不平衡的样本应该就是导致过拟合的最主要因素，当我们的样本数据占比不平衡的时候，模型就会偏向于多数类的结果。  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ1NTU0MDEw_size_16_color_FFFFFF_t_70]

# 利用imblearn库处理不平衡样本 #

针对不平衡数据，最简单的一种方法就是生成少数类的样本，这其中最基本的方法就是从少数类的样本中进行随机采样来增加新的样本。使用`imblearn.over_sampling.RandomOverSampler()`可以通过简单的随机采样少数类的样本，使得每类样本的比例为1。

from imblearn.over_sampling import RandomOverSampler
    
    
    ...
    
    ros = RandomOverSampler()
    data = pd.read_csv(DATA_PATH, encoding='utf-8')
    labels = data['label']
    images = data.drop(columns=['label'])
    images, labels = ros.fit_resample(images, labels)
    print('The size of dataset: {}'.format(images.shape[0]))

The size of dataset: 46935

# 第二次尝试 (CNN) #

## my\_dataset.py ##

import numpy as np
    from torch.utils.data import Dataset
    
    
    class my_dataset(Dataset):
        def __init__(self, df, labels, transform):
            self.df = df.values.reshape(-1, 28, 28, 3).astype(np.uint8)
            self.labels = labels
            self.transform = transform
    
        def __len__(self):
            return len(self.df)
    
        def __getitem__(self, index):
            return self.transform(self.df[index]), self.labels[index]

## train\_CNN.py ##

import os
    import tqdm
    
    from imblearn.over_sampling import RandomOverSampler
    import matplotlib.pyplot as plt
    import numpy as np
    import pandas as pd
    from sklearn.model_selection import train_test_split
    import torch
    from torch.autograd import Variable
    import torch.nn as nn
    import torch.optim as optim
    from torch.utils.data import DataLoader
    from torchsummary import summary
    from torchvision import transforms
    
    from model import CNN
    from my_dataset import my_dataset
    
    
    if not os.path.exists('./model/'):
        os.mkdir('./model/')
    
    BATCH_SIZE = 128
    DATA_PATH = './data/hmnist_28_28_RGB.csv'
    EPOCH = 100
    LEARNING_RATE = 1e-3
    GPU_IS_OK = torch.cuda.is_available()
    
    device = torch.device('cuda' if GPU_IS_OK else 'cpu')
    print('SkinCancerMNIST-HAM10000 is being trained using the {}.'.format('GPU' if GPU_IS_OK else 'CPU'))
    
    ros = RandomOverSampler()
    transform = transforms.Compose([
        transforms.ToTensor(),
        transforms.Normalize(mean=[0.5], std=[0.5])
    ])
    
    print('Loading the dataset...')
    data = pd.read_csv(DATA_PATH, encoding='utf-8')
    labels = data['label']
    images = data.drop(columns=['label'])
    images, labels = ros.fit_resample(images, labels)
    print('The size of dataset: {}'.format(images.shape[0]))
    X_train, X_eval, y_train, y_eval = train_test_split(images, labels, train_size=0.8, random_state=21)
    y_train = torch.from_numpy(np.array(y_train)).type(torch.LongTensor)
    y_eval = torch.from_numpy(np.array(y_eval)).type(torch.LongTensor)
    train_data = my_dataset(df=X_train, labels=y_train, transform=transform)
    eval_data = my_dataset(df=X_eval, labels=y_eval, transform=transform)
    data_loader_train = DataLoader(train_data, batch_size=BATCH_SIZE, shuffle=True)
    data_loader_eval = DataLoader(eval_data, batch_size=BATCH_SIZE, shuffle=True)
    print('Over.')
    
    my_model = CNN().to(device)
    print('The model structure and parameters are as follows:')
    summary(my_model, input_size=(3, 28, 28))
    criterion = nn.CrossEntropyLoss()
    optimizer = optim.Adam(my_model.parameters(), lr=LEARNING_RATE, betas=(0.9, 0.999), eps=1e-8)
    
    print('Training...')
    for epoch in range(1, EPOCH + 1):
        # train
        train_loss = 0
        train_acc = 0
        train_step = 0
        my_model.train()
        for image, label in tqdm.tqdm(data_loader_train):
            image = Variable(image.to(device))
            label = Variable(label.to(device))
            output = my_model(image)
            loss = criterion(output, label)
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
            train_loss += loss.item()
            _, pred = output.max(1)
            num_correct = (pred == label).sum().item()
            acc = num_correct / BATCH_SIZE
            train_acc += acc
            train_step += 1
        train_loss /= train_step
        train_acc /= train_step
        # eval
        eval_loss = 0
        eval_acc = 0
        eval_step = 0
        my_model.eval()
        for image, label in data_loader_eval:
            image = Variable(image.to(device))
            label = Variable(label.to(device))
            output = my_model(image)
            loss = criterion(output, label)
            eval_loss += loss.item()
            _, pred = output.max(1)
            num_correct = (pred == label).sum().item()
            acc = num_correct / BATCH_SIZE
            eval_acc += acc
            eval_step += 1
        eval_loss /= eval_step
        eval_acc /= eval_step
        # print the loss and accuracy
        print('[{:3d}/{:3d}] Train Loss: {:11.9f} | Train Accuracy: {:6.4f} | Eval Loss: {:11.9f} | Eval Accuracy: {:6.4f}'
              .format(epoch, EPOCH, train_loss, train_acc, eval_loss, eval_acc))
        # save the model
        torch.save(my_model.state_dict(), './model/cnn_epoch{}.pth'.format(epoch))
    
    print('Training completed.')

训练结果如下：

可以看到，模型的过拟合问题有了很大的改善（虽然仍有一些轻微过拟合），验证集精度在第59轮达到了最高值0.9787。

# 第三次尝试 (ResNet18预训练模型) #

随后我又进行了第三次尝试，使用`torchvision.models`中的ResNet18预训练模型进行迁移学习。考虑到HAM10000数据集的图片与预训练模型的数据集图片差异较大，我将`pratrained`参数设置为`True`，但并没有冻结前面的网络层参数，而是将模型整体进行微调。

## train\_ResNet18.py ##

代码总体上与`train_CNN.py`差别不大，只是在定义模型的地方不同，另外将学习率`LEARNING_RATE`由先前的`1e-3`调整为`1e-4`。

import os
    import tqdm
    
    from imblearn.over_sampling import RandomOverSampler
    import matplotlib.pyplot as plt
    import numpy as np
    import pandas as pd
    from sklearn.model_selection import train_test_split
    import torch
    from torch.autograd import Variable
    import torch.nn as nn
    import torch.optim as optim
    from torch.utils.data import DataLoader
    from torchsummary import summary
    from torchvision import transforms
    from torchvision.models import resnet18
    
    from my_dataset import my_dataset
    
    
    if not os.path.exists('./model/'):
        os.mkdir('./model/')
    
    BATCH_SIZE = 128
    DATA_PATH = './data/hmnist_28_28_RGB.csv'
    EPOCH = 100
    LEARNING_RATE = 1e-4
    GPU_IS_OK = torch.cuda.is_available()
    
    device = torch.device('cuda' if GPU_IS_OK else 'cpu')
    print('SkinCancerMNIST-HAM10000 is being trained using the {}.'.format('GPU' if GPU_IS_OK else 'CPU'))
    
    ros = RandomOverSampler()
    transform = transforms.Compose([
        transforms.ToTensor(),
        transforms.Normalize(mean=[0.5], std=[0.5])
    ])
    
    print('Loading the dataset...')
    data = pd.read_csv(DATA_PATH, encoding='utf-8')
    labels = data['label']
    images = data.drop(columns=['label'])
    images, labels = ros.fit_resample(images, labels)
    print('The size of dataset: {}'.format(images.shape[0]))
    X_train, X_eval, y_train, y_eval = train_test_split(images, labels, train_size=0.8, random_state=21)
    y_train = torch.from_numpy(np.array(y_train)).type(torch.LongTensor)
    y_eval = torch.from_numpy(np.array(y_eval)).type(torch.LongTensor)
    train_data = my_dataset(df=X_train, labels=y_train, transform=transform)
    eval_data = my_dataset(df=X_eval, labels=y_eval, transform=transform)
    data_loader_train = DataLoader(train_data, batch_size=BATCH_SIZE, shuffle=True)
    data_loader_eval = DataLoader(eval_data, batch_size=BATCH_SIZE, shuffle=True)
    print('Over.')
    
    ResNet18 = resnet18(pretrained=True)
    ResNet18.fc = nn.Linear(ResNet18.fc.in_features, 7)
    ResNet18 = ResNet18.to(device)
    print('The model structure and parameters are as follows:')
    summary(ResNet18, input_size=(3, 28, 28))
    criterion = nn.CrossEntropyLoss()
    optimizer = optim.Adam(ResNet18.parameters(), lr=LEARNING_RATE, betas=(0.9, 0.999), eps=1e-8)
    
    print('Training...')
    for epoch in range(1, EPOCH + 1):
        # train
        train_loss = 0
        train_acc = 0
        train_step = 0
        ResNet18.train()
        for image, label in tqdm.tqdm(data_loader_train):
            image = Variable(image.to(device))
            label = Variable(label.to(device))
            output = ResNet18(image)
            loss = criterion(output, label)
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()
            train_loss += loss.item()
            _, pred = output.max(1)
            num_correct = (pred == label).sum().item()
            acc = num_correct / BATCH_SIZE
            train_acc += acc
            train_step += 1
        train_loss /= train_step
        train_acc /= train_step
        # eval
        eval_loss = 0
        eval_acc = 0
        eval_step = 0
        ResNet18.eval()
        for image, label in data_loader_eval:
            image = Variable(image.to(device))
            label = Variable(label.to(device))
            output = ResNet18(image)
            loss = criterion(output, label)
            eval_loss += loss.item()
            _, pred = output.max(1)
            num_correct = (pred == label).sum().item()
            acc = num_correct / BATCH_SIZE
            eval_acc += acc
            eval_step += 1
        eval_loss /= eval_step
        eval_acc /= eval_step
        # print the loss and accuracy
        print('[{:3d}/{:3d}] Train Loss: {:11.9f} | Train Accuracy: {:6.4f} | Eval Loss: {:11.9f} | Eval Accuracy: {:6.4f}'
              .format(epoch, EPOCH, train_loss, train_acc, eval_loss, eval_acc))
        # save the model
        torch.save(ResNet18.state_dict(), './model/ResNet18_epoch{}.pth'.format(epoch))
    
    print('Training completed.')

训练结果如下：

SkinCancerMNIST-HAM10000 is being trained using the GPU.
    Loading the dataset...
    The size of dataset: 46935
    Over.
    The model structure and parameters are as follows:
    ----------------------------------------------------------------
            Layer (type)               Output Shape         Param #
    ================================================================
                Conv2d-1           [-1, 64, 14, 14]           9,408
           BatchNorm2d-2           [-1, 64, 14, 14]             128
                  ReLU-3           [-1, 64, 14, 14]               0
             MaxPool2d-4             [-1, 64, 7, 7]               0
                Conv2d-5             [-1, 64, 7, 7]          36,864
           BatchNorm2d-6             [-1, 64, 7, 7]             128
                  ReLU-7             [-1, 64, 7, 7]               0
                Conv2d-8             [-1, 64, 7, 7]          36,864
           BatchNorm2d-9             [-1, 64, 7, 7]             128
                 ReLU-10             [-1, 64, 7, 7]               0
           BasicBlock-11             [-1, 64, 7, 7]               0
               Conv2d-12             [-1, 64, 7, 7]          36,864
          BatchNorm2d-13             [-1, 64, 7, 7]             128
                 ReLU-14             [-1, 64, 7, 7]               0
               Conv2d-15             [-1, 64, 7, 7]          36,864
          BatchNorm2d-16             [-1, 64, 7, 7]             128
                 ReLU-17             [-1, 64, 7, 7]               0
           BasicBlock-18             [-1, 64, 7, 7]               0
               Conv2d-19            [-1, 128, 4, 4]          73,728
          BatchNorm2d-20            [-1, 128, 4, 4]             256
                 ReLU-21            [-1, 128, 4, 4]               0
               Conv2d-22            [-1, 128, 4, 4]         147,456
          BatchNorm2d-23            [-1, 128, 4, 4]             256
               Conv2d-24            [-1, 128, 4, 4]           8,192
          BatchNorm2d-25            [-1, 128, 4, 4]             256
                 ReLU-26            [-1, 128, 4, 4]               0
           BasicBlock-27            [-1, 128, 4, 4]               0
               Conv2d-28            [-1, 128, 4, 4]         147,456
          BatchNorm2d-29            [-1, 128, 4, 4]             256
                 ReLU-30            [-1, 128, 4, 4]               0
               Conv2d-31            [-1, 128, 4, 4]         147,456
          BatchNorm2d-32            [-1, 128, 4, 4]             256
                 ReLU-33            [-1, 128, 4, 4]               0
           BasicBlock-34            [-1, 128, 4, 4]               0
               Conv2d-35            [-1, 256, 2, 2]         294,912
          BatchNorm2d-36            [-1, 256, 2, 2]             512
                 ReLU-37            [-1, 256, 2, 2]               0
               Conv2d-38            [-1, 256, 2, 2]         589,824
          BatchNorm2d-39            [-1, 256, 2, 2]             512
               Conv2d-40            [-1, 256, 2, 2]          32,768
          BatchNorm2d-41            [-1, 256, 2, 2]             512
                 ReLU-42            [-1, 256, 2, 2]               0
           BasicBlock-43            [-1, 256, 2, 2]               0
               Conv2d-44            [-1, 256, 2, 2]         589,824
          BatchNorm2d-45            [-1, 256, 2, 2]             512
                 ReLU-46            [-1, 256, 2, 2]               0
               Conv2d-47            [-1, 256, 2, 2]         589,824
          BatchNorm2d-48            [-1, 256, 2, 2]             512
                 ReLU-49            [-1, 256, 2, 2]               0
           BasicBlock-50            [-1, 256, 2, 2]               0
               Conv2d-51            [-1, 512, 1, 1]       1,179,648
          BatchNorm2d-52            [-1, 512, 1, 1]           1,024
                 ReLU-53            [-1, 512, 1, 1]               0
               Conv2d-54            [-1, 512, 1, 1]       2,359,296
          BatchNorm2d-55            [-1, 512, 1, 1]           1,024
               Conv2d-56            [-1, 512, 1, 1]         131,072
          BatchNorm2d-57            [-1, 512, 1, 1]           1,024
                 ReLU-58            [-1, 512, 1, 1]               0
           BasicBlock-59            [-1, 512, 1, 1]               0
               Conv2d-60            [-1, 512, 1, 1]       2,359,296
          BatchNorm2d-61            [-1, 512, 1, 1]           1,024
                 ReLU-62            [-1, 512, 1, 1]               0
               Conv2d-63            [-1, 512, 1, 1]       2,359,296
          BatchNorm2d-64            [-1, 512, 1, 1]           1,024
                 ReLU-65            [-1, 512, 1, 1]               0
           BasicBlock-66            [-1, 512, 1, 1]               0
    AdaptiveAvgPool2d-67            [-1, 512, 1, 1]               0
               Linear-68                    [-1, 7]           3,591
    ================================================================
    Total params: 11,180,103
    Trainable params: 11,180,103
    Non-trainable params: 0
    ----------------------------------------------------------------
    Input size (MB): 0.01
    Forward/backward pass size (MB): 1.09
    Params size (MB): 42.65
    Estimated Total Size (MB): 43.75
    ----------------------------------------------------------------
    Training...
    100%|██████████| 294/294 [00:27<00:00, 10.86it/s]
    [  1/100] Train Loss: 0.483066778 | Train Accuracy: 0.8372 | Eval Loss: 0.149059397 | Eval Accuracy: 0.9429
    100%|██████████| 294/294 [00:26<00:00, 11.00it/s]
    [  2/100] Train Loss: 0.092949655 | Train Accuracy: 0.9690 | Eval Loss: 0.088184921 | Eval Accuracy: 0.9625
    100%|██████████| 294/294 [00:26<00:00, 10.96it/s]
    [  3/100] Train Loss: 0.048947430 | Train Accuracy: 0.9825 | Eval Loss: 0.075887994 | Eval Accuracy: 0.9664
    100%|██████████| 294/294 [00:26<00:00, 10.99it/s]
    [  4/100] Train Loss: 0.047908589 | Train Accuracy: 0.9830 | Eval Loss: 0.078713355 | Eval Accuracy: 0.9673
    100%|██████████| 294/294 [00:26<00:00, 10.92it/s]
    [  5/100] Train Loss: 0.036838357 | Train Accuracy: 0.9863 | Eval Loss: 0.074998591 | Eval Accuracy: 0.9707
    100%|██████████| 294/294 [00:26<00:00, 10.91it/s]
    [  6/100] Train Loss: 0.035881876 | Train Accuracy: 0.9859 | Eval Loss: 0.069636245 | Eval Accuracy: 0.9722
    100%|██████████| 294/294 [00:26<00:00, 10.93it/s]
    [  7/100] Train Loss: 0.015354439 | Train Accuracy: 0.9928 | Eval Loss: 0.062996378 | Eval Accuracy: 0.9760
    100%|██████████| 294/294 [00:26<00:00, 10.93it/s]
    [  8/100] Train Loss: 0.013381565 | Train Accuracy: 0.9936 | Eval Loss: 0.071176288 | Eval Accuracy: 0.9750
    100%|██████████| 294/294 [00:26<00:00, 10.93it/s]
    [  9/100] Train Loss: 0.025142998 | Train Accuracy: 0.9893 | Eval Loss: 0.072122657 | Eval Accuracy: 0.9716
    100%|██████████| 294/294 [00:26<00:00, 10.95it/s]
    [ 10/100] Train Loss: 0.034815519 | Train Accuracy: 0.9857 | Eval Loss: 0.073852423 | Eval Accuracy: 0.9726
    100%|██████████| 294/294 [00:26<00:00, 10.94it/s]
    [ 11/100] Train Loss: 0.019632874 | Train Accuracy: 0.9914 | Eval Loss: 0.083632919 | Eval Accuracy: 0.9722
    100%|██████████| 294/294 [00:26<00:00, 10.93it/s]
    [ 12/100] Train Loss: 0.016087983 | Train Accuracy: 0.9928 | Eval Loss: 0.055468219 | Eval Accuracy: 0.9781
    100%|██████████| 294/294 [00:26<00:00, 10.96it/s]
    [ 13/100] Train Loss: 0.008893993 | Train Accuracy: 0.9952 | Eval Loss: 0.074550826 | Eval Accuracy: 0.9755
    100%|██████████| 294/294 [00:26<00:00, 10.93it/s]
    [ 14/100] Train Loss: 0.018936129 | Train Accuracy: 0.9919 | Eval Loss: 0.068807984 | Eval Accuracy: 0.9756
    100%|██████████| 294/294 [00:26<00:00, 10.90it/s]
    [ 15/100] Train Loss: 0.024428692 | Train Accuracy: 0.9901 | Eval Loss: 0.053713250 | Eval Accuracy: 0.9804
    100%|██████████| 294/294 [00:27<00:00, 10.78it/s]
    [ 16/100] Train Loss: 0.008889921 | Train Accuracy: 0.9952 | Eval Loss: 0.052808468 | Eval Accuracy: 0.9793
    100%|██████████| 294/294 [00:27<00:00, 10.88it/s]
    [ 17/100] Train Loss: 0.005512334 | Train Accuracy: 0.9960 | Eval Loss: 0.063315019 | Eval Accuracy: 0.9779
    100%|██████████| 294/294 [00:27<00:00, 10.87it/s]
    [ 18/100] Train Loss: 0.014360317 | Train Accuracy: 0.9934 | Eval Loss: 0.075652671 | Eval Accuracy: 0.9750
    100%|██████████| 294/294 [00:27<00:00, 10.64it/s]
    [ 19/100] Train Loss: 0.015307088 | Train Accuracy: 0.9928 | Eval Loss: 0.069055489 | Eval Accuracy: 0.9762
    100%|██████████| 294/294 [00:27<00:00, 10.86it/s]
    [ 20/100] Train Loss: 0.013735209 | Train Accuracy: 0.9932 | Eval Loss: 0.068658131 | Eval Accuracy: 0.9756
    100%|██████████| 294/294 [00:27<00:00, 10.79it/s]
    [ 21/100] Train Loss: 0.022175656 | Train Accuracy: 0.9909 | Eval Loss: 0.058698054 | Eval Accuracy: 0.9775
    100%|██████████| 294/294 [00:27<00:00, 10.87it/s]
    [ 22/100] Train Loss: 0.009645415 | Train Accuracy: 0.9945 | Eval Loss: 0.055439501 | Eval Accuracy: 0.9795
    100%|██████████| 294/294 [00:27<00:00, 10.78it/s]
    [ 23/100] Train Loss: 0.006096516 | Train Accuracy: 0.9959 | Eval Loss: 0.069358642 | Eval Accuracy: 0.9757
    100%|██████████| 294/294 [00:27<00:00, 10.72it/s]
    [ 24/100] Train Loss: 0.007846275 | Train Accuracy: 0.9952 | Eval Loss: 0.073728980 | Eval Accuracy: 0.9760
    100%|██████████| 294/294 [00:27<00:00, 10.88it/s]
    [ 25/100] Train Loss: 0.011561618 | Train Accuracy: 0.9943 | Eval Loss: 0.111599398 | Eval Accuracy: 0.9689
    100%|██████████| 294/294 [00:27<00:00, 10.88it/s]
    [ 26/100] Train Loss: 0.020321033 | Train Accuracy: 0.9910 | Eval Loss: 0.065664590 | Eval Accuracy: 0.9772
    100%|██████████| 294/294 [00:27<00:00, 10.87it/s]
    [ 27/100] Train Loss: 0.005860286 | Train Accuracy: 0.9960 | Eval Loss: 0.061213112 | Eval Accuracy: 0.9794
    100%|██████████| 294/294 [00:26<00:00, 10.91it/s]
    [ 28/100] Train Loss: 0.003614324 | Train Accuracy: 0.9967 | Eval Loss: 0.064343554 | Eval Accuracy: 0.9773
    100%|██████████| 294/294 [00:27<00:00, 10.87it/s]
    [ 29/100] Train Loss: 0.004452085 | Train Accuracy: 0.9968 | Eval Loss: 0.063765692 | Eval Accuracy: 0.9792
    100%|██████████| 294/294 [00:27<00:00, 10.89it/s]
    [ 30/100] Train Loss: 0.009551118 | Train Accuracy: 0.9954 | Eval Loss: 0.080716959 | Eval Accuracy: 0.9746
    100%|██████████| 294/294 [00:26<00:00, 10.91it/s]
    [ 31/100] Train Loss: 0.025784245 | Train Accuracy: 0.9899 | Eval Loss: 0.060676569 | Eval Accuracy: 0.9779
    100%|██████████| 294/294 [00:27<00:00, 10.89it/s]
    [ 32/100] Train Loss: 0.004817960 | Train Accuracy: 0.9966 | Eval Loss: 0.060224885 | Eval Accuracy: 0.9799
    100%|██████████| 294/294 [00:26<00:00, 10.90it/s]
    [ 33/100] Train Loss: 0.002682492 | Train Accuracy: 0.9969 | Eval Loss: 0.064649559 | Eval Accuracy: 0.9802
    100%|██████████| 294/294 [00:27<00:00, 10.88it/s]
    [ 34/100] Train Loss: 0.005738550 | Train Accuracy: 0.9960 | Eval Loss: 0.057705094 | Eval Accuracy: 0.9789
    100%|██████████| 294/294 [00:26<00:00, 10.89it/s]
    [ 35/100] Train Loss: 0.017126426 | Train Accuracy: 0.9925 | Eval Loss: 0.049599137 | Eval Accuracy: 0.9797
    100%|██████████| 294/294 [00:26<00:00, 10.90it/s]
    [ 36/100] Train Loss: 0.010584366 | Train Accuracy: 0.9943 | Eval Loss: 0.045030481 | Eval Accuracy: 0.9812
    100%|██████████| 294/294 [00:27<00:00, 10.89it/s]
    [ 37/100] Train Loss: 0.007059852 | Train Accuracy: 0.9954 | Eval Loss: 0.055950528 | Eval Accuracy: 0.9797
    100%|██████████| 294/294 [00:26<00:00, 10.90it/s]
    [ 38/100] Train Loss: 0.005999635 | Train Accuracy: 0.9962 | Eval Loss: 0.063413396 | Eval Accuracy: 0.9802
    100%|██████████| 294/294 [00:26<00:00, 10.90it/s]
    [ 39/100] Train Loss: 0.003029934 | Train Accuracy: 0.9969 | Eval Loss: 0.064003758 | Eval Accuracy: 0.9803
    100%|██████████| 294/294 [00:26<00:00, 10.90it/s]
    [ 40/100] Train Loss: 0.012572614 | Train Accuracy: 0.9943 | Eval Loss: 0.061227709 | Eval Accuracy: 0.9779
    100%|██████████| 294/294 [00:27<00:00, 10.89it/s]
    [ 41/100] Train Loss: 0.011098052 | Train Accuracy: 0.9946 | Eval Loss: 0.067416840 | Eval Accuracy: 0.9778
    100%|██████████| 294/294 [00:26<00:00, 10.89it/s]
    [ 42/100] Train Loss: 0.010188664 | Train Accuracy: 0.9946 | Eval Loss: 0.051657797 | Eval Accuracy: 0.9798
    100%|██████████| 294/294 [00:26<00:00, 10.91it/s]
    [ 43/100] Train Loss: 0.001425890 | Train Accuracy: 0.9974 | Eval Loss: 0.055644791 | Eval Accuracy: 0.9799
    100%|██████████| 294/294 [00:26<00:00, 10.90it/s]
    [ 44/100] Train Loss: 0.002125683 | Train Accuracy: 0.9972 | Eval Loss: 0.059153810 | Eval Accuracy: 0.9788
    100%|██████████| 294/294 [00:26<00:00, 10.90it/s]
    [ 45/100] Train Loss: 0.003516213 | Train Accuracy: 0.9967 | Eval Loss: 0.080817615 | Eval Accuracy: 0.9761
    100%|██████████| 294/294 [00:26<00:00, 10.91it/s]
    [ 46/100] Train Loss: 0.011380621 | Train Accuracy: 0.9942 | Eval Loss: 0.085672604 | Eval Accuracy: 0.9719
    100%|██████████| 294/294 [00:26<00:00, 10.89it/s]
    [ 47/100] Train Loss: 0.014756009 | Train Accuracy: 0.9935 | Eval Loss: 0.082796592 | Eval Accuracy: 0.9737
    100%|██████████| 294/294 [00:26<00:00, 10.89it/s]
    [ 48/100] Train Loss: 0.008038490 | Train Accuracy: 0.9953 | Eval Loss: 0.050740134 | Eval Accuracy: 0.9792
    100%|██████████| 294/294 [00:26<00:00, 10.90it/s]
    [ 49/100] Train Loss: 0.006135156 | Train Accuracy: 0.9958 | Eval Loss: 0.059173986 | Eval Accuracy: 0.9803
    100%|██████████| 294/294 [00:27<00:00, 10.89it/s]
    [ 50/100] Train Loss: 0.002710547 | Train Accuracy: 0.9969 | Eval Loss: 0.054478022 | Eval Accuracy: 0.9809
    100%|██████████| 294/294 [00:26<00:00, 10.89it/s]
    [ 51/100] Train Loss: 0.006189135 | Train Accuracy: 0.9959 | Eval Loss: 0.059071087 | Eval Accuracy: 0.9796
    100%|██████████| 294/294 [00:26<00:00, 10.90it/s]
    [ 52/100] Train Loss: 0.004576677 | Train Accuracy: 0.9964 | Eval Loss: 0.063089357 | Eval Accuracy: 0.9779
    100%|██████████| 294/294 [00:27<00:00, 10.89it/s]
    [ 53/100] Train Loss: 0.005469601 | Train Accuracy: 0.9960 | Eval Loss: 0.054615553 | Eval Accuracy: 0.9789
    100%|██████████| 294/294 [00:26<00:00, 10.90it/s]
    [ 54/100] Train Loss: 0.007072183 | Train Accuracy: 0.9956 | Eval Loss: 0.045009875 | Eval Accuracy: 0.9805
    100%|██████████| 294/294 [00:26<00:00, 10.90it/s]
    [ 55/100] Train Loss: 0.006000444 | Train Accuracy: 0.9959 | Eval Loss: 0.052580673 | Eval Accuracy: 0.9803
    100%|██████████| 294/294 [00:26<00:00, 10.89it/s]
    [ 56/100] Train Loss: 0.004500941 | Train Accuracy: 0.9966 | Eval Loss: 0.056490892 | Eval Accuracy: 0.9810
    100%|██████████| 294/294 [00:26<00:00, 10.89it/s]
    [ 57/100] Train Loss: 0.003007725 | Train Accuracy: 0.9968 | Eval Loss: 0.062376558 | Eval Accuracy: 0.9789
    100%|██████████| 294/294 [00:27<00:00, 10.88it/s]
    [ 58/100] Train Loss: 0.009814496 | Train Accuracy: 0.9950 | Eval Loss: 0.077214261 | Eval Accuracy: 0.9757
    100%|██████████| 294/294 [00:26<00:00, 10.90it/s]
    [ 59/100] Train Loss: 0.012665438 | Train Accuracy: 0.9941 | Eval Loss: 0.051051934 | Eval Accuracy: 0.9821
    100%|██████████| 294/294 [00:26<00:00, 10.89it/s]
    [ 60/100] Train Loss: 0.005280341 | Train Accuracy: 0.9963 | Eval Loss: 0.056703273 | Eval Accuracy: 0.9798
    100%|██████████| 294/294 [00:27<00:00, 10.84it/s]
    [ 61/100] Train Loss: 0.004021642 | Train Accuracy: 0.9969 | Eval Loss: 0.060731210 | Eval Accuracy: 0.9792
    100%|██████████| 294/294 [00:27<00:00, 10.51it/s]
    [ 62/100] Train Loss: 0.003342969 | Train Accuracy: 0.9967 | Eval Loss: 0.064883126 | Eval Accuracy: 0.9800
    100%|██████████| 294/294 [00:27<00:00, 10.65it/s]
    [ 63/100] Train Loss: 0.004120748 | Train Accuracy: 0.9967 | Eval Loss: 0.087833898 | Eval Accuracy: 0.9765
    100%|██████████| 294/294 [00:27<00:00, 10.83it/s]
    [ 64/100] Train Loss: 0.005887351 | Train Accuracy: 0.9961 | Eval Loss: 0.055562493 | Eval Accuracy: 0.9817
    100%|██████████| 294/294 [00:27<00:00, 10.82it/s]
    [ 65/100] Train Loss: 0.003840348 | Train Accuracy: 0.9968 | Eval Loss: 0.072857055 | Eval Accuracy: 0.9765
    100%|██████████| 294/294 [00:27<00:00, 10.78it/s]
    [ 66/100] Train Loss: 0.009082971 | Train Accuracy: 0.9951 | Eval Loss: 0.052342930 | Eval Accuracy: 0.9804
    100%|██████████| 294/294 [00:27<00:00, 10.63it/s]
    [ 67/100] Train Loss: 0.001961522 | Train Accuracy: 0.9973 | Eval Loss: 0.048264342 | Eval Accuracy: 0.9813
    100%|██████████| 294/294 [00:27<00:00, 10.73it/s]
    [ 68/100] Train Loss: 0.004715381 | Train Accuracy: 0.9965 | Eval Loss: 0.082956100 | Eval Accuracy: 0.9770
    100%|██████████| 294/294 [00:28<00:00, 10.47it/s]
    [ 69/100] Train Loss: 0.009326266 | Train Accuracy: 0.9950 | Eval Loss: 0.053404679 | Eval Accuracy: 0.9804
    100%|██████████| 294/294 [00:27<00:00, 10.54it/s]
    [ 70/100] Train Loss: 0.003306112 | Train Accuracy: 0.9966 | Eval Loss: 0.057575880 | Eval Accuracy: 0.9802
    100%|██████████| 294/294 [00:27<00:00, 10.77it/s]
    [ 71/100] Train Loss: 0.006009721 | Train Accuracy: 0.9959 | Eval Loss: 0.047661654 | Eval Accuracy: 0.9817
    100%|██████████| 294/294 [00:27<00:00, 10.82it/s]
    [ 72/100] Train Loss: 0.002376151 | Train Accuracy: 0.9969 | Eval Loss: 0.042407131 | Eval Accuracy: 0.9828
    100%|██████████| 294/294 [00:27<00:00, 10.82it/s]
    [ 73/100] Train Loss: 0.006238399 | Train Accuracy: 0.9959 | Eval Loss: 0.053447751 | Eval Accuracy: 0.9789
    100%|██████████| 294/294 [00:27<00:00, 10.74it/s]
    [ 74/100] Train Loss: 0.002654893 | Train Accuracy: 0.9970 | Eval Loss: 0.060148940 | Eval Accuracy: 0.9797
    100%|██████████| 294/294 [00:27<00:00, 10.82it/s]
    [ 75/100] Train Loss: 0.003384080 | Train Accuracy: 0.9968 | Eval Loss: 0.050685377 | Eval Accuracy: 0.9812
    100%|██████████| 294/294 [00:27<00:00, 10.80it/s]
    [ 76/100] Train Loss: 0.005460243 | Train Accuracy: 0.9960 | Eval Loss: 0.064673200 | Eval Accuracy: 0.9785
    100%|██████████| 294/294 [00:27<00:00, 10.79it/s]
    [ 77/100] Train Loss: 0.004674682 | Train Accuracy: 0.9965 | Eval Loss: 0.048508364 | Eval Accuracy: 0.9811
    100%|██████████| 294/294 [00:27<00:00, 10.74it/s]
    [ 78/100] Train Loss: 0.006081517 | Train Accuracy: 0.9959 | Eval Loss: 0.061809725 | Eval Accuracy: 0.9793
    100%|██████████| 294/294 [00:27<00:00, 10.85it/s]
    [ 79/100] Train Loss: 0.003128574 | Train Accuracy: 0.9969 | Eval Loss: 0.050054868 | Eval Accuracy: 0.9814
    100%|██████████| 294/294 [00:27<00:00, 10.87it/s]
    [ 80/100] Train Loss: 0.002726834 | Train Accuracy: 0.9968 | Eval Loss: 0.067210749 | Eval Accuracy: 0.9798
    100%|██████████| 294/294 [00:27<00:00, 10.87it/s]
    [ 81/100] Train Loss: 0.003748428 | Train Accuracy: 0.9969 | Eval Loss: 0.046661229 | Eval Accuracy: 0.9826
    100%|██████████| 294/294 [00:27<00:00, 10.87it/s]
    [ 82/100] Train Loss: 0.001142720 | Train Accuracy: 0.9975 | Eval Loss: 0.049936761 | Eval Accuracy: 0.9825
    100%|██████████| 294/294 [00:27<00:00, 10.88it/s]
    [ 83/100] Train Loss: 0.003226624 | Train Accuracy: 0.9969 | Eval Loss: 0.074197290 | Eval Accuracy: 0.9760
    100%|██████████| 294/294 [00:27<00:00, 10.87it/s]
    [ 84/100] Train Loss: 0.013108548 | Train Accuracy: 0.9942 | Eval Loss: 0.044454515 | Eval Accuracy: 0.9811
    100%|██████████| 294/294 [00:27<00:00, 10.88it/s]
    [ 85/100] Train Loss: 0.002160195 | Train Accuracy: 0.9971 | Eval Loss: 0.048371719 | Eval Accuracy: 0.9807
    100%|██████████| 294/294 [00:27<00:00, 10.87it/s]
    [ 86/100] Train Loss: 0.002619434 | Train Accuracy: 0.9971 | Eval Loss: 0.050486882 | Eval Accuracy: 0.9813
    100%|██████████| 294/294 [00:27<00:00, 10.85it/s]
    [ 87/100] Train Loss: 0.006864140 | Train Accuracy: 0.9961 | Eval Loss: 0.056692976 | Eval Accuracy: 0.9792
    100%|██████████| 294/294 [00:27<00:00, 10.87it/s]
    [ 88/100] Train Loss: 0.004875265 | Train Accuracy: 0.9963 | Eval Loss: 0.041186074 | Eval Accuracy: 0.9832
    100%|██████████| 294/294 [00:27<00:00, 10.88it/s]
    [ 89/100] Train Loss: 0.000652768 | Train Accuracy: 0.9976 | Eval Loss: 0.047177161 | Eval Accuracy: 0.9815
    100%|██████████| 294/294 [00:27<00:00, 10.87it/s]
    [ 90/100] Train Loss: 0.000488987 | Train Accuracy: 0.9977 | Eval Loss: 0.045264324 | Eval Accuracy: 0.9824
    100%|██████████| 294/294 [00:27<00:00, 10.89it/s]
    [ 91/100] Train Loss: 0.000341539 | Train Accuracy: 0.9977 | Eval Loss: 0.034529524 | Eval Accuracy: 0.9848
    100%|██████████| 294/294 [00:27<00:00, 10.88it/s]
    [ 92/100] Train Loss: 0.002224435 | Train Accuracy: 0.9971 | Eval Loss: 0.057096185 | Eval Accuracy: 0.9810
    100%|██████████| 294/294 [00:27<00:00, 10.89it/s]
    [ 93/100] Train Loss: 0.009456459 | Train Accuracy: 0.9949 | Eval Loss: 0.057099932 | Eval Accuracy: 0.9805
    100%|██████████| 294/294 [00:27<00:00, 10.88it/s]
    [ 94/100] Train Loss: 0.004690705 | Train Accuracy: 0.9963 | Eval Loss: 0.054541877 | Eval Accuracy: 0.9821
    100%|██████████| 294/294 [00:26<00:00, 10.89it/s]
    [ 95/100] Train Loss: 0.003725891 | Train Accuracy: 0.9968 | Eval Loss: 0.056962815 | Eval Accuracy: 0.9809
    100%|██████████| 294/294 [00:27<00:00, 10.89it/s]
    [ 96/100] Train Loss: 0.003008096 | Train Accuracy: 0.9970 | Eval Loss: 0.046588485 | Eval Accuracy: 0.9816
    100%|██████████| 294/294 [00:27<00:00, 10.89it/s]
    [ 97/100] Train Loss: 0.004697346 | Train Accuracy: 0.9962 | Eval Loss: 0.058122337 | Eval Accuracy: 0.9794
    100%|██████████| 294/294 [00:26<00:00, 10.89it/s]
    [ 98/100] Train Loss: 0.002204622 | Train Accuracy: 0.9971 | Eval Loss: 0.043696313 | Eval Accuracy: 0.9834
    100%|██████████| 294/294 [00:26<00:00, 10.89it/s]
    [ 99/100] Train Loss: 0.001267484 | Train Accuracy: 0.9974 | Eval Loss: 0.045243341 | Eval Accuracy: 0.9830
    100%|██████████| 294/294 [00:26<00:00, 10.89it/s]
    [100/100] Train Loss: 0.011427559 | Train Accuracy: 0.9942 | Eval Loss: 0.051266292 | Eval Accuracy: 0.9795
    Training completed.

利用ResNet18预训练模型进行训练，模型的验证集精度很快就达到了0.97以上。在第91轮训练中，验证集精度达到了最高值0.9848，相较于前面的CNN模型提高了0.0061，从结果来看略优于CNN，从训练的学习速度来看要比自定义CNN进行训练快很多。

[Skin Cancer MNIST_ HAM10000]: https://www.kaggle.com/kmader/skin-cancer-mnist-ham10000
[https_challenge2018.isic-archive.com]: https://challenge2018.isic-archive.com
[https_dataverse.harvard.edu_dataset.xhtml_persistentId_doi_10.7910_DVN_DBW86T]: https://dataverse.harvard.edu/dataset.xhtml?persistentId=doi:10.7910/DVN/DBW86T
[https_arxiv.org_abs_1902.03368]: https://arxiv.org/abs/1902.03368
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzQ1NTU0MDEw_size_16_color_FFFFFF_t_70]: /images/20221020/255d5fad0a4d4793b0b9416b4a41ac83.png