深度学习-Pytorch：构建DNN神经网络模型【构建方式：自定义函数及参数、nn.Module()、nn.Module()&nn.Sequential()】、Visdom可视化

Love The Way You Lie 2023-10-05 17:52 4阅读 0赞

## 一、nn.Module()的作用 ##

### 1、nn.Module()包含现成的Layer可供调用： ###

*  nn.Linear
 *  nn.Conv2d
 *  nn.ReLU
 *  nn.Sigmoid
 *  nn.ConvTransposed2d
 *  nn.Dropout  
    …

### 2、nn.Module()作为一个Container ###

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70_pic_center]

### 3、parameters ###

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70_pic_center 1]

### 4、modules ###

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70_pic_center 2]

### 5、to(device) ###

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70_pic_center 3]

### 6、save/load模型 ###

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70_pic_center 4]

### 7、train/test模式切换 ###

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70_pic_center 5]

### 8、implement own layer ###

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70_pic_center 6]  
![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70_pic_center 7]

## 二、DNN神经网络模型的创建模式 ##

### 1、手工创建参数、函数–>梯度优化(手写数字识别) ###

import torch
    import torch.nn as nn
    import torch.nn.functional as F
    import torch.optim as optim
    from torchvision import datasets, transforms
    
    batch_size = 20000
    learning_rate = 0.01
    epochs = 3
    
    # 一、获取手写数字辨识训练数据集
    db_train = datasets.MNIST('../data', train=True, download=True, transform=transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))]))
    db_test = datasets.MNIST('../data', train=False, transform=transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))]))
    # 1.1 拆分训练集为训练集、验证集
    db_train, db_val = torch.utils.data.random_split(db_train, [50000, 10000])
    # 1.2 将训练集、验证集转为batch集合
    train_loader = torch.utils.data.DataLoader(db_train, batch_size=batch_size, shuffle=True)
    val_loader = torch.utils.data.DataLoader(db_val, batch_size=batch_size, shuffle=True)
    
    # 二、创建权重参数[W1,W2,W3,B1,B2,B3]
    W1 = torch.randn(200, 784, requires_grad=True)
    B1 = torch.zeros(200, requires_grad=True)
    W2 = torch.randn(200, 200, requires_grad=True)
    B2 = torch.zeros(200, requires_grad=True)
    W3 = torch.randn(10, 200, requires_grad=True)
    B3 = torch.zeros(10, requires_grad=True)
    
    # 三、使用kaiming法初始化W1、W2、W3
    torch.nn.init.kaiming_normal_(W1)
    torch.nn.init.kaiming_normal_(W2)
    torch.nn.init.kaiming_normal_(W3)
    
    
    # 四、构建全连接神经网络各层
    def forward(X):
        # 第01层Layer
        X = X @ W1.t() + B1
        X = F.relu(X)
        # 第02层Layer
        X = X @ W2.t() + B2
        X = F.relu(X)
        # 第03层Layer
        X = X @ W3.t() + B3
        X = F.relu(X)
        return X
    
    
    # 五、梯度下降优化器设置
    optimizer = optim.SGD([W1, B1, W2, B2, W3, B3], lr=learning_rate)
    
    # 六、实例化损失函数
    criteon = nn.CrossEntropyLoss()
    
    # 七、训练
    for epoch in range(epochs):
        # *****************************训练*****************************
        for batch_index, (data, target) in enumerate(train_loader):
            data = data.view(-1, 28 * 28)
            logits = forward(data)
            loss = criteon(logits, target)
            optimizer.zero_grad()
            loss.backward()
            # print(W1.grad.norm(), W2.grad.norm())
            optimizer.step()
            print('Training Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(epoch, batch_index * len(data), len(train_loader.dataset), 100. * batch_index / len(train_loader), loss.item()))
        val_loss = 0
        correct = 0
        # *****************************模型评估*****************************
        for data, target in val_loader:
            data = data.view(-1, 28 * 28)
            logits = forward(data)
            val_loss += criteon(logits, target).item()
            pred = logits.data.max(1)[1]
            correct += pred.eq(target.data).sum()
        val_loss /= len(val_loader.dataset)
        print('\nValidation set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(val_loss, correct, len(val_loader.dataset), 100. * correct / len(val_loader.dataset)))

打印结果：

Training Epoch: 0 [0/50000 (0%)]	Loss: 2.676057
    Training Epoch: 0 [20000/50000 (33%)]	Loss: 2.536507
    Training Epoch: 0 [20000/50000 (67%)]	Loss: 2.472484
    
    Validation set: Average loss: 0.0002, Accuracy: 1355/10000 (14%)
    
    Training Epoch: 1 [0/50000 (0%)]	Loss: 2.411102
    Training Epoch: 1 [20000/50000 (33%)]	Loss: 2.368159
    Training Epoch: 1 [20000/50000 (67%)]	Loss: 2.332945
    
    Validation set: Average loss: 0.0002, Accuracy: 1793/10000 (18%)
    
    Training Epoch: 2 [0/50000 (0%)]	Loss: 2.301854
    Training Epoch: 2 [20000/50000 (33%)]	Loss: 2.280679
    Training Epoch: 2 [20000/50000 (67%)]	Loss: 2.242718
    
    Validation set: Average loss: 0.0002, Accuracy: 2183/10000 (22%)
    
    
    Process finished with exit code 0

### 2、nn.Module()创建神经网络结构–>梯度优化(手写数字识别) ###

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70_pic_center 8]

import torch
    from torch import nn
    from torch.nn import functional as F
    from torch import optim
    import torchvision
    from matplotlib import pyplot as plt
    
    batch_size = 500
    
    
    # 画曲线工具
    def plot_curve(data):
        fig = plt.figure()
        plt.plot(range(len(data)), data, color='blue')
        plt.legend(['value'], loc='upper right')
        plt.xlabel('step')
        plt.ylabel('value')
        plt.show()
    
    
    # 画图像工具
    def plot_image(img, label, name):
        fig = plt.figure()
        for i in range(6):
            plt.subplot(2, 3, i + 1)
            plt.tight_layout()
            plt.imshow(img[i][0] * 0.3081 + 0.1307, cmap='gray', interpolation='none')
            plt.title("{}: {}".format(name, label[i].item()))
            plt.xticks([])
            plt.yticks([])
        plt.show()
    
    
    # ont-hot编码
    def one_hot(label, depth=10):
        out = torch.zeros(label.size(0), depth)
        idx = torch.LongTensor(label).view(-1, 1)
        out.scatter_(dim=1, index=idx, value=1)
        return out
    
    
    # 一、获取手写数字辨识训练数据集
    db_train = datasets.MNIST('../data', train=True, download=True, transform=transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))]))
    db_test = datasets.MNIST('../data', train=False, transform=transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))]))
    # 1.1 拆分训练集为训练集、验证集
    db_train, db_val = torch.utils.data.random_split(db_train, [50000, 10000])
    # 1.2 将训练集、验证集转为batch集合
    train_loader = torch.utils.data.DataLoader(db_train, batch_size=batch_size, shuffle=True)
    val_loader = torch.utils.data.DataLoader(db_val, batch_size=batch_size, shuffle=True)
    print('train_loader = {0}, val_loader ={1}'.format(train_loader, val_loader))
    X_sample, Y_sample = next(iter(train_loader))
    print('X_sample.shape = {0}, Y_sample.shape ={1}, X_sample.min() = {2}, X_sample.max() = {3}'.format(X_sample.shape, Y_sample.shape, X_sample.min(), X_sample.max()))
    plot_image(X_sample, Y_sample, 'image sample')
    
    
    # 二、构建神经网络结构：Linear 表示全连接神经网络，激活函数用 relu
    class MLP(nn.Module):
        def __init__(self):
            super(MLP, self).__init__()
            self.fullcon1 = nn.Linear(28 * 28, 256)  # 全连接层（输入维度为28*28，输出维度为256）
            self.fullcon2 = nn.Linear(256, 64)  # 全连接层（输入维度为256，输出维度为64）
            self.fullcon3 = nn.Linear(64, 10)  # 全连接层（输入维度为64，输出维度为10）
    
        def forward(self, X):
            # h1 = relu(xw1+b1)
            h1 = F.relu(self.fullcon1(X))  # [b, 1, 28*28]=>[b, 1, 256]
            # h2 = relu(h1w2+b2)
            h2 = F.relu(self.fullcon2(h1))  # [b, 1, 256]=>[b, 1, 64]
            # output = h2w3+b3
            output = self.fullcon3(h2)
            # output = F.softmax(self.fullcon3(h2))  # [b, 1, 64]=>[b, 1, 10]
            return output
    
    
    # 三、实例化神经网络结构
    mlp= MLP()
    optimizer = optim.SGD(mlp.parameters(), lr=0.01, momentum=0.9)  # mlp.parameters(): [w1, b1, w2, b2, w3, b3]
    
    
    # 四、整体数据集进行一次梯度下降来更新模型参数，整体数据集迭代一次，一般用epoch。每个epoch中含有batch_step_no个step，每个step中就是设置的每个batch所含有的样本数量。
    def train_epoch(epoch_no):
        print('++++++++++++++++++++++++++++++++++++++++++++第{0}轮Epoch-->Training 阶段：开始++++++++++++++++++++++++++++++++++++++++++++'.format(epoch_no))
        for batch_step_index, (X_batch, Y_batch) in enumerate(train_loader):
            # x: [b, 1, 28, 28], y: [512]
            # [b, 1, 28, 28] => [b, 784]
            X_batch = X_batch.view(X_batch.size(0), 28 * 28)
            # => [b, 10]
            out_logits = mlp(X_batch)
            # [b, 10]
            Y_batch_onehot = one_hot(Y_batch)
            # loss = mse(out, y_onehot)
            MSE_Loss = F.mse_loss(out_logits, Y_batch_onehot)
            optimizer.zero_grad()  # 清零梯度值
            MSE_Loss.backward()  # 计算梯度值
            optimizer.step()  # 根据梯度值更新参数：w' = w - lr*grad
            train_loss.append(MSE_Loss.item())
            if batch_step_index % 50 == 0:
                print('epoch_no = {0}, batch_step_index = {1}, MSE_Loss.item() = {2}'.format(epoch_no, batch_step_index, MSE_Loss))
        print('++++++++++++++++++++++++++++++++++++++++++++第{0}轮Epoch-->Training 阶段：结束++++++++++++++++++++++++++++++++++++++++++++'.format(epoch_no))
    
    
    # 五、模型评估 evluation
    def evluation(epoch_no):
        print('++++++++++++++++++++++++++++++++++++++++++++第{0}轮Epoch-->Evluation 阶段：开始++++++++++++++++++++++++++++++++++++++++++++'.format(epoch_no))
        total_correct = 0
        for X_batch, Y_batch in val_loader:
            X_batch = X_batch.view(X_batch.size(0), 28 * 28)
            out_logit = mlp(X_batch)
            # out: [b, 10] => pred: [b]
            out_pred = out_logit.argmax(dim=1)  # 查找最大值所在的索引位置
            correct = out_pred.eq(Y_batch).sum().float().item()
            total_correct += correct
        total_num = len(val_loader.dataset)
        acc = total_correct / total_num
        print('epoch_no = {0}, 模型评估：准确度 = {1}, '.format(epoch_no, acc))
        print('++++++++++++++++++++++++++++++++++++++++++++第{0}轮Epoch-->Evluation 阶段：结束++++++++++++++++++++++++++++++++++++++++++++'.format(epoch_no))
    
    
    # 六、整体数据迭代多次梯度下降来更新模型参数
    def train():
        epoch_count = 3  # epoch_count为整体数据集迭代梯度下降次数
        for epoch_no in range(1, epoch_count + 1):
            print('\n\n利用整体数据集进行模型的第{0}轮Epoch迭代开始:**********************************************************************************************************************************'.format(epoch_no))
            train_epoch(epoch_no)
            evluation(epoch_no)
            print('利用整体数据集进行模型的第{0}轮Epoch迭代结束:**********************************************************************************************************************************'.format(epoch_no))
        print('train_loss = {0}'.format(train_loss))
        print('\n***********************************************打印 train_loss 曲线***********************************************')
        plot_curve(train_loss)
    
    
    # 七、测试
    def test():
        print('\n***********************************************测试***********************************************')
        X_batch, _ = next(iter(val_loader))
        out_logit = mlp(X_batch.view(X_batch.size(0), 28 * 28))
        out_pred = out_logit.argmax(dim=1)
        plot_image(X_batch, out_pred, 'Test out_pred ')
    
    
    if __name__ == '__main__':
        train_loss = []
        train()  # we get optimal [w1, b1, w2, b2, w3, b3]
        test()

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70 1]

打印结果：

train_loader = <torch.utils.data.dataloader.DataLoader object at 0x0000013B7AE49908>, val_loader =<torch.utils.data.dataloader.DataLoader object at 0x0000013B7C9D1DD8>
    X_sample.shape = torch.Size([500, 1, 28, 28]), Y_sample.shape =torch.Size([500]), X_sample.min() = -0.4242129623889923, X_sample.max() = 2.821486711502075
    
    
    利用整体数据集进行模型的第1轮Epoch迭代开始:**********************************************************************************************************************************
    ++++++++++++++++++++++++++++++++++++++++++++第1轮Epoch-->Training 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
    epoch_no = 1, batch_step_index = 0, MSE_Loss.item() = 0.1291128247976303
    epoch_no = 1, batch_step_index = 50, MSE_Loss.item() = 0.0706515833735466
    ++++++++++++++++++++++++++++++++++++++++++++第1轮Epoch-->Training 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
    ++++++++++++++++++++++++++++++++++++++++++++第1轮Epoch-->Evluation 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
    epoch_no = 1, 模型评估：准确度 = 0.714, 
    ++++++++++++++++++++++++++++++++++++++++++++第1轮Epoch-->Evluation 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
    利用整体数据集进行模型的第1轮Epoch迭代结束:**********************************************************************************************************************************
    
    
    利用整体数据集进行模型的第2轮Epoch迭代开始:**********************************************************************************************************************************
    ++++++++++++++++++++++++++++++++++++++++++++第2轮Epoch-->Training 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
    epoch_no = 2, batch_step_index = 0, MSE_Loss.item() = 0.05567050352692604
    epoch_no = 2, batch_step_index = 50, MSE_Loss.item() = 0.046730369329452515
    ++++++++++++++++++++++++++++++++++++++++++++第2轮Epoch-->Training 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
    ++++++++++++++++++++++++++++++++++++++++++++第2轮Epoch-->Evluation 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
    epoch_no = 2, 模型评估：准确度 = 0.8222, 
    ++++++++++++++++++++++++++++++++++++++++++++第2轮Epoch-->Evluation 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
    利用整体数据集进行模型的第2轮Epoch迭代结束:**********************************************************************************************************************************
    
    
    利用整体数据集进行模型的第3轮Epoch迭代开始:**********************************************************************************************************************************
    ++++++++++++++++++++++++++++++++++++++++++++第3轮Epoch-->Training 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
    epoch_no = 3, batch_step_index = 0, MSE_Loss.item() = 0.04250795394182205
    epoch_no = 3, batch_step_index = 50, MSE_Loss.item() = 0.04056573286652565
    ++++++++++++++++++++++++++++++++++++++++++++第3轮Epoch-->Training 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
    ++++++++++++++++++++++++++++++++++++++++++++第3轮Epoch-->Evluation 阶段：开始++++++++++++++++++++++++++++++++++++++++++++
    epoch_no = 3, 模型评估：准确度 = 0.8614, 
    ++++++++++++++++++++++++++++++++++++++++++++第3轮Epoch-->Evluation 阶段：结束++++++++++++++++++++++++++++++++++++++++++++
    利用整体数据集进行模型的第3轮Epoch迭代结束:**********************************************************************************************************************************
    train_loss = [0.1291128247976303, 0.13044174015522003, 0.1272156536579132, 0.12160427123308182, 0.1211298480629921, 0.11663255840539932, 0.11270908266305923, 0.10814651846885681, 0.10669085383415222, 0.10295026004314423, 0.10101114213466644, 0.09815928339958191, 0.09584306925535202, 0.09468168765306473, 0.09207087755203247, 0.09191320091485977, 0.09076083451509476, 0.08995824307203293, 0.08734000474214554, 0.08807621151208878, 0.08676423877477646, 0.08637084811925888, 0.08454613387584686, 0.08528003096580505, 0.08443577587604523, 0.08353377133607864, 0.08330731838941574, 0.08190210163593292, 0.08293816447257996, 0.08074264228343964, 0.08093489706516266, 0.08032829314470291, 0.0803324282169342, 0.07852856069803238, 0.0785190537571907, 0.0769621804356575, 0.07708759605884552, 0.0773153081536293, 0.0756053626537323, 0.07448747754096985, 0.075688935816288, 0.07464658468961716, 0.0732233002781868, 0.07418092340230942, 0.07457191497087479, 0.07434888184070587, 0.07403060048818588, 0.0729573592543602, 0.06997241079807281, 0.07219536602497101, 0.0706515833735466, 0.07082699984312057, 0.070269376039505, 0.06963413208723068, 0.07065454125404358, 0.06963349133729935, 0.07031571865081787, 0.06744655221700668, 0.06730636954307556, 0.06825980544090271, 0.06762295216321945, 0.06645598262548447, 0.06583689153194427, 0.0669311061501503, 0.06721851229667664, 0.06420829892158508, 0.06466630846261978, 0.06396583467721939, 0.06614362448453903, 0.06366819143295288, 0.06328243762254715, 0.06476756185293198, 0.06399714946746826, 0.06503674387931824, 0.062194567173719406, 0.0633464977145195, 0.06282727420330048, 0.06075088679790497, 0.06093635782599449, 0.06095554307103157, 0.061674751341342926, 0.06101302057504654, 0.05921223759651184, 0.05976448953151703, 0.05893367901444435, 0.05907851457595825, 0.058452509343624115, 0.05963074415922165, 0.05974317714571953, 0.05802195891737938, 0.06064910814166069, 0.057840559631586075, 0.05612918734550476, 0.05583397299051285, 0.05664679408073425, 0.05561935529112816, 0.056890808045864105, 0.057227905839681625, 0.05527901649475098, 0.057134959846735, 0.05567050352692604, 0.056415606290102005, 0.05691703408956528, 0.05264866352081299, 0.055786360055208206, 0.05290836840867996, 0.05422500520944595, 0.05349273234605789, 0.053789496421813965, 0.05485423654317856, 0.051740508526563644, 0.05348716303706169, 0.0548003651201725, 0.05272025242447853, 0.054838329553604126, 0.05209319293498993, 0.05160987377166748, 0.05209755897521973, 0.05201425775885582, 0.050856031477451324, 0.05327674001455307, 0.05023441091179848, 0.05085492506623268, 0.048322878777980804, 0.050107188522815704, 0.0502738319337368, 0.05057230591773987, 0.05106339603662491, 0.04920419305562973, 0.04879516735672951, 0.04774750396609306, 0.049592312425374985, 0.04933563992381096, 0.04949263483285904, 0.04793641343712807, 0.04780952259898186, 0.04809335991740227, 0.04800635948777199, 0.04842348396778107, 0.04933779314160347, 0.05077265575528145, 0.047386206686496735, 0.047962553799152374, 0.04873092472553253, 0.05061652511358261, 0.04552381485700607, 0.04688310623168945, 0.046897538006305695, 0.04753261059522629, 0.046342331916093826, 0.046730369329452515, 0.04627261683344841, 0.04688314348459244, 0.04574425891041756, 0.044676411896944046, 0.04721922427415848, 0.04717138782143593, 0.04380550608038902, 0.04529754817485809, 0.04511982202529907, 0.04532311111688614, 0.04658367857336998, 0.045253925025463104, 0.04433036968111992, 0.043740443885326385, 0.04555383324623108, 0.044463757425546646, 0.04347236454486847, 0.045887626707553864, 0.046037618070840836, 0.04274151846766472, 0.042166028171777725, 0.04345878213644028, 0.0429772324860096, 0.043307844549417496, 0.04480277746915817, 0.04343221336603165, 0.0434730090200901, 0.04290395975112915, 0.043878987431526184, 0.045538414269685745, 0.043192896991968155, 0.04353556036949158, 0.04181528836488724, 0.041755884885787964, 0.04251985624432564, 0.044771455228328705, 0.04295073822140694, 0.04253952205181122, 0.04324943199753761, 0.042303454130887985, 0.0421823225915432, 0.04329197481274605, 0.04329170659184456, 0.04388098791241646, 0.040907829999923706, 0.04287262633442879, 0.04263708367943764, 0.04127754271030426, 0.04103789106011391, 0.04250795394182205, 0.04061960428953171, 0.04150346666574478, 0.04289315640926361, 0.04266168922185898, 0.04082883521914482, 0.04131302610039711, 0.04201776161789894, 0.04192619398236275, 0.040360044687986374, 0.040700457990169525, 0.042395561933517456, 0.040383704006671906, 0.03860364481806755, 0.03840060904622078, 0.041741594672203064, 0.03972075134515762, 0.041619688272476196, 0.04171542450785637, 0.041514765471220016, 0.039556656032800674, 0.04057379439473152, 0.04328334331512451, 0.038404252380132675, 0.038783613592386246, 0.042881954461336136, 0.03846145421266556, 0.03959750756621361, 0.040078021585941315, 0.036042604595422745, 0.04113992676138878, 0.039557136595249176, 0.03800525516271591, 0.040941301733255386, 0.03929487243294716, 0.03862420469522476, 0.03850855305790901, 0.04139616712927818, 0.03725165128707886, 0.039298247545957565, 0.03992690518498421, 0.041072919964790344, 0.03858896344900131, 0.03939962759613991, 0.03696129098534584, 0.03778659924864769, 0.041128259152173996, 0.03856305778026581, 0.03721937909722328, 0.03888898342847824, 0.04056573286652565, 0.0364970900118351, 0.03817414119839668, 0.03743955120444298, 0.03759564831852913, 0.037728626281023026, 0.037812069058418274, 0.03692056983709335, 0.03642625734210014, 0.03823438286781311, 0.03734381124377251, 0.038524847477674484, 0.03840061277151108, 0.03610086813569069, 0.03859890624880791, 0.03691568225622177, 0.03608223423361778, 0.03722282499074936, 0.03553231433033943, 0.03682379797101021, 0.03806352615356445, 0.035934928804636, 0.03614666685461998, 0.0389103889465332, 0.03705954551696777, 0.035829052329063416, 0.035150352865457535, 0.036663081496953964, 0.03527655079960823, 0.03467680886387825, 0.038327015936374664, 0.03711606189608574, 0.034700050950050354, 0.03661731630563736, 0.03701657056808472, 0.036406390368938446, 0.03650892153382301, 0.034515105187892914, 0.035059090703725815, 0.03613708168268204, 0.03532600402832031, 0.036248888820409775, 0.034991368651390076, 0.036527663469314575, 0.03678029775619507, 0.036512721329927444, 0.03455689176917076, 0.037010710686445236, 0.03427138179540634, 0.03327715024352074]
    
    ***********************************************打印 train_loss 曲线***********************************************
    
    ***********************************************测试***********************************************

### 3、nn.Module()&nn.Sequential()创建神经网络结构–>梯度优化(手写数字识别) ###

nn.Module()包含现成的Layer可供调用：

*  nn.Linear
 *  nn.Conv2d
 *  nn.ReLU
 *  nn.Sigmoid
 *  nn.ConvTransposed2d
 *  nn.Dropout  
    …

nn.Sequential()可将不同的Layer(包括自定义的Layer)按照顺序串联起来

import torch
    import torch.nn as nn
    import torch.optim as optim
    from torchvision import datasets, transforms
    
    batch_size = 20000
    learning_rate = 0.01
    epochs = 3
    
    # 一、获取手写数字辨识训练数据集
    db_train = datasets.MNIST('../data', train=True, download=True, transform=transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))]))
    db_test = datasets.MNIST('../data', train=False, transform=transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))]))
    # 1.1 拆分训练集为训练集、验证集
    db_train, db_val = torch.utils.data.random_split(db_train, [50000, 10000])
    # 1.2 将训练集、验证集转为batch集合
    train_loader = torch.utils.data.DataLoader(db_train, batch_size=batch_size, shuffle=True)
    val_loader = torch.utils.data.DataLoader(db_val, batch_size=batch_size, shuffle=True)
    
    # 二、构建神经网络结构：Linear 表示全连接神经网络，激活函数用 relu
    class MLP(nn.Module):
        def __init__(self):
            super(MLP, self).__init__()
            self.model = nn.Sequential(
                nn.Linear(784, 200),
                nn.ReLU(inplace=True),	 # inplace=True 使用原数据的内存地址，节省内存使用。
                nn.Linear(200, 200),
                nn.ReLU(inplace=True),	 # inplace=True 使用原数据的内存地址，节省内存使用。
                nn.Linear(200, 10),
                nn.ReLU(inplace=True),	 # inplace=True 使用原数据的内存地址，节省内存使用。
            )
    
        def forward(self, X):
            X = self.model(X)
            return X
    
    
    # 三、实例化多层感知机MLP
    mlp = MLP()
    
    # 四、梯度下降优化器设置
    optimizer = optim.SGD(mlp.parameters(), lr=learning_rate)
    
    # 五、实例化损失函数
    criteon = nn.CrossEntropyLoss()
    
    # 六、训练
    for epoch in range(epochs):
        # *****************************训练*****************************
        for batch_index, (data, target) in enumerate(train_loader):
            data = data.view(-1, 28 * 28)
            logits = mlp(data)
            loss = criteon(logits, target)
            optimizer.zero_grad()
            loss.backward()
            # print(w1.grad.norm(), w2.grad.norm())
            optimizer.step()
            print('Train Epoch{}-batch_index{} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(epoch, batch_index, batch_index * len(data), len(train_loader.dataset), 100. * batch_index / len(train_loader), loss.item()))
        # *****************************模型评估*****************************
        val_loss = 0
        correct = 0
        for data, target in val_loader:
            data = data.view(-1, 28 * 28)
            logits = mlp(data)
            val_loss += criteon(logits, target).item()
            pred = logits.data.max(1)[1]
            correct += pred.eq(target.data).sum()
        val_loss /= len(val_loader.dataset)
        print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(val_loss, correct, len(val_loader.dataset), 100. * correct / len(val_loader.dataset)))

打印结果：

Train Epoch0-batch_index0 [0/50000 (0%)]	Loss: 2.296899
    Train Epoch0-batch_index1 [20000/50000 (33%)]	Loss: 2.295931
    Train Epoch0-batch_index2 [20000/50000 (67%)]	Loss: 2.294703
    
    Test set: Average loss: 0.0002, Accuracy: 1423/10000 (14%)
    
    Train Epoch1-batch_index0 [0/50000 (0%)]	Loss: 2.293654
    Train Epoch1-batch_index1 [20000/50000 (33%)]	Loss: 2.292577
    Train Epoch1-batch_index2 [20000/50000 (67%)]	Loss: 2.291245
    
    Test set: Average loss: 0.0002, Accuracy: 1661/10000 (17%)
    
    Train Epoch2-batch_index0 [0/50000 (0%)]	Loss: 2.289515
    Train Epoch2-batch_index1 [20000/50000 (33%)]	Loss: 2.288878
    Train Epoch2-batch_index2 [20000/50000 (67%)]	Loss: 2.287304
    
    Test set: Average loss: 0.0002, Accuracy: 1934/10000 (19%)
    
    
    Process finished with exit code 0

### 4、Visdom可视化 ###

import  torch
    import  torch.nn as nn
    import  torch.nn.functional as F
    import  torch.optim as optim
    from    torchvision import datasets, transforms
    # Visdom离线包：https://github.com/fossasia/visdom
    # 安装下载包：PS C:\Users\Admin\Downloads\visdom-master> pip install -e .
    # 开启Visdom的web服务器：python -m visdom.server
    from visdom import Visdom
    
    
    batch_size=200
    learning_rate=0.01
    epochs=10
    
    # 一、获取手写数字辨识训练数据集
    train_loader = torch.utils.data.DataLoader(
        datasets.MNIST('../data', train=True, download=True,
                       transform=transforms.Compose([
                           transforms.ToTensor(),
                           # transforms.Normalize((0.1307,), (0.3081,))
                       ])),
        batch_size=batch_size, shuffle=True)
    val_loader = torch.utils.data.DataLoader(
        datasets.MNIST('../data', train=False, transform=transforms.Compose([
            transforms.ToTensor(),
            # transforms.Normalize((0.1307,), (0.3081,))
        ])),
        batch_size=batch_size, shuffle=True)
    
    # 二、构建神经网络结构：Linear 表示全连接神经网络，激活函数用 relu
    class MLP(nn.Module):
        def __init__(self):
            super(MLP, self).__init__()
            self.model = nn.Sequential(
                nn.Linear(784, 200),
                nn.LeakyReLU(inplace=True),
                nn.Linear(200, 200),
                nn.LeakyReLU(inplace=True),
                nn.Linear(200, 10),
                nn.LeakyReLU(inplace=True),
            )
        def forward(self, x):
            x = self.model(x)
            return x
    
    # 三、定义GPU
    device = torch.device('cuda:0')
    
    # 四、实例化多层感知机MLP
    mlp = MLP().to(device)
    
    # 五、梯度下降优化器设置
    optimizer = optim.SGD(mlp.parameters(), lr=learning_rate)
    
    # 六、实例化损失函数
    criteon = nn.CrossEntropyLoss().to(device)
    
    # 七、实例化可视化窗口
    viz = Visdom()
    viz.line(Y=[0.], X=[0.], win='train_loss', opts=dict(title='train loss'))   # 初始化小窗口，id为”train_loss“；title为”train loss“
    viz.line(Y=[[0.0, 0.0]], X=[0.], win='test', opts=dict(title='test loss&acc.', legend=['loss', 'acc.']))   # 初始化小窗口，id为”test“；title为”test loss&acc.“；Y=[[0.0, 0.0]]表示Y轴为2条曲线
    global_step = 0
    
    # 八、训练
    for epoch in range(epochs):
        # *****************************训练*****************************
        for batch_index, (data, target) in enumerate(train_loader):
            data = data.view(-1, 28*28)
            data, target = data.to(device), target.cuda()
            logits = mlp(data)
            loss = criteon(logits, target)
            optimizer.zero_grad()
            loss.backward()
            # print(w1.grad.norm(), w2.grad.norm())
            optimizer.step()
            global_step += 1
            viz.line([loss.item()], [global_step], win='train_loss', update='append')   # 更新id为”train_loss“的小窗口，更新方式为：”append“；global_step为X轴；loss.item()为Y轴
            if batch_index % 100 == 0:
                print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(epoch, batch_index * len(data), len(train_loader.dataset), 100. * batch_index / len(train_loader), loss.item()))
        # *****************************模型评估*****************************
        val_loss = 0
        correct = 0
        for data, target in val_loader:
            data = data.view(-1, 28 * 28)
            data, target = data.to(device), target.cuda()
            logits = mlp(data)
            val_loss += criteon(logits, target).item()
            pred = logits.argmax(dim=1)
            correct += pred.eq(target).float().sum().item()
        viz.line([[val_loss, correct / len(val_loader.dataset)]], [global_step], win='test', update='append')     # 更新id为”test“的小窗口，更新方式为：”append“；global_step为X轴；val_loss, correct / len(val_loader.dataset)]分别作Y轴
        viz.images(data.view(-1, 1, 28, 28), win='x')
        viz.text(str(pred.detach().cpu().numpy()), win='pred', opts=dict(title='pred'))
        val_loss /= len(val_loader.dataset)
        print('\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\n'.format(val_loss, correct, len(val_loader.dataset), 100. * correct / len(val_loader.dataset)))

![在这里插入图片描述][watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70 2]  
打印结果：

Setting up a new session...
    Train Epoch: 0 [0/60000 (0%)]	Loss: 2.301227
    Train Epoch: 0 [20000/60000 (33%)]	Loss: 2.280302
    Train Epoch: 0 [40000/60000 (67%)]	Loss: 2.231285
    
    Test set: Average loss: 0.0108, Accuracy: 4614.0/10000 (46%)
    
    Train Epoch: 1 [0/60000 (0%)]	Loss: 2.191999
    Train Epoch: 1 [20000/60000 (33%)]	Loss: 2.057158
    Train Epoch: 1 [40000/60000 (67%)]	Loss: 1.867862
    
    Test set: Average loss: 0.0080, Accuracy: 6438.0/10000 (64%)
    
    Train Epoch: 2 [0/60000 (0%)]	Loss: 1.578062
    Train Epoch: 2 [20000/60000 (33%)]	Loss: 1.327201
    Train Epoch: 2 [40000/60000 (67%)]	Loss: 1.031852
    
    Test set: Average loss: 0.0043, Accuracy: 8017.0/10000 (80%)
    
    Train Epoch: 3 [0/60000 (0%)]	Loss: 0.868925
    Train Epoch: 3 [20000/60000 (33%)]	Loss: 0.845542
    Train Epoch: 3 [40000/60000 (67%)]	Loss: 0.662352
    
    Test set: Average loss: 0.0030, Accuracy: 8435.0/10000 (84%)
    
    Train Epoch: 4 [0/60000 (0%)]	Loss: 0.622172
    Train Epoch: 4 [20000/60000 (33%)]	Loss: 0.517312
    Train Epoch: 4 [40000/60000 (67%)]	Loss: 0.503740
    
    Test set: Average loss: 0.0024, Accuracy: 8702.0/10000 (87%)
    
    Train Epoch: 5 [0/60000 (0%)]	Loss: 0.490726
    Train Epoch: 5 [20000/60000 (33%)]	Loss: 0.477141
    Train Epoch: 5 [40000/60000 (67%)]	Loss: 0.455091
    
    Test set: Average loss: 0.0021, Accuracy: 8824.0/10000 (88%)
    
    Train Epoch: 6 [0/60000 (0%)]	Loss: 0.461932
    Train Epoch: 6 [20000/60000 (33%)]	Loss: 0.551792
    Train Epoch: 6 [40000/60000 (67%)]	Loss: 0.398757
    
    Test set: Average loss: 0.0020, Accuracy: 8887.0/10000 (89%)
    
    Train Epoch: 7 [0/60000 (0%)]	Loss: 0.467709
    Train Epoch: 7 [20000/60000 (33%)]	Loss: 0.295131
    Train Epoch: 7 [40000/60000 (67%)]	Loss: 0.435595
    
    Test set: Average loss: 0.0018, Accuracy: 8947.0/10000 (89%)
    
    Train Epoch: 8 [0/60000 (0%)]	Loss: 0.323253
    Train Epoch: 8 [20000/60000 (33%)]	Loss: 0.368885
    Train Epoch: 8 [40000/60000 (67%)]	Loss: 0.278219
    
    Test set: Average loss: 0.0018, Accuracy: 9007.0/10000 (90%)
    
    Train Epoch: 9 [0/60000 (0%)]	Loss: 0.478840
    Train Epoch: 9 [20000/60000 (33%)]	Loss: 0.325814
    Train Epoch: 9 [40000/60000 (67%)]	Loss: 0.418261
    
    Test set: Average loss: 0.0017, Accuracy: 9028.0/10000 (90%)
    
    
    Process finished with exit code 0

[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70_pic_center]: https://img-blog.csdnimg.cn/20210120205643589.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE=,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70_pic_center 1]: https://img-blog.csdnimg.cn/20210120205700254.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE=,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70]: https://img-blog.csdnimg.cn/20210120205743264.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE=,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70_pic_center 2]: https://img-blog.csdnimg.cn/2021012020575135.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE=,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70_pic_center 3]: https://img-blog.csdnimg.cn/20210120205757863.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE=,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70_pic_center 4]: https://img-blog.csdnimg.cn/20210120205801900.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE=,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70_pic_center 5]: https://img-blog.csdnimg.cn/20210120205806127.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE=,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70_pic_center 6]: https://img-blog.csdnimg.cn/20210120205810400.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE=,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70_pic_center 7]: https://img-blog.csdnimg.cn/20210120205815157.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE=,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70_pic_center 8]: https://img-blog.csdnimg.cn/20210120204058567.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE=,size_16,color_FFFFFF,t_70#pic_center
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70 1]: https://img-blog.csdnimg.cn/2021012018063399.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE=,size_16,color_FFFFFF,t_70
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE_size_16_color_FFFFFF_t_70 2]: https://img-blog.csdnimg.cn/20210120173641701.png?x-oss-process=image/watermark,type_ZmFuZ3poZW5naGVpdGk,shadow_10,text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3UwMTMyNTA4NjE=,size_16,color_FFFFFF,t_70