[Pytorch专栏]基本概念之nn.Module

朱雀 2023-06-11 11:22 7阅读 0赞

pytorch在深度学习领域的重要性我就不再多罗嗦了，相信你看到这里的时候也一定对torch、pytorch有了些许了解，并且希望深入学习，在这个专栏带领大家一起玩转pytorch。

参考资料：[https://pytorch.org/tutorials/][https_pytorch.org_tutorials]

基本概念这一部分讲解使用pytorch模型的基本组成元素，其中包括nn.Module、DataLoader、Tranformer、visualization、DistributedTraining等部分，在后续章节中会分别进行介绍

## nn.Module ##

训练模型，必须首先创建一个模型，而创建模型的所需要的父类就是nn.Module。

class SimpleNet(nn.Module):
        def __init__(self, num_classes=1000):
            super(SimpleNet, self).__init__()
            self.features = nn.Sequential(
                nn.Conv2d(3, 64, kernel_size=11, stride=4, padding=2),
                nn.ReLU(inplace=True),
                nn.MaxPool2d(kernel_size=3, stride=2),
                nn.Conv2d(64, 192, kernel_size=5, padding=2),
                nn.ReLU(inplace=True),
                nn.MaxPool2d(kernel_size=3, stride=2),
                nn.Conv2d(192, 384, kernel_size=3, padding=1),
                nn.ReLU(inplace=True),
                nn.Conv2d(384, 256, kernel_size=3, padding=1),
                nn.ReLU(inplace=True),
                nn.Conv2d(256, 256, kernel_size=3, padding=1),
                nn.ReLU(inplace=True),
                nn.MaxPool2d(kernel_size=3, stride=2),
            )
            self.avgpool = nn.AvgPool2d((1, 1))
            self.classifier = nn.Sequential(
                nn.Dropout(),
                nn.Linear(256 * 6 * 6, 4096),
                nn.ReLU(inplace=True),
                nn.Dropout(),
                nn.Linear(4096, 4096),
                nn.ReLU(inplace=True),
                nn.Linear(4096, num_classes),
            )
    
        def forward(self, x):
            x = self.features(x)
            x = self.avgpool(x)
            x = x.view(x.size(0), 256 * 6 * 6)
            x = self.classifier(x)
            return x

上面代码创建了一个简单的分类模型，每一个模型都是按照如上格式进行模型的。创建好网络后可以使用print打印一下

> SimpleNet(  
> (features): Sequential(  
> (0): Conv2d(3, 64, kernel\_size=(11, 11), stride=(4, 4), padding=(2, 2))  
> (1): ReLU(inplace)  
> (2): MaxPool2d(kernel\_size=3, stride=2, padding=0, dilation=1, ceil\_mode=False)  
> (3): Conv2d(64, 192, kernel\_size=(5, 5), stride=(1, 1), padding=(2, 2))  
> (4): ReLU(inplace)  
> (5): MaxPool2d(kernel\_size=3, stride=2, padding=0, dilation=1, ceil\_mode=False)  
> (6): Conv2d(192, 384, kernel\_size=(3, 3), stride=(1, 1), padding=(1, 1))  
> (7): ReLU(inplace)  
> (8): Conv2d(384, 256, kernel\_size=(3, 3), stride=(1, 1), padding=(1, 1))  
> (9): ReLU(inplace)  
> (10): Conv2d(256, 256, kernel\_size=(3, 3), stride=(1, 1), padding=(1, 1))  
> (11): ReLU(inplace)  
> (12): MaxPool2d(kernel\_size=3, stride=2, padding=0, dilation=1, ceil\_mode=False)  
> )  
> (avgpool): AvgPool2d(kernel\_size=(1, 1), stride=(1, 1), padding=0)  
> (classifier): Sequential(  
> (0): Dropout(p=0.5)  
> (1): Linear(in\_features=9216, out\_features=4096, bias=True)  
> (2): ReLU(inplace)  
> (3): Dropout(p=0.5)  
> (4): Linear(in\_features=4096, out\_features=4096, bias=True)  
> (5): ReLU(inplace)  
> (6): Linear(in\_features=4096, out\_features=1000, bias=True)  
> )  
> )

如果前后层尺寸不对应的话，打印是看不出来错误的，只能在训练或者前向时才可以看出来。还可以打印网络层名称和权重

for name, parameters in net.named_parameters():
        print('name: {}, param: {}'.format(name, parameters))

网络层名称和权重输出结果如下，从下面输出结果可以看出，name即为网络层名称，也就是在定义网络的时候写的层名，当加载权重的时候也是通过匹配name进行权重赋值的，param就是层参数，参数的基本单元是tensor，tensor是高维数组，可以与numpy进行互相转换，对于卷积层和全连接层，有权重和偏置两个参数。激活层ReLU和MaxPool层没有参数

> name: features.3.bias, param: Parameter containing:  
> tensor(\[ 1.4957e-02, 9.2688e-04, -1.5689e-02, -1.7813e-02, 2.0555e-02,  
> 2.4371e-02, -1.5469e-02, 1.3342e-02, 3.3973e-03, 2.4930e-02,  
> 2.0547e-03, 1.8555e-02, -3.7325e-03, 2.0697e-02, 1.6567e-02,  
> \-2.0579e-02, 1.5670e-02, -5.5653e-03, 9.8694e-03, 1.6047e-02,  
> \-5.5923e-03, -8.6149e-03, -1.9673e-03, -1.7893e-02, 2.4934e-02,  
> \-1.4138e-02, 8.2390e-03, -1.4437e-02, 2.0114e-02, -1.4466e-02,  
> \-1.3437e-02, -8.9125e-03, -2.1675e-02, 8.2867e-04, -1.4810e-02,  
> \-1.3505e-02, -4.0385e-04, -4.7128e-03, -1.5937e-02, 1.0475e-02,  
> 1.0923e-02, 8.2931e-03, 2.0249e-02, -2.2489e-02, -2.3102e-02,  
> \-2.3941e-02, 1.3586e-02, 9.4410e-03, -1.3969e-02, 1.8650e-02,  
> 6.5202e-03, -2.0268e-02, 2.0344e-02, -2.4513e-02, -2.1452e-02,  
> \-2.2310e-02, -1.4287e-02, 2.3486e-02, 3.5223e-03, 1.7498e-02,  
> \-6.4333e-03, -1.8280e-02, 1.6511e-02, 3.1823e-03, -1.0159e-02,  
> \-1.8306e-02, 8.8460e-04, 4.2371e-03, -2.4310e-02, 6.1531e-03,  
> 1.7120e-02, 9.4392e-03, 3.9597e-03, -1.5277e-02, 1.1027e-02,  
> \-1.8415e-02, 1.9837e-02, 1.2948e-02, -1.6227e-02, -1.8143e-02,  
> \-1.1087e-02, 1.5955e-02, -1.3796e-02, -7.9715e-03, -8.2901e-03,  
> \-1.1397e-02, 2.1885e-02, -1.3258e-02, -9.6440e-03, 2.0368e-02,  
> \-5.2773e-04, -3.4875e-03, -1.5337e-02, 2.3905e-02, 1.8523e-02,  
> 3.4253e-03, -2.4514e-02, -1.9495e-02, -1.5818e-02, 7.3031e-03,  
> \-1.9106e-02, -1.1330e-03, -2.3167e-02, 2.0370e-02, -2.5699e-05,  
> \-2.0461e-02, -2.1582e-02, 1.5271e-02, 1.7045e-02, -2.1056e-02,  
> 1.7383e-03, -3.1803e-03, 1.5177e-02, -6.6120e-03, 2.3724e-02,  
> 9.1531e-03, -7.9457e-03, -1.3581e-02, -2.4429e-02, 2.2770e-02,  
> \-4.8633e-03, 1.3242e-02, 2.3412e-02, 1.6137e-02, -1.0654e-02,  
> \-2.3665e-02, -6.4708e-03, 5.3640e-03, -5.2605e-03, -1.2003e-02,  
> \-1.5830e-02, 1.8685e-02, 1.7946e-02, -2.2646e-02, -2.3571e-02,  
> 3.1388e-03, -1.5248e-02, 2.2037e-02, 1.4381e-02, 8.1348e-03,  
> \-2.3800e-02, 1.9604e-02, 2.5116e-03, -1.1058e-03, -1.9687e-02,  
> 1.2248e-02, 2.3413e-02, 2.1731e-02, 5.0117e-03, 8.7472e-04,  
> \-7.5963e-03, -1.1021e-02, 1.4873e-02, -2.2508e-02, 2.1154e-03,  
> \-9.1743e-04, -1.7245e-02, 1.7589e-02, 1.1800e-02, 8.5025e-03,  
> \-2.3779e-02, -2.0176e-02, -2.0294e-02, -1.1143e-02, 4.0771e-03,  
> \-1.5206e-02, 2.2434e-02, 1.0912e-02, -1.6116e-02, 9.9414e-03,  
> 2.7820e-03, -5.8191e-03, -3.6565e-03, 4.7741e-03, -5.2534e-03,  
> 2.2425e-02, -3.1049e-03, -9.2849e-05, -5.2352e-03, -1.5395e-02,  
> \-2.3302e-02, 1.3749e-02, 7.7268e-03, -1.9604e-03, -1.3374e-02,  
> 1.9139e-02, -1.8436e-02, -3.1385e-03, 1.9631e-02, 2.1478e-02,  
> 1.8698e-02, -1.4259e-02\], requires\_grad=True)  
> name: features.6.weight, param: Parameter containing:  
> tensor(\[\[\[\[ 1.4673e-02, -1.2573e-03, -4.9999e-03\],  
> \[-1.1767e-04, -2.3442e-03, 9.0190e-03\],  
> \[-2.2670e-02, 9.2900e-04, 2.3845e-02\]\],
> 
> \[\[ 7.1682e-03, 1.0164e-02, 7.7916e-03\],  
> \[-1.2084e-02, 8.5199e-03, 1.7568e-02\],  
> \[-1.6802e-02, -7.6553e-03, -1.4794e-02\]\],
> 
> \[\[ 2.3868e-02, 5.9265e-03, 1.8999e-02\],  
> \[ 1.0245e-02, -5.0897e-03, -9.1414e-03\],  
> \[-1.2725e-02, 1.5891e-02, -2.2689e-05\]\],
> 
> ...,

从上面的结果我们看不出来网络层feature map的具体情况，还可以将网络进行可视化，下面使用hiddenlayer对网络进行可视化，[hiddenlayer][]借助onnx生成静态图然后使用graphviz完成网络结构图绘制

hl_graph = hl.build_graph(net, torch.zeros([1, 3, 224, 224]))
    hl_graph.theme = hl.graph.THEMES["blue"].copy()  # Two options: basic and blue
    hl_graph.save("simplenet", format="png")

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpZGlhbnpoaW1lbmc_size_16_color_FFFFFF_t_70][]

当使用pytorch1.3版本的时候可能会出现一下问题，当将版本回退到1.0就没有问题了 ，[参考这里][Link 1]，具体原因还没搞清楚

> AttributeError: ‘torch.\_C.Value’ object has no attribute ‘uniqueName’

也可能会遇到一下问题，应该是onnx不支持导致，可以[参考这里][Link 2]，将nn.AdaptiveAvgPool2d((6, 6))改为nn.AvgPool2d((1, 1))

> AssertionError: Only output\_size=\[1, 1\] is supported

基本上创建了一个网络，最后还可以尝试输入一个数据看看输出结果

input_image = torch.FloatTensor(10, 3, 224, 224)
    input_image = Variable(input_image)
    
    output = net(input_image)
    print('output: {}'.format(output))
    print(output.shape)
    print(output.size())

> output: tensor(\[\[-0.0007, 0.0012, 0.0134, ..., 0.0062, -0.0009, 0.0047\],  
> \[ 0.0005, 0.0043, 0.0097, ..., 0.0084, -0.0027, 0.0035\],  
> \[-0.0009, 0.0066, 0.0108, ..., 0.0078, -0.0074, 0.0050\],  
> ...,  
> \[-0.0013, 0.0022, 0.0134, ..., 0.0118, -0.0027, 0.0049\],  
> \[-0.0013, 0.0021, 0.0120, ..., 0.0082, -0.0027, 0.0055\],  
> \[ 0.0021, 0.0031, 0.0131, ..., 0.0122, -0.0021, 0.0033\]\],  
> grad\_fn=<AddmmBackward>)  
> torch.Size(\[10, 1000\])  
> torch.Size(\[10, 1000\])

写道这里基本完成了pytorch创建一个分类模型，提一个问题，大家知道上面是哪个经典模型吗？

[https_pytorch.org_tutorials]: https://pytorch.org/tutorials/
[hiddenlayer]: https://github.com/waleedka/hiddenlayer
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3hpZGlhbnpoaW1lbmc_size_16_color_FFFFFF_t_70]: /images/20230531/1015ed6273eb4a35b4070f85f53dd6b9.png
[Link 1]: https://blog.csdn.net/ZZJ__/article/details/100142097
[Link 2]: https://discuss.pytorch.org/t/when-using-torch-onnx-export-errors-occurs-assertionerror-only-output-size-1-1-is-supported/38528/5