Pytorch 训练网络时 fix 住部分层参数

素颜马尾好姑娘i 2022-01-26 13:29 297阅读 0赞

先粘贴一段非常好的pytorch forum回复，来自 [https://discuss.pytorch.org/t/how-the-pytorch-freeze-network-in-some-layers-only-the-rest-of-the-training/7088/2][https_discuss.pytorch.org_t_how-the-pytorch-freeze-network-in-some-layers-only-the-rest-of-the-training_7088_2]

I faced this just a few days ago, so I’m sure this code should be up to date. Here’s my answer for Resnet, but this answer can be used for literally any model.

The basic idea is that all models have a function model.children() which returns it’s layers. Within each layer, there are parameters (or weights), which can be obtained using .param() on any children (i.e. layer). Now, every parameter has an attribute called requires\_grad which is by default True. True means it will be backpropagrated and hence to freeze a layer you need to set requires\_grad to False for all parameters of a layer. This can be done like this -

model_ft = models.resnet50(pretrained=True)
    ct = 0
    for child in model_ft.children():
    ct += 1
    if ct < 7:
        for param in child.parameters():
            param.requires_grad = False

This freezes layers 1-6 in the total 10 layers of Resnet50. Hope this helps!

但是这样只是 requires\_grad 设置成 false，最好还是filter掉这些参数，虽然他们不会被训练了

optimizer.SGD(filter(lambda p: p.requires\_grad, model.parameters()), lr=1e-3)

参考：

[https://zhuanlan.zhihu.com/p/34147880][https_zhuanlan.zhihu.com_p_34147880]

[https://blog.csdn.net/guotong1988/article/details/79739775][https_blog.csdn.net_guotong1988_article_details_79739775]

[https_discuss.pytorch.org_t_how-the-pytorch-freeze-network-in-some-layers-only-the-rest-of-the-training_7088_2]: https://discuss.pytorch.org/t/how-the-pytorch-freeze-network-in-some-layers-only-the-rest-of-the-training/7088/2
[https_zhuanlan.zhihu.com_p_34147880]: https://zhuanlan.zhihu.com/p/34147880
[https_blog.csdn.net_guotong1988_article_details_79739775]: https://blog.csdn.net/guotong1988/article/details/79739775