固定部分参数进行训练

参考:pytorch固定部分参数进行网络训练

有两种实现场景:

  1. 仅训练某些层
  2. 暂时固定某些层

仅训练某些层

直接在优化器中输入要训练的层参数即可

第一步:在模型中设置不训练的层参数的require_gradsFalse

class AlexNet(nn.Module):

    def __init__(self, num_classes=1000):
        super(AlexNet_SPP, self).__init__()
        self.features = nn.Sequential(
            nn.Conv2d(3, 64, kernel_size=11, stride=4, padding=2),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=3, stride=2),
            nn.Conv2d(64, 192, kernel_size=5, padding=2),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=3, stride=2),
            nn.Conv2d(192, 384, kernel_size=3, padding=1),
            nn.ReLU(inplace=True),
            nn.Conv2d(384, 256, kernel_size=3, padding=1),
            nn.ReLU(inplace=True),
            nn.Conv2d(256, 256, kernel_size=3, padding=1),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=3, stride=2),
        )
        self.avgpool = nn.AdaptiveAvgPool2d((6, 6))
        # 只训练分类器,设置之前参数的require_grads为False
        for p in self.parameters():
            p.requires_grad = False

        self.classifier = nn.Sequential(
            nn.Dropout(),
            nn.Linear(256 * 50, 4096),
            nn.ReLU(inplace=True),
            nn.Dropout(),
            nn.Linear(4096, 4096),
            nn.ReLU(inplace=True),
            nn.Linear(4096, num_classes),
        )

    def forward(self, x):
        x = self.features(x)
        x = self.avgpool(x)
        x = torch.flatten(x, 1)
        x = self.classifier(x)
        return x

第二步:在optimizer中输入待训练的层参数

optimizer = optim.Adam(filter(lambda p: p.requires_grad, model.parameters()), lr=1e-4)

Note:使用filterlambda函数过滤不训练的层参数

暂时固定某些层

使用named_parameters过滤

for k, v in model.named_parameters():
        print(k, v.requires_grad)
        if 'classifier' not in k:
                v.requires_grad = False