fast-reid复现知识蒸馏部分（Market1501）--- 2020.08.31版本

╰+哭是因爲堅強的太久メ 2022-12-03 08:30 255阅读 0赞

由于想继续使用pytorch1.1.0，所以可能会涉及到一些pytorch相关的修改，会注明。

## 0.环境 ##

ubuntu16.04
    cuda9.0
    python3.6
    torch==1.1.0
    torchvision==0.3.0
    Cython
    yacs
    tensorboard
    future
    termcolor
    sklearn
    tqdm
    opencv-python==4.1.0.25
    matplotlib
    scikit-image
    faiss-gpu==1.6.3
    tabulate
    gdown

## 1.下载对应版本的fast-reid ##

[https://download.csdn.net/download/qq\_35975447/12797752][https_download.csdn.net_download_qq_35975447_12797752]

## 2.修改 ##

### 2.1 模型输出 ###

这个地方应该是代码未更新完整，导致的接口不一致。

![20200902111628693.png][]

# kd_trainer.py
    # line70
    
    outputs, targets = self.model(data)
    改为：
    outputs = self.model(data) # add
    
    #line80-82
    with torch.no_grad():
        outputs_t, _ = self.model_t(data)
    改为：
    with torch.no_grad():
        outputs_t = self.model_t(data) # add
        # outputs_t, _ = self.model_t(data)

### 2.2 DataParallel问题 ###

# kd_trainer.py
    # line72-76
    
    # Compute reid loss
    if isinstance(self.model, DistributedDataParallel):
        loss_dict = self.model.module.losses(outputs, targets)
    else:
        loss_dict = self.model.losses(outputs, targets)
    
    改为：
    loss_dict = self.model.module.losses(outputs, targets)

## 2.3 disttill\_loss ##

# kd_trainer.py
    # line141
    
    loss = F.kl_div(p_s, p_t, reduction='sum') * (t ** 2) / y_s.shape[0]
    改为：
    loss = F.kl_div(p_s.cpu(), p_t.cpu(), reduction='sum') * (t ** 2) / y_s.shape[0]

### 2.4 cross\_entropy\_loss ###

# cross_entropy_loss.py
    # line71-73
    
    with torch.no_grad():
        non_zero_cnt = max(loss.nonzero(as_tuple=False).size(0), 1)
    
    改为：
    with torch.no_grad():
        non_zero_cnt = max(loss.nonzero().size(0), 1) # add
        # non_zero_cnt = max(loss.nonzero(as_tuple=False).size(0), 1)

### 2.4 adam.py中所有的memory\_format=torch.preserve\_format去掉 ###

adam.py在fast-reid-master\\fastreid\\solver\\optim\\adam.py 中。

因为memory\_format=torch.preserve\_format应该是pytorch1.6版本里面的参数，fast-reid作者新支持pytorch1.6.

这个参数可以去掉（[https://www.geek-book.com/src/docs/pytorch/pytorch/pytorch.org/docs/stable/tensors.html][https_www.geek-book.com_src_docs_pytorch_pytorch_pytorch.org_docs_stable_tensors.html]），中文网站中有提到这个是默认的方式。我们可以不用。

![watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70][]

#所有的下载语句修改：
    torch.zeros_like(p, memory_format=torch.preserve_format)
    改为：
    torch.zeros_like(p)

## 3.运行 ##

这个地方作者给的参考很好用。[https://github.com/JDAI-CV/fast-reid/blob/master/projects/DistillReID/README.md][https_github.com_JDAI-CV_fast-reid_blob_master_projects_DistillReID_README.md]

# a demo on Market1501 dataset
    # please see more in ./configs
    # train BagTricksIBN50 as teacher model
    
    #单gpu训练
    # 教师模型
    CUDA_VISIBLE_DEVICES=0 python ./tools/train_net.py --config-file ./projects/DistillReID/configs-bagtricks-ibn-market1501/bagtricks_R50-ibn.yml 
    # train BagTricksIBN18 as student model 
    CUDA_VISIBLE_DEVICES=0 python ./projects/DistillReID/train_net.py --kd --config-file ./projects/DistillReID/configs-bagtricks-ibn-market1501/KD-bot50ibn-bot18ibn.yml
    
    #多gpu训练
    # 教师模型
    CUDA_VISIBLE_DEVICES='0,1' python ./tools/train_net.py --config-file ./projects/DistillReID/configs-bagtricks-ibn-market1501/bagtricks_R50-ibn.yml  --num-gpus 2 
    # train BagTricksIBN18 as student model 
    CUDA_VISIBLE_DEVICES='0,1' python ./projects/DistillReID/train_net.py --kd --config-file ./projects/DistillReID/configs-bagtricks-ibn-market1501/KD-bot50ibn-bot18ibn.yml --num-gpus 2
    
    #测试学生模型
    CUDA_VISIBLE_DEVICES=0 python tools/train_net.py --config-file "logs/market1501/KD-bot50ibn-bot18ibn/config_caffe.yaml" --eval-only  MODEL.WEIGHTS "logs/market1501/KD-bot50ibn-bot18ibn/model_final.pth"

[https_download.csdn.net_download_qq_35975447_12797752]: https://download.csdn.net/download/qq_35975447/12797752
[20200902111628693.png]: /images/20221123/decf92c1748d46ae806787de9228979c.png
[https_www.geek-book.com_src_docs_pytorch_pytorch_pytorch.org_docs_stable_tensors.html]: https://www.geek-book.com/src/docs/pytorch/pytorch/pytorch.org/docs/stable/tensors.html
[watermark_type_ZmFuZ3poZW5naGVpdGk_shadow_10_text_aHR0cHM6Ly9ibG9nLmNzZG4ubmV0L3FxXzM1OTc1NDQ3_size_16_color_FFFFFF_t_70]: /images/20221123/a3930962eda84e6ba5aebcaf4a1ab01f.png
[https_github.com_JDAI-CV_fast-reid_blob_master_projects_DistillReID_README.md]: https://github.com/JDAI-CV/fast-reid/blob/master/projects/DistillReID/README.md