Google的bert预训练模型下载地址＋将tensorflow版本的预训练模型转为pytorch版本进行加载

刺骨的言语ヽ痛彻心扉 2023-07-16 03:36 26阅读 0赞

google的bert预训练模型：

*  [`BERT-Large, Uncased (Whole Word Masking)`][BERT-Large_ Uncased _Whole Word Masking]: 24-layer, 1024-hidden, 16-heads, 340M parameters
 *  [`BERT-Large, Cased (Whole Word Masking)`][BERT-Large_ Cased _Whole Word Masking]: 24-layer, 1024-hidden, 16-heads, 340M parameters
 *  [`BERT-Base, Uncased`][BERT-Base_ Uncased]: 12-layer, 768-hidden, 12-heads, 110M parameters
 *  [`BERT-Large, Uncased`][BERT-Large_ Uncased]: 24-layer, 1024-hidden, 16-heads, 340M parameters
 *  [`BERT-Base, Cased`][BERT-Base_ Cased]: 12-layer, 768-hidden, 12-heads , 110M parameters
 *  [`BERT-Large, Cased`][BERT-Large_ Cased]: 24-layer, 1024-hidden, 16-heads, 340M parameters
 *  [`BERT-Base, Multilingual Cased (New, recommended)`][BERT-Base_ Multilingual Cased _New_ recommended]: 104 languages, 12-layer, 768-hidden, 12-heads, 110M parameters
 *  [`BERT-Base, Multilingual Uncased (Orig, not recommended)`][BERT-Base_ Multilingual Uncased _Orig_ not recommended] (Not recommended, use `Multilingual Cased`instead): 102 languages, 12-layer, 768-hidden, 12-heads, 110M parameters
 *  [`BERT-Base, Chinese`][BERT-Base_ Chinese]: Chinese Simplified and Traditional, 12-layer, 768-hidden, 12-heads, 110M parameters
 *  pytorch的bert预训练模型（pretrained\_model\_name\_or\_path）：

PRETRAINED_VOCAB_ARCHIVE_MAP = {
        'bert-base-uncased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-vocab.txt",
        'bert-large-uncased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-large-uncased-vocab.txt",
        'bert-base-cased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased-vocab.txt",
        'bert-large-cased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-large-cased-vocab.txt",
        'bert-base-multilingual-uncased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-multilingual-uncased-vocab.txt",
        'bert-base-multilingual-cased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-multilingual-cased-vocab.txt",
        'bert-base-chinese': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-chinese-vocab.txt",
    }

PRETRAINED_MODEL_ARCHIVE_MAP = {
        'bert-base-uncased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased.tar.gz",
        'bert-large-uncased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-large-uncased.tar.gz",
        'bert-base-cased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-cased.tar.gz",
        'bert-large-cased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-large-cased.tar.gz",
        'bert-base-multilingual-uncased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-multilingual-uncased.tar.gz",
        'bert-base-multilingual-cased': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-multilingual-cased.tar.gz",
        'bert-base-chinese': "https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-chinese.tar.gz",
    }

[https://huggingface.co/transformers/converting\_tensorflow\_models.html][https_huggingface.co_transformers_converting_tensorflow_models.html]

因为下载的模型为tensorflow版本，我们如果想用pytorch加载。做一下操作

①: pip install transformers ②:解压完后直接用一条命令: transformers bert bert\_model.ckpt bert\_config.json pytorch\_model.bin

[BERT-Large_ Uncased _Whole Word Masking]: https://storage.googleapis.com/bert_models/2019_05_30/wwm_uncased_L-24_H-1024_A-16.zip
[BERT-Large_ Cased _Whole Word Masking]: https://storage.googleapis.com/bert_models/2019_05_30/wwm_cased_L-24_H-1024_A-16.zip
[BERT-Base_ Uncased]: https://storage.googleapis.com/bert_models/2018_10_18/uncased_L-12_H-768_A-12.zip
[BERT-Large_ Uncased]: https://storage.googleapis.com/bert_models/2018_10_18/uncased_L-24_H-1024_A-16.zip
[BERT-Base_ Cased]: https://storage.googleapis.com/bert_models/2018_10_18/cased_L-12_H-768_A-12.zip
[BERT-Large_ Cased]: https://storage.googleapis.com/bert_models/2018_10_18/cased_L-24_H-1024_A-16.zip
[BERT-Base_ Multilingual Cased _New_ recommended]: https://storage.googleapis.com/bert_models/2018_11_23/multi_cased_L-12_H-768_A-12.zip
[BERT-Base_ Multilingual Uncased _Orig_ not recommended]: https://storage.googleapis.com/bert_models/2018_11_03/multilingual_L-12_H-768_A-12.zip
[BERT-Base_ Chinese]: https://storage.googleapis.com/bert_models/2018_11_03/chinese_L-12_H-768_A-12.zip
[https_huggingface.co_transformers_converting_tensorflow_models.html]: https://huggingface.co/transformers/converting_tensorflow_models.html