我要评论huggingface镜像网站：https://hf-mirror.com （感觉大佬们）
通过修改环境变量访问镜像网站加载模型
import os
os.environ['hf_endpoint'] = 'https://hf-mirror.com'注意：修改环境变量一定要放到最上方！！！放到 import transformers前修改！！！
完整代码
import os
os.environ['hf_endpoint'] = 'https://hf-mirror.com'
from transformers import autotokenizer
check_point = 'distilbert-base-uncased-finetuned-sst-2-english'
tokenizer = autotokenizer.from_pretrained(check_point)
inputs = [
'i‘m hunger',
'nice to meet you, lihua'
]
input = tokenizer(inputs, padding=true, truncation=true, return_tensors='pt')
print('input: ',input)打印结果
input: {'input_ids': tensor([[ 101, 1045, 1520, 1049, 9012, 102, 0, 0, 0],
[ 101, 3835, 2000, 3113, 2017, 1010, 5622, 14691, 102]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 0, 0, 0],
[1, 1, 1, 1, 1, 1, 1, 1, 1]])}
发表评论