AI开发平台MODELARTS-推理场景介绍:支持的模型列表和权重文件

时间：2024-06-17 17:57:09

AI开发平台MODELARTS 主流开源大模型基于DevServer适配PyTorch NPU推理指导（6.3.905）

支持的模型列表和权重文件

本方案支持vLLM的v0.3.2版本。不同vLLM版本支持的模型列表有差异，具体如表3所示。

表3 支持的模型列表和权重获取地址
序号	模型名称	支持vLLM v0.3.2	开源权重获取地址
1	llama-7b	√	https://huggingface.co/huggyllama/llama-7b
2	llama-13b	√	https://huggingface.co/huggyllama/llama-13b
3	llama-65b	√	https://huggingface.co/huggyllama/llama-65b
4	llama2-7b	√	https://huggingface.co/meta-llama/Llama-2-7b-chat-hf
5	llama2-13b	√	https://huggingface.co/meta-llama/Llama-2-13b-chat-hf
6	llama2-70b	√	https://huggingface.co/meta-llama/Llama-2-70b-hf https://huggingface.co/meta-llama/Llama-2-70b-chat-hf (推荐)
7	llama3-8b	√	https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct
8	llama3-70b	√	https://huggingface.co/meta-llama/Meta-Llama-3-70B-Instruct
9	yi-6b	√	https://huggingface.co/01-ai/Yi-6B-Chat
10	yi-9b	√	https://huggingface.co/01-ai/Yi-9B
11	yi-34b	√	https://huggingface.co/01-ai/Yi-34B-Chat
12	deepseek-llm-7b	√	https://huggingface.co/deepseek-ai/deepseek-llm-7b-chat
13	deepseek-coder-instruct-33b	√	https://huggingface.co/deepseek-ai/deepseek-coder-33b-instruct
14	deepseek-llm-67b	√	https://huggingface.co/deepseek-ai/deepseek-llm-67b-chat
15	qwen-7b	√	https://huggingface.co/Qwen/Qwen-7B-Chat
16	qwen-14b	√	https://huggingface.co/Qwen/Qwen-14B-Chat
17	qwen-72b	√	https://huggingface.co/Qwen/Qwen-72B-Chat
18	qwen1.5-0.5b	√	https://huggingface.co/Qwen/Qwen1.5-0.5B-Chat
19	qwen1.5-7b	√	https://huggingface.co/Qwen/Qwen1.5-7B-Chat
20	qwen1.5-1.8b	√	https://huggingface.co/Qwen/Qwen1.5-1.8B-Chat
21	qwen1.5-14b	√	https://huggingface.co/Qwen/Qwen1.5-14B-Chat
22	qwen1.5-32b	√	https://huggingface.co/Qwen/Qwen1.5-32B/tree/main
23	qwen1.5-72b	√	https://huggingface.co/Qwen/Qwen1.5-72B-Chat
24	qwen1.5-110b	√	https://huggingface.co/Qwen/Qwen1.5-110B-Chat
25	baichuan2-7b	√	https://huggingface.co/baichuan-inc/Baichuan2-7B-Chat
26	baichuan2-13b	√	https://huggingface.co/baichuan-inc/Baichuan2-13B-Chat
27	chatglm2-6b	√	https://huggingface.co/THUDM/chatglm2-6b
28	chatglm3-6b	√	https://huggingface.co/THUDM/chatglm3-6b
29	gemma-2b	√	https://huggingface.co/google/gemma-2b
30	gemma-7b	√	https://huggingface.co/google/gemma-7b
31	mistral-7b	√	https://huggingface.co/mistralai/Mistral-7B-v0.1