Google GCPとDataRobotでファインチューニングされたLlama 2¶
There are a wide variety of open source large language models (LLMs). For example, there has been a lot of interest in Llama and variations such as Alpaca, Vicuna, Falcon, and Mistral. Because these LLMs require expensive GPUs, users often want to compare cloud providers to find the best hosting option. In this accelerator you will work with Google Cloud Platform to host Llama 2.
You may also want to integrate with the cloud provider that hosts your Virtual Private Cloud (VPC) so that you can ensure proper authentication and access it only from within the VPC. While this accelerator uses authentication over the public internet, it is possible to leverage Google's cloud infrastructure to adjust and suit your cloud architectural needs, including provisioning scaleout policies.
Finally, by leveraging Vertex AI in a managed format, you can integrate that infrastructure into your existing stack to meet monitoring needs—things like monitoring service health, CPU usage, and low-level alerting to billing, cost attribution, and account management and, using GCP's tools to route information into BigQuery for ad hoc analytics, log exploration, and more.
Llama 2¶
For information about Llama 2 you can read:
- The model card on HuggingFace.
- The paper released on Arxiv.
Llama is available from Meta for download.
Lllama 13B-Instruct¶
The Llama-13b-instruct model has been fine-tuned on datasets available from HuggingFace and is designed specifically for instruction-based use cases. It was trained to use [INST] and [/INST] control tokens around a user message as well as to begin with system ID (<s>). 例：
<s> [INST] What is your favorite condiment? [/INST]
GCPの概要¶
The GCP instance types listed below can host Llama-13B with acceleration:
- g2-standard-8（L4 GPU 1個、vCPU 8個、32GB RAM、月額$623ドル）
- n1-standard-16（V100 GPU 2個、vCPU 16個、60GB RAM、月額$388ドル）
- n1-standard-16（T4 GPU 2個、vCPU 16個、60GB RAM + 32GB + 32GB、月額$388ドル）
- a2-highgpu-1g（A100 GPU 1個、vCPU 12個、85GB RAM、月額$2,682ドル）