Vicuna is an open-source chatbot developed by the Vicuna Team, fine-tuned on user-shared conversations collected from ShareGPT. Preliminary evaluations show that Vicuna-13B achieves more than 90% quality of OpenAI ChatGPT and Google Bard, outperforming other models like LLaMA and Stanford Alpaca in over 90% of cases.

Features

• Performance: After fine-tuning Vicuna with 70K user-shared ChatGPT conversations, the chatbot can generate more detailed and well-structured answers compared to Alpaca.
• Evaluation System: The team proposed an automated evaluation framework for benchmark generation and performance assessments based on GPT-4.
• Serving System: A lightweight distributed serving system was built, capable of serving multiple models with distributed workers.

Use Cases

Vicuna-13B can be used as a chatbot system, generating detailed and well-structured responses. It can serve as an open starting point for future research to tackle limitations in tasks involving reasoning or mathematics, accurately identifying itself or ensuring the factual accuracy of its outputs.

Limitations

Like other large language models, Vicuna has certain limitations. It is not good at tasks involving reasoning or mathematics, and may have limitations in accurately identifying itself or ensuring the factual accuracy of its outputs. It has not been sufficiently optimized to guarantee safety or mitigate potential toxicity or bias.

Release and License

The training, serving, and evaluation code is released on a GitHub repo and the Vicuna-13B model weights have also been released. The online demo is a research preview intended for non-commercial use only, and the code is released under the Apache License 2.0.