Machine Learning - Learning/Language Models

Nous-Hermes-Llama2-13b is currently the highest ranked 13B LLaMA finetune on the Open LLM Leaderboard.

Model Description

Nous-Hermes-Llama2-13b is a state-of-the-art language model fine-tuned on over 300,000 instructions. This model was fine-tuned by Nous Research, with Teknium and Emozilla leading the fine tuning process and dataset curation, Redmond AI sponsoring the compute, and several other contributors.

This Hermes model uses the exact same dataset as Hermes on Llama-1. This is to ensure consistency between the old Hermes and new, for anyone who wanted to keep Hermes as similar to the old one, just more capable.

This model stands out for its long responses, lower hallucination rate, and absence of OpenAI censorship mechanisms. The fine-tuning process was performed with a 4096 sequence length on an 8x a100 80GB DGX machine.

Announcements

• https://twitter.com/NousResearch/status/1682458324804009987
• https://twitter.com/Teknium1/status/1682459395853279232