Nvidia has unveiled Nemotron-4 340B, a family of open-source language models designed to generate high-quality synthetic data and develop powerful artificial intelligence applications across various industries.
The Nemotron-4 340B family includes three key models: Base, Instruct, and Reward, which work together to create synthetic data used in training new large-scale language models (LLMs). The Instruct model generates high-quality synthetic data and was trained with 98% synthetic data, while the Reward model filters this data to select the highest quality examples.
The Nemotron-4 models have proven to be competitive and even superior to other open-source models like Llama-3, Mixtral, and Qwen-2 in various benchmark tests. Additionally, Nvidia has released Mamba-2 Hybrid, a selective state space model (SSM) that has outperformed transformer-based LLM models in accuracy.
Nvidia is not only providing a family of open-source models that match the capabilities of its main competitors but also excels in creating the synthetic data needed to further advance the development of new LLMs. The chip manufacturing giant continues to solidify its position as a powerhouse in the field of artificial intelligence.
Launch of the Nemotron-4 340B Family
Nvidia has announced that the Nemotron-4 340B models are optimized to work with Nvidia NeMo, an open-source framework for end-to-end model training, and the Nvidia TensorRT-LLM open-source library for inference.
Developers can download Nemotron-4 340B from Hugging Face and will soon be available on ai.nvidia.com, where they will be packaged as a Nvidia NIM microservice with a standard API interface that can be deployed anywhere.
Generating Synthetic Data with Nemotron
LLM models can help developers generate synthetic training data in scenarios where access to large and diverse labeled datasets is limited. The Instruct model of Nemotron-4 340B creates diverse synthetic data that mimic real-world data characteristics, improving the quality of data to enhance the performance and robustness of custom LLMs in various domains.
To further enhance the quality of data generated by AI, developers can use the Reward model of Nemotron-4 340B to filter high-quality responses. This model evaluates responses on five attributes: utility, correctness, coherence, complexity, and verbosity. It currently ranks first on the RewardBench leaderboard by AI2, hosted on Hugging Face.
Optimization and Fine-tuning with NeMo and TensorRT-LLM
By leveraging the open-source tools Nvidia NeMo and Nvidia TensorRT-LLM, developers can optimize the efficiency of their Instruct and Reward models for generating synthetic data and evaluating responses.
All Nemotron-4 340B models are optimized with TensorRT-LLM to harness tensor parallelism, a type of model parallelism where individual weight matrices are split across multiple GPUs and servers, enabling efficient inference at scale.
Security Evaluation and Getting Started
The Instruct model of Nemotron-4 340B has undergone thorough security evaluation, including adversarial testing, and has performed well across a wide range of risk indicators. However, users are advised to carefully assess the model outputs to ensure that the synthetically generated data is suitable, safe, and accurate for their use case.
Developers can download the Nemotron-4 340B models via Hugging Face and access more details in the model and dataset research papers. This Nvidia innovation promises to transform synthetic data generation and AI application development across multiple sectors.