Cohere open-sources a coding agent that runs on a single H100
Summary
- TITLE: Open-Source Coding Agent Can Run on a Single High-Powered Computer HOMEPAGE: A new open-source coding agent called North Mini Code can help software engineers do their jobs more efficiently.
- This agent can perform complex coding tasks and run on a single powerful computer, unlike other similar models that require multiple machines.
- SUMMARY: A company called Cohere has created an open-source coding agent that can run on a single high-powered computer.
- The agent, called North Mini Code, is designed to help software engineers with complex coding tasks such as code review and terminal-based tasks.
- It can also analyze and map systems architecture and perform code review across large codebases.
- North Mini Code is a 30 billion parameter model that was trained on over 70,000 verifiable tasks and can generate three times the output tokens of comparable models.
- It is available for free under an open-source license.
- North Mini Code is a sparse mixture-of-experts model with 128 experts, of which 8 activate per token.
- The compute requirement at inference time is closer to a 3 billion parameter model despite 30 billion total parameters.
- The model is trained through two stages of supervised fine-tuning followed by reinforcement learning with verifiable rewards across more than 70,000 verifiable tasks.
- North Mini Code is a 256,000 token context window with a 64,000 token maximum generation length.
- It is available on Hugging Face under an Apache 2.0 license.
- WHY IT MATTERS: This new open-source coding agent can make a big difference for software engineers who want to automate complex coding tasks.
- With North Mini Code, they can focus on more high-level tasks and let the agent handle the details.
- This can lead to faster development times and improved code quality.
- North Mini Code also shows that open-source models can be competitive with proprietary ones, which could lead to more innovation in the field.
- EXPLANATION: Mixture-of-Experts (MoE) model: Imagine you have a team of experts who can all do different tasks.
- A MoE model is like a single expert who can combine the knowledge of all those experts to solve a problem.
- In this case, North Mini Code is a MoE model with 128 experts, of which 8 activate per token.
- Sparse model: A sparse model is like a library with only the books you need on a particular topic.
- North Mini Code is a sparse model because it only uses 3 billion parameters at inference time, even though it has 30 billion total parameters.
- This makes it more efficient and faster.
- Reinforcement learning with verifiable rewards: Imagine you're training a dog to do tricks.
- You give it treats when it does the trick correctly, but not when it doesn't.
- That's similar to reinforcement learning with verifiable rewards.
- Cohere used this approach to train North Mini Code, where the rewards were verifiable tasks that the model performed correctly.
Save articles to read later — View Saved
MORE FROM THIS EDITION