Code testing is an essential part of software development, but it is often a task that developers do not enjoy. Additionally, there is a need for independent verification to ensure the quality of the code. This has led to the rise of generative AI startups focusing on code testing, such as Antithesis, CodiumAI, QA Wolf, and Momentic.
One such startup is Nova AI, which aims to break the rules of traditional startup operations. Unlike many startups that start small, Nova AI targets mid-size to large enterprises with complex code-bases and an immediate need for testing solutions. Founder and CEO Zach Smith explains that their customers are mainly late-stage venture-backed startups in ecommerce, fintech, or consumer products with heavy user experiences. For these companies, downtime can be costly.
Nova AI’s approach involves using GenAI to automatically build tests by sifting through the customers’ code. Their tools are particularly suited for continuous integration and continuous delivery/deployment (CI/CD) environments where engineers are constantly shipping code into production.
The inspiration for Nova AI came from Smith and his co-founder Jeffrey Shih’s experiences as engineers in big tech companies. Smith, a former Googler, worked on cloud-related teams that focused on automation technology. Shih, who previously worked at Meta, Unity, and Microsoft, had expertise in synthetic data and AI. They have since added AI data scientist Henry Li as a third co-founder.
While many AI startups rely on OpenAI’s GPT models, Nova AI takes a different approach. They use OpenAI’s Chat GPT-4 sparingly, mainly for code generation and labeling tasks. Smith explains that large enterprises do not trust OpenAI with their data, even if OpenAI promises not to use paid business plan data for training. OpenAI is currently facing lawsuits from individuals and organizations concerned about their work being used without authorization or compensation.
To avoid these concerns, Nova AI heavily relies on open source models like Llama developed by Meta and StarCoder from the BigCoder community. They also build their own models and have tested Google’s Gemma with positive results. By using open source models, Nova AI can address specific tasks without sending customer data to OpenAI.
Smith emphasizes that open source AI models are not only more trustworthy but also cost-effective for targeted tasks like code testing. While massive models like GPT-4 may have their uses, Nova AI’s models are fine-tuned for writing tests, which is their primary focus.
The open source AI model industry is advancing rapidly, with Meta’s new version of Llama gaining recognition in technology circles. This progress may encourage more AI startups to explore alternatives to OpenAI.
In conclusion, the field of generative AI startups focusing on code testing is growing rapidly. Nova AI stands out by targeting mid-size to large enterprises and prioritizing end-to-end testing tools for complex code-bases. They break from the norm by using open source models instead of relying heavily on OpenAI, addressing concerns about data privacy and cost-effectiveness. As open source models continue to evolve, more startups may consider these alternatives to meet specific needs.