Earlier this year, GitHub rolled out Copilot Chat, a ChatGPT-like programming-centric chatbot for organizations subscribed to Copilot for Business. Copilot Chat more recently came to individual Copilot customers — those paying $10 per month — in beta. And now, GitHub’s launching Chat in general availability for all users.
As of today, Copilot Chat is available in the sidebar in Microsoft’s IDEs, Visual Studio Code and Visual Studio — included as a part of GitHub Copilot paid tiers and free for verified teachers, students and maintainers of certain open source projects.
“As home to the world’s developers, we’ve brought to market what is now the most widely adopted AI developer tool in history,” Shuyin Zhao, VP of product management at GitHub, told TechCrunch in an email interview. “And code complete was just the beginning.”
Little else about Copilot Chat has changed since the beta.
The chatbot’s still powered by GPT-4, OpenAI’s flagship generative AI model, fine-tuned specifically for dev scenarios. Developers can prompt Copilot Chat in natural language to get real-time guidance, for example asking Copilot Chat to explain concepts, detect vulnerabilities or write unit tests.
Like all generative AI models, the model underpinning Copilot Chat, GPT-4, was trained on publicly available data — some of which is copyrighted or under a restrictive license. Vendors including, GitHub, argue fair use doctrine shields them from copyright claims. But that hasn’t stopped coders from filing class action lawsuits against GitHub, Microsoft (GiHub’s parent company) and OpenAI over what they allege are open source licensing and IP violations.
I asked Zhao whether codebase owners will have a chance to opt out of training, now, in the event that they wish to do so. She said that there’s no new mechanism for this with the broader launch of Copilot Chat and instead suggested that codebase owners make their repositories private to prevent them from being included in future training sets.
I have to imagine codebase owners won’t take too kindly to that suggestion — there are many reasons for keeping copyrighted code public, least of which is crowdsourcing bug hunting. But GitHub’s evidently not willing to budge on training data opt-outs — or not yet, at least.
Generative AI models, including GPT-4, also have a tendency to hallucinate, or confidently make up facts — which is particularly problematic in the coding realm. According to a recent Stanford study, developers who use AI assistants to code tend to produce code that’s less secure compared to those who don’t use AI assistants, in part because the AI assistants introduce buggy or deprecated code snippets.
Zhao said that GPT-4 performs “better” against hallucinations compared to the older model that once powered Copilot and pointed to exploit-mitigating features such as filters for insecure code patterns, which notify Copilot Chat users of vulnerabilities like hardcoded credentials, SQL injections and path injections. But she stressed the importance of close human review of any AI-suggested code.
“GitHub Copilot is powered by OpenAI’s models, which we’ve found to be the best models for the services we offer today,” Zhao said. “We’re in a really strong position to continue empowering developers with the AI tools they need to build better, more secure software at scale — and to have fun while they’re doing it.”
In October, Microsoft CEO Satya Nadella told analysts that Copilot had 1 million paying users and ~37,000 enterprise clients. But it’s incumbent on GitHub to make Copilot even more attractive lest it lose ground to competitors — and, for that matter, lose cash.
According to a Wall Street Journal piece, Copilot loses an average of $20 a month per user, with some customers costing GitHub as much as $80 a month. The high price of running the underlying AI models is reportedly to blame — a problem GenAI coding startup Kite ran into, also, forcing it to shut down early last December.
As GitHub struggles to make Copilot profitable, Amazon continues upgrading CodeWhisperer, perhaps Copilot’s best-resourced rival.
In April, Amazon made CodeWhisperer free of charge to developers without any usage restrictions. That month also saw the launch of CodeWhisperer Professional Tier, which added single sign-on with AWS Identity and Access Management integration as well as higher limits on scanning for security vulnerabilities. An enterprise plan for CodeWhisperer launched in September. And in early November, Amazon “optimized” CodeWhisperer to provide “enhanced” suggestions for app development on MongoDB, the open source database management program.
Aside from CodeWhisperer, Copilot has competition in startups like Magic, Tabnine, Codegen and Laredo, as well as open source models like Meta’s Code Llama and Hugging Face’s and ServiceNow’s StarCoder.
Comment