As OpenAI’s multimodal API launches broadly, research shows it’s still flawed

Today during its first-ever dev conference, OpenAI released new details of a version of GPT-4, the company’s flagship text-generating AI model, that can understand the context of images as well as text. This version, which OpenAI calls “GPT-4 with vision,” can caption and even interpret relatively complex images — for example identifying a Lightning Cable adapter from a picture of a plugged-in iPhone.

GPT-4 with vision had previously only been available to select users of Be My Eyes, an app designed to help vision-impaired people navigate the world around them; subscribers to the premium tiers of OpenAI’s AI-powered chatbot, ChatGPT; and “red teamers” charged with probing GPT-4 with vision for signs of unintended behavior. That’s because OpenAI held back GPT-4 with vision after unveiling it in early March, reportedly on fears about how it might be abused — and violate privacy.

Now, OpenAI’s seemingly confident enough in its mitigations to let the wider dev community build GPT-4 with vision into their apps, products and services. GPT-4 with vision will become available in the coming weeks, the company said this morning, via the newly launched GPT-4 Turbo API.

The question is whether GPT-4 with vision’s actually safer than it was before, though.

In October, a few weeks before OpenAI began rolling out GPT-4 with vision to ChatGPT subscribers, the company published a whitepaper detailing the model’s limitations and more… questionable tendencies (e.g. discriminating against certain body types). But the paper was co-authored by OpenAI scientists — not outside testers who might bring a more impartial perspective to the table.

Luckily, OpenAI provided several researchers — the aforementioned red teamers — early access to GPT-4 with vision for evaluation purposes. At least two, Chris Callison-Burch, an associate professor of computer science at the University of Pennsylvania, and Alyssa Hwang, Callison-Burch’s PhD student, published their early impressions this afternoon at OpenAI’s conference.

A PR firm connected TechCrunch with Callison-Burch and Hwang via email.

“I experimented with GPT-4 with vision for a variety of tasks, from question-answering about images to having it help select 3D objects for scenes in video games to describing the compositions and artistic styles of fine art paintings,” Callison-Burch, who said he’s had access to GPT-4 with vision since July, told TechCrunch in an interview. “Each time, it nailed it. The descriptions are incredibly good, and are a clear advance over the previous state-of-the-art in image captioning.”

But Hwang, who conducted a more systematic review of GPT-4 with vision’s capabilities, found that the model remains flawed in several significant — and problematic, in some cases — ways.

“I discovered that GPT-4 with vision often correctly described the positions of elements [in an image] but was less successful with their structural or relative relationships,” Hwang told TechCrunch in an email. “For example, it once correctly said that two curves on a line graph leaned upward, but incorrectly said which one was higher than the other. And it made quite a few errors with graphs in general, from incorrectly estimating the values on a bar or line graph to misinterpreting the colors in a legend.”

Hwang documents many other instances of GPT-4 with vision making mistakes in a draft study published on the preprint server Arxiv.org. Her work focuses primarily on GPT-4 with vision’s ability to describe figures in academic papers, a potentially quite useful application of the tech — but one where accuracy matters. A lot.

Unfortunately, accuracy isn’t GPT-4 with vision’s strong suit where it concerns scientific interpretation.

Hwang writes that GPT-4 with vision makes errors when reproducing mathematical formulas, oftentimes leaving out subscripts or printing them incorrectly. Counting objects in illustrations poses another problem for the model, as does describing colors — particularly the colors of objects next to each other, which GPT-4 with vision sometimes mixes up.

Some of GPT-4 with vision’s more serious, broader shortcomings lie in the factual accuracy department.

GPT-4 with vision can’t reliably extract text from an image. To demonstrate, in the study, Hwang gave the model a spread with a list of recipes and asked it to copy down each recipe in writing. GPT-4 with vision made mistakes in parsing the recipe titles, writing things like “Eggs Red Velvet Cake” instead of “Eggless Red Velvet Cake” and “Sesame Pork Medallions” instead of “Sesame Pork Milanese.”

A related challenge for GPT-4 with vision is summarizing. When asked for the gist of, say, a scan of a document, GPT-4 with vision might poorly paraphrase sentences in that document — omitting information in the process. Or it might alter direct quotes in misleading ways, leaving out parts such that it affects the text’s meaning.

That’s not to suggest GPT-4 with vision is a total failure of a multimodal model. Hwang praises its analytical capabilities, noting that the model shines when asked to describe even fairly complicated scenes. It’s clear why OpenAI and Be My Eyes saw GPT-4 with vision as possibly useful for accessibility — it’s a natural fit.

But Hwang’s findings confirm what the OpenAI paper hinted at: that GPT-4 with vision remains a work in progress. Far from a universal problem solver, GPT-4 with vision makes basic mistakes that a human wouldn’t — and potentially introduces biases along the way.