Anthropic’s Post

Introducing Claude 3.5 Sonnet—our most intelligent model yet. This is the first release in our 3.5 model family. Claude 3.5 Sonnet now outperforms competitor models on key evaluations, at twice the speed of Claude 3 Opus and one-fifth the cost. Try it for free: https://claude.ai. Claude 3.5 Sonnet is now our strongest vision model. Sonnet now surpasses Claude 3 Opus across all standard vision benchmarks. Improvements are most noticeable in tasks requiring visual reasoning, like interpreting charts, graphs, or transcribing text from imperfect images. Claude 3.5 Sonnet sets new industry benchmarks for graduate-level reasoning (GPQA), undergraduate-level knowledge (MMLU), and coding proficiency (HumanEval). It shows marked improvement in grasping nuance, humor, and complex instructions, all while writing with a natural tone. Claude 3.5 Sonnet is available for free on claude.ai and the Claude iOS app. Claude Pro and Team subscribers benefit from significantly higher rate limits. Sonnet is also available via the Anthropic API, Amazon Bedrock, and Google Cloud's Vertex AI. To complete the Claude 3.5 model family, we'll be releasing Claude 3.5 Haiku and Claude 3.5 Opus later this year. In addition, we're developing new modalities and features for businesses, alongside rigorous safety testing. Read more: https://lnkd.in/eddane8F

You can now fine-tune Claude 3 Haiku—our fastest and most cost-effective model—in Amazon Bedrock: https://lnkd.in/e8NX_F-g. In testing, we fine-tuned Haiku to moderate comments on internet forums. Fine-tuning improved classification accuracy from 81.5% to 99.6% and reduced tokens per query by 89%. Early customers, like SK Telecom, have used fine-tuning to create custom Claude 3 models. These models deliver more effective responses across a range of use cases, from customer support to legal operations. Fine-tuning is currently available for Claude 3 Haiku in preview.

Artifacts made with Claude can now be published and shared. You can also remix Artifacts shared by others. This is an experimental feature. Remix an example Artifact to try it out: https://lnkd.in/eND9zqxp. You can ask Claude to generate docs, code, mermaid diagrams, vector graphics, or even simple games. Artifacts appear next to your chat, letting you see, iterate, and build on your creations in real-time. We look forward to seeing what you create: https://claude.ai.

We've added new features to the Anthropic Console: https://lnkd.in/ertrbfcU. Claude can generate prompts, create test variables, and show you the outputs of prompts side by side. Use Claude to generate input variables for your prompt. Then run the prompt to see Claude’s response. The new Evaluate tab enables you to automatically create test cases to evaluate your prompt against real-world inputs. Modify your test cases as needed, then run all of them in one click. We’ve also added the ability to compare the outputs of two or more prompts side by side. As you iterate on different versions of the prompt, your subject matter experts can compare responses and grade them on a 5-point scale. Test case generation and output comparison features are available to all users today.

Claude 3.5 Sonnet shows marked improvement in grasping nuance, humor, and complex instructions, all while writing with a natural tone. Pen some new ideas today: https://claude.ai. Sonnet also sets new industry benchmarks for graduate-level reasoning (GPQA), undergraduate-level knowledge (MMLU), and coding proficiency (HumanEval).

Claude 3.5 Sonnet—our most intelligent model yet—is also our strongest vision model. Claude 3.5 Sonnet surpasses Claude 3 Opus across all standard vision benchmarks. All this, available to try for free: https://claude.ai. Improvements are most noticeable in tasks requiring visual reasoning, like interpreting charts, graphs, or transcribing text from imperfect images.

We're introducing a feature preview of Artifacts on claude.ai. This is in addition to the launch of Claude 3.5 Sonnet—our most intelligent model yet. You can now ask Claude to generate docs, code, mermaid diagrams, vector graphics, or even simple games. Artifacts appear in a dedicated window next to your chat, letting you see, iterate, and build on your creations in real-time. We look forward to seeing what you create with Claude: https://claude.ai

Today, we're sharing a sample of red teaming methods we’ve used to test our AI systems. We detail challenges, findings, and the need to work towards common industry standards: https://lnkd.in/eR-6jd7Y

Today, we're publishing details about the processes we use to test and mitigate elections-related risks. We're also sharing samples of the evaluations we use to test our models: https://lnkd.in/eQc5gSCQ