Large Language Models (LLMs) have achieved remarkable advancements in natural language processing (NLP), enabling applications in text generation, summarization, and question-answering. However, their reliance on token-level processing—predicting one word at a time—presents challenges. This approach contrasts with human communication, which often operates at higher levels of abstraction, such as sentences or ideas. Token-level modeling also struggles with tasks requiring long-context understanding and may produce outputs with inconsistencies. Moreover, extending these models to multilingual and multimodal applications is computationally expensive and data-intensive. To address these issues, researchers at Meta AI have proposed a new approach: Large Concept Models (LCMs).

Large Concept Models

Meta AI’s Large Concept Models (LCMs) represent a shift from traditional LLM architectures. LCMs bring two significant innovations:

1. High-dimensional Embedding Space Modeling: Instead of operating on discrete tokens, LCMs perform computations in a high-dimensional embedding space. This space represents abstract units of meaning, referred to as concepts, which correspond to sentences or utterances. The embedding space, called SONAR, is designed to be language- and modality-agnostic, supporting over 200 languages and multiple modalities, including text and speech.

2. Language- and Modality-agnostic Modeling: Unlike models tied to specific languages or modalities, LCMs process and generate content at a purely semantic level. This design allows seamless transitions across languages and modalities, enabling strong zero-shot generalization.

At the core of LCMs are concept encoders and decoders that map input sentences into SONAR’s embedding space and decode embeddings back into natural language or other modalities. These components are frozen, ensuring modularity and ease of extension to new languages or modalities without retraining the entire model.

Technical Details and Benefits of LCMs

LCMs introduce several innovations to advance language modeling:

- Hierarchical Architecture: LCMs employ a hierarchical structure, mirroring human reasoning processes. This design improves coherence in long-form content and enables localized edits without disrupting broader context.

- Diffusion-based Generation: Diffusion models were identified as the most effective design for LCMs. These models predict the next SONAR embedding based on preceding embeddings. Two architectures were explored:

- One-Tower: A single Transformer decoder handles both context encoding and denoising.

- Two-Tower: Separates context encoding and denoising, with dedicated components for each task.

- Scalability and Efficiency: Concept-level modeling reduces sequence length compared to token-level processing, addressing the quadratic complexity of standard Transformers and enabling more efficient handling of long contexts.

- Zero-shot Generalization: LCMs exhibit strong zero-shot generalization, performing well on unseen languages and modalities by leveraging SONAR’s extensive multilingual and multimodal support.

- Search and Stopping Criteria: A search algorithm with a stopping criterion based on distance to an “end of document” concept ensures coherent and complete generation without requiring fine-tuning.

Insights from Experimental Results

Meta AI’s experiments highlight the potential of LCMs. A diffusion-based Two-Tower LCM scaled to 7 billion parameters demonstrated competitive performance in tasks like summarization. Key results include:

- Multilingual Summarization: LCMs outperformed baseline models in zero-shot summarization across multiple languages, showcasing their adaptability.

- Summary Expansion Task: This novel evaluation task demonstrated the capability of LCMs to generate expanded summaries with coherence and consistency.

- Efficiency and Accuracy: LCMs processed shorter sequences more efficiently than token-based models while maintaining accuracy. Metrics such as mutual information and contrastive accuracy showed significant improvement, as detailed in the study’s results.

Conclusion

Meta AI’s Large Concept Models present a promising alternative to traditional token-based language models. By leveraging high-dimensional concept embeddings and modality-agnostic processing, LCMs address key limitations of existing approaches. Their hierarchical architecture enhances coherence and efficiency, while their strong zero-shot generalization expands their applicability to diverse languages and modalities. As research into this architecture continues, LCMs have the potential to redefine the capabilities of language models, offering a more scalable and adaptable approach to AI-driven communication.

Today, we're open-sourcing the Model Context Protocol (MCP), a new standard for connecting AI assistants to the systems where data lives, including content repositories, business tools, and development environments. Its aim is to help frontier models produce better, more relevant responses.

As AI assistants gain mainstream adoption, the industry has invested heavily in model capabilities, achieving rapid advances in reasoning and quality. Yet even the most sophisticated models are constrained by their isolation from data—trapped behind information silos and legacy systems. Every new data source requires its own custom implementation, making truly connected systems difficult to scale.

MCP addresses this challenge. It provides a universal, open standard for connecting AI systems with data sources, replacing fragmented integrations with a single protocol. The result is a simpler, more reliable way to give AI systems access to the data they need.

The Model Context Protocol is an open standard that enables developers to build secure, two-way connections between their data sources and AI-powered tools. The architecture is straightforward: developers can either expose their data through MCP servers or build AI applications (MCP clients) that connect to these servers.

Today, we're introducing three major components of the Model Context Protocol for developers:

- The Model Context Protocol specification and SDKs

- Local MCP server support in the Claude Desktop apps

- An open-source repository of MCP servers

Claude 3.5 Sonnet is adept at quickly building MCP server implementations, making it easy for organizations and individuals to rapidly connect their most important datasets with a range of AI-powered tools. To help developers start exploring, we’re sharing pre-built MCP servers for popular enterprise systems like Google Drive, Slack, GitHub, Git, Postgres, and Puppeteer.

Early adopters like Block and Apollo have integrated MCP into their systems, while development tools companies including Zed, Replit, Codeium, and Sourcegraph are working with MCP to enhance their platforms—enabling AI agents to better retrieve relevant information to further understand the context around a coding task and produce more nuanced and functional code with fewer attempts.

"At Block, open source is more than a development model—it’s the foundation of our work and a commitment to creating technology that drives meaningful change and serves as a public good for all,” said Dhanji R. Prasanna, Chief Technology Officer at Block. “Open technologies like the Model Context Protocol are the bridges that connect AI to real-world applications, ensuring innovation is accessible, transparent, and rooted in collaboration. We are excited to partner on a protocol and use it to build agentic systems, which remove the burden of the mechanical so people can focus on the creative.”

Instead of maintaining separate connectors for each data source, developers can now build against a standard protocol. As the ecosystem matures, AI systems will maintain context as they move between different tools and datasets, replacing today's fragmented integrations with a more sustainable architecture.

Getting started

Developers can start building and testing MCP connectors today. Existing Claude for Work customers can begin testing MCP servers locally, connecting Claude to internal systems and datasets. We'll soon provide developer toolkits for deploying remote production MCP servers that can serve your entire Claude for Work organization.

To start building:

- Install pre-built MCP servers through the Claude Desktop app

- Follow our quickstart guide to build your first MCP server

- Contribute to our open-source repositories of connectors and implementations

An open community

We’re committed to building MCP as a collaborative, open-source project and ecosystem, and we’re eager to hear your feedback. Whether you’re an AI tool developer, an enterprise looking to leverage existing data, or an early adopter exploring the frontier, we invite you to build the future of context-aware AI together.

With Gemini, users can play music by specifying song titles, artist names, album names, playlist names, or even activities. However, it currently does not support creating playlists or radio stations on Spotify. Notably, if another music service, such as YouTube Music, is already linked, users must specify their desired service in their requests, as Gemini will default to the last used service.

To utilize this extension, users must link their Spotify and Google accounts and enable Gemini Apps Activity, which retains AI queries for up to 72 hours. It is important to note that the Spotify extension is not available within Google Messages, the Gemini web app, or the Gemini app on iOS. Additionally, the functionality is limited to English language settings for Gemini.

The newly established “Scaled Solutions” division aims to connect businesses with “nuanced analysts, testers, and independent data operators” through its platform. This division builds on an existing internal team based in the US and India, which has been responsible for feature testing and converting restaurant menus for Uber Eats.

Uber has already been employing artificial intelligence and machine learning for its operations and is now making these capabilities available to external clients for a fee. The company is recruiting gig workers for tasks such as data labeling, testing, and localization for various companies, including Aurora, Luma AI, and Niantic.

Training AI models necessitates the involvement of numerous human workers to perform repetitive tasks. A significant aspect of AI model training involves tedious activities such as selecting the most human-like chatbot responses or labeling obstacles, like pedestrians, in self-driving car footage on a frame-by-frame basis. Companies developing AI models frequently hire workers from developing countries to carry out these tasks, compensating them with minimal payments for each completed activity. For instance, an engineer in India shared with Bloomberg that they were responsible for evaluating and rating the accuracy of AI-generated responses to complex coding problems, earning approximately 200 rupees per set, equivalent to about $2.37.

Currently, Uber is onboarding individuals from Canada, India, Poland, Nicaragua, and the US, offering differing pay rates per task completed, with earnings distributed to workers on a monthly basis. The company is also seeking individuals from diverse cultural backgrounds to enhance AI adaptability across various markets.

This initiative is not Uber's first venture into the AI space. The company previously invested billions in developing its own self-driving car technology but halted the project following a tragic incident involving one of its vehicles and a pedestrian. Additionally, in 2016, Uber acquired an AI research lab founded by cognitive scientist Gary Marcus and several other computer science professors.

Anthropic is adding custom styles to Claude that quickly adjust the tone and length of responses. Anthropic is adding a new feature to its Claude AI assistant that will give users more control over how the chatbot responds to different writing tasks. The new custom styles are available to all Claude AI users, enabling anyone to train it to match their own communication style or select from preset options to quickly adjust the tone and level of detail it provides.

This update aims to personalize the chatbot’s replies and make them feel more natural or appropriate for specific applications, such as writing detailed technical documents or professional emails. Three preset styles are available: Formal for “clear and polished” text, Concise for shorter and more direct responses, and Explanatory for educational replies that need to include additional detail. If these don’t suit your requirements, Claude can also generate custom styles that are trained to mimic other writing mannerisms. Anthropic says users need to upload “sample content that reflects your preferred way of communicating” to the chatbot, and then instruct it on how to match the writing style.

Creating a custom style is effectively an easy way to automate how you engineer prompts to make responses sound more like your own personal style. “You might want in-depth explanations when you’re learning something new, or quick, straight-to-the-point answers when you’re in a hurry,” Claude’s Product leader Scott White said in the announcement. “You might prefer Claude to be more formal in some contexts, or use a friendly, conversational tone in others. The great thing is, you can now set these preferences once and have every interaction feel just right for you.”

It’s a welcome update that may make it less obvious when you’re using Claude to write different kinds of text, but the feature is not unique to Anthropic’s chatbot. OpenAI’s ChatGPT and Google’s Gemini have similar features that allow users to tailor responses based on their personal style of writing, for example, and Gemini can quickly adjust the tone or detail of Gmail drafts. The Writing Tools feature in Apple Intelligence also provides similar style presets.

Subscribe now