Researchers from Stanford and UC Berkeley Introduce TokenGate: Token-Gated Attention Makes Long-Context — 2026-10-25

SPEAKER_1 (0:00)

Meta's Muse AI agent is now managing money for over 10 million users in the US and Canada. But should you trust it with your finances? We'll explore that, plus how open-weight DeepSeek is outperforming Google's Gemini model in bug detection and Anthropic's new OSS Scanner for open-source security. Also, Accenture and Google public sector's Gemini Launchpads are transforming federal missions, and OpenAI's ChatGPT Voice is simplifying Diwali preparations. Later, we'll dive into how Stanford and UC Berkeley's TokenGate is revolutionizing long-context processing.

Meta's Muse AI agent claims to manage your money, but should you let it? Meta's Muse AI agent, launched in September, is already managing finances for over 10 million users in the US and Canada. The AI assistant can handle tasks like sending emails and making phone calls, all while ensuring user privacy and security. However, financial experts urge caution, emphasizing the importance of human oversight in financial decisions. While Muse AI offers convenience, users should remain vigilant about its limitations and potential risks. As AI continues to evolve, the balance between automation and human judgment remains crucial.

Openweight DeepSeek outperforms Google's gated Gemini model in bug detection. DeepSeek, an openweight AI model, is making waves by outperforming Google's Gemini model in finding software bugs. While Google restricted access to its Gemini model due to cybersecurity concerns, DeepSeek is freely available and has proven more effective in bug detection. This development highlights the potential of openweight models to compete with proprietary systems, offering a cost-effective solution for developers seeking to enhance software security without relying on large, expensive AI models.

Anthropic launches OSS Scanner, a free AI vulnerability scanner for open-source projects. Anthropic has introduced OSS Scanner, a free tool designed to identify vulnerabilities in open-source projects. By leveraging its strongest AI models, Anthropic aims to provide thorough security scans without human review, allowing for faster detection of critical issues. While this approach accelerates the identification process, it places the responsibility of triaging findings on project maintainers. With 114 projects already enrolled, OSS Scanner is poised to enhance open-source security, though maintainers must be prepared to handle the increased workload. Accenture and Google Public Sector Launch Gemini Launchpads for Federal Missions Accenture Federal Services and Google Public Sector have unveiled Gemini Launchpads, a new initiative to accelerate AI adoption in Federal Missions. These Launchpads aim to transition agencies from experimentation to large-scale execution, offering pre-built agents for HR, finance and supply chain tasks. By deploying AI solutions tailored to Federal needs, the initiative seeks to enhance operational efficiency and mission success. As AI becomes integral to government operations, initiatives like Gemini Launchpads are crucial for modernizing public sector capabilities.

OpenAI's ChatGPT Voice simplifies Diwali preparations with personalized assistance. OpenAI is promoting ChatGPT Voice as a helpful tool for Diwali preparations, offering assistance with budgeting, shopping, and hosting family gatherings. Featuring Indian Voice options Neer and Leher, ChatGPT Voice aims to streamline festive planning by providing personalized suggestions and managing tasks. As households across India gear up for celebrations, this AI tool offers a convenient way to handle the myriad responsibilities that come with the festive season, making it easier for families to focus on enjoying the festivities.

Stanford and UC Berkeley's TokenGate is transforming long context processing by making it up to 11.3 times faster.

Researchers from Stanford and UC Berkeley have introduced TokenGate, a groundbreaking module that significantly accelerates long context processing in language models. By implementing learned token pruning, TokenGate allows models to efficiently skim through input, focusing only on the most relevant tokens. This innovation results in a time-to-first token, TTFT speedup, ranging from 1.4 times to 11.3 times, with no loss in accuracy. Additionally, it reduces peak pre-fill memory usage by up to 3.4 times. TokenGate's approach involves a small MLP gate and lower I, which are integrated into existing models like Qwen3-1.7B and Qwen3.5-2B.

This integration enables the models to handle contexts up to 128,000 tokens efficiently. The module's performance has been tested on various tasks, proving to be Pareto optimal against multiple baselines. The most significant gains were observed with NIAH 16k, achieving a 13 times TTFT improvement at 128k tokens. Unlike traditional transformers that process every token through every layer, TokenGate's learned token pruning offers a more efficient alternative without the need for retraining from scratch. This makes it a versatile drop-in component for enhancing existing language models. The code and checkpoints for TokenGate are available on GitHub under the Apache 2 license, making it accessible for developers and researchers to implement and experiment with. As AI models continue to evolve, innovations like TokenGate are crucial for managing the increasing complexity and size of data inputs. By optimizing processing efficiency, TokenGate not only speeds up computations, but also reduces the computational resources required, making it a valuable tool for developers working with large-scale language models. As the AI community explores new frontiers and model capabilities, TokenGate represents a significant step forward in balancing performance with resource efficiency.