Sakana AI and NVIDIA Introduce TwELL with CUDA Kernels for 20.5% Inference and 21.9% Training Speedup in LLMs
Researchers from Sakana AI and NVIDIA developed TwELL, a sparse tensor format that exploits activation sparsity in large language model feedforward layers. Using L1 regularization to induce over 99% sparsity with minimal accuracy impact, they created custom CUDA kernels that operate within existing matmul epilogues to achieve real GPU throughput gains. The innovation targets batched GEMM operations with thousands of tokens, covering both training and high-throughput inference regimes. Benchmarks show scaling benefits: 0.5B models see +17.0% inference speedup, while 2B models achieve +20.5% inference and +21.9% training speedup on H100 GPUs.
Primary source: arXiv:2603.23198 [Sparser, Faster, Lighter Transformer Language Models]
A Coding Implementation to Build Agent-Native Memory Infrastructure with Memori for Persistent Multi-User and Multi-Session LLM Applications
Memori provides an LLM-agnostic memory infrastructure layer that turns agent execution and conversation into structured, persistent state for production systems. The system integrates with existing software infrastructure without requiring changes to agent code or prompts, automatically capturing structured memory from conversation and agent execution after each turn. Key features include entity-based scoping, process_id for agent personas, session management for grouping related turns, and support for both synchronous and asynchronous LLM clients. Memori was evaluated on the LoCoMo benchmark, achieving 81.95% overall accuracy while using just 4.97% of the full-context footprint, demonstrating efficient structured memory preservation.
Primary source: GitHub repository for Memori agent-native memory infrastructure
Best Vector Databases in 2026: Pricing, Scale Limits, and Architecture Tradeoffs Across Nine Leading Systems
Vector databases have become mission-critical infrastructure for RAG pipelines, semantic search systems, and agentic AI workflows in 2026. The analysis compares nine production options across architecture, performance, pricing, and use cases, ranging from fully managed services like Pinecone and MongoDB Atlas Vector Search to open-source solutions like Milvus, Qdrant, Weaviate, and specialized libraries like Faiss. Key trends include GPU acceleration for billion-scale deployments (Milvus/Zilliz Cloud’s Cardinal engine), hybrid search capabilities (Weaviate), and serverless multimodal support (LanceDB). The guide emphasizes choosing based on existing infrastructure, scale requirements, and budget, with specific recommendations for PostgreSQL-native teams (pgvector), MongoDB users (Atlas Vector Search), and LLM-native prototyping (Chroma).
Primary source: Pinecone vector database platform
Primary source: Milvus open-source vector database
Primary source: Qdrant vector similarity search engine
Primary source: Weaviate open-source vector database
Primary source: pgvector PostgreSQL extension for vector similarity search
