Univ. of Oxford & Univ. College London present Craftax, a JAX-based RL benchmark outperforming others in speed. It offers Craftax-Classic, solvable by a basic PPO agent in 51 mins, encouraging higher timesteps gain. Despite disappointing existing approaches, Craftax aims to facilitate RL research with limited resources. Craftax-Classic serves as an entry point for Crafter users.

StarCoder2, an advanced code generation model, derives from the BigCode project, led by researchers from 30+ institutions. Trained on a vast dataset including GitHub repositories, it offers models of varying sizes (3B, 7B, 15B) with exceptional performance in code generation. The project prioritizes transparency, releasing model weights and training data details to encourage collaboration and…

Intersection of AI and arts, particularly music, is a significant study due to its impact on human creativity, with researchers focusing on creating music through language models. Skywork AI and Hong Kong University developed ChatMusician, outperforming GPT-4, but facing challenges in music variety and open-ended tasks. The open-source project aims to spur cooperation in this…

Salesforce AI Researchers introduced the SFR-Embedding-Mistral model to improve text-embedding models for natural language processing (NLP) tasks. It leverages multi-task training, task-homogeneous batching, and hard negatives to enhance performance significantly, particularly in retrieval tasks. The model demonstrates state-of-the-art results across diverse NLP benchmarks.

The emergence of Large Language Models (LLMs) like GPT and LLaMA has prompted a growing need for proprietary LLMs, but their resource-intensive development remains a challenge. FUSECHAT, a novel chat-based LLM integration approach, leverages knowledge fusion techniques and the VARM merging method to outperform individual models and fine-tuned baselines. It offers a practical and efficient…

A novel framework called CyberDemo is introduced to address the challenges in robotic manipulation. It leverages simulated human demonstrations, remote data collection, and simulator-exclusive data augmentation to enhance task performance and surpass the limitations of real-world data. CyberDemo demonstrates significant improvements in manipulation tasks and outperforms traditional methods, showcasing the untapped potential of simulation data.

The integration of advanced technological tools is increasingly essential in urban planning, particularly with the emergence of specialized large language models like PlanGPT. Developed by researchers, PlanGPT offers a customized solution for urban and spatial planning, outperforming existing models by improving precision and relevance in tasks essential for urban planning professionals.

Recent advancements in AI and deep learning have led to significant progress in generative modeling. Autoregressive and diffusion models have limitations in text generation, but the new SEDD model challenges these, offering high-quality and controlled text production. It competes with autoregressive models like GPT-2, showing promise in NLP generative modeling. [50 words]

Cancer therapy is a constantly evolving field, aiming to improve patient outcomes through innovative treatments. Off-label and off-guideline usage plays a significant role, providing alternative pathways for patients. A recent study by Stanford University, Genentech, and the University of Southern California analyzes real-world data to reveal insights into unconventional cancer treatments, highlighting the potential for…

PDETime, a new approach to long-term multivariate time series forecasting, reimagines the problem by treating the data as spatiotemporal phenomena sampled from continuous dynamical systems. It outperforms traditional models, incorporating spatial and temporal information through a PDE-based framework and achieving superior predictive accuracy. This research represents a significant advancement in forecasting.