Large language models (LLMs) aligning with human expectations is crucial for societal benefits. Reinforcement learning from human feedback (RLHF) and direct alignment from preferences (DAP) are approaches discussed. A new study introduces Online AI Feedback (OAIF) for DAP, combining online flexibility and efficiency. Empirical comparisons demonstrate OAIF’s effectiveness, especially in aligning LLMs online.

This research from UC Berkeley analyzes the evolving role of large language models (LLMs) in the digital ecosystem, highlighting the complexities of in-context reward hacking (ICRH). It discusses the limitations of static benchmarks in understanding LLM behavior and proposes dynamic evaluation recommendations to anticipate and mitigate risks. The study aims to enhance the development of…

Infographics and user interfaces share design concepts and visual languages. To address the complexity of each, Google Research introduced ScreenAI, a Vision-Language Model (VLM) capable of comprehending UIs and infographics. ScreenAI achieved remarkable performance on various tasks and released three new datasets to advance the field. Learn more in the research paper.

Large Language Models (LLMs) such as GPT, PaLM, and LLaMa have enhanced AI and NLP by enabling machines to comprehend and produce human-like content. Finetuning is crucial to adapt these generalist models to specialized activities. Approaches include Parameter Efficient Fine Tuning (PEFT), Supervised Finetuning with hyperparameter tweaking, transfer learning, and few-shot learning, and Reinforcement Learning…

This survey explores the burgeoning field of prompt engineering, which leverages task-specific instructions to enhance the adaptability and performance of language and vision models. Researchers present a systematic overview of over 29 techniques, categorizing advancements by application area and emphasizing the transformative impact of prompt engineering on model capabilities. Despite notable successes, challenges such as…

Studying scaling laws in large language models is crucial for optimizing their performance in tasks like translation. Challenges include determining the impact of pretraining data size on downstream tasks and developing strategies to enhance model performance. New scaling laws by researchers predict translation quality based on pretraining data size, offering insights for effective model training…

Reinforcement learning encompasses model-based (MB) and model-free (MF) algorithms. The Diffusion World Model (DWM) is a novel approach addressing inaccuracies in world modeling. DWM predicts long-horizon outcomes and enhances RL performance. By combining MB and MF strengths, DWM achieves state-of-the-art results, bridging the gap between the two approaches. This new framework presents promising advancements in…

CodeCompose, utilized by Meta developers, enhanced its AI-powered code authoring tool to provide multiline suggestions. The transition addressed challenges such as workflow disruption and latency concerns. Model-hosting optimizations improved multiline suggestion latency by 2.5 times, with significant productivity gains. Despite minor opt-outs, multiline suggestions have proven effective, aiding code completion and discovery.

Researchers have introduced the Listwise Preference Optimization (LiPO) framework, reshaping language model alignment as a listwise ranking challenge. LiPO-λ emerges as a powerful tool leveraging listwise data to enhance alignment, bridging LM preference optimization and Learning-to-Rank, setting new benchmarks, and driving future research. This approach signals a new era of language model development. [45 words]

Adobe introduces AI Assistant in Adobe Acrobat, a generative AI technology integrated into document workflows. This powerful tool offers productivity benefits for a wide range of users, from project managers to students. Adobe emphasizes responsible AI development and outlines a vision for future AI-powered document experiences, including intelligent creation and collaboration support.