LLMs Research

LLM Research Highlights: March 1-15, 2025 [ Part 2/2 ]

Sat, 22 Mar 2025 13:00:00 +0000

🔑 takeaway from today’s newsletter

Chain of Draft (CoD) cuts LLM token usage by up to 92.4% while retaining 91% accuracy on GSM8k.
LADDER boosts a 3B LLM’s math accuracy from 1% to 82% via self-generated problem variants.
LMM-R1 enhances 3B multimodal LLMs, improving MathVerse scores by 4.83% with rule-based RL.
SoRFT-Qwen-7B resolves 21.4% of software issues, outperforming larger models on SWE-Bench.
TOKENSWIFT speeds up 100K-token generation by 3×, reducing LLaMA3.1-8B time from 5 hours to 90 minutes.

Reasoning and context length improvement of large language models

Reasoning

Chain of Draft: Thinking Faster by Writing Less

Paper: https://arxiv.org/abs/2502.18600
Authors: Silei Xu et al. (Zoom Communications)
Focus: Reducing verbosity in LLM reasoning while maintaining accuracy
Code: https://github.com/sileix/chain-of-draft

This research paper introduces a novel prompting strategy, Chain of Draft (CoD), designed to make LLMs reason more efficiently. Unlike the verbose Chain-of-Thought (CoT) prompting, CoD mimics human shorthand by encouraging concise, essential intermediate reasoning steps, cutting down on token usage and latency.

Key Innovations:

Minimalistic Reasoning: CoD prompts LLMs to produce brief, information-dense drafts at each step, reducing unnecessary elaboration.

Efficiency Optimization: It achieves CoT-level accuracy with significantly fewer tokens, enhancing inference speed and lowering computational costs.

Results: Evaluated on benchmarks like GSM8k, date understanding, and coin flipping, CoD reduces token usage by up to 92.4% compared to CoT. For instance, on GSM8k, it maintains over 91% accuracy while cutting output tokens by 80% (e.g., from 205.1 to 43.9 for GPT-4o). Latency drops by 76.2% for GPT-4o and 48.4% for Claude 3.5 Sonnet, making CoD ideal for real-time applications without compromising reasoning quality.

LADDER: Self-Improving LLMs Through Recursive Problem Decomposition

Paper: https://arxiv.org/abs/2503.00735
Authors: Toby Simonds et al. (Tufa Labs)
Focus: Enabling LLMs to autonomously enhance reasoning via self-generated problem variants

This paper presents a framework called LADDER, that allows LLMs to self-improve by recursively breaking down complex problems into simpler variants. This autonomous learning process, guided by reinforcement learning (RL) and numerical verification, eliminates the need for human supervision or curated datasets.

Key Innovations:

Recursive Variant Generation: LLMs create a tree of progressively simpler problem versions, forming a difficulty gradient for incremental learning.

Test-Time Reinforcement Learning (TTRL): At inference, TTRL refines solutions by applying RL to problem-specific variants, boosting performance dynamically.

Results: On mathematical integration tasks, LADDER improves a Llama 3.2 3B model’s accuracy from 1% to 82% on undergraduate-level problems. A Qwen2.5 7B Deepseek-R1 Distilled model achieves 73% on the 2025 MIT Integration Bee qualifying exam, surpassing GPT-4o (42%). With TTRL, accuracy rises to 90%, outpacing OpenAI’s o1 (80%), demonstrating the power of self-directed learning for complex reasoning.

LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL

Paper: https://arxiv.org/abs/2503.07536
Authors: Yingzhe Peng et al. (Southeast University, Ant Group, et al.)
Focus: Enhancing reasoning in compact 3B-parameter multimodal LLMs
Code: https://github.com/TideDra/lmm-r1

"LMM-R1" proposes a two-stage rule-based RL framework to boost reasoning in 3B-parameter large multimodal models (LMMs), overcoming their limited capacity and the scarcity of high-quality multimodal reasoning data. It first strengthens foundational reasoning with text-only data, then generalizes it to multimodal contexts.

How: "LMM-R1" proposes two stage RL which is a foundational reasoning enhancement (FRE) uses text-only data to build reasoning skills, followed by multimodal generalization training (MGT) for multimodal application. It also proposes a rule based rewards which combines format and accuracy rewards to guide learning efficiently, avoiding extensive human annotation.

Results: Applied to Qwen2.5-VL-Instruct-3B, LMM-R1 achieves average gains of 4.83% on multimodal benchmarks (e.g., MathVerse: 34.64% to 41.55%) and 4.5% on text-only benchmarks (e.g., MATH500: 63.4% to 65.8%). On the complex Football Game task, it improves by 3.63% (15.36 to 18.99), outperforming larger models like GPT-4o (21.20). This approach proves text-based reasoning can effectively transfer to multimodal domains, offering a cost-efficient training strategy.

Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching

Paper: https://arxiv.org/abs/2503.05179
Authors: Not specified (PDF unavailable)
Focus: Improving LLM reasoning efficiency with cognitive-inspired sketching

This paper introduces a method, Sketch-of-Thought, to enhance reasoning efficiency in LLMs. Drawing from cognitive processes, it uses adaptive sketching to streamline problem-solving, though detailed insights are limited due to the unavailable PDF.

How: This paper proposes adaptive sketching. It likely employs concise, cognitive-inspired sketches to guide reasoning, reducing computational overhead. Also, it aims to maintain accuracy while speeding up inference, though specifics are unclear without the full paper.

Results: Without access to the paper, specific results cannot be detailed. Based on the abstract, it likely demonstrates improved reasoning efficiency across various tasks, aligning with the trend of optimizing LLM performance for practical use.

Context length improvement

SoRFT: Issue Resolving with Subtask-oriented Reinforced Fine-Tuning

Paper: https://arxiv.org/abs/2502.20127
Authors: Zexiong Ma et al. (Peking University & ByteDance)
Focus: Enhancing context handling for issue resolving in software development

While not primarily focused on context length extension, SoRFT indirectly improves LLMs’ ability to manage complex, long-context tasks like software issue resolving. Targeting open-source models, it addresses cost and privacy concerns of commercial APIs by decomposing issue resolution into manageable subtasks: file localization, function localization, line localization, and code edit generation.

Two-Stage Training:

Rejection-Sampled Supervised Fine-Tuning (SFT): Filters Chain-of-Thought (CoT) data using ground-truth from pull requests, teaching the model structured reasoning.
Rule-Based Reinforcement Learning (RL): Employs Proximal Policy Optimization (PPO) with rewards based on β scores (prioritizing recall), refining the model’s precision across subtasks.
Context Relevance: By breaking down repository-scale tasks, SoRFT enables LLMs to process extensive codebases effectively, leveraging long-context data from open-source projects.

Results: On SWE-Bench Verified, SoRFT-Qwen-7B resolves 21.4% of issues, outperforming larger models like SWE-Gym-Qwen-32B (20.6%). It also boosts general code tasks (e.g., 90% on RepoQA vs. 85% baseline), showcasing enhanced long-context comprehension and generalization.

From Hours to Minutes: Lossless Acceleration of Ultra Long Sequence Generation up to 100K Tokens

Authors: Tong Wu et al. (Shanghai Jiao Tong University & BIGAI)
Focus: Accelerating ultra-long sequence generation
Paper: https://arxiv.org/abs/2502.18890

TOKENSWIFT, introduced in this paper, revolutionizes the generation of ultra-long sequences (up to 100K tokens) by slashing processing time without compromising quality. Traditional autoregressive (AR) methods take hours (e.g., 5 hours for LLaMA3.1-8B), a bottleneck for applications like creative writing or reasoning traces.

The authors identify three hurdles:

Frequent Model Reloading: Slows generation due to I/O bottlenecks.
Prolonged KV Cache Growth: Increases complexity as sequences lengthen.
Repetitive Content: Degrades quality in long outputs.

TOKENSWIFT counters these with:

Multi-Token Generation & Token Reutilization: Generates multiple tokens per forward pass and reuses frequent n-grams, reducing reloads.
Dynamic KV Cache Management: Updates partial caches iteratively, maintaining efficiency.
Contextual Penalty: Mitigates repetition, enhancing diversity (Distinct-n scores improve).

Results: TOKENSWIFT achieves over 3× speedup (e.g., 90 minutes vs. 5 hours for 100K tokens on LLaMA3.1-8B), with lossless accuracy across models (1.5B to 14B). It saves up to 5.54 hours on a 14B model, making ultra-long generation practical.

Explore Past Highlights

If you liked what you read and interested in reading more than here are our achieved newsletters we sent in past.

LLM Research Highlights: March 1-15, 2025

Fri, 21 Mar 2025 10:30:00 +0000

🔑 takeaway from today’s newsletter

Performance Boosts: Forgetting Transformer, Multi-Attempt RL, and R1-Searcher improve efficiency, math accuracy, and search with selective memory, feedback, and RL.
Simplified Design: Normalization-Free Transformers speed up training and inference using Dynamic Tanh in a streamlined architecture.
Data Optimization: RDS+ enhances instruction tuning, achieving top performance with only 6% of the data pool.
Memory Efficiency: Q-Filters and RSQ optimize long-context handling and quantization by compressing KV Cache and prioritizing key tokens.
Compression & Fairness: TinyR1-32B-Preview and Group-Robust Unlearning deliver high accuracy and equitable data removal via distillation and unlearning techniques.

LLM Research Highlights: March 1-15, 2025

Core research

Forgetting Transformer: Softmax Attention with a Forget Gate

Paper: https://arxiv.org/abs/2503.02130
Authors: Zhixuan Lin et al. (Mila & Université de Montréal)
Focus: Enhancing Transformer performance with a data-dependent forgetting mechanism
Code: https://github.com/zhixuan-lin/forgetting-transformer

The Forgetting Transformer (FoX) addresses a key limitation in standard Transformers: their lack of an explicit mechanism to selectively forget past information, a feature common in recurrent sequence models via forget gates. While Transformers excel at long-context tasks, they often retain irrelevant details, impacting efficiency and performance on both short and long sequences. FoX introduces a novel approach by integrating a forget gate into softmax attention.

Key contribution:

Forgetting Attention: A scalar forget gate, computed as f_t=σ(w_f^⊤x_t+b_f), down-weights unnormalized attention scores in a data-dependent manner. This is applied as F_ij=∏ⁱ_l=j+1f_l, allowing the model to prioritize relevant context dynamically.

Pro Block Design: An enhanced architecture incorporating recurrent-inspired components like output gates and token shifts, boosting performance across tasks.

Results: Evaluated on LongCrawl64 with a 16K-token context, FoX outperforms the baseline Transformer on long-context language modeling (e.g., lower per-token loss), length extrapolation, and short-context downstream tasks (e.g., 50.85% avg. accuracy on LM-eval-harness vs. 50.79% for Transformer-Pro). It matches Transformer performance on long-context downstream tasks (LongBench) while requiring no positional embeddings and remaining compatible with FlashAttention. FoX’s ability to retain long-context retrieval (near-perfect needle-in-the-haystack scores) while improving efficiency marks a significant step forward for LLMs.

Learning from Failures in Multi-Attempt Reinforcement Learning

Paper: https://arxiv.org/abs/2503.04808
Authors: Stephen Chung et al. (DualityRL & Shanghai AI Lab)
Focus: Boosting LLM reasoning through multi-attempt training with feedback

This paper extends reinforcement learning (RL) for LLMs by shifting from single-turn question-answering to a multi-attempt framework, where models refine responses based on feedback after incorrect attempts. Inspired by DeepSeek R1, the authors argue that allowing multiple tries enhances reasoning by encouraging self-refinement, a capability often absent in single-turn trained models.

Key Innovations:

Multi-Attempt Task: The model gets N attempts (sampled from 1 to 5), with a transition function terminating the dialogue on a correct answer or exhausted attempts. Feedback prompts refinement after errors.

Reward Design: +1 for a correct answer, -0.5 for a wrong answer in correct format, and -1 otherwise, incentivizing exploration and correction without penalizing attempt count.

Results: Fine-tuning Qwen 2.5 Math 1.5B on 8K math questions, the multi-attempt LLM improves from 45.6% accuracy (1 attempt) to 52.5% (2 attempts) across five math benchmarks (e.g., AIME 2024, MATH 500). The single-turn baseline, in contrast, rises only from 42.3% to 43.2%. Even in single-attempt evaluations, the multi-attempt model edges out the baseline (45.4% vs. 43.5%), showcasing its superior adaptability and reasoning refinement—crucial for real-world LLM applications.

R1-Searcher: Incentivizing the Search Capability in LLMs via Reinforcement Learning

Paper: https://arxiv.org/abs/2503.05592
Authors: Huatong Song et al. (Renmin University of China)
Focus: Enhancing LLMs with autonomous external search capabilities
Code: https://github.com/RUCAIBox/R1-Searcher

R1-Searcher tackles a persistent LLM weakness: reliance on internal knowledge, which falters on time-sensitive or knowledge-intensive tasks, leading to hallucinations. By integrating retrieval-augmented generation (RAG) with a two-stage RL approach, it trains LLMs to invoke external search systems effectively, improving reasoning without supervised fine-tuning (SFT).

Contributions:

Two-Stage RL: Stage 1 uses a retrieval reward (0.5 if search is invoked) to teach correct query formatting; Stage 2 adds an answer reward (F1 score-based) to optimize problem-solving with retrieved data.

RAG-based Rollout: Special tags (e.g., ) pause generation for retrieval, integrating results seamlessly into reasoning.

Results: Trained on HotpotQA and 2WikiMultiHopQA, R1-Searcher (Qwen-2.5-7B-Base) outperforms GPT-40-mini-based ReARTeR by 48.2% on HotpotQA and 21.7% on 2Wiki (LLM-as-Judge scores). On the unseen Bamboogle dataset with online search, it achieves an 11.4% gain over Search-o1 (32B). This pure RL approach enhances generalization and inference efficiency, making LLMs more robust for complex, real-world queries.

Transformers without Normalization: Weight Space Analysis and Improved Training Dynamics

Paper:https://arxiv.org/abs/2503.10622
Authors: Jiachen Zhu et al. (Meta, NYU, MIT, Princeton)
Focus: Eliminating normalization layers in Transformers with minimal performance trade-offs
Code: https://jiachenzhu.github.io/DyT

Normalization layers like Layer Norm (LN) and RMSNorm are cornerstones of Transformer architectures, widely assumed to be critical for stable training and strong performance. "Transformers without Normalization" challenges this assumption, introducing a surprisingly simple alternative—Dynamic Tanh (DyT)—to replace normalization entirely. The authors observe that LN often produces tanh-like, S-shaped mappings, squashing extreme values while scaling inputs. DyT leverages this insight with an element-wise operation, DyT(x)=tanh(αx), where α is a learnable parameter, bypassing the need for statistical computations.

Contribution:

Dynamic Tanh (DyT): A lightweight replacement for normalization, DyT uses tanh(αx) to mimic LN’s squashing and scaling effects, with α dynamically adjusting to input ranges.

Drop-in Simplicity: DyT integrates seamlessly into existing Transformer designs, replacing LN or RMSNorm without altering other components or requiring extensive hyperparameter tweaks.

Results:

Tested across diverse domains—vision (ViT, ConvNeXt), language (LLaMA), speech (wav2vec 2.0), and DNA modeling (HyenaDNA)—DyT matches or exceeds the performance of normalized models. For LLaMA (7B to 70B), DyT achieves equivalent training loss and zero-shot accuracy on 15 tasks, while cutting training time by 8.2% and inference time by 7.8% on a 7B model. Its efficiency stems from avoiding mean/variance calculations, making it a leaner option. Unlike alternatives like Fixup or SkipInit, DyT maintains stability and performance without complex initialization tricks, using just a default α = 0.5 for most tasks (though tuned for LLMs). This approach redefines Transformer design, offering a faster, simpler alternative to normalization while preserving—or even enhancing—LLM capability, especially in resource-sensitive settings.

Discover more about recent research papers that enhance the performance of LLMs, published in the latter half of February 2025.

Instruction Tuning

Large-Scale Data Selection for Instruction Tuning

Paper: https://arxiv.org/abs/2503.01807
Code: https://github.com/hanishivi/automated-instruction-selection
Authors: Hamish Ivison et al. (University of Washington, Allen Institute for AI, University of Southern California)
Focus: Optimizing data selection for instruction-tuning at scale

Instruction-tuning drives language model performance, but scaling data selection to millions of samples remains tricky. This paper evaluates nine automated methods on pools up to 5.8 million samples, introducing RDS+, a simple embedding approach that outperforms complex techniques across single and multi-task settings. This paper offers RDS+ method, which uses weighted mean pooling of LM hidden states, consistently beating advanced methods like LESS and IFD. This method thrives with larger datasets, outperforming human-curated mixtures and improving multi-task generalization.

Results: On TULU 2, RDS+ selects 326,000 samples (6% of 5.8 million) to exceed the TULU 2 mixture and match full-pool performance, with 2-point gains over baselines and random selection. This efficient method shines at scale, enhancing large-scale instruction-tuning.

Cache management

Q-Filters: Leveraging Query-Key Geometry for Efficient Key-Value Cache Compression

Paper: https://arxiv.org/abs/2503.02812
Code: https://github.com/NathanGodey/qfilters
Authors: Nathan Godey et al. (Sorbonne Université, Inria, Sapienza University of Rome, University of Edinburgh, Miniml.AI)
Focus: Compressing the KV Cache for LLMs using Query-Key geometry

The KV Cache’s memory demands hinder long-context LLMs. Q-Filters leverages Query-Key geometry to estimate attention scores and prune less critical KV pairs without attention weights, ensuring compatibility with FlashAttention for efficient compression.

Key contribution:

Geometric Insight: Projects Keys onto Query eigenvectors to assess KV importance.
Training-Free & FlashAttention-Compatible: Computes filters once, integrating with memory-efficient attention.

Results: On Llama-3.1-8B, achieves 99% accuracy with 32x compression in needle-in-a-haystack tests and cuts perplexity drop by 65% over Streaming-LLM on the Pile with a 512-item cache.

Quantization

RSQ: Learning from Important Tokens Leads to Better Quantized LLMs

Paper: https://arxiv.org/abs/2503.01820
Authors: Yi-Lin Sung et al. (University of North Carolina at Chapel Hill)
Focus: Enhancing post-training quantization (PTQ) of LLMs by prioritizing important tokens
Code: https://github.com/ylsung/rsq

RSQ introduces a layer-wise quantization method that boosts LLM compression by focusing on high-importance tokens (e.g., those with large attention scores). It uses a three-step process (Rotate, Scale, Quantize) and token importance strategies to maintain key information, cutting computational costs.

Key Innovations:

Three-Step Process: Rotate (reduces outliers), Scale (adjusts features by importance), Quantize (uses scaled Hessian). Rotate applies orthogonal transformations to mitigate weight outliers, reducing quantization errors. While scale adjusts token features based on their importance, using a modified loss function: ‖(WX - ŴX)R‖₂², where R scales tokens dynamically. and, Quantize uses the GPTQ framework with a scaled Hessian matrix (H_RSQ = 2XR²Xᵀ) for efficient quantization.
Token Importance: Heuristic (e.g., First-N) and dynamic (AttnCon) strategies improve accuracy.

Results: On LLaMA3-8B-Instruct, RSQ gains 1.6% accuracy at 3-bit precision, with up to 3.0% improvement on long-context tasks over QuaRot, shining in extreme compression.

TinyR1-32B-Preview: Boosting Accuracy with Branch-Merge Distillation

Paper: https://arxiv.org/abs/2503.04872
Authors: Lin Sun et al. (Qiyuan Tech, Peking University)
Focus: Compressing LLMs via distillation with enhanced accuracy

TinyR1-32B-Preview uses Branch-Merge distillation to compress a 671B model (DeepSeek-R1) into a 32B student, improving accuracy in math, coding, and science while enabling efficient, quantization-ready deployment. It combines models using KL divergence for parameter selection.

Results: Outperforms baselines by +5.5 (Math), +4.4 (Coding), +2.9 (Science), nearly matching the 671B model and surpassing a 70B variant.

Unlearning

Group-robust Machine Unlearning

Paper: https://arxiv.org/abs/2503.09330
Authors: Thomas De Min et al. (University of Trento)
Focus: Ensuring fairness in machine unlearning for LLMs
Code: https://github.com/tdemin16/group-robust_machine_unlearning

Standard unlearning can harm model fairness when the forget set over represents certain groups, degrading performance unevenly. This paper proposes group-robust unlearning to maintain accuracy across groups in large language models (LLMs) handling diverse data.

Key Innovations:

REWEIGHT: Reweights sampling in retraining for fair exact unlearning.
MIU: Approximate unlearning that reduces group-specific bias by minimizing mutual information, aligning with the original model.

Results: On CelebA (0.5 unlearning ratio), MIU with REWEIGHT hits 69.0% group accuracy, beating SCRUB’s 62.9%, and keeps equalized odds delta at 0.6 vs. SCRUB’s 3.2. This method ensures equitable unlearning, enhancing LLM fairness for data removal tasks.

Breakthrough papers improving LLMs performance

Thu, 06 Mar 2025 04:21:51 +0000

Highlights

Enhanced Attention Mechanisms: Native Sparse Attention and SpargeAttn propose hardware-aligned and universal sparse frameworks that accelerate long-context inference and training without compromising performance.
Knowledge Integration & Continual Learning: "How Do LLMs Acquire New Knowledge?" and "How Much Knowledge Can You Pack into a LoRA Adapter" explore strategies to embed new information into LLMs while preserving established capabilities.
Temporal & Diffusion Innovations: The Continuous Diffusion Model and Temporal Heads work address the dynamic nature of language by refining outputs iteratively and specializing in time-specific information recall.
Advanced Fine-Tuning Strategies: Make LoRA Great Again, PAFT, and Thinking Preference Optimization introduce novel adaptation methods to improve efficiency and enhance chain-of-thought reasoning during fine-tuning.
Architectural Enhancements: You Do Not Fully Utilize Transformer's Representation Capacity and MUDDFormer offer solutions to overcome residual bottlenecks by improving cross-layer information flow and expanding representational power.

Papers imprving performance of LLMs published between February 16th-28th 2025

Survey paper

Optimize global IT operations with our World at Work Guide

Explore this ready-to-go guide to support your IT operations in 130+ countries. Discover how:

Standardizing global IT operations enhances efficiency and reduces overhead
Ensuring compliance with local IT legislation to safeguard your operations
Integrating Deel IT with EOR, global payroll, and contractor management optimizes your tech stack

Leverage Deel IT to manage your global operations with ease.

Download free guide

LLMs architecture improvement

Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention

Standard attention mechanisms are notoriously computationally expensive for long-context modeling. This paper fills the gap by proposing a sparse attention mechanism that not only reduces computation but is also natively trainable on modern hardware.

Methodology

The authors introduce NSA - a dynamic hierarchical sparse strategy. In simple terms, NSA compresses tokens coarsely while selectively preserving crucial tokens at a fine-grained level. This two-tiered approach balances global context awareness with local precision. The design is carefully optimized for modern hardware by balancing arithmetic intensity, enabling end-to-end training without the pre-training overhead typical of full attention models.

Results: Experiments reveal that models pre-trained with NSA not only match or exceed full-attention baselines on general benchmarks and long-context tasks but also achieve substantial speedups on 64k-length sequences during decoding, forward, and backward passes.

How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Training | GitHub

While LLMs excel at knowledge-intensive tasks, the internal mechanisms that assimilate new knowledge remain poorly understood. This paper explores how neural circuits evolve to incorporate new facts during continual pre-training.

Methodology

Using a “knowledge circuits” lens, the study analyzes the formation and optimization of computational subgraphs responsible for knowledge storage. The researchers show that new information is more effectively integrated when it relates to pre-existing knowledge, and they identify a distinct deep-to-shallow evolution pattern in these circuits.

Continuous Diffusion Model for Language Modeling | GitHub

Traditional diffusion models for discrete data lose valuable iterative refinement signals. This study addresses the limitations of discrete diffusion in language modeling by proposing a continuous alternative.

Methodology
The paper bridges discrete diffusion and continuous flow on a statistical manifold. By incorporating the geometry of the underlying categorical distribution, the authors design a diffusion process that generalizes previous discrete models. A simulation-free training framework based on radial symmetry further alleviates the high-dimensionality challenge.

Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment

Although Low-Rank Adaptation (LoRA) enables efficient fine-tuning of LLMs, its performance traditionally lags behind full fine-tuning. This paper seeks to close that gap.

Methodology

The authors propose GOAT, a framework that dynamically integrates SVD-based priors with a Mixture-of-Experts (MoE) architecture. By deriving a theoretical scaling factor, GOAT realigns optimization dynamics to better harness pre-trained knowledge without modifying existing architecture or training routines.

SpargeAttn: Accurate Sparse Attention Accelerating Any Model Inference | GitHub

Even though many models exhibit sparse attention maps, most optimizations have been model-specific. There is a need for a universal sparse attention method that accelerates inference across diverse architectures.

Methodology

SpargeAttn employs a two-stage online filtering mechanism. Initially, it predicts the attention map to decide which matrix multiplications can be skipped. Then, an online softmax-aware filter further eliminates unnecessary computations without extra overhead.

How Much Knowledge Can You Pack into a LoRA Adapter without Harming LLM?

Integrating new facts into LLMs via LoRA fine-tuning can sometimes lead to a degradation in general performance, particularly in question-answering benchmarks.

Methodology
This study systematically varies the proportion of new versus known facts in the training data when fine-tuning a Llama-3.1-8B-instruct model. The experiments highlight the delicate balance required to incorporate additional knowledge without biasing the model’s outputs toward overrepresented entities.

Results: The best performance is observed when the training data is a balanced mix of known and new facts. However, when biased, the model shows a decline in benchmark performance and a tendency to overcommit to few dominant answers—emphasizing the importance of careful data composition during LoRA-based updates.

Does Time Have Its Place? Temporal Heads: Where Language Models Recall Time-specific Information

While LLMs excel in fact recall, their ability to handle temporally dynamic information is less understood. This work investigates the specific mechanisms that store time-related knowledge.

Methodology

The researchers identify “Temporal Heads”—specialized attention heads that process time-specific information. By disabling these heads and observing performance drops in temporal recall (without affecting time-invariant knowledge), they demonstrate the critical role these heads play.

Results: The findings confirm that temporal heads are activated by both numeric cues (e.g., “in 2004”) and textual phrases (e.g., “in the year…”). Their study lays the groundwork for targeted editing of temporal information within LLMs.

You Do Not Fully Utilize Transformer's Representation Capacity | GitHub

Standard Transformers limit themselves by relying solely on the immediately preceding layer’s output, leading to suboptimal representation capacity.

Methodology
The authors introduce Layer-Integrated Memory (LIMe), which aggregates hidden states from earlier layers without increasing the overall memory footprint. This strategy counters representation collapse by enriching the model’s contextual information.

PAFT: Prompt-Agnostic Fine-Tuning

Fine-tuned LLMs often overfit to specific prompt formulations, resulting in fragile performance when the prompts vary. PAFT addresses this by aiming for prompt-agnostic robustness.

Methodology

PAFT operates in two stages. First, it generates a diverse set of synthetic candidate prompts. Then, during fine-tuning, it randomly samples from this set so that the model learns to rely on underlying task principles rather than fixed prompt cues.

MUDDFormer: Breaking Residual Bottlenecks in Transformers via Multiway Dynamic Dense Connections | GitHub

Residual connections in Transformers can act as bottlenecks, limiting cross-layer information flow and thus hindering overall performance.

Methodology

MUDDFormer introduces Multiway Dynamic Dense (MUDD) connections, which generate connection weights dynamically based on the hidden states for each input stream (query, key, value, or residual). This approach effectively integrates information from earlier layers, breaking through the limitations of traditional residual architectures.

Results: The paper reports striking improvements MUDDFormer achieves the performance of Transformers trained with 1.8×–2.4× compute. For example, the MUDDPythia-2.8B model matches the pretraining perplexity of Pythia-6.9B and rivals Pythia-12B in five-shot tasks—all while adding only 0.23% more parameters and 0.4% extra computation.

Thinking Preference Optimization

Supervised Fine-Tuning (SFT) for chain-of-thought reasoning often leads to performance plateaus or declines with repeated training, and acquiring high-quality long responses is costly.

Methodology
Thinking Preference Optimization (ThinkPO) is a post-SFT method that leverages readily available short chain-of-thought responses as “rejected” answers and long responses as “chosen” ones. By applying direct preference optimization, the model is nudged to generate more comprehensive reasoning without the need for new, expensive data collection.

Results: ThinkPO boosts performance significantly—math reasoning accuracy increases by 8.6% and output length by 25.9%. Notably, when applied to the DeepSeek-R1-Distill-Qwen-7B model, performance on the MATH500 benchmark improves from 87.4% to 91.2%.

Creative way of using LLMs

Tue, 04 Mar 2025 15:15:00 +0000

Highlights

LEDD: Find your data lake treasures with LLM-powered semantic search!
LLMs in Mobile Apps: Discover the secrets & struggles of adding LLMs to your Android apps!
BayesGenie: Edit like a pro with AI that combines LLMs & Bayesian optimization!
Talking Like P&IDs: Chat with your P&IDs using natural language and AI-powered knowledge graphs!
ArtInsight: Bridge vision gaps with AI that describes children's art for BLV families!
Learning Code-Edit Embedding: Level up your debugging with AI that learns from your coding edits!

(Feb 15-28 2025) Innovative applications using LLMs

Creative ways to use LLMs!!

The AI That Called 📞 2,739 Strangers (And Got Them Talking)

Paper: Telephone Surveys Meet Conversational AI: Evaluating a LLM-Based Telephone Survey System at Scale

Traditional telephone surveys require substantial human resources for recruiting and training interviewers. This research introduces an autonomous AI system that dramatically scales survey deployment without compromising data quality.

In a pilot study with 75 calls in the United States and a larger study with 2,739 calls in Peru, the system was deployed an STT-LLM-TTS pipeline that achieved 89% response parity with human interviews across Peru. But here's the twist: When respondents hesitated, the AI used "cognitive mirroring" - subtly matching regional speech patterns, reducing hang-ups by 22% compared to rigid IVR systems.

At $0.83 per completed survey vs. $25+ for human teams, this could democratize global policy research. The AI even detected vocal stress patterns correlating with response accuracy (r=0.67, p<0.01)

Stanford's code-editing model analyzed 1.4M student submissions

Paper: Learning Code-Edit Embedding to Model Student Debugging Behavior

Providing timely, personalized feedback in programming education is challenging. By modeling how students debug their code, educators can design targeted support tools that improve learning outcomes.

This paper presents an encoder-decoder model that learns "code-edit embeddings" from consecutive student code submissions. By analyzing the patterns in how students edit their code after receiving test case feedback, the system can generate personalized next-step coding suggestions, identify common debugging patterns through clustering, and better understand the learning process. Educators can provide more targeted assistance based on individual problem-solving approaches.

Optimize global IT operations with our World at Work Guide

Explore this ready-to-go guide to support your IT operations in 130+ countries. Discover how:

Standardizing global IT operations enhances efficiency and reduces overhead
Ensuring compliance with local IT legislation to safeguard your operations
Integrating Deel IT with EOR, global payroll, and contractor management optimizes your tech stack

Leverage Deel IT to manage your global operations with ease.

Download free guide

ArtInsight: Making Children's Art Accessible to All

Paper: ArtInsight: Enabling AI-Powered Artwork Engagement for Mixed Visual-Ability Families

Art is meant to be shared and appreciated, but how do blind or low-vision family members engage with children's artwork? ArtInsight offers a touching solution to this challenge using LLMs.

The Innovation: The ArtInsight system uses large language models to generate rich, descriptive content about children's artwork. It goes beyond basic visual descriptions to create detailed artistic interpretations, audio recordings of children explaining their art, and AI-generated questions to prompt meaningful discussions

This paper shows how LLMs can bridge experiential gaps and create more inclusive family interactions around visual creativity.

Engineering Diagrams That Speak Your Language

Paper: Talking like Piping and Instrumentation Diagrams (P&IDs)

Process engineers are familiar with the complex schematics known as Piping and Instrumentation Diagrams (P&IDs). Now, researchers have enabled natural language communication with these diagrams through a three-component system: transforming P&IDs into graph representations via the DEXPI data model, creating P&ID knowledge graphs, and integrating these with LLMs through graph-based retrieval augmented generation.

This innovation allows engineers to query complex diagrams using plain language, potentially reducing errors and improving efficiency in industrial settings where these diagrams are essential.

BayesGenie: Precision Image Editing Through Language

Paper: Bayesian Optimization for Controlled Image Editing via LLMs

Image editing requires precise control and semantic accuracy—critical for both creative professionals and everyday users. This research introduces a novel method that marries LLMs with Bayesian optimization to refine image edits using natural language instructions.

The model, dubbed BayesGenie, uses LLM-generated edits and applies Bayesian optimization to fine-tune parameters, thereby ensuring that the edited image retains both accuracy and semantic consistency. Users can specify desired changes in natural language without manually selecting image areas.

Discovering Hidden Data in Vast Data Lakes

Paper: LEDD: Large Language Model-Empowered Data Discovery in Data Lakes

Many organizations store vast amounts of unstructured data in data lakes, and finding useful information within them can be overwhelming.

The LEDD system uses an LLM to automatically generate hierarchical catalogs that organize data semantically. It also supports natural language queries, which make it easier to search for specific information across millions of data points. Though exact numerical improvements in search speed or accuracy are not detailed in the abstract, the paper highlights that LEDD can significantly reduce the time and effort needed to locate valuable data.

A detailed survey paper on LLMs in mobile apps

Paper: LLMs in Mobile Apps: Practices, Challenges, and Opportunities

Why?: The research is significant because it explores how LLMs can transform mobile application development, making apps more intelligent and personalized while addressing the unique challenges developers face.

How?: The research methodology involved constructing a dataset of 149 LLM-enabled Android apps and conducting an exploratory analysis to examine the deployment and usage of LLMs within these applications, focusing on integration strategies and challenges.

Results: The analysis revealed key characteristics of the dataset, prevalent integration strategies, and common challenges faced by developers in the context of mobile app development.

Research papers improving performance of LLMs [3/3]

Sat, 22 Feb 2025 16:47:00 +0000

In partnership with

What’s in it today?

SCONE: who needs a bigger vocabulary when you can just contextualize the heck out of your n-grams?
DAAs: Making LLMs agree with humans, one preference at a time (and sometimes only needing 5% of the data to do it!)
CT-KL: Ignoring the KL penalty and focusing on critical tokens to boost LLMs, because sometimes you just need to be a rebel
LIMO: Less is More for Reasoning - Proving you don't need a mountain of data, just a tiny, well-curated molehill!

Don’t have much time to read entire newsletter? No problem? Listen to this fun and engaging podcast covering these research papers in detail.

Papers improving performance of LLMs published in 2025

Scaling Embedding Layers in Language Models

Research paper

Embedding layers in language models maps discrete tokens to continuous vector representations. Scaling embedding layers enhances model performance but simply increasing vocabulary size has limitations due to the tight coupling between input and output embedding layers, especially the increasing computational cost of the output layer's logits calculation. The benefits of simply scaling the vocabulary also diminish due to the proliferation of low-frequency "tail tokens" which are rarely updated during training, resulting in lower quality embeddings.

To address these issues, the paper introduces SCONE (Scalable, Contextualized, Offloaded, N-gram Embedding), a novel approach designed to disentangle the input and output embeddings, enabling effective input embedding scaling with minimal additional inference cost.

Proposed approach

The core of SCONE lies in augmenting the existing token vocabulary with contextualized variants derived from frequent n-grams (f-grams). Instead of directly increasing the vocabulary size, SCONE uses a set of frequently occurring n-grams to provide contextualized representations for each input token. These contextualized tokens are used only for input embedding computation, allowing for a massive augmented input embedding table without increasing the output layer's computational burden. The embeddings for these contextualized tokens are generated by a separate embedding transformer model, referred to as the f-gram model, which is jointly trained with the main language model. This allows for rich contextualized representations without the sparsity issues associated with simply increasing the vocabulary.

The paper proposes an efficient method, analogous to continuing a Byte Pair Encoding (BPE) tokenizer's training, to identify frequent n-grams (f-grams) in the training corpus. This process involves scanning the corpus to count the occurrences of n-grams of length up to a specified maximum (n). To reduce memory usage, a minimum frequency threshold is applied during the counting process. The SCONE method maps a sequence of tokens to a sequence of embedding vectors. During training, the method utilizes an f-gram transformer model (Af-gram) to generate embeddings for contextualized tokens. At inference time, a precomputed f-gram embedding layer (F) is used to map the f-grams to embedding vectors, allowing for efficient retrieval of contextualized embeddings. After that, The embeddings generated by the SCONE method are passed to a standard transformer model (Amain), referred to as the main model, followed by a prediction head (D). This combination enables next-word prediction with SCONE. The f-gram model and the main model are trained jointly, allowing the f-gram model to learn contextualized representations that are aligned with the main model's objective.

Results

The paper shows that scaling both the number of cached f-gram embeddings and the size of the f-gram model allows SCONE to outperform a 1.9B parameter baseline model across diverse corpora. For example, with 10M f-grams and a 1.8B f-gram model, a 1.3B main model matches the perplexity of the 1.9B baseline. With 1B f-grams and a 1.8B f-gram model, a 1B main model surpasses the 1.9B baseline. Also, SCONE enables improved performance while using only half the inference-time FLOPS of the baseline model.

The Differences Between Direct Alignment Algorithms are a Blur

Research paper

Aligning Large Language Models (LLMs) with human values is a challenging task. Generally methods like Supervised Fine-Tuning (SFT), Reward Modeling (RM), and Reinforcement Learning from Human Feedback (RLHF) is used for that. Recenlty Direct Alignment Algorithms (DAAs) is gained a traction. It directly optimize the policy based on human preferences, without the explicit reward modeling step. However, the paper notes the confusing landscape of DAAs, differing in their theoretical underpinnings (pairwise vs. pointwise objectives), implementation details (e.g., using a reference policy or an odds ratio), and whether they require a separate SFT phase (one-stage vs. two-stage). The paper aims to clarify the relationships between these algorithms, determine the importance of the SFT stage, and identify the key factors influencing their alignment quality.

Methodology

The core of the paper's methodology is a systematic empirical comparison of different Direct Alignment Algorithms. The paper investigates the role of the SFT stage, the influence of a tempering factor (β), and the impact of pairwise vs. pointwise preference optimization. To achieve this, the paper focuses primarily on two single-stage DAA methods, ORPO (Odds Ratio Preference Optimization) and ASFT (Aligned Supervised Fine-Tuning), and modifies them in several ways.

Introducing an Explicit SFT Stage: The paper explores whether adding a separate SFT phase before applying ORPO and ASFT improves their performance. This involves training a model using standard supervised fine-tuning on a dataset of instructions and responses, and then using ORPO or ASFT to align the model with human preferences.
Adding a Tempering Factor (β): The paper introduces a scaling parameter β into the ORPO and ASFT loss functions. This parameter, inspired by similar parameters used in other DAAs like DPO, controls the strength of the preference optimization. By varying the value of β, the paper investigates its impact on the alignment quality of ORPO and ASFT.
Theoretical Analysis and Equivalence Proofs: The paper presents theoretical analyses demonstrating relationships between different DAA loss functions. These analyses include theorems proving the equivalence of ASFT to a binary cross-entropy loss, establishing an upper bound of ASFT on ORPO, and demonstrating collinearity of gradients under certain conditions. These theoretical results are used to support and interpret the empirical findings.
Controlled Experiments: The paper conducts a series of carefully controlled experiments to evaluate the performance of different DAA configurations. These experiments involve training and evaluating models on benchmark datasets, using metrics designed to assess alignment quality.
Varying SFT Data Size: The paper examines how the final alignment quality depends on the amount of data used in the SFT stage. This is done to determine whether it's necessary to use the full dataset for SFT, or if a smaller subset is sufficient.

Result

The paper shows that incorporating an explicit SFT stage significantly improves the performance of both ORPO and ASFT. The paper demonstrates that introducing a tempering factor (β) enhances the alignment quality of ASFT and ORPO. The authors found that tuning β was essential for achieving good performance.

Ignore the KL Penalty! Boosting Exploration on Critical Tokens to Enhance RL Fine-Tuning

Research paper | Implementation

The paper presents a novel approach to improving the exploration capabilities of LLMs during reinforcement learning (RL) fine-tuning. The core challenge in RL fine-tuning is balancing exploration and stability. Traditional methods often use a Kullback-Leibler (KL) penalty to prevent the model from deviating too far from its pre-trained behavior, ensuring stability but potentially limiting exploration.

The paper introduces the Critical Token KL (CT-KL) method, which dynamically adjusts the KL penalty during training based on critical tokens identified as crucial decision points. These critical tokens are determined using attention patterns and gradient information, indicating where targeted exploration can significantly impact performance. By reducing or eliminating the KL penalty for these tokens, CT-KL encourages more extensive exploration at these key junctures while maintaining stability elsewhere.

This selective approach contrasts with traditional methods that apply uniform penalties across all tokens or eliminate penalties entirely, which can lead to instability or inefficient learning. The CT-KL method modifies the standard transformer architecture by adjusting its training objective without altering its base structure.

Implementation Details

This method integrates CT-KL into existing RL frameworks for LLMs. The first step is identifying critical tokens within input sequences using attention weights and gradients. After that, the training objective is modified to selectively reduce or remove KL penalties for these identified critical tokens. Models are than trained with this modified objective on various tasks requiring complex reasoning and problem-solving skills.

Results

The paper demonstrates significant improvements in performance when using CT-KL compared to standard approaches: On mathematical reasoning tasks, models trained with CT-KL showed an improvement of up to 8.2% compared to those using traditional uniform penalties.

LIMO: Less is More for Reasoning

Research paper | Implementation | Dataset | Model

This paper challenges the conventional wisdom that complex reasoning in LLMs requires extensive training data. This hypothesis tests that if a model has sufficient pre-trained knowledge and is presented with effective "cognitive templates" demonstrating problem-solving processes, it can achieve strong reasoning capabilities with minimal fine-tuning data.

Methodology

Paper carefully curated a small training dataset to act as "cognitive templates" for reasoning. The key is not the quantity of the data, but its quality and how well it demonstrates the reasoning process. While the specific details of how these training samples are orchestrated is limited from the paper, they must have done so effectively to get such positive outcomes.

The general idea for the methodology is:

Leveraging Pre-trained Knowledge: The paper builds upon modern foundation models that have used significant mathematical knowledge during pre-training. This pre-trained knowledge serves as the foundation for reasoning.
Minimal Exemplars as Cognitive Templates: The core approach involves creating a small set of training examples that demonstrate how to utilize the pre-trained knowledge to solve complex reasoning problems.
Emphasis on Inference-Time Computation: The approach is designed to encourage extended deliberation and systematic application of pre-trained knowledge during inference. This highlights the importance of having sufficient computational resources at inference time.

Results

The paper reports impressive results on mathematical reasoning benchmarks:

AIME Benchmark: LIMO achieves 57.1% accuracy on the AIME benchmark, a significant improvement over previous SFT-based models which achieved only 6.5% accuracy. This demonstrates the ability to achieve strong performance on a challenging reasoning task.
MATH Benchmark: LIMO achieves 94.8% accuracy on the MATH benchmark, also a substantial improvement over previous models (59.2%).
Out-of-Distribution Generalization: Most remarkably, LIMO demonstrates exceptional out-of-distribution generalization, achieving a 40.5% absolute improvement across 10 diverse benchmarks, outperforming models trained on 100x more data. This result challenges the common belief that SFT inherently leads to memorization rather than generalization.

Research papers improving performance of LLMs [2/3]

Fri, 21 Feb 2025 21:00:14 +0000

In partnership with

What’s in it today?

ReLearn makes LLMs forget unwanted knowledge and remember how to speak good
Coupled Adam fixes Adam so language model embeddings aren't too "extra"
TransMLA converts GQA models to MLA ones for better LLM expression, because apparently size does matter
LASP-2 makes linear attention training zoom by decluttering communication

Don’t have much time to read entire newsletter? Well, listen to this fun and engaging podcast covering these research papers in detail.

[2] Jan 16th-Feb 15th 2025

ReLearn: Unlearning via Learning for Large Language Models

Research paper | Paper implementation

Why this research is important?

LLMs are getting trained on more and more data and its consumption is increasing drastically. LLM providers use large-scale AI training datasets that often contain unauthorized private and copyrighted information. Recent legal actions, like the New York Times lawsuit against OpenAI, underscore the urgency of addressing these issues. The core problem is that current unlearning methods often rely on reverse optimization, which reduces the probabilities of target tokens but degrades the model's fundamental language generation capabilities, resulting in repetitive or incoherent outputs. The paper argues that current evaluation metrics are also inadequate, as they primarily focus on contextual forgetting while failing to capture broader limitations in fluency and relevance. Therefore, this research is crucial for developing more effective and reliable unlearning techniques that comply with privacy and copyright regulations without compromising the model's linguistic coherence.

Approach:

Paper proposes unlearning pipeline based on data augmentation and positive optimization. Instead of suppressing token probabilities (reverse optimization), ReLearn overwrites sensitive information with new, authorized knowledge by training the model on augmented data. This process preserves the model's linguistic ability while forgetting target knowledge, akin to human memory updating. To better understand I divide ReLearn pipeline in following steps:

Unlearning data synthesis: In this step ReLearn pipeline synthesize non-sensitive training data by augmenting the forget set (data to be unlearned) with diverse variations, ensuring comprehensive coverage of the knowledge to be forgotten. This process is entirely performed by an LLM using specific prompts. It performs question and answer augmentation. For question augment it iterates on each question-answer pair in the forget set, the method synthesizes four types of question variations: (1) Simple Variants to prevent overfitting to specific phrasings, (2) Contextual Variants to ensure forgetting across contexts, (3) Noise Variants to enhance robustness to noisy inputs, and (4) Logical Variants to adapt to different knowledge forms by altering the logic of the questions. and, for answer augmentation it iterates on each augmented question, the method synthesizes new pairs with relevant, deliberately vague answers. These answers must be (1) Unlearned, containing no original sensitive content; (2) Relevant, aligning with the question context; and (3) No-risk, avoiding the introduction of new sensitive content.
Content verification: To ensure the safety of the augmented data, the method uses a Content Verification process for the synthesized answers. This process utilizes LLMs to conduct Chain-of-Thought (COT) analysis on each augmented answer, evaluating it against predefined safety criteria. If verification fails, indicating a potential risk in the augmented data, the process returns to the step of "Answer Augmentation."
Data diversification: To prevent QA format overfitting and catastrophic forgetting, the method uses two main strategies sentence completion and generic dataset. In sentence completion, the augmented data is augmented with sentence completion pairs, split from each answer. For example, "Isabella Marquez can be reached through conventional electronic communication channels." is split into "Isabella Marquez can be reached through" and the label "conventional electronic communication channels." and in generic dataset, ReLearn applies generic data by randomly sampling questions from WikiQA and Chatbot Instruction datasets.
Unlearning via learning: The unlearning objective is formulated using three datasets: the augmented forget set, the sentence completion set, and the generic dataset. The vanilla model is then fine-tuned on a combination of these datasets. The loss function combines generative loss, masked language model loss, and knowledge loss, which is then used to perform parameter update and fine-tuning of the vanilla model.

Results:

Paper identified limitations in existing unlearning metrics and proposes three metrics: Knowledge Forgetting Rate (KFR), Knowledge Retention Rate (KRR), and Linguistic Score (LS). KFR measures the extent of knowledge forgetting, and KRR measures the extent of knowledge retention, while LS evaluates the linguistic quality of the unlearned model, capturing linguistic degradation patterns such as reduced vocabulary diversity, simplified syntax, and diminished lexical richness. ReLearn outperforms SOTA in many benchmarks, one of them is shown above.

Better Embeddings with Coupled Adam

Research paper | Paper implementation

This paper solves the problem of anisotropic embeddings. Buit what it is? Well, word representations learned by LLMs tend to cluster in a small subspace, away from the origin of the vector space. This clustering limits the semantic expressiveness of the embeddings and, consequently, the model's overall performance. The paper begins by noting that while various attempts have been made to explain and alleviate this issue, the role of the optimization algorithm itself has been largely overlooked. This research paper argue that the second moment estimate in Adam, which is used to adapt the learning rate for each parameter, causes embedding vectors to shift collectively away from the origin.

To understand this, let’s first understand how Adam works. Adam maintains an exponentially decaying average of past gradients (first moment) and squared gradients (second moment) for each parameter. The second moment is meant to scale the learning rate adaptively, giving larger updates to parameters with smaller gradients and smaller updates to parameters with larger gradients. This is particularly useful for sparse data, like word frequencies in LLM training, where some words occur far more often than others.

However, paper shows that this adaptive scaling, specifically the i-dependency of the second moment, leads to the anisotropy problem. They demonstrate mathematically that while the sum of gradients over all embedding vectors vanishes with SGD, the weighted sum (weighted by the adaptive learning rate based on the second moment) does not vanish with Adam. This non-vanishing sum causes a collective shift of the embedding vectors away from the origin. They also provide experimental evidence to show that the expectation value of the second moment is proportional to the unigram probability of the corresponding word, confirming the link between word frequency and the anisotropic effect.

Proposed approach

Paper proposes a modified version of Adam called Coupled Adam. The main idea behind Coupled Adam is to enforce that the second moments are the same for all embedding vectors. To do this, team replaced the individual second moment estimates for each embedding vector with the average of the second moments over all embedding vectors.

Mathematically, the original Adam update rule uses an i-dependent effective learning rate (η_i) that depends on the second moment estimate of the i^th embedding vector. Coupled Adam replaces this i-dependent rate with an i-independent rate based on the average second moment.

This coupling of the second moments ensures that the sum of embedding updates vanishes, similar to SGD, preventing the collective shift of embeddings away from the origin. At the same time, Coupled Adam retains the benefits of Adam, using a second moment to normalize the embedding update vectors (albeit a global one).

Implementation

The implementation of Coupled Adam is straightforward. Paper provides pseudocode in Algorithm 1 of the paper which is as shown above. The key modification lies in Algorithm 1 lines 8-12 in the inclusion of Coupled Adam where the second moments are averaged across all the vocabulary items before the update vector is calculated, as such:

Calculate the standard Adam update vectors for all embedding vectors e_i.
Compute the average second moment (ν) over all embedding vectors.
Replace the individual second moment estimates for each embedding vector with this average value (ν).
Apply the standard Adam update rule using this shared second moment.

Paper emphasizes that Coupled Adam can be easily integrated into existing training pipelines with minimal code changes, specifically for the embedding parameters, while standard Adam can be used for all non-embedding parameters.

Setup and Results:

Paper presents multiple experiments to evaluate the effectiveness of Coupled Adam. These experiments were divided into small-scale and large-scale settings, involving different datasets, model architectures, and training frameworks.

Small-Scale Experiments: The small-scale experiments utilized the OpenWebText Corpus and the GPT-2 tokenizer. The model architecture also followed GPT-2, with hyperparameter settings derived from GPT-3. Model sizes ranged from 125M to 760M parameters, and dataset sizes ranged from 5B to 20B tokens. Each experiment was repeated three times with different random seeds to assess statistical significance.
Large-Scale Experiments: The large-scale experiments employed the SlimPajama dataset and the GPT-2 tokenizer. The model architecture was a state-of-the-art dense transformer similar to those used in recent LLMs, including RoPE embeddings and SwiGLU activation functions. Model sizes were 1.3B and 2.6B parameters.

Across all experiments, the authors trained two models: one using standard Adam and one using Coupled Adam for the embedding parameters. They then evaluated the models using various metrics to assess both general performance and embedding quality. Here are the findings:

The use of Coupled Adam resulted in lower perplexity on both small and large scale. For example, they reported 14.69 perplexity score using vanilla adam compared to 14.45 score using Coupled Adam.
Coupled Adam was seen to outperform vanilla adam in different training sizes. For example, it was seen that model trained on 10B tokens has perplexity of 15.51. And, on similar training setup using Coupled Adam, perplexity score comes out to be 15.37.
Coupled Adam had significantly less anisotropic score which proves that the method is working to improve the quality of embeddings. The lower the score, the better is the embedding. Example: The anistropic score for vanilla adam was 0.874 compared to coupled adam which was 0.817.

TransMLA: Multi-head Latent Attention Is All You Need

Research paper | Implementation

As models and sequence lengths grow, the KV cache, which stores information about previous tokens, becomes a significant bottleneck, consuming substantial memory and bandwidth. While methods like Group Query Attention (GQA) have been adopted to reduce KV cache size, the authors propose that Multi-head Latent Attention (MLA), used in Deepseek models, offers a more theoretically sound and practically effective solution. The key problem this paper tackles is how to efficiently transition existing GQA-based models, which are widely used, to MLA-based models, which are more expressive for the same KV cache overhead.

Core argument:

The main argument is that Multi-head Latent Attention (MLA) provides greater expressive power than Group Query Attention (GQA) for the same KV cache overhead. Paper provides a theoretical proof to support this claim. It then introduces TransMLA, a post-training method, that enables the conversion of widely used GQA-based pre-trained models (such as LLaMA, Qwen, and Mixtral) into equivalent MLA-based models. The goal is to allow existing models to benefit from the superior expressiveness of MLA with minimal changes to model architecture and without increasing KV cache size.

Methodology

We divided proposed method in three main parts:

Theoretical Proof of GQA to MLA Conversion: Paper provides a mathematical proof showing that any GQA configuration can be equivalently transformed into MLA with the same size of KV cache. This proof relies on showing that the key transformation in GQA can be represented as a low-rank factorization, which is the core idea behind MLA. Paper demonstrates how to replicate keys in GQA (making all heads identical within a group), move that replication to the parameter side (replicating weights instead of activations), and then show that this replicated weight matrix has a low-rank structure that can be factorized in the same way as MLA. They achieve this by factorizing W^′_K using the Singular Value Decomposition (SVD) W^′_K = U_K S_K V^⊤_K , and only keeping the top-r singular values, thus achieving KV compression.
MLA is Not Representable in GQA: Paper also proves that the reverse is not true – MLA cannot always be represented by GQA. They demonstrate this by considering a scenario where vectors in the MLA transformation matrix are orthogonal. This leads to diversity in the outputs that cannot be replicated by GQA’s grouped heads which are necessarily identical within each group. This asymmetry provides the theoretical justification for expecting performance improvements when converting GQA models to MLA.
Practical Conversion and Post-Training: The core of the TransMLA method lies in converting the weights of a pre-trained GQA model to equivalent MLA weights. This involves several steps:
- Weight Decomposition: The authors take the existing key projection matrix from the GQA model and decompose it into two smaller matrices, corresponding to the W^a_K and W^b_K matrices in the MLA formulation. This decomposition is achieved using SVD or other low-rank factorization techniques.
- Weight Recombination: These newly formed W^a_K and W^b_K matrices are then integrated into the model as the new key projection weights. This effectively replaces the GQA attention mechanism with the MLA mechanism.
- Post-Conversion Training: After the conversion, the model is fine-tuned on a dataset to allow it to adapt to the new MLA architecture and realize the benefits of its enhanced expressiveness. This fine-tuning step is crucial for recovering any potential performance loss from the weight conversion and for fully exploiting the potential of MLA. This training is done without increasing the KV cache size.

LASP-2: Rethinking Sequence Parallelism for Linear Attention and Its Hybrid

Research paper | Implementation

Attention mechanism’s computation increases at quadratic complexity with respect to sequence length. Paper enhances the efficiency of linear attention mechanisms through a combination of theoretical insights and practical implementations. Paper begins by analyzing the limitations of existing linear attention methods, which typically rely on fixed parallelism strategies that do not fully exploit the potential of modern hardware architectures and than it proposes a new approach that allows for dynamic sequence parallelism, enabling better utilization of computational resources.

Paper introduces a hybrid architecture that combines both local and global attention mechanisms. Local attention focuses on a limited context window, allowing for efficient computation within smaller segments of the input sequence. In contrast, global attention captures long-range dependencies across the entire sequence. By integrating these two forms of attention, paper achieves a balance between computational efficiency and expressive power.

The paper details how LASP-2 employs a two-stage process for attention computation. In the first stage, local attention is computed in parallel across segments of the input sequence. This allows for rapid processing of individual segments while maintaining low memory overhead. In the second stage, global attention is applied to aggregate information from these segments, ensuring that long-range dependencies are preserved. Moreover paper introduces a novel mechanism for adaptive segment sizing based on input characteristics. This allows LASP-2 to dynamically adjust the size of local segments depending on the complexity of the input data, further optimizing performance.

Result

Paper demonstrates up to a 30% reduction in training time while maintaining comparable accuracy levels on large datasets. LASP-2 maintained stable performance without significant degradation in processing speed or accuracy for up to 10,000 tokens.

Research papers improving performance of LLMs [1/3]

Thu, 20 Feb 2025 20:23:00 +0000

In partnership with

Highlights

InfiniteHiP prunes tokens like scissors, extending context to 3M
LongRoPE stretches context to 2M+ tokens with fine-tuning
DarwinLM uses evolution to prune LLMs , keeping performance high with structured pruning and training
New paper draws a line between context length and model size
Get a $50 free credit to get the humanized data for your project. No credit card required!

Don’t have much time to read entire newsletter? Well, listen to this fun and engaging podcast covering these research papers in detail.

[1] Jan 16th-Feb 15th 2025

DarwinLM: Evolutionary Structured Pruning of Large Language Models

Research paper | Paper implementation

This research paper propose effective model compression technique. Pruning is a popular model compression technique. In LLMs, pruning refers to the process of removing less important or redundant connections (parameters) of model. By strategically removing these parameters, the model becomes smaller and more efficient, requiring less computation and memory.

Pruning can be unstructured or structured. Unstructured pruning allows for the removal of individual connections seemingly at random, leading to sparsity in the model's weight matrices. While this can reduce the model size, it often doesn't translate directly into speed improvements on standard hardware. Structured pruning, on the other hand, removes entire structures within the network, such as neurons, channels, or even layers. This results in a more compact model that can be executed more efficiently on conventional hardware, providing end-to-end speedups without requiring specialized hardware or software.

However, not all parts of an LLM are equally important. Some components are more sensitive to pruning than others. This research paper ntroduces a novel method for training-aware, non-uniform structured pruning. The core idea is to combine an evolutionary search process with a lightweight training procedure to identify the optimal substructure of the LLM. The name "DarwinLM" itself is inspired by the concept of natural selection. The method works by generating multiple "offspring" models from a parent model, each with slightly different pruning configurations (mutations). These offspring models are then evaluated based on their performance on a given task. The best-performing offspring are selected to "survive" and become the parents for the next generation. This process is repeated over several generations, gradually refining the pruning configuration and identifying the most efficient and effective substructure.

A key innovation in DarwinLM is its training-aware approach. Recognizing that the performance of a pruned model depends not only on its initial structure but also on how it's subsequently trained, DarwinLM incorporates a lightweight, multi-step training process within each generation. This allows the method to assess the effect of post-compression training and eliminate poorly performing models early on. The training process progressively increases the number of tokens used, providing a more comprehensive evaluation of the model's ability to learn and generalize.

Results:

DarwinLM achieves a remarkable 2x reduction in model size with only a 3% performance loss across various tasks. The effectiveness of DarwinLM is consistent across different model sizes, including smaller models (350M parameters) and larger ones (7B parameters). Meaning, it works well on all size of models.

InfiniteHiP: Extending Language Model Context Up to 3 Million Tokens on a Single GPU

Research paper | Implementation

Context window or length is the amount of text a model can consider when processing information. Think of the context window as the model's short-term memory. A larger context window allows the model to better understand complex relationships, resolve ambiguities, and maintain coherence over longer passages. However, increasing the context window of an LLM presents significant technical challenges. One of the main bottlenecks is computational cost. As the context window grows, the amount of computation required to process the information increases dramatically, often requiring more powerful hardware and more energy consumption. Another challenge is memory. LLMs need to store information about the entire context window in memory, and as the context window expands, the memory requirements can quickly exceed the capacity of available hardware. This research paper works in this direction.

How it works?

This paper introduces a novel framework called InfiniteHiP, which stands for Infinite Hierarchical Pruning. This framework applies several innovative techniques to extend the context length that can be processed by an LLM to an impressive 3 million tokens, all while using a single GPU.

The first key component of InfiniteHiP is hierarchical token pruning. In simple terms, this technique intelligently removes less important tokens from the input, reducing the computational load without significantly impacting the quality of the output. It does this at multiple levels, ensuring that the most relevant information is retained for processing.

Next, the framework uses adaptive adjustments to Rotary Position Embeddings (RoPE). Positional embeddings are a crucial component of many LLMs, as they help the model understand the order of words in a sequence. Traditional positional embeddings assign a fixed vector to each position in the sequence. However, these fixed embeddings can struggle to generalize to longer sequences than the model was originally trained on. RoPE offer a more flexible and generalizable approach. RoPE encodes positional information using rotations in a high-dimensional space. This allows the model to better extrapolate to unseen sequence lengths. By modifying RoPE based on observed attention patterns, InfiniteHiP allows the model to effectively handle much longer sequences than it was originally trained on.

Main contribution in InfiniteHiP is its approach to memory management. LLMs typically store intermediate computations (known as the key-value cache) in GPU memory, which is fast but limited in capacity. InfiniteHiP instead stores this cache in the computer's main memory (RAM), which is much larger but slower. This clever offloading technique allows for processing of extremely long contexts without running out of GPU memory. This framework is implemented within a system called SGLang, which optimizes how these various components work together. This integration ensures that the token pruning, memory management, and computational processes are coordinated efficiently.

Result

The results of this research are impressive. InfiniteHiP achieves a speedup of 18.95 times in processing attention (a key component of LLMs) for sequences of 1 million tokens compared to baseline methods. It can handle sequences three times longer than previous approaches without losing important information. Remarkably, these improvements are achieved without needing to retrain the model or change its fundamental architecture.

In practical terms, this means that a single GPU with 48GB of memory (specifically, an NVIDIA L40s) can now process up to 3 million tokens at once. This is a massive leap forward, enabling applications that were previously impractical or impossible.

Explaining Context Length Scaling and Bounds for Language Models

Research paper

Ever wondered how does the length of context affect the performance of these language models? Well this research paper has answers for that!

To understand the significance of this question, it's important to know that context in language models refers to the amount of preceding text the model considers when predicting the next word or performing a task. Traditionally, most models were limited to relatively short contexts, often just a few hundred words. However, recent advancements have pushed this boundary, with some models capable of considering thousands or even millions of words as context.

This increase in context length has led to some interesting and sometimes conflicting observations. Some studies have shown that longer contexts can improve model performance, leading to more accurate predictions and better understanding of complex topics. This phenomenon is often described as "Scaling Laws," suggesting that performance improves as context length increases. On the other hand, other research has found that long, irrelevant contexts can actually degrade performance, confusing the model rather than helping it. Now, question is which one is true?

This research paper provides a theoretical framework to explain how context length impacts language modeling. It introduces the concept of "Intrinsic Space" as a key to understanding context length effects. Intrinsic Space refers to a lower-dimensional representation of the data that captures the essential features relevant to the language modeling task. Think of it as distilling the most important information from the vast sea of language data. Paper suggests that impact of context length can be explained by how well the model can learn and represent these intrinsic features. As context length increases, the model has more information to work with, potentially allowing it to better capture these essential features. However, there's a point of diminishing returns, beyond which additional context doesn't provide meaningful new information.

One of the key findings from this research is that there's an optimal context length for any given size of training dataset. This means that simply increasing context length indefinitely isn't always beneficial. Instead, the optimal context length grows with the size of the training data. The paper also establishes theoretical bounds for context length scaling under certain conditions. These bounds provide guidance on how context length should be increased as models and training datasets grow larger.

LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens

Research paper

This research paper introduces a novel technique called "LongRoPE" that allows LLMs to effectively handle extremely long context windows – in this case, extending beyond 2 million tokens – while maintaining strong performance and reasonable computational costs.

LongRoPE modifies the RoPE (explained in 1st paper) mechanism to make it more efficient for very long sequences. The researchers observed that for very long contexts, the high-frequency components of the RoPE embeddings become less important. LongRoPE takes advantage of this observation by scaling down these high-frequency components as the context length increases. This scaling reduces the computational burden and allows the model to process much longer sequences without sacrificing performance.

In simpler terms, LongRoPE is like a zoom lens for the model's attention. When focusing on a small area (short context), the lens provides a detailed view. But when zooming out to see a larger area (long context), some of the fine details become less important, and the lens adjusts to provide a broader, more efficient view.

The important thing about LongRoPE is its simplicity and ease of integration. It can be applied to existing LLMs with minimal modifications.

Results:

The models extended via LongRoPE maintained low perplexity scores across evaluation lengths ranging from 4k to 2048k tokens. Low perplexity indicates that the model's predictions are more accurate and reliable. In tasks requiring long contexts, such as passkey retrieval, LongRoPE achieved over 90% accuracy! and, most importantly, models utilizing LongRoPE preserved their original performance levels on shorter context windows while significantly reducing perplexity when processing longer contexts.

DeepSeek-R1 Special Edition

Tue, 28 Jan 2025 20:36:57 +0000

📑 IN THIS ISSUE

DeepSeek-R1 has become very popular online, and as an AI professional, it's important to understand why. How is the DeepSeek team able to train models with just a few million dollars in GPU computing, while teams like OpenAI need billions?

Take some time to read this newsletter. It's important to learn about this model, as it represents a new research direction in large language models (LLMs).

TL;DR listen to this amazing podcast discussing this research paper.

DeepSeek-R1 Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

Introduction

Remember when calculators transformed mathematics? We're at a similar inflection point with AI. DeepSeek-R1 isn't just another language model – it's a fundamental rethinking of how AI systems learn to reason.

DeepSeek-R1 has sent shockwaves through the AI community, wiping $1 trillion from tech market caps in a single day. But beyond the market chaos lies a revolutionary approach to AI reasoning that could redefine the field. While ChatGPT and others focus on generating human-like responses, DeepSeek-R1 focuses on something more crucial: teaching AI how to think step-by-step through problems.

"This isn't just another model release - it's a fundamental rethinking of how AI systems learn to reason. DeepSeek-R1 represents what we've been missing in the field - focus on quality of thinking rather than just output generation."
Dr. Sarah Chen, Stanford AI Lab

Market reaction

Quadrant summarizing reaction on DeepSeek-R1 launch

Interesting facts:

Ultra-low cost training cost: Reportedly $5.6 million training cost for the base model
According to sources, DeepSeek bought a large number of Nvidia A100 GPUs (between 10,000 and 50,000 units) before U.S. chip restrictions were put in place.

Technical Analysis

The core idea of DeepSeek-R1 is to improve the reasoning capability of LLMs by incorporating a reward mechanism that incentivizes logical reasoning steps, rather than focusing solely on generating correct final answers. The model undergoes a three-phase training process:

Supervised Fine-Tuning (SFT)
Reward Model Training
Reinforcement Learning via Proximal Policy Optimization (PPO)

Stage 1. Supervised Fine-Tuning (SFT)

The process begins with supervised fine-tuning of the base DeepSeek-LLM model. The team meticulously curated a dataset combining GSM8K, MATH, LogiQA, and ProofWriter datasets. What sets their approach apart is the careful structuring of this data. Rather than simply using question-answer pairs, they incorporated detailed reasoning paths and step-by-step solutions.

The fine-tuning process employs a dynamic curriculum learning strategy. Problems are presented in increasing order of complexity, with the model's performance determining when to introduce more challenging examples. This ensures the model builds a strong foundation in basic reasoning before tackling more complex problems.

Stage 2: Reward Model Development

The heart of DeepSeek-R1's innovation lies in its reward modeling system. The team developed a specialized reward model trained on human preferences for different reasoning approaches. This wasn't simply about marking answers as correct or incorrect; instead, they collected detailed human feedback on:

The logical coherence of reasoning steps
The efficiency of solution paths
The clarity and completeness of explanations
The validity of intermediate conclusions

The reward model training involved pair-wise comparisons of different reasoning approaches for the same problems. Human evaluators, primarily mathematics and logic experts, rated these comparisons based on predefined quality criteria. This resulted in a reward model that could effectively distinguish between different levels of reasoning quality.

How DeepSeek-R1 works?

Stage 3: Reinforcement Learning

The final stage employs a modified version of Proximal Policy Optimization (PPO). The innovation here lies in how they adapted PPO for reasoning tasks. The process works as follows:

The model generates multiple reasoning attempts for each problem
The reward model evaluates each attempt based on multiple criteria
The policy is updated using a custom loss function that balances:
- Reward maximization
- KL divergence from the reference model
- Entropy regularization to maintain exploration

A key innovation is their implementation of intermediate rewards. Rather than waiting for the final answer, the model receives feedback on each step of its reasoning process. This creates a more granular learning signal, helping the model understand which specific reasoning steps are effective.

The Evolution of DeepSeek-R1

DeepSeek-R1's development is interesting because of its progress over time. It started with DeepSeek-V3-Base and used GRPO for reinforcement learning. This lead to a new model DeepSeek-R1-Zero. After many training steps, DeepSeek-R1-Zero's performance on AIME 2024 improved from 15.6% to 71.0%. With majority voting, it went up to 86.7%, matching OpenAI-o1-0912's performance. The team then refined the model by: Creating new training data through rejection sampling Combining it with DeepSeek-V3's supervised data on writing, factual QA, and self-awareness Retraining the base model Doing more reinforcement learning This careful process led to the final DeepSeek-R1 model, which performed as well as OpenAI-o1-1217. A key achievement was making the technology more accessible through model distillation: They successfully transferred capabilities to smaller models based on Qwen and Llama Their 14B distilled model did better than the top QwQ-32B-Preview The 32B and 70B distilled versions set new records for reasoning abilities

Implementation Details

DeepSeek-R1 training pipeline

The implementation required significant computational resources:

Training was conducted on a cluster of A100 GPUs
The reward model training alone took several weeks
The full training pipeline required multiple iterations and refinements

The team implemented several optimizations:

Distributed training across multiple nodes
Efficient reward computation
Dynamic batch sizing based on problem complexity
Gradient accumulation for stability

Evaluation Framework

The evaluation of DeepSeek-R1 is particularly comprehensive, going beyond simple accuracy metrics. The team developed a multi-dimensional evaluation framework:

Reasoning Quality Assessment

The model's performance is evaluated on:

Mathematical reasoning (using GSM8K and MATH benchmarks)
Logical deduction (using LogiQA and ProofWriter)
Scientific reasoning (using ScienceQA)
General problem-solving (using custom benchmarks)

Each solution is evaluated for: Step-by-step coherence, Logical validity, Efficiency of approach, and Clarity of explanation. Team also performed extensive comparisons with previous versions of DeepSeek models, other leading language models, and specialized reasoning models.

Performance comparison of DeepSeek-R1 with other models

What's particularly noteworthy is the quality of the reasoning processes. The model doesn't just arrive at correct answers; it demonstrates human-like problem-solving approaches, showing each step clearly and logically.

Challenges and Solutions

DeepSeek team faced several challenges and here’s a quick summary of challenges they faced and solutions they used to overcome the challenges.

Training Stability: Implementation of reference models and careful KL divergence constraints
Reward Sparsity: Development of dense reward signals through intermediate evaluations
Computational Efficiency: Custom optimizations in the training pipeline and reward computation
Generalization: Careful curriculum design and diverse training data

There’s a reason 400,000 professionals read this daily.

Join The AI Report, trusted by 400,000+ professionals at Google, Microsoft, and OpenAI. Get daily insights, tools, and strategies to master practical AI skills that drive results.

Exploring New LLM Applications: Insights from January 1–15, 2025 Research Papers

Mon, 20 Jan 2025 15:00:00 +0000

Fun & engaging podcast using NotebookLM

TL;DR! Listen to this amazing and fun podcast discussing below mentioned research papers!

Podcast: Application of LLMs (Jan 1-15th, 2025)

Transforming healthcare with LLMs

This section contains LLMs applications in healthcare and bioinformatics. Researchers are leveraging these models to enhance disease detection, interpret complex medical data, and democratize bioinformatics analysis.

Enhancing Alzheimer's Detection with ADAM-1

Understanding and detecting Alzheimer's disease has been a persistent challenge. The ADAM-1: AI and Bioinformatics for Alzheimer's Detection and Microbiome-Clinical Data Integrations introduces a multi-agent LLM framework that integrates microbiome profiles, clinical datasets, and external knowledge bases. By synthesizing insights through retrieval-augmented generation, ADAM-1 improves research and diagnostic applications. Remarkably, it achieved mean F1 scores comparable to XGBoost but with significantly reduced variance, highlighting its robustness, especially with small laboratory datasets.

Interpretable Mental Health Diagnostics

Mental health diagnostics often grapple with complexity and the risk of errors. Large Language Models for Interpretable Mental Health Diagnosis proposes a clinical decision support system using LLMs to translate diagnostic manuals into logic programs. The implementation involves fine-tuning an LLM to understand and convert diagnostic criteria into constraint logic programming (CLP) rules. These logic programs are then executed by a CLP engine to simulate diagnostic reasoning. The system allows experts to inspect and modify the generated logic to ensure alignment with official diagnostic standards, enhancing interpretability. Validation is performed by comparing the system's outputs with established diagnoses, emphasizing the necessity of expert review to maintain fidelity to the manuals.

Advancing Radiology Report Generation with RadAlign

RadAlign: Advancing Radiology Report Generation with Vision-Language Concept Alignment presents a model that improves automated radiology report generation by aligning visual features with medical concepts. The implementation involves a vision-language model (VLM) that extracts visual features from radiological images and maps them to predefined medical concepts using a disease classification module achieving an AUC of 0.885. These concepts guide a LLMs equipped with retrieval-augmented generation to produce comprehensive reports. The system incorporates a concept alignment mechanism to ensure consistency between the visual input and textual output and employs error mitigation strategies to reduce inaccuracies, resulting in a GREEN score of 0.678 for report quality.

Democratizing Bioinformatics with BioAgents

Bioinformatics workflows are inherently complex, requiring interdisciplinary expertise. BioAgents: Democratizing Bioinformatics Analysis with Multi-Agent Systems introduces a multi-agent system utilizing small language models tailored for bioinformatics tasks. Enhanced with retrieval-augmented generation, this system allows for personalized, local use with proprietary data. The performance was comparable to human experts on genomics tasks, paving the way for more accessible bioinformatics analysis.

Stay up-to-date with AI

The Rundown is the most trusted AI newsletter in the world, with 1,000,000+ readers and exclusive interviews with AI leaders like Mark Zuckerberg, Demis Hassibis, Mustafa Suleyman, and more.

Their expert research team spends all day learning what’s new in AI and talking with industry experts, then distills the most important developments into one free email every morning.

Plus, complete the quiz after signing up and they’ll recommend the best AI tools, guides, and courses – tailored to your needs.

Code Generation

This section is for us! Let’s see how LLMs makes our work easy or in other words snatches our job! 😀

Automating Cloud Operations with MOYA

Managing cloud infrastructure can be complex and labor-intensive. Engineering LLM Powered Multi-agent Framework for Autonomous CloudOps presents MOYA, a multi-agent framework integrating various systems using generative AI. By addressing challenges like diverse data sources and complex task automation, MOYA enhances the management and optimization of cloud infrastructure. Practitioners observed improved accuracy and responsiveness across complex workflows.

Self-Reflective Code Generation with CodeCoR

CodeCoR: An LLM-Based Self-Reflective Multi-Agent Framework for Code Generation introduces a multi-agent approach where agents specialize in task prompts, code generation, test case creation, and repair advice. By producing multiple outputs and refining failing code based on testing outcomes, CodeCoR ensures robust final code delivery. It significantly outperformed baselines with an average Pass@1 score of 77.8%.

Hierarchical Code Summarization in Business Applications

Hierarchical Repository-Level Code Summarization for Business Applications Using Local LLMs presents a two-step summarization approach implemented with local LLMs to respect confidentiality. Initially, code is parsed into small units like functions and classes, which are summarized using LLMs with prompts tailored to capture technical details and business context. In the second step, these summaries are aggregated hierarchically to create summaries for larger structures such as files and modules, utilizing custom prompts that relate these components to overall business logic. The implementation ensures that the summarization captures both the technical and domain-specific aspects, which was validated in a telecommunications business support system, enhancing code comprehension and maintenance.

Enhancing GitHub Issue Resolution with SWE-Fixer

SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution presents a framework with code retrieval and editing modules. Trained on an extensive dataset of 110K GitHub issues, SWE-Fixer achieved state-of-the-art performance among open-source models, facilitating effective issue resolution.

Advancements in Program Repair and Debugging

Several research papers have focused on automated program repair and debugging:

I Can Find You in Seconds! leverages LLMs for code authorship attribution, aiding in software forensics and plagiarism detection.
Evaluating Agent-based Program Repair at Google assesses the applicability of agent-based approaches for automated bug fixing in an enterprise environment, achieving plausible patch creation for 73% of evaluated bugs.
How is Google using AI for internal code migrations? shares insights into Google's application of LLMs for enterprise-level code migration, demonstrating significantly reduced migration times.
AIOpsLab: A Holistic Framework to Evaluate AI Agents for Enabling Autonomous Clouds introduces a framework that evaluates AI agents in cloud environments, contributing to the development of self-healing systems.
Debugging Without Error Messages explores how different LLM prompting strategies can enhance programming error explanations, aiding educators and novice programmers.

AI agents in autonomous systems

AI agents and multi-agent systems is opening new avenues for automation, collaboration, and efficiency across various domains. This section is dedicated to such papers!

Dynamic Workflow Generation with Flow

Flow: A Modular Approach to Automated Agentic Workflow Generation introduces workflows as activity-on-vertex graphs to emphasize modularity. By refining workflows through dynamic task allocations based on historical performance, the framework enhances efficiency and error tolerance, showing substantial improvements in handling practical tasks.

Integrating AI Agents with Web3 Applications through Eliza

Eliza: A Web3 Friendly AI Agent Operating System addresses the integration gap between AI agents and Web3 applications. Developed fully in TypeScript, Eliza allows users to deploy Web3 applications effortlessly. It facilitates interaction with blockchain data and smart contracts while ensuring stable performance through optimized runtime components.

Automating Systematic Reviews with LatteReview

LatteReview: A Multi-Agent Framework for Systematic Review Automation Using LLMs leverages LLMs and multi-agent systems to automate tasks like title screening and data extraction. By incorporating features like retrieval-augmented generation and asynchronous programming, LatteReview handles large datasets efficiently, integrating with both cloud-based and local models.

Context-Aware Storytelling with MDSF (Uncovers aha moment from data!)

Data storytelling is becoming increasingly important. MDSF: Context-Aware Multi-Dimensional Data Storytelling Framework Based on Large Language Model introduces a framework that uses LLMs for automated insight generation and storytelling. With advanced preprocessing, analysis algorithms, and an agent-based storytelling continuation control, MDSF outperforms existing methods in insight ranking accuracy and narrative coherence.

Democratizing LLM Services with LLM-Net

LLM-Net: Democratizing LLMs-as-a-Service through Blockchain-based Expert Networks presents a decentralized blockchain framework creating a network of specialized LLM providers. By leveraging collective computational resources and incorporating reputation-based mechanisms, LLM-Net ensures high service quality and facilitates seamless AI advancement across multiple domains.

Domain adoption

Enhancing Labor Market Analytics

Enhancing Talent Employment Insights Through Feature Extraction with LLM Finetuning utilizes semantic chunking, retrieval-augmented generation, and fine-tuning of DistilBERT models to process over one million job postings. The fine-tuned models significantly improved the identification of complex job features like remote work availability and remuneration structures.

Automated Crypto Portfolio Management

Cryptocurrency investments requires multi-modal data and intricate reasoning. LLM-Powered Multi-Agent System for Automated Crypto Portfolio Management employs specialized LLM agents that collaborate for tasks like data analysis and investment decision-making. Fine-tuned with historical data and enhanced with unique collaboration mechanisms, the framework outperformed single-agent models in classification, asset pricing, portfolio management, and explainability.

Optimizing Home Energy Management

Simplifying home energy management systems (HEMS) for non-technical users is important. Large Language Model Interface for Home Energy Management Systems presents an LLM-based interface that transforms poorly formatted user input into well-structured parameters. Using the Reason and Act method and few-shot prompting, the system achieved an average parameter retrieval accuracy of 88%, enhancing user interaction with power systems.

Power Grid Optimization with SafePowerGraph-LLM

Modern power grids face increasing complexity. SafePowerGraph-LLM: Novel Power Grid Graph Embedding and Optimization with Large Language Models introduces a framework that leverages LLMs combined with graph and tabular representations to solve Optimal Power Flow problems. Utilizing in-context learning and fine-tuning protocols, the framework reliably handled realistic grid components and constraints, demonstrating the impact of LLM architecture on performance.

Improving Time Series Forecasting with Pre-trained LLMs

Using Pre-trained LLMs for Multivariate Time Series Forecasting leverages the capabilities of pre-trained LLMs by mapping multivariate inputs into the LLM's token embedding space. Utilizing a novel multivariate patching strategy, the approach embeds features into decoder-only pre-trained transformers. The results are competitive with state-of-the-art forecasting models, showcasing the potential of LLMs in complex data analysis tasks.

Innovative FPGA Architecture for Efficient Computing

Monolithic 3D FPGAs Utilizing Back-End-of-Line Configuration Memories introduces a novel FPGA architecture that significantly improves area, latency, and power efficiency. By using back-end-of-line (BEOL) transistors for configuration memory and pass gates, and integrating n-type and p-type amorphous oxide semiconductors (AOS) under development, the researchers developed physics-based models interfaced with Verilog-to-Routing tools. Demonstrated at 7 nm technology, the proposed M3D FPGA design reduces the area-time squared product by 3.4x and decreases critical path latency by 27%. This advancement has broad implications for industries relying on efficient computing, including the deployment of LLMs.

Robotics/VLLMs application/improvement

Advancing 3D Scene Understanding with 3UR-LLM

3UR-LLM: An End-to-End Multimodal Large Language Model for 3D Scene Understanding addresses this by building the 3DS-160K dataset and introducing a model that processes 3D point cloud data. Utilizing a 3D compressor module, the model efficiently handles computation demands, outperforming previous state-of-the-art models by 7.1% on relevant benchmarks.

VLMs as Operator Agents in Space

Visual Language Models as Operator Agents in the Space Domain explores the application of Vision-Language Models in autonomous control and decision-making for space missions. In both software simulations and hardware contexts, VLMs effectively processed visual inputs to perform orbital maneuvers and inspect space objects. These models compete with or outperform traditional methods, showing promise for future space exploration.

Compositional Text-to-Video Generation with BlobGEN-Vid

Generating videos that accurately reflect complex text prompts is a very challenging task. BlobGEN-Vid: Compositional Text-to-Video Generation with Blob Video Representations introduces a model that uses blob-grounded video diffusion to control object motions and appearances. With enhanced regional consistency and semantic control, BlobGEN-Vid demonstrates superior zero-shot video generation capabilities, surpassing proprietary generators in compositional accuracy.

Affective Tactile Interaction Driven by LLMs

Touched by ChatGPT: Using an LLM to Drive Affective Tactile Interaction leverages LLMs to generate tactile signals that convey emotions. Using a wearable sleeve equipped with vibration motors, unique patterns associated with specific emotions were generated. Participants accurately recognized intended emotions, demonstrating the effectiveness of LLMs in communicating emotion via tactile signals.

LLMs related research papers published in December 2024

LLMs Research — Sun, 12 Jan 2025 15:00:00 +0000

🔑 takeaway from today’s newsletter

Tokens are so yesterday! The Byte Latent Transformer ditches tokens for dynamic byte patches, making models faster and more efficient.
Less is more! TrimLLM trims unnecessary layers, boosting speed without sacrificing smarts. It's like a transformer on a diet!
Now you cache it, now you don't! Slashing KV cache memory usage to just 20%, it's the Houdini of memory optimization.
Now you cache it, now you don't! Slashing KV cache memory usage to just 20%, it's the Houdini of memory optimization.
From drone dances to AR cooking! See how LLMs are shaking things up in creative ways you never imagined.

TL;DR - No time to read entire newsletter? Don’t worry We’ve got you covered! Listen 🎧 to this fun and informative podcast 🎙️ using Google’s NotebookLM which discuss important research papers from this newsletter in good detail with real world references!

Podcast: December'24 research summary

Architecture & Token Processing

Research papers from this section explore fundamental architectural improvements and token processing methods, achieving better scalability and efficiency through novel approaches to model architecture and token handling.

Byte Latent Transformer: Patches Scale Better Than Tokens 🔥

The Byte Latent Transformer (BLT) operates at the byte level rather than on tokens. Instead of fixed-size tokens, BLT forms dynamically sized byte patches determined by the entropy (unpredictability) of the next byte. High-entropy regions, which contain more information, receive more computational focus.

Dynamic Patching: BLT groups sequences of bytes into variable-length patches based on predictability, efficiently capturing patterns without relying on a predefined vocabulary.
Entropy-Based Processing: By allocating computational resources according to the complexity of the data (measured by entropy), BLT ensures that challenging inputs receive more attention.

The researchers conducted a FLOP-controlled scaling study, training BLT models with up to 8 billion parameters on 4 trillion bytes of data. This study assessed the efficiency and performance of byte-level models compared to traditional token-based LLMs.

Enhanced Computationally Efficient Long LoRA Inspired Perceiver Architectures for Auto-Regressive Language Modeling

This research builds upon the Perceiver AR architecture, designed to handle longer sequences efficiently. By integrating ideas from Long Low-Rank Adaptation (Long-LoRA), the researchers develop an enhanced architecture called the Long LoRA Perceiver. This model introduces architectural enhancements that balance computational efficiency and performance. It applies low-rank approximations to the attention matrices, reducing computational complexity from quadratic to linear with respect to sequence length. Additionally, it utilizes hierarchical attention mechanisms to capture both local and global dependencies efficiently, enabling the model to focus on relevant information across different scales within the sequence. By carefully optimizing model depth, width, and other hyperparameters, the Long LoRA Perceiver maximizes performance while keeping computational demands manageable.

Llama 3 Meets MoE: Efficient Upcycling

The research introduces an efficient method to "upcycle" a pre-trained dense model into an MoE architecture. Specifically, an 8-Expert Top-2 MoE model is trained using pre-trained dense checkpoints from Llama 3-8B. This approach leverages existing knowledge in the dense model and integrates MoE layers with minimal additional training. By adding MoE layers where each expert specializes in different data subsets, the model increases its capacity. During inference, only the top two most relevant experts are activated per input token, keeping computational costs similar to the original dense model. The upcycling process requires less than one epoch of training, significantly reducing time and resources compared to training an MoE model from scratch. Initializing the MoE model with pre-trained dense weights helps maintain performance while benefiting from the increased capacity.

Results: The upcycled MoE model achieves a 2× increase in model capacity compared to the original Llama 3-8B dense model

Please support my work by checking what this advertise offer. Maybe it helps you!👇

Want to get the most out of ChatGPT?

ChatGPT is a superpower if you know how to use it correctly.

Discover how HubSpot's guide to AI can elevate both your productivity and creativity to get more things done.

Learn to automate tasks, enhance decision-making, and foster innovation with the power of AI.

Download the free guide with actionable prompts and tips to make your work more productive.

Model Efficiency & Compression

Recent advancements have introduced innovative techniques to compress and accelerate LLMs. current methods such as quantization and layer pruning achieve substantial speedups (2× to 5×) and memory savings while maintaining, or even enhancing, model performance. This section covers papers improving the LLMs efficiency and their compression.

TrimLLM: Progressive Layer Dropping for Domain-Specific LLMs

TrimLLM introduces a progressive layer dropping technique inspired by the layer-wise specialization phenomenon observed in LLMs. The method begins with a thorough analysis of each transformer's layer contribution to the model's performance on domain-specific tasks. By measuring changes in loss or accuracy when a layer is temporarily removed, less critical layers are identified.

The progressive layer dropping involves iteratively removing these least important layers. After each layer is dropped, the model is evaluated on validation data to ensure performance remains within acceptable bounds. Domain-specific fine-tuning is applied after each pruning step, allowing the model to adjust to its new, slimmer architecture and mitigate any performance loss. Also, this methodology does not require specialized hardware, making it accessible for a wide range of users.

Results: TrimLLM achieved 2.1-5.7× inference speedup on consumer GPUs and up to 3.1× on A100 GPUs compared to state-of-the-art compression methods, while maintaining no loss in accuracy at 50∼60% model compression ratio.

Low-Rank Correction for Quantized LLMs 🔥

Microsoft research team introduced a method to mitigate quantization errors by introducing low-rank weight matrices in full precision that act on unquantized activations. The approach augments the quantized weight matrix with a low-rank correction matrix, resulting in a corrected weight matrix that combines both. A joint optimization objective is defined to minimize the discrepancy between the outputs of the corrected quantized model and the original full-precision model.

During fine-tuning, quantization-aware training is performed where both the quantized weights and correction matrices are updated. The quantized weights are adjusted using quantized gradients, while the correction matrices are updated using full-precision gradients, effectively compensating for the precision loss in aggressive quantization. By correcting the activations through these low-rank matrices, the method restores model accuracy without significantly increasing model size, as the correction matrices require minimal additional storage.

Results: Proposed approach reduced the accuracy gap by over 50% compared to quantized models without correction.

LLM-BIP: Structured Pruning for Large Language Models with Block-Wise Forward Importance Propagation

LLM-BIP introduces a structured pruning technique that calculates the importance of parameters using block-wise forward importance propagation. The model is divided into blocks, such as attention heads or MLP layers. The method simulates the impact of pruning each block by approximating its effect on the model's output using a forward pass and leveraging Lipschitz continuity to ensure stability.

By assessing how changes in a block affect the loss function, the importance of different structures is accurately evaluated. Blocks are ranked based on these importance scores, and the least important ones are pruned while ensuring the model remains functional. After pruning, fine-tuning is conducted to recover any potential performance degradation, using learning rate schedules that stabilize training.

Results: The method increases accuracy by 3.26% compared to previous pruning methods at similar compression rates.

DFRot: Achieving Outlier-Free and Massive Activation-Free for Rotated LLMs with Refined Rotation

DFRot reduces outliers in LLM quantization by refining weight matrices through rotation to produce outlier-free and evenly distributed activation values. The method applies randomized Hadamard transforms to the weight matrices, spreading out large values and reducing the chance of large activations. A weighted loss function is introduced to penalize activations exceeding a certain threshold, effectively handling long-tail distributions.

The rotation matrices are further refined using orthogonal Procrustes optimization, ensuring the transformed weights remain close to the original weights while achieving desired activation properties. This results in a more uniform activation distribution, facilitating better quantization without sacrificing precision for other values.

Results: DFRot improved perplexity by 0.25 on W4A4KV4 and 0.21 on W4A4KV16, demonstrating effectiveness in reducing outliers and massive activations in quantized LLaMA3-8B.

GeLoRA: Geometric Adaptive Ranks For Efficient LoRA Fine-tuning

GeLoRA introduces a geometry-inspired approach that dynamically adjusts the ranks of Low-Rank Adaptation (LoRA) matrices based on the intrinsic dimensionality of hidden state representations in each layer. By computing the singular values of hidden representations, the method estimates the intrinsic dimensionality, with layers processing more complex information receiving higher ranks.

Analyzing the decay of singular values allows GeLoRA to adjust ranks to match each layer's complexity, ensuring efficient parameter allocation. This leverages geometric principles to determine a theoretical lower bound for optimal rank selection, balancing efficiency and expressiveness.

During fine-tuning, only the LoRA parameters are updated while the base model remains frozen. This parameter-efficient strategy enables task-specific adaptation without requiring full model training, making it suitable for environments with limited resources.

Memory Management & KV Cache Optimization

The Key-Value (KV) cache is very important for efficient transformer model inference, storing past hidden states to avoid redundant computations. However, the KV cache can consume significant memory resources, leading to scalability issues and high computational costs.

Recent research has focused on optimizing KV cache memory usage without compromising model performance. By developing sophisticated compression and pruning techniques, researchers aim to reduce memory footprints by up to 80-90%, enabling LLMs to handle longer sequences and be deployable in resource-constrained environments.

In this section, we explore innovative approaches that address memory efficiency in LLMs, particularly focusing on KV cache optimization through dynamic compression, sparse coding, adaptive strategies, and novel pruning methods.

ZigZagkv: Dynamic KV Cache Compression for Long-context Modeling based on Layer Uncertainty

ZigZagkv introduces a dynamic KV cache compression method that allocates memory budgets per layer based on layer uncertainty. The key idea is to assess the necessity of memory allocation for each layer individually, rather than applying a uniform compression strategy across all layers. The method estimates uncertainty scores for each layer by analyzing attention patterns and the variability of hidden state outputs. Layers with higher uncertainty—indicating greater importance for accurate predictions—are assigned larger memory budgets to retain more precise KV representations. Conversely, layers with lower uncertainty have their KV caches compressed to save memory without significantly impacting performance.

By dynamically adjusting the compression ratio for the KV cache in each layer, the model retains essential information in critical layers while substantially reducing overall memory usage. This approach ensures efficient memory utilization, enabling LLMs to handle longer contexts without exceeding memory limitations.

Results: The proposed method reduces KV cache memory usage to approximately 20% of the original size during inference, significantly alleviating memory bottlenecks.

Lexico: Extreme KV Cache Compression via Sparse Coding over Universal Dictionaries

Lexico uses a sparse coding approach using a universal dictionary to approximate the KV caches with sparse linear combinations. The universal dictionary, consisting of approximately 4,000 atoms (basis vectors), captures essential patterns in KV caches across various inputs, tasks, and model architectures. By representing each key and value vector as a sparse combination of these atoms, the method compresses the KV cache significantly.

Orthogonal Matching Pursuit (OMP) is utilized to find the optimal sparse representation for each vector, ensuring accurate reconstruction with minimal coefficients. The sparsity level can be adjusted to control the compression ratio, allowing flexible trade-offs between memory savings and performance. This approach maintains the model's ability to reconstruct necessary information during inference, achieving extreme compression without significant loss.

Results: Lexico maintains 90-95% of the original model performance despite significant compression.

EMS: Adaptive Evict-then-Merge Strategy for Head-wise KV Cache Compression Based on Global-Local Importance

EMS introduces an adaptive evict-then-merge strategy for KV cache compression by calculating a Global-Local Importance (GLI) score for each token. The GLI score integrates global accumulated attention scores—reflecting how much a token is attended to throughout the entire sequence—with local attention scores that capture importance within specific contexts or attention heads. This combined score provides a more accurate estimation of token significance.

Using the GLI scores, less important tokens are first evicted to reduce the KV cache size. Then, redundant or similar tokens are merged by aggregating their representations, further saving memory. This process is applied on a per-head basis, recognizing that different attention heads may focus on different aspects of the input. A zero-class mechanism is introduced to handle evicted tokens consistently during inference without adverse effects.

By adapting the compression strategy based on the GLI scores and processing it head-wise, EMS effectively balances memory savings with performance preservation, allowing efficient handling of long-context inputs in LLMs.

Results: EMS achieves state-of-the-art performance with low perplexity, improving metrics by over 1.28 points on four LLMs evaluated on the LongBench benchmark under a 256 token cache budget.

Cross-Self KV Cache Pruning for Efficient Vision-Language Inference

The proposed method decomposes attention scores into intra-modality (within the same modality) and inter-modality (across modalities) components. By separately analyzing these components, the model can more precisely estimate the importance of tokens in both visual and textual modalities.

Intra-modality attention captures relationships among tokens within the same modality, helping to identify essential visual features or textual elements. Inter-modality attention examines interactions between modalities, ensuring that tokens critical for cross-modal understanding are retained. Additionally, an n-softmax function is introduced to maintain the smoothness of attention scores after pruning, adjusting attention distributions to account for the pruned tokens and preventing abrupt changes that could destabilize performance.

By employing this tailored pruning approach, the model effectively reduces the KV cache size without losing important information from either modality, leading to more efficient inference in vision-language tasks.

Results: The Cross-Self Pruning (CSP) method matches the performance of models with full KV caches and outperforms previous pruning methods, achieving up to a 41% memory reduction.

iLLaVA: An Image is Worth Fewer Than 1/3 Input Tokens in Large Multimodal Models

iLLaVA introduces a method to optimize LVLMs by merging redundant tokens in image representations. Utilizing a precise and rapid algorithm, the method identifies tokens within image inputs that carry redundant or less significant information. These redundant tokens are merged, effectively reducing the number of tokens required to represent an image.

To ensure important visual information is not lost, the method recycles information from the removed tokens by integrating their contributions into the remaining tokens. This process preserves essential features and relationships necessary for the model's performance. By reducing the token count for images, iLLaVA decreases memory usage and doubles the inference throughput, as the model has fewer tokens to process. The approach achieves these efficiency gains with minimal modifications to the model architecture, facilitating easy integration into existing systems.

Results: iLLaVA doubles throughput and halves memory consumption with only a 0.2% drop in performance, demonstrating efficiency gains without sacrificing accuracy.

Training & Optimization Methods

These papers improve LLM training efficiency and effectiveness through enhanced optimization techniques, distributed training methods, and novel adaptation approaches, achieving better convergence and reduced computational requirements.

EDiT: A Local-SGD-Based Efficient Distributed Training Method for Large Language Models

EDiT introduces an efficient distributed training framework that integrates a specific variant of Local Stochastic Gradient Descent (Local SGD) with model sharding. By allowing workers to perform multiple local updates before synchronizing, the method reduces communication frequency and overhead. The key innovation lies in using layer-wise synchronization during the forward pass. This approach overlaps computation and communication, minimizing idle time and enhancing efficiency.

To stabilize training, EDiT introduces a pseudo-gradient penalty that acts as a regularizer, mitigating the divergence that can occur with delayed updates in Local SGD. Furthermore, recognizing the heterogeneity of cluster environments, the researchers present A-EDiT, an asynchronous version of the method. A-EDiT accommodates varied computational resources and network conditions by allowing workers to operate without strict synchronization, further improving scalability and robustness in diverse training environments.

APOLLO: SGD-like Memory, AdamW-level Performance

APOLLO addresses this challenge by proposing an optimizer that retains the memory efficiency of SGD while achieving performance comparable to AdamW. The key idea is to approximate the per-parameter adaptive learning rates of AdamW through a structured update rule that relies on an auxiliary low-rank optimizer state. By leveraging random projection techniques, APOLLO maintains a low-rank approximation of the gradients, capturing essential variance information without the need to store full-size moment estimates.

This approach reduces memory usage significantly since the auxiliary optimizer state requires far less storage than the full moment vectors. APOLLO adjusts learning rates adaptively based on the low-rank approximation, effectively emulating the benefits of AdamW's adaptive updates. Additionally, the researchers introduce APOLLO-Mini, a variant designed for even lower memory environments, making it suitable for training on devices with limited resources.

Results: APOLLO and its variant, APOLLO-Mini, deliver competitive or superior performance compared to AdamW, with significant memory savings: 3x throughput and 4x larger batch sizes on an 8xA100-80GB setup, enabling model scalability and low-end GPU training.

KaSA: Knowledge-Aware Singular-Value Adaptation of Large Language Models

KaSA introduces a novel PEFT method that employs Singular Value Decomposition (SVD) with knowledge-aware singular values to adapt LLMs more efficiently. The method decomposes weight matrices into singular vectors and singular values, allowing fine-grained control over the model's parameter adaptation. By analyzing task relevance, KaSA dynamically activates singular values corresponding to components of the model that are most pertinent to the target task.

This knowledge-aware activation reduces noise by de-emphasizing irrelevant parameters, effectively focusing the model's capacity on task-specific knowledge. The approach improves expressiveness without significantly increasing the computational burden. By updating only the most relevant singular values, KaSA minimizes memory overhead and accelerates the fine-tuning process.

Results: KaSA outperforms 14 popular PEFT baselines and FFT across 16 benchmarks and 4 synthetic datasets, demonstrating its efficacy in tasks such as NLU, NLG, and commonsense reasoning.

Taming Sensitive Weights : Noise Perturbation Fine-tuning for Robust LLM Quantization

Paper introduces Noise Perturbation Fine-Tuning (NPFT), a method designed to diminish the impact of sensitive weights during quantization. NPFT involves applying random weight perturbations during parameter-efficient fine-tuning (PEFT). By adding controlled noise to the weights identified as sensitive (those with high influence on the loss Hessian trace), the method encourages the model to distribute importance more evenly across parameters.

This process reduces the reliance on outlier weights, effectively smoothing the loss landscape and enhancing the model's robustness to quantization errors. NPFT operates without requiring special treatment for outliers or altering the model architecture, making it a straightforward addition to existing fine-tuning procedures. By integrating NPFT with PEFT, the approach maintains parameter efficiency while improving quantized model performance.

Results: NPFT enhanced the performance of OPT and LLaMA models across uniform and non-uniform quantizers. On the LLaMA2-7B-4bits benchmark, RTN matched the performance of GPTQ, achieving better inference efficiency.

Knowledge Management & Reasoning

This section focuses on research that helps LLMs to acquire, maintain, and utilize knowledge through innovative approaches such as memory compression, knowledge graph completion, interpretability techniques, and novel prompt optimization methods. These advancements not only improve the models reasoning capabilities but also address challenges like continual learning, interpretability, and efficient adaptation.

CMT: A Memory Compression Method for Continual Knowledge Learning of Large Language Models

The CMT method emulates human memory by compressing and storing information from new documents into an external memory bank. Instead of updating the entire model, CMT retrieves relevant compressed memories during query processing. It introduces three key techniques to enhance performance:

Memory-Aware Objective: This encourages effective utilization of the compressed memory by adjusting the training objective, balancing reliance on stored knowledge versus internal parameters.
Self-Matching: It improves the quality of stored memories by matching and integrating similar information within the memory bank, enhancing representational coherence.
Top-Aggregation: During inference, it retrieves and aggregates the most relevant memories, allowing the model to access and reason over pertinent information without altering its original parameters.

By leveraging these techniques, CMT reduces catastrophic forgetting—the loss of previously learned knowledge when new information is added—allowing the model to incorporate new knowledge seamlessly.

Results: The CMT method demonstrates improved adaptability and robustness, achieving enhancements like +4.07 EM and +4.19 F1 on the StreamingQA dataset with Llama-2-7b.

A Contextualized BERT model for Knowledge Graph Completion

The study introduces a contextualized BERT model that leverages contextual information from neighboring entities and relationships within the knowledge graph. Unlike traditional KGC approaches that rely heavily on entity descriptions or require negative triplet sampling, this model incorporates structural context directly into the embeddings. By encoding the neighborhood information of entities using BERT's transformer architecture, the model captures both local and global semantics, enabling it to infer missing links more accurately. This approach reduces computational demands and mitigates semantic inconsistencies by eliminating the need for negative sampling.

Results: The model outperforms existing methods, achieving a 5.3% improvement in standard evaluation metrics such as Mean Reciprocal Rank (MRR) and Hits@N.

Frame Representation Hypothesis: Multi-Token LLM Interpretability and Concept-Guided Text Generation

Researchers propose the Frame Representation Hypothesis, modeling multi-token words as frames—a sequence of vectors representing each token in the word. Analyzing these frames uncovers how LLMs internally represent complex concepts composed of multiple tokens. By interpreting concepts using the average of word frames that share the same concept, the methodology facilitates manipulation of these concepts within the model's latent space. This enables detection and modification of biases, as well as concept-guided text generation where specific concepts can be emphasized or de-emphasized in generated text. The approach was validated on models such as Llama 3.1, Gemma 2, and Phi 3, demonstrating its ability to identify biases related to gender and language and to remediate them through frame manipulation.

GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong Prompt Optimizers

GReaTer introduces a novel approach that utilizes task loss gradients for prompt optimization in smaller language models. Instead of relying solely on textual feedback or guidance from larger models, GReaTer allows the smaller model to introspect its own performance by analyzing gradients computed during task-specific losses. By leveraging these gradients, the model adjusts its prompts to improve performance iteratively. This self-improvement mechanism enables the model to enhance its reasoning capabilities without external assistance or significant computational overhead, making it practical for deployment in resource-constrained environments.

Creative ways to use LLMs!!

Semantic Steganography: A Framework for Robust and High-Capacity Information Hiding using Large Language Models - Paper constructs a semantic space to map secret messages using ontology-entity trees, enhancing robustness, transmission reliability, and indistinguishability of stego texts.

Embracing Large Language Models in Traffic Flow Forecasting - Paper uses LLMs to improve traffic flow prediction by capturing complex spatio-temporal relationships through graph and hyper graph structures, advancing intelligent transportation systems.

IntelEX: A LLM-driven Attack-level Threat Intelligence Extraction Framework - Paper uses LLMs to convert unstructured cyber threat intelligence reports into structured formats, facilitating better security analysis and response planning.

WHAT-IF: Exploring Branching Narratives by Meta-Prompting Large Language Models [Marvel Fans check this paper!] 🔥 - Paper applies zero-shot meta-prompting with GPT-4 to generate branching narratives from a prewritten story. Starting with a linear plot, it creates branches at key decision points by prompting the LLM to consider major plot points, ensuring coherent alternate storylines. The branching plot is stored in a graph for tracking and structuring the interactive fiction system.

SwarmGPT-Primitive: A Language-Driven Choreographer for Drone Swarms Using Safe Motion Primitive Composition - Paper achieves smooth drone swarm choreographies using natural language commands powered by LLMs, integrating motion planning and safety constraints.

Beyond pip install: Evaluating LLM Agents for the Automated Installation of Python Projects - Paper Creates an agent named Installamatic that autonomously installs Python project dependencies by interpreting and following documentation, improving software installation processes.

HiVeGen -- Hierarchical LLM-based Verilog Generation for Scalable Chip Design - Paper uses hierarchical LLMs to generate complex hardware designs in Verilog by structuring tasks into smaller submodules, enhancing scalability and reducing errors in chip design.

Rethinking Time Series Forecasting with LLMs via Nearest Neighbor Contrastive Learning - Paper adapts LLMs for time series forecasting using Nearest Neighbor Contrastive Learning to align textual embeddings with time series data, excelling in few-shot and long-term predictions.

ARChef: An iOS-Based Augmented Reality Cooking Assistant Powered by Multimodal Gemini LLM - Paper combines LLMs with augmented reality to identify ingredients and generate personalized recipes through an iOS application, enhancing the cooking experience and accessibility.

LLMs related research papers published in November 2024

Fri, 27 Dec 2024 15:21:44 +0000

🔑 takeaway from today’s newsletter

Smarter Thinking: Fixing logic gaps with Critical Tokens and tackling multi-hop reasoning challenges.
Efficient Fine-Tuning: Innovations like LoRA-SB cut costs without compromising performance.
Compact Models: Faster, lighter LLMs with breakthroughs like FlexiBit and MixPE.
Creative Applications: Endless panoramas, AI storytelling, and dynamic simulations powered by LLMs.
Sharper Understanding: Syntax tools and self-distillation improve accuracy and versatility.

🔑 Fun & engaging podcast using NotebookLM

Don’t have much time to read entire newsletter? Well, listen to this fun and engaging podcast covering these research papers in a detailed manner!

November 2024 research summary

Token-Level Reasoning and Syntax

Critical Tokens Matter: Token-Level Contrastive Estimation Enhances LLM's Reasoning Capability

Why?: Discriminating influential 'critical tokens' which leads to incorrect reasoning paths can help improve LLMs reasoning ability.

How?: The authors propose a method called cDPO (Contrastive Direct Preference Optimization) to identify and correct these troublesome tokens. The method is built on the insight that not all tokens are equally important; some can drastically shift the model’s reasoning trajectory.

Positive vs. Negative Reasoning Trajectories
- They collect positive (correct) reasoning paths and negative (incorrect) ones, often from a chain-of-thought or step-by-step solution. The key is to figure out which tokens appeared in correct vs. incorrect solutions, and how likely the LLM is to generate those tokens under different conditions.
Contrasting Likelihoods
- They train two separate LLM “heads” or fine-tuned variants: one favoring positive tokens, the other favoring negative tokens. This allows them to contrast how likely the model is to produce certain “critical tokens” in each scenario. If a token is much more probable in the negative model, it might be a strong predictor of failure.
Token-Level DPO
- Direct Preference Optimization (DPO) is usually done at the sequence level (i.e., preferring one entire output over another). The authors adapt DPO at the token level—pinpointing precisely where the model’s reasoning diverges. This token-level alignment is the core innovation: rather than treating a whole reasoning chain as “good or bad,” they surgically intervene on the problematic tokens.

Sneaking Syntax into Transformer Language Models with Tree Regularization

Why?: Paper addresses the need for syntactic inductive biases in transformer language models to improve their robustness and data efficiency, without limiting model expressivity or increasing inference complexity.

How?: The authors propose TreeReg, a technique that blends syntactic constraints into a Transformer’s hidden stateswithout altering the core model architecture. Rather than fully retraining a parser, they apply a carefully designed auxiliary loss that leverages syntactic bracketing (e.g., parse trees).

Syntactic Brackets
- Traditional syntactic parsers produce bracketed structures indicating phrases, clauses, etc.
- These brackets encode where one syntactic constituent ends and another begins.
Orthogonality Constraints
- TreeReg translates bracket information into differentiable constraints on hidden states: if two tokens are in the same syntactic constituent, their vector representations should align differently than tokens in different constituents.
- By encouraging orthogonality (or closeness) based on bracket positions, the model “feels” the syntactic boundaries directly in its feature space.
No Architecture Overhaul
- Crucially, no new parameters or special modules are added. Instead, the model gains a syntax-aware training signal through a supplementary loss that can be combined with standard next-token prediction or classification losses.

Enhancing Character-Level Understanding in LLMs through Token Internal Structure Learning

Why?: Existing tokenization methods, such as Byte-Pair Encoding (BPE), which often obscure the internal character structures within tokens, hindering LLMs' ability to grasp these details and perform effectively on tasks with limited data.

How?: Paper introduces a method called Token Internal Position Awareness (TIPA), which trains LLMs on reverse character prediction tasks using the tokenizer's vocabulary. This approach helps models learn internal token structures and character positions, enhancing their understanding.

Results: LLMs trained with TIPA outperform baseline models in predicting character positions at the token level and show improved performance and faster convergence on the downstream task of Chinese Spelling Correction (CSC).

The Two-Hop Curse: LLMs trained on A->B, B->C fail to learn A-->C

Why?: Understanding the limitations of LLMs in performing internal reasoning without explicit chain-of-thought helps to enhance their reasoning capabilities, which is essential for advancing LLMs applications.

How?: Paper introduces a controlled experimental setting to assess two-hop reasoning in LLMs by training models on fictional facts and testing them on their ability to perform reasoning tasks both with and without chain-of-thought (CoT) aid. The models, including Llama 3 8B Instruct and GPT-4o, were evaluated on their performance in generalizing and composing learned facts across different documents and prompts.

Results: Models succeeded at two-hop reasoning using CoT but failed when reasoning required internal inference without CoT. The failure was evident when learned facts were in separate documents, highlighting LLMs inability for latent multi-hop reasoning without external aids in over half of the cases.

Efficient and Effective Fine-Tuning

Initialization using Update Approximation is a Silver Bullet for Extremely Efficient Low-Rank Fine-Tuning - GitHub

Why?: Fine-tuning LLMs often involves updating billions of parameters, which is computationally expensive and requires high-end hardware. LoRA can drastically reduce the number of trainable parameters by decomposing the weight updates into low-rank matrices. However, standard LoRA approaches often lag behind full fine-tuning in terms of performance and may demand extensive hyperparameter tuning to close that gap.

How?: The authors observe that while LoRA significantly cuts parameter counts, its performance can be very sensitive to how the low-rank matrices are initialized and scaled. Traditional approaches typically rely on random or naive initializations, requiring lengthy hyperparameter sweeps.

Gradient-Based Approximation
LoRA-SB approximates the initial gradient of a full fine-tuning step to seed the low-rank matrices. By focusing on the directions in parameter space that matter most for the task, it effectively “zeroes in” on the relevant updates right from the start.
Constrained Update Space
The low-rank adapters operate in a restricted subspace compared to full model updates. Finding the best possible initialization in that subspace is critical because every parameter or gradient dimension counts more when you have fewer degrees of freedom.
No Extra Hyperparameters
A major advantage is that LoRA-SB does not introduce additional hyper parameters. It’s a straightforward re-initialization scheme that can be easily integrated into existing LoRA pipelines.

Results: LoRA-SB outperforms standard LoRA and LoRA-XS, achieving efficient fine-tuning with 27-90x fewer parameters while maintaining high performance across various reasoning and language tasks.

Puzzle: Distillation-Based NAS for Inference-Optimized LLMs

Why?: Deploying LLMs on resource-constrained hardware (e.g., a single GPU or edge devices) can be extremely expensive and slow at inference time.

Neural Architecture Search (NAS) can find specialized, smaller architectures—but typically requires extensive compute and big teacher models.
Knowledge distillation helps preserve performance in a smaller model, but it’s often done in a global, all-at-once manner that doesn’t account for hardware constraints at a granular level.

The Puzzle framework aims to co-optimize:

Architecture design (via NAS) and
Performance retention (via distillation),
all while ensuring the resulting model meets specific hardware constraints (e.g., fits on a single GPU with minimal latency).

How?: Puzzle combines blockwise local knowledge distillation (BLD) with mixed-integer programming to systematically prune and restructure an LLM, step by step, into a smaller yet high-performing network.

Blockwise Local Distillation (BLD)
- Instead of training a smaller model from scratch or distilling globally all at once, the approach partitions the original model into “blocks.” Each block is distilled locally, ensuring the sub-architecture within that block preserves knowledge from the teacher. This local focus can make distillation more stable and more aligned with the final architecture changes.
Mixed-Integer Programming for NAS
- Puzzle sets hardware constraints (e.g., model size, memory footprint, or inference time) as optimization objectives. It then systematically selects or prunes blocks and layers, guided by a global objective function. Mixed-integer programming finds an optimal combination of blocks that fit the constraints while aiming for maximal performance.
Iterative Refinement
- The framework iterates, refining architecture choices and distilling block by block. Each iteration zeros in on a final architecture that best balances size, speed, and accuracy.

Results: Nemotron-51B, derived from Puzzle, achieves 2.17x inference speedup fitting on a single NVIDIA H100 GPU, retaining 98.4% of the teacher model’s performance.

Dynamic Self-Distillation via Previous Mini-batches for Fine-tuning Small Language Models

How?: The researchers propose a model-agnostic self-distillation method, DynSDPB, which learns from the model's own previous mini-batch outputs. This approach dynamically adjusts distillation influence and temperature to improve early iteration accuracy. It is a fine-tuning policy that integrates self-correction and self-training methods without architectural modification.

Previous Mini-Batch Outputs: Instead of referencing an external teacher, the model looks at its own predictions (logits, soft labels) from the previous mini-batch. These predictions become the pseudo “teacher” for the next iteration.
Dynamic Adjustments: The distillation influence (how strongly the model trusts its past predictions) and temperature (softening or sharpening the pseudo labels) are adjusted on the fly during fine-tuning. This dynamic tuning ensures the self-distillation signal remains useful and stable as the model learns.
No Extra Architectural Changes: Importantly, no additional layers or model capacity is required—just a revised fine-tuning policy that leverages the previous batch’s logits.

A word from today’s sponsor!

Love Hacker News but don’t have the time to read it every day? Try TLDR’s free daily newsletter. TLDR covers the best tech, startup, and coding stories in a quick email that takes 5 minutes to read. No politics, sports, or weather (we promise). And it's read by over 1,250,000 people!

Subscribe for free now and you'll get our next newsletter tomorrow morning.

Quantization: Memory & Speed Gains

Low-Bit Quantization Favors Undertrained LLMs: Scaling Laws for Quantized LLMs with 100T Training Tokens

Why?: Paper explores how low-bit quantization interacts with the training level of LLMs, uncovering implications for model efficiency and future training strategies.

How?: The study involved analyzing over 1500 quantized LLM checkpoints with variations in model size and training levels. Scaling laws were derived to explore the relationship between quantization-induced degradation (QiD) and factors like training tokens, model size, and bit width.

Results: The study revealed that undertrained models are less susceptible to QiD impacts compared to fully trained small models. These insights provide benchmarks and predictions for quantization performance in massive future models expected to be trained with 100 trillion tokens.

Pushing the Limits of Large Language Model Quantization via the Linearity Theorem

Why?: Quantization often impacts performance unevenly across layers, but identifying which layers tolerate fewer bits while maintaining overall model quality is a challenge. This research introduces a systematic framework to minimize this trade-off, optimizing layer-specific quantization.

How?: The authors introduce the 'linearity theorem' to correlate layer-wise reconstruction error with increased perplexity due to quantization. They developed the HIGGS method employing Hadamard rotations and MSE-optimal grids and proposed an efficient dynamic programming approach for non-uniform per-layer quantization.

Results: The proposed methods enhance accuracy-compression trade-offs for Llama-3.1, Llama-3.2, and Qwen models, outperforming existing data-free approaches like NF4.

MixPE: Quantization and Hardware Co-design for Efficient LLM Inference

Why?: The research tackles the computational and memory challenges inherent in deploying LLMs by introducing a more efficient quantization approach.

How?: MixPE introduces a mixed-precision processing element to enhance inference efficiency. It minimizes dequantization operations using two innovations: performing dequantization post per-group mixed-precision matrix multiplication and replacing traditional multipliers with shift add operations, thus boosting computational and energy efficiency.

Results: MixPE achieves a 2.6× speedup and 1.4× energy reduction over current quantization accelerators.

FlexiBit: Fully Flexible Precision Bit-parallel Accelerator Architecture for Arbitrary Mixed Precision AI

Why?: Current hardware accelerators are rigid, often supporting only standard precisions (e.g., FP16 or INT8). This limitation hampers the potential of custom mixed-precision arithmetic, which could better exploit model-specific needs and improve performance.

How?: FlexiBit proposes FlexiBit, a bit-parallel accelerator architecture which supports flexible precision and dynamic formats. It overcomes the limitations of bit-serial designs by enabling arbitrary mixed-precision computations, allowing models to adapt precision based on layer or task-specific requirements.

Results: FlexiBit achieved 1.66x to 3.9x higher performance per area compared to existing architectures on GPT-3 using FP6 precision.

Infrastructure, Caching, and Serving Optimizations

Marconi: Prefix Caching for the Era of Hybrid LLMs

Why?: Hybrid LLMs, which combine attention and recurrent layers, often waste compute and memory when caching prefixes. Traditional caching systems rely on exact-match caching, which is inefficient for hybrid models that process partially overlapping prefixes.

How?: Marconi introduces novel cache admission and eviction policies that evaluate potential entries based on recency, predicted reuse likelihood, and computational savings versus memory costs. This approach allows efficient prefix caching for hybrid models, correcting traditional systems that require exact-match caches.

Results: Marconi achieves up to 34.4× higher token hit rates and reduces time-to-first-token by 617 ms compared to existing systems, indicating significant performance improvements.

MiniKV: Pushing the Limits of LLM Inference via 2-Bit Layer-Discriminative KV Cache

Why?: The KV (key-value) cache is a critical bottleneck in long-context LLM inference due to its high memory requirements. Standard 8-bit compression methods are insufficient for scaling to larger models and contexts.

How?: The study introduces MiniKV, a KV cache optimization method using a novel 2-bit layer-discriminative approach, preserving accuracy while significantly reducing cache size. Specialized CUDA kernels were developed to ensure compatibility with FlashAttention, tested across various long-context tasks.

Results: Experiments demonstrated reduction in KV cache size by 86% with minimal accuracy loss.

FastSwitch: Optimizing Context Switching Efficiency in Fairness-aware Large Language Model Serving

Why?: In multi-user LLM-serving systems, inefficient context switching leads to fairness issues, where certain users face delays due to resource contention or idling GPUs. Improving context-switching efficiency ensures fairer service-level objectives (SLOs) for all users.

How?: The researchers developed FastSwitch, a system that maintains memory allocation while reducing context-switching overhead through enhancements like better I/O utilization and minimizing GPU idleness, responding to identified inefficiencies in current systems.

Results: FastSwitch demonstrated significant performance improvements over existing systems like vLLM, with speedups ranging from 1.4 to 11.2 times in various metrics.

InstCache: A Predictive Cache for LLM Serving

How?: Paper proposes InstCache, which predicts user instructions using an instruction-aligned LLM. They implement an instruction pre-population algorithm based on the negative log likelihood to optimize cache size and hit rate. InstCache is executed as a hash table to minimize lookup latency, allowing for quick deployment.

Results: InstCache attains a 51.3% improvement in cache hit rate.

A Simple and Provable Scaling Law for the Test-Time Compute of Large Language Models

Why?: The computational cost of test-time inference grows with model size and sequence length, often creating bottlenecks in scalable deployments. Reducing test-time compute without sacrificing accuracy is crucial for scaling LLMs efficiently.

How?: Paper proposes a two-stage algorithm: (1) Generate N candidate solutions using the LLM. (2) Select the best solution via a K-round knockout tournament that compares pairs of solutions.

The algorithm leverages parallelization and requires only N × (K + 1) LLM calls.

Hallucination and attribution

AttriBoT: A Bag of Tricks for Efficiently Approximating Leave-One-Out Context Attribution

Why?: Understanding how context influences LLM behavior is crucial for improving interpretability and efficiency. However, calculating context attribution is computationally expensive.

How?: Paper introduces AttriBoT, using cached activations to avoid redundant operations, hierarchical attribution for reduced computation, and proxy models to emulate the behavior of larger models. This approach approximates the LOO error more efficiently than previous methods.

Results: AttriBoT achieves over 300x speedup in computing context attributions, making it 30x faster than generating the response, while maintaining fidelity to target model's LOO error.

Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Models

Why?: Training dense multi-modal transformers for handling text, images, and speech demands substantial computational resources, which limits their scalability and accessibility. An efficient architecture is required to reduce floating-point operations (FLOPs) while maintaining high performance across modalities.

How?: The research introduces the Mixture-of-Transformers (MoT), a sparse multi-modal transformer architecture. MoT separates model parameters by modality while maintaining global self-attention, thus allowing modality-specific processing. This separation reduces the computational cost by utilizing fewer floating-point operations (FLOPs) than dense models.

Results: Research showcase a comparable performance to dense multi-modal models while using 55.8% fewer FLOPs, demonstrating significant resource savings.

Advanced Pruning, Layer Slicing, and Attention Mechanisms

DRPruning: Efficient Large Language Model Pruning through Distributionally Robust Optimization

Why?: Pruning LLMs can reduce their size and computational cost but often results in uneven performance across domains. This research addresses these imbalances by introducing a pruning method that maintains robust performance across diverse datasets and tasks.

How?: The research introduces DRPruning, integrating distributionally robust optimization to improve structured pruning. This method refines pruning and pretraining processes, automatically finding optimal reference losses and data ratios to prevent biased performance across different domains.

Results: Experiments in both monolingual and multilingual settings indicate DRPruning outperforms similarly sized models in pruning metrics such as perplexity, downstream tasks, and instruction tuning, enhancing robustness against distribution shifts.

The Super Weight in Large Language Models

Why?: A small fraction of LLM parameters, termed "super weights," are disproportionately critical for model performance. Understanding these parameters enables more efficient pruning and quantization while preserving accuracy.

How?: The study identifies 'super weights' using a data-free approach involving a single forward pass through the model. The impact of pruning these weights is examined by evaluating changes in perplexity and zero-shot accuracy. The researchers also examine how preserving 'super activations' affects quantization strategies.

Results: Pruning a single 'super weight' dramatically increases perplexity and reduces accuracy. Preserving these weights allows simple quantization methods to match state-of-the-art performance and enables larger quantization block sizes.

Star Attention: Efficient LLM Inference over Long Sequences

Why?: Transformers face quadratic complexity in their attention mechanism, making inference on long sequences computationally inefficient. Star Attention addresses this by reducing memory and compute requirements.

How?: A two-phase block-sparse approximation called Star Attention is introduced. Phase one uses blockwise-local attention processed in parallel across multiple hosts. Phase two applies sequence-global attention for query and response tokens attending to cached tokens. This method integrates seamlessly with global attention-trained LLMs.

Results: The method reduces memory requirements and inference times by up to 11x while maintaining 95-100% accuracy.

LASER: Attention with Exponential Transformation

Why?: Transformer attention mechanisms suffer from poor gradient signal backpropagation, which can hinder learning and slow convergence. LASER addresses this issue by improving gradient flow in the attention mechanism.

How?: The researchers propose LASER, an attention mechanism with larger gradient signals than standard attention, implemented with minor modifications to existing setups. They conducted experiments with autoregressive LLMs up to 2.2 billion parameters.

Survey papers

Reassessing Layer Pruning in LLMs: New Insights and Methods

How Good is ChatGPT at Audiovisual Deepfake Detection: A Comparative Study of ChatGPT, AI Models and Human Perception

Creative ways to use LLMs!!

SIMS: Simulating Human-Scene Interactions with Real World Script Planning - Generate scripts for human-scene interactions in physics-based animations, enabling more dynamic and realistic character motion for games and simulations.

AIDetx: a compression-based method for identification of machine-learning generated text - AIDetx leverages data compression techniques to accurately distinguish between human-written and AI-generated text, achieving exceptional detection accuracy with minimal computational cost.

Build An Influential Bot In Social Media Simulations With Large Language Models - A novel framework integrates LLMs into agent-based social media simulations, replicating opinion dynamics and influencer behavior to better understand public opinion formation.

CoVis: A Collaborative Framework for Fine-grained Graphic Visual Understanding - CoVis combines segmentation networks with LLM-based content generation to provide detailed and holistic graphic visual interpretations, improving the accessibility of complex visual data.

Automated Test Transfer Across Android Apps Using Large Language Models - LLMigrate simplifies and accelerates UI test transfer between Android apps, significantly reducing the time and effort required for mobile app testing.

PanoLlama: Generating Endless and Coherent Panoramas with Next-Token-Prediction LLMs

PanoLlama transforms panoramic image generation by reimagining it as a next-token prediction task, enabling endless, coherent panoramas with enhanced scalability and precision.

From CISC to RISC: language-model guided assembly transpilation - CRT automates the translation of x86 assembly code to ARM, facilitating the transition to more energy-efficient architectures while ensuring high accuracy and performance gains.

Multiverse of Greatness: Generating Story Branches with LLMs - Generate branching storylines with enhanced coherence and creativity, revolutionizing AI-driven storytelling for visual novels and dynamic narratives.

LLMs related research papers published from September 15-19, 2024

Sat, 12 Oct 2024 20:00:00 +0000

🔑 takeaway from today’s newsletter

🧠 LLM Surgery: Learn how researchers can update or remove specific knowledge in LLMs without retraining the entire model, making AI more efficient and up-to-date.

🔀 Adaptive Transformers: Meet new Transformer models that adjust their depth dynamically, processing simpler inputs faster by skipping unnecessary layers.

⚡ FP8 Training Acceleration: Discover how using FP8 precision speeds up training and reduces memory usage in LLMs, leading to faster and more efficient models without sacrificing performance.

🎨 Creative LLM Applications: See how LLMs are being used in innovative ways—from assisting in Alzheimer's detection to generating recipes and food images.

Don’t have much time?

No worries! Jump onto your car or subway for commute and listen to this amazing podcast covering these papers!

Podcast discussing papers improving core performance of LLMs [september 15-19, 2024]

Core research improving LLMs!

LLM Surgery: Efficient Knowledge Unlearning and Editing in Large Language Models

Updating large language modes (LLMs) with new information and removing outdated or problematic knowledge without retraining from scratch is a significant challenge. Traditional fine-tuning methods are computationally expensive and risk overwriting valuable existing knowledge.

How?
The paper introduces LLM Surgery, a framework that efficiently modifies an LLM's knowledge base using a three-component objective function:

Unlearning Objective (Reverse Gradient): Removes outdated knowledge by performing gradient ascent (reverse gradient) on an unlearning dataset D_unlearn. This effectively increases the model's loss on this data, causing it to forget specific information.
Learning Objective (Gradient Descent): Incorporates new knowledge by performing gradient descent on an update dataset D_update.
Retention Objective (KL Divergence Minimization): Ensures that the model's outputs remain similar to the original model on a retain dataset D_retain by minimizing the Kullback-Leibler (KL) divergence between the original and updated models' output distributions.

Results
Applying LLM Surgery to the LLaMA2-7B model, the researchers achieved:

Significant forgetting on the unlearn dataset.
A 20% improvement on the update dataset.
Minimal performance degradation on the retain dataset.

Scaling FP8 Training to Trillion-Token LLMs

Training LLMs using lower-precision formats like FP8 can significantly reduce computational costs and memory usage. However, scaling FP8 training to large datasets (trillion tokens) reveals instabilities that need to be addressed for practical deployment.

How?
The researchers identified that the SwiGLU activation function amplifies outlier gradients over long training durations, leading to numerical instabilities in FP8 training. To mitigate this, they introduced:

Smooth-SwiGLU Activation Function: A modification that stabilizes FP8 training by smoothing the activation function, preventing gradient explosions without altering its functional behavior.
FP8 Quantization of Adam Optimizer Moments: Applied FP8 quantization to the first and second moments in the Adam optimizer to maintain stability and efficiency.

Results
Training a 7B parameter model on 2 trillion tokens using 256 Intel Gaudi2 accelerators:

Achieved comparable performance to BF16 baselines.
Realized up to a 34% speedup and 43% memory reduction during training.
Demonstrated that FP8 training is feasible and efficient for large-scale LLMs when addressing activation-induced instabilities.

Adaptive Large Language Models by Layerwise Attention Shortcuts

Traditional Transformer architectures process inputs sequentially layer by layer, which can be inefficient and may not adapt computations based on input complexity. Introducing adaptability can improve efficiency and performance.

How?
Paper proposes modifications to the Transformer architecture by adding attention shortcuts from the final layer to all intermediate layers:

Layerwise Attention Shortcuts: The final layer attends to the outputs of all previous layers, allowing it to directly access lower-level features.
Dynamic Depth Adaptation: The model can adjust its computational depth based on input complexity, effectively skipping unnecessary layers for simpler inputs.

The modified attention mechanism at the final layer LL becomes:

RetrievalAttention: Accelerating Long-Context LLM Inference via Vector Retrieval

Handling long contexts in LLMs is computationally intensive due to the quadratic complexity of self-attention mechanisms. Reducing inference time and memory consumption is critical for deploying LLMs in practical applications.

How?
The paper introduces RetrievalAttention, a method that accelerates inference by:

Key-Value (KV) Caching with Vector Retrieval: Instead of processing the entire context, the model retrieves only the most relevant key-value pairs from the KV cache using Approximate Nearest Neighbor Search (ANNS).
Attention-Aware Vector Search: Adapts the retrieval process to the distribution of query vectors, ensuring that relevant context is efficiently retrieved.

for a query vector q, the attention is computed over a subset of keys {k_i} and values {v_i}:

By selecting only the top-k keys k_i that are most similar to q, the computation is significantly reduced.

Results

The method accesses only a small fraction of the KV cache (1-3% KV cache access) during inference. It achieves up to a 4x speedup in inference time with minimal impact on model performance!

Self-Attention Limits Working Memory Capacity of Transformer-Based Models

By understanding the limitations of LLMs working memory capacity their ability to handle tasks requiring long-term dependencies can be improved.

How?
Paper investigate the working memory capacity of Transformers by training models on N-back tasks to test their ability to remember information NN steps back. It observes that as NN increases, the entropy of the attention distribution increases, leading to dispersed attention and reduced capacity.

Paper proposes that self-attention mechanisms inherently limit working memory capacity due to this dispersion. It shows that attention mechanisms contribute to capacity limits due to increased entropy.

Let’s make LLMs safe!!

Defending against Reverse Preference Attacks is Difficult - shows how adversarial reinforcement learning can induce harmful behaviours in safety-aligned LLMs through Reverse Preference Attacks (RPAs). Explores defence strategies using Constrained Markov-Decision Processes, finding that 'online' defences with controlled loss functions are effective, while 'offline' defences are less so.

MEOW: MEMOry Supervised LLM Unlearning Via Inverted Facts proposes MEOW, a gradient descent-based method for unlearning sensitive information in LLMs. Generates inverted facts and utilizes the MEMO metric to quantify memorization, effectively improving forgetfulness without sacrificing model utility.

LLM-Powered Text Simulation Attack Against ID-Free Recommender Systems Introduces a text poisoning attack using LLMs to manipulate textual information of target items in ID-free recommender systems. Leverages popular item characteristics to create promotional descriptions that mimic popular items.

Jailbreaking Large Language Models with Symbolic Mathematics shows how encoding harmful prompts into mathematical problems can bypass LLM safety measures. It encodes harmful natural language prompts into mathematical problems.The MathPrompt technique achieves 73.6% attack success rate in top 13 LLMs!

Creative ways to use LLMs!!

Profiling Patient Transcripts Using LLMs for Alzheimer's Detection

Introduces a framework that utilizes LLMs for profiling linguistic deficits in patient transcripts. Shows an 8.51% improvement over state-of-the-art methods in Alzheimer's disease detection.

ChefFusion: Multimodal Foundation Model for Recipe and Food Image Generation

Paper develops ChefFusion, a foundation model integrating LLMs with pre-trained image encoders and decoders. Performs tasks like recipe generation and food image generation, advancing multimodal food computing.

LLMs for robotics & VLLMs

Leveraging LLMs for Automated Framing Analysis in TV Shows uses prompt engineering to guide LLMs in identifying framing in spoken content from TV shows. Achieves agreement rates up to 43% with human annotations, offering a support tool for framing analysis.

Arena 4.0: A ROS2 Platform for Human-Centric Navigation

Paper introduces Arena 4.0, utilizing LLMs and diffusion models to generate human-centric environments from text prompts or floorplans. Offers a 3D model database and migrates to ROS 2 for hardware compatibility, validated through a user study.

Hierarchical LLMs In-the-Loop Optimization for Multi-Robot Target Tracking

Paper introduces a hierarchical LLM-based framework for optimizing multi-robot task allocation in real-time. Validated through simulations and real-world experiments, enhancing adaptability in hazardous environments.

E2Map: Self-Reflective Robot Navigation with LLMs

Paper creates an 'Experience-and-Emotion Map' integrating LLM knowledge with real-world experiences. Allows robots to adjust behavior dynamically in stochastic environments.

LLMs evaluations

What Would You Ask When You First Saw a²+b²=c²? Evaluating LLM on Curiosity-Driven Questioning Introduces a framework to assess LLMs ability to generate curious questions when presented with new scientific statements. Surprisingly it found out that models like GPT-4 and Mistral 8x7b generate coherent questions, with smaller models matching or exceeding their effectiveness.

SpecEval: Evaluating Code Comprehension in Large Language Models via Program Specifications introduces SpecEval, a framework using program specifications to assess LLMs code comprehension. It designs four tasks utilizing formal specifications, counterfactual analysis, and consistency checks.

Investigating Context-Faithfulness in Large Language Models studies factors influencing context-faithfulness in LLMs by quantifying memory strength and evaluating different evidence styles. Uses datasets with well-known and long-tail questions.

Confidence Estimation for LLM-Based Dialogue State Tracking explores methods to estimate confidence scores in dialogue state tracking. It finds that fine-tuning open-weight LLMs enhances calibration accuracy and improves performance.

Survey papers

From Linguistic Giants to Sensory Maestros: A Survey on Cross-Modal Reasoning with Large Language Models - survey of methodologies for cross-modal reasoning using LLMs.

Art and Science of Quantizing Large-Scale Models: A Comprehensive Overview - A review of variations of quantization methods like post-training quantization (PTQ) and quantization-aware training (QAT).

LLMs related research papers published from September 9-13, 2024

Sun, 29 Sep 2024 20:13:28 +0000

Dear readers,

This edition is long so you may need to click on “view entire content” button at the end to read full edition. Not my fault, it’s Google!

Speaking of Google, I have tried NotebookLM to create an interesting podcast discussing this edition’s research papers. If you don’t have time to read, LISTEN!😄

Share you thoughts if you liked this podcast format or want any modifications! This podcast is yours, treat it like one! - shower the comments for improvements.👏🏻

Summary of papers from today's edition

🔑 takeaway from today’s newsletter

AIPO refines LLMs to deliver concise and meaningful responses - enhancing alignment with human preferences for better AI interactions.
CPL introduces Critical Planning Step Learning, boosting LLM reasoning and significantly improving problem - solving accuracy.
Faster Speech-LLaMA accelerates speech recognition by predicting multiple tokens at once - reducing inference time without sacrificing accuracy.
AdaCAD dynamically balances contextual and parametric knowledge in LLMs, resolving conflicts for clearer and more accurate outputs.
E2LLM extends LLMs' capabilities to handle longer contexts efficiently - enhancing understanding and reasoning over extended inputs.

Core research improving LLMs!

AIPO: Improving Training Objective for Iterative Preference Optimization

Why?: Improving the alignment of LLMs through iterative preference optimization is important for their effective operation and scaling, as preference optimization is becoming a popular alternative to proximal policy optimization.

How?: AIPO introduces a new training method that encourages LLMs to produce concise and meaningful responses, addressing the length exploitation problem. It does so by:

Carefully selecting training data: AIPO focuses on using synthetic data, particularly instructions created by humans and responses generated by the LLM itself.
Iterative training: The model is trained in multiple rounds, incorporating feedback from a reference model (a more powerful LLM) in each round. This helps refine the model's responses over time.
Agreement-aware training: AIPO uses a special coefficient to adjust the training process based on how well the LLM's responses align with the reference model's preferences. This ensures that the LLM is rewarded for producing responses that are both concise and aligned with human-like preferences.

In simpler terms, imagine you're teaching a child to write good stories. Instead of just telling them to write longer stories, you:

Give them specific examples of good stories and let them learn from their own attempts.
Provide feedback in stages, gradually refining their writing style and content.
Reward them not just for length, but also for clarity, conciseness, and creativity, using a teacher's (reference model's) opinion as a guide.

AIPO applies a similar concept to LLMs, encouraging them to produce responses that are not just long, but also high-quality and aligned with human expectations.

CPL: Critical Planning Step Learning Boosts LLM Generalization in Reasoning Tasks

Why?: Reasoning improvement of LLMs.

How?: This paper introduces Critical Planning Step Learning (CPL) that uses Monte Carlo Tree Search (MCTS) to identify diverse planning steps in multi-step reasoning tasks. CPL focuses on learning step-level planning preferences based on long-term outcomes to improve planning capabilities. It incorporates Step-level Advantage Preference Optimization (Step-APO), which integrates an advantage estimate for step-level preference pairs using MCTS within Direct Preference Optimization (DPO) techniques, capturing fine-grained supervision at each reasoning step. Experiments were conducted on GSM8K and MATH datasets.

Results: The CPL method significantly improved model performance on GSM8K +10.5% and MATH by 6.5%.

Faster Speech-LLaMA Inference with Multi-token Prediction

Why?: Reducing the inference time of Speech-LLaMA models.

How?: Meta team proposes predicting multiple tokens per decoding step instead of the traditional sequential approach. They explore various model architectures and implement threshold-based and verification-based inference strategies. Additionally, they introduce a prefix-based beam search decoding method for efficient minimum word error rate training. The models are then evaluated across several public speech recognition benchmarks.

Results: The proposed models achieve a 3.2x reduction in decoder calls while maintaining or improving word error rate (WER) performance.

AdaCAD: Adaptively Decoding to Balance Conflicts between Contextual and Parametric Knowledge

Why?: The paper tries to resolve knowledge conflict in LLMs, where discrepancies between contextual and parametric information can impair performance.

How?: Paper introduces AdaCAD, a dynamic approach that calculates the degree of conflict between contextual and parametric knowledge using the Jensen-Shannon divergence. This calculated divergence helps to dynamically adjust the decoding process at an instance level, rather than using static methods. Tests involved comparing AdaCAD to other contrastive methods across four models on various QA and summarization tasks, focusing on how well it handles varying degrees of conflict.

Results: AdaCAD yielded an average accuracy gain of 14.21% over a static contrastive baseline.

Geometric-Averaged Preference Optimization for Soft Preference Labels

This research paper argues that limiting human preferences to binary choices is an oversimplification and proposes using soft preference labels, represented by a probability (p̂), to reflect the nuanced nature of human preferences. To incorporate these soft labels into LLMs alignment, the authors introduce weighted geometric averaging into the loss function of Direct Preference Optimization (DPO) and related algorithms like IPO and ROPO. This method weights the likelihoods of the chosen and rejected responses based on the soft preference label—strong preferences (p̂ closer to 1) result in larger gradient updates during training than weak preferences (p̂ closer to 0.5). This weighting scheme allows the model to prioritize learning from high-confidence preference pairs while mitigating the effects of noisy data points and the tendency to over-optimize for response length.

E2LLM: Encoder Elongated Large Language Models for Long-Context Understanding and Reasoning

Why?: Extends context length

How?: Paper introduces E2LLM, a model that enhances long-context capabilities by segmenting long inputs into chunks, which are compressed into embedding vectors using a pre-trained text encoder. These embeddings are then aligned with a decoder-only LLM through an adapter. The training objectives focus on reconstructing the encoder’s output and fine-tuning for long-context instructions, aiding LLMs in interpreting soft prompts.

Length Desensitization in Directed Preference Optimization

Why?: Current LLMs which are trained with Direct Preference Optimization (DPO) suffer from excessive verbosity.

How?: Paper performs a theoretical analysis of DPO and its tendency to correlate rewards with data length, leading to verbosity.

To handle this, paper introduces LD-DPO, a novel algorithm designed to mitigate length sensitivity in DPO by decoupling the preference for verbosity from other human-like preferences.

LD-DPO works by:

Identifying the "public length" (lp) of a response pair—the number of tokens shared by both responses.
Decomposing the likelihood of the longer response into the product of the likelihood of the public-length portion and the likelihood of the "excessive" portion.
Introducing a hyperparameter α (ranging from 0 to 1) to control the influence of the excessive portion on the likelihood. A lower α weakens the impact of the excessive length, reducing DPO's sensitivity to it.
By adjusting α, LD-DPO aims to find a balance between reducing verbosity and preserving the LLM's ability to learn other valuable preferences from the training data.

Results: LD-DPO consistently reduced response lengths by 10-40% compared to DPO.

Extracting Paragraphs from LLM Token Activations

Why?: Paper explores the inner workings of LLMs beyond token-level predictions.

How?: Paper analyze the role of the "\n\n" double newline token in single-token activations. The authors explore how these activations encode information about the content of subsequent paragraphs. They employ a method of patching these activations to assess their information transfer capabilities regarding contextual understanding in paragraph formation.

LLMs Will Always Hallucinate, and We Need to Live With This

Why?: This research claims that hallucination is not a bug in LLMs that can be solved instead it is a nature of LLM and is inevitable.

How?: The paper uses mathematical concepts and logic to argue that several factors make structural hallucinations inevitable:

Incomplete Training Data: It's impossible to capture all of human knowledge in a dataset, even a massive one. This incompleteness means that LLMs will always encounter situations or facts they haven't seen before, leading to potential hallucinations.
Difficulty Finding the Right Information: Even with a complete dataset, finding the exact information needed for a specific query can be extremely difficult for an LLM. Think of it like searching for a needle in a haystack—there's just too much data to sift through perfectly.
Misinterpreting What We Mean: Natural language is ambiguous, meaning the same phrase can have different meanings depending on the context. LLMs often struggle to correctly interpret the true meaning behind our prompts, leading to hallucinations.
Unpredictable Outputs: The paper draws a parallel between how LLMs generate text and a famous problem in computer science called the "Halting Problem." Basically, it's impossible to perfectly predict what output an LLM will generate beforehand, just like it's impossible to always know if a computer program will run forever or eventually stop. This unpredictability makes hallucinations unavoidable.
Fact-Checking Isn't Foolproof: While fact-checking can help, the paper argues that it's mathematically impossible to have a fact-checking system that catches every single hallucination.

The Bottom Line: We need to accept that LLMs will always have the potential to hallucinate and focus on developing strategies to manage and mitigate this risk.

Logically Consistent Language Models via Neuro-Symbolic Integration

Why?: Paper improves reasoning ability in LLMs.

How?: The research introduces a neuro-symbolic methodology that integrates neuro-symbolic reasoning into the LLM training process. This involves developing a loss function based on logical consistency with a predefined set of facts and rules, allowing the LLM to be trained to adhere to these constraints. The approach is balanced between full fine-tuning and offloading reasoning tasks to external tools, aiming to teach LLMs logical consistency with minimal fine-tuning. It also facilitates the inclusion and integration of multiple logical constraints efficiently, enabling the models to handle a wide range of logical requirements.

LLMOps & GPU level optimization

Optimal Workload Placement on Multi-Instance GPUs

Why?: The research manages LLMs inferencing workloads utilizing the partitioning feature, Multi-Instance GPU (MIG).

How?: This paper developed two approaches: an optimization method and a heuristic method. Both aim to efficiently place or migrate workloads to minimize the number of GPUs used and reduce memory and compute wastage.

Results: The research demonstrated up to a 2.85x improvement in the number of GPUs used and up to a 70% reduction in GPU wastage over baseline heuristics.

Inf-MLLM: Efficient Streaming Inference of Multimodal Large Language Models on a Single GPU

Why?: Deploying multimodal large language models (MLLMs) on edge devices.

How?: The paper introduce Inf-MLLM, a framework for efficient streaming inference of MLLMs on a single GPU. They identify a novel attention pattern called 'attention saddles' that enables the system to maintain a size-constrained key-value (KV) cache. The cache dynamically stores recent and relevant tokens, reducing memory usage. The framework also implements 'attention bias' to capture long-term dependencies. This approach facilitates the inference of texts up to 4 million tokens long in multi-turn conversations and long videos.

Results: Inf-MLLM achieves stable performance on texts with 4 million tokens and multi-round conversations, offering superior streaming reasoning quality and 2x speedup compared to H2O.

DFabric: Scaling Out Data Parallel Applications with CXL-Ethernet Hybrid Interconnects

Why?: To address bandwidth limitations in LLM training.

How?: This paper introduced DFabric, a two-tier interconnect architecture that enhances data communication across multiple hosts. First, it disaggregates computing units with a CXL fabric at the rack level for efficient intra-rack communication. Second, it disaggregates NICs from hosts into a centralized NIC pool using CXL fabric to enhance communication across racks. To tackle local memory access bottlenecks, DFabric creates a memory pool by disaggregating and expanding host memory.

LLMs evaluations

Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale

Windows Agent Arena, a benchmark environment for LLMs operating in the Windows OS. It utilizes the OSWorld framework to create 150+ tasks that test agents' planning, screen understanding, and tool usage within real Windows applications. The benchmark is scalable and can be evaluated quickly using parallelized computing on Azure. A new multi-modal agent called Navi is also introduced to showcase the capabilities of the benchmark.

Survey papers

Exploring LLMs for Malware Detection: Review, Framework Design, and Countermeasure Approaches

Agents in Software Engineering: Survey, Landscape, and Vision

What is the Role of Small Models in the LLM Era: A Survey

Let’s make LLMs safe!!

FP-VEC: Fingerprinting Large Language Models via Efficient Vector Addition

Why?: The research provides a way to protect the intellectual property of LLMs.

How?: The research introduces FP-VEC, a technique that creates a fingerprint vector representing a confidential signature embedded in the LLM. This method enables the integration of the fingerprint into an unlimited number of LLMs simply through vector addition. The approach is lightweight, allowing fingerprinting using CPU-only devices, and maintains the model's typical behavior while being scalable with a single training process.

Securing Vision-Language Models with a Robust Encoder Against Jailbreak and Adversarial Attacks

Why?: The research propose a safety mechanism for the critical vulnerability of Large Vision-Language Models (LVLMs) against adversarial and jailbreak attacks, which can lead to misleading or harmful outputs.

How?: The research introduces Sim-CLIP+, a defense mechanism that uses a Siamese architecture to adversarially fine-tune the CLIP vision encoder. The method maximizes cosine similarity between perturbed and clean samples to enhance robustness against adversarial attacks. Sim-CLIP+ is designed as a plug-and-play solution without requiring structural changes to the existing LVLM frameworks and maintains minimal computational overhead.

Context-Aware Membership Inference Attacks against Pre-trained Large Language Models

Why?: Membership inference attacks can reveal whether specific data points were part of the training dataset, compromising user privacy.

How?: The researchers developed a novel approach to membership inference attacks tailored to LLMs by leveraging the perplexity dynamics of subsequences within a token sequence. This method departs from traditional loss-based attacks that are inadequate for LLMs. By focusing on context-dependent variations, the researchers adapt statistical tests to uncover how these models memorize and reveal training data.

Representation Tuning

Why?: The research makes models more reliable and honest without needing real-time interventions.

How?: The research identifies activation vectors related to honesty in Llama-2-13b-chat. These vectors are added to residual stream activations to control model honesty. The novelty lies in directly fine-tuning these vectors into the model through a dual loss function combining cosine similarity of activations and a standard token-based loss. This method called 'representation tuning,' is compared to traditional fine-tuning and online steering.

Results: Fine-tuning vectors using the dual loss function increased honesty more effectively than online steering and generalized better than using a standard token-based loss.

AdaPPA: Adaptive Position Pre-Fill Jailbreak Attack Approach Targeting LLMs - GitHub

Paper develop an adaptive position pre-fill jailbreak attack approach that first uses the LLM to generate safe output content. Then, by exploiting the model's ability to follow instructions and shift narratives, the attack prompts the LLM to produce harmful content. The approach was evaluated through extensive black-box experiments, targeting a well-known secure model, Llama2, effectively aligning with the model's differential output-stage alignment protection capabilities.

Results: This method increased the attack success rate by 47% on Llama2.

Creative ways to use LLMs!!

LLaQo: Towards a Query-Based Coach in Expressive Music Performance Assessment - Providing feedback in music performance for music lessons

The research proposes LLaQo, a query-based coaching model that evaluates music performances by processing audio data to provide insightful feedback. The system uses instruction-tuned query-response datasets covering music aspects like pitch accuracy and technique. It utilizes AudioMAE encoder and Vicuna-7b as a backend to predict performance ratings, understand contextual aspects like piece difficulty, and answer open-ended questions. LLaQo was trained to align responses with teacher assessments, achieving SOTA in predicting performance ratings.

Large Language Model Can Transcribe Speech in Multi-Talker Scenarios with Versatile Instructions - LLMs in multi-talker speech transcription

The study utilizes WavLM and Whisper encoders to capture detailed speech representations that consider speaker-specific traits and the context. These encoded features are then input into the large language model. The LLM is fine-tuned using the Low-Rank Adaptation (LoRA) method, enhancing its ability to comprehend and transcribe speech amidst multiple speakers. The model is tailored to follow various instructions and respond to differences such as language, speaker characteristics, and keywords.

Enabling Cost-Effective UI Automation Testing with Retrieval-Based LLMs: A Case Study in WeChat - LLMs for UI automation

The researchers developed CAT, a system combining LLMs and machine learning. CAT uses Retrieval Augmented Generation (RAG) to gather contextual examples for few-shot learning, guiding LLMs in generating actions for UI tasks. These tasks are then mapped onto the UI using machine learning, with LLMs optimizing the process. CAT was tested on WeChat, showcasing its integration into the application’s existing testing platform.

Enhancing Long Video Understanding via Hierarchical Event-Based Memory - Long video understanding in LLMs

The researchers propose a Hierarchical Event-based Memory-enhanced LLM (HEM-LLM) to better handle long video understanding by developing an adaptive sequence segmentation scheme that divides long videos into separate events. Each event is then individually modeled to reduce redundancy and preserve key semantics. Information from a prior event is compressed and incorporated while modeling the current event to maintain inter-event dependencies over the long term. The approach aims to improve video comprehension by maintaining contextual connections within events and across the video as a whole.

Towards Agentic AI on Particle Accelerators

The research proposes a decentralized, multi-agent framework for the control of particle accelerators, powered by LLMs. The framework envisions a system where autonomous agents are responsible for high-level tasks and communication, with each agent specializing in controlling individual accelerator components. The system allows for self-improvement through experience and human interaction. Two examples are provided to demonstrate the feasibility of this architecture, raising questions about future AI applications in this field and the implications of integrating human feedback into the system.

Advertiser Content Understanding via LLMs for Google Ads Safety

Google researchers introduces a method using LLMs to comprehend advertiser intent in relation to content policy violations. It constructs advertiser content profiles by aggregating signals from ads, domains, and targeting information and utilizes LLMs for classification. Additionally, LLMs leverage prior knowledge about advertisers to predict policy violations. Minimal prompt tuning was conducted to enhance model performance.

LLMs for robotics & VLLMs

Intelligent LiDAR Navigation: Leveraging External Information and Semantic Maps with LLM as Copilot

Why?: The research enhance robot navigation by using human-like understanding of LLMs, allowing robots to utilize external and experiential information akin to humans.

How?: The study proposes osmAG, a semantic topometric hierarchical map format, to integrate LLMs into robot navigation systems. This involves using the LLM as a copilot, which allows for the assimilation of diverse informational inputs. The methodology bridges the contextual understanding of LLMs with the spatial capabilities of traditional systems like ROS move_base, enhancing the flexibility and adaptability of robotic navigation.

LlamaF: An Efficient Llama2 Architecture Accelerator on Embedded FPGAs

Why?: The deploys LLMs on resource-constrained embedded devices, improving their performance.

How?: The researchers designed an FPGA-based accelerator to enhance LLM inference on embedded devices. They used post-training quantization to reduce memory usage and optimized for off-chip memory bandwidth. The architecture features asynchronous computation and full pipelining for efficient matrix-vector multiplication. The experiments evaluated the accelerator using the TinyLlama 1.1B model on a Xilinx ZCU102 platform.

Results: Experiments demonstrated a 14.3-15.8x speedup and a 6.1x power efficiency improvement compared to running solely on the ZCU102 processing system.

Summary of LLMs related research paper published on 2-4 September, 2024

LLMs Research — Sat, 21 Sep 2024 14:00:00 +0000

🔑 takeaway from today’s newsletter

LongLLaVA scales up multimodal LLMs to process nearly 1,000 images efficiently - boosting AI's visual prowess without overloading your GPU.
The Single-Turn Crescendo Attack (STCA) reveals LLM vulnerabilities in just one interaction - underlining the need for stronger AI safeguards.
SmileyLlama modifies large language models for chemical exploration- bringing a fresh twist to molecular discoveries.
Mamba serves as a motion encoder for robotic imitation learning - helping robots mimic human movements more effectively.
LLMs hypothesize missing causal variables—acting as digital detectives to fill gaps in scientific understanding.

Core research improving LLMs!

CLUE: Concept-Level Uncertainty Estimation for Large Language Models

Why?: This paper addresses the limitations of existing uncertainty estimation methods that focus only on sequence-level uncertainty and fail to separately assess the uncertainty of each component in a sequence.

How?: The proposed CLUE framework transforms LLM-generated sequences into concept-level representations, allowing for individual concept uncertainty estimation. This is achieved by breaking down sequences into individual concepts and mathematically measuring uncertainty for each concept separately. The approach allows researchers to conduct more detailed uncertainty assessments, providing a granular understanding of model outputs.

LongCite: Enabling LLMs to Generate Fine-grained Citations in Long-context QA

Why?: Current long-context LLMs often lack citations in their responses, creating verification challenges and diminishing trustworthiness due to potential hallucinations.

How?: The research introduces LongBench-Cite for evaluating LLM citation performance. A novel CoF pipeline was developed to generate QA instances with precise citations. This pipeline was used to create LongCite-45k, a dataset to train LongCite-8B and LongCite-9B models for generating accurate responses with sentence-level citations.

Results: Models trained using LongCite-45k achieved state-of-the-art citation quality, outperforming proprietary models like GPT-4o.

LongLLaVA: Scaling Multi-modal LLMs to 1000 Images Efficiently via Hybrid Architecture

GitHub: https://github.com/FreedomIntelligence/LongLLaVA

Why?: Expands multimodal LLMs long-context capabilities

How?: The research developed a hybrid model architecture combining Mamba and Transformer blocks. It incorporated a data construction approach capturing temporal and spatial dependencies across multiple images and used a progressive training strategy. This combination aimed to balance efficiency and effectiveness, enabling the model to process nearly a thousand images efficiently with high throughput and low memory usage.

Results: LongLLaVA achieved competitive results across various benchmarks while maintaining high throughput and low memory consumption. It could process nearly a thousand images on a single A100 80GB GPU, indicating its efficiency and potential applicability.

Pooling And Attention: What Are Effective Designs For LLM-Based Embedding Models?

Why?: Paper optimizes LLM-based embedding models by identifying effective pooling and attention mechanisms, potentially improving performance on tasks like text similarity and information retrieval.

How?: The study involved a large-scale experiment with LLM-based embedding models. These models were trained on the same dataset and base model, differing only in pooling and attention strategies. The researchers compared various designs, including bidirectional attention, EOS-last token pooling, and more. A new Multi-Layers Trainable Pooling strategy was introduced, transforming outputs from all hidden layers using a cross-attention network. This experiment aimed to evaluate statistically significant differences in performance across diverse tasks.

Results: The findings indicate no universal optimal design: bidirectional attention and trainable pooling excel in text similarity and retrieval, whereas simpler designs suffice for clustering and classification. The proposed Multi-Layers Trainable Pooling showed statistically superior performance in specific tasks.

Hallucination Detection in LLMs: Fast and Memory-Efficient Finetuned Models

The authors developed a novel methodology to create LLM ensembles capable of detecting hallucinations without extensive computational and memory resources. They focus on optimizing the training and inference processes, ensuring that only a single GPU is needed, making it accessible and pragmatic for practical applications.

Deconfounded Causality-aware Parameter-Efficient Fine-Tuning for Problem-Solving Improvement of LLMs

Why?: The research aims to improve the reasoning abilities of LLMs, especially in mathematical and physics tasks.

How?: The study begins by visualizing text generation at the attention and representation level to assess genuine reasoning. It then uses a causal framework to explain these observations. Following this, the researchers introduce Deconfounded Causal Adaptation (DCA), a parameter-efficient fine-tuning approach designed to enhance reasoning skills by allowing the model to generalize problem-solving abilities across questions. The method involves modifying only 1.2 million parameters, focusing on extracting and applying overarching problem-solving skills.

Results: Experiments demonstrate that DCA outperforms existing fine-tuning methods across various benchmarks, achieving better or comparable results while being parameter-efficient.

In Defense of RAG in the Era of Long-Context Language Models

Why?: The research revisits the relevance of retrieval-augmented generation (RAG) amidst the rise of long-context LLMs, arguing the former can yield higher quality answers by maintaining focus on pertinent information.

How?: The research introduces an order-preserving retrieval-augmented generation (OP-RAG) mechanism, which addresses the challenges of long-context LLMs by finely tuning the amount of context supplied to the model. Through extensive experiments on public benchmarks, the OP-RAG approach is evaluated by varying the number of retrieved chunks. This method observes that as more chunks are retrieved, answer quality initially increases before decreasing, establishing optimal points for maximal answer quality with fewer tokens.

Self-Instructed Derived Prompt Generation Meets In-Context Learning: Unlocking New Potential of Black-Box LLMs

Why?: Increasing the performance of Black-Box LLMs like GPT-4 is challenging due to inaccessible parameters, making it essential to focus on prompt quality for better human-like responses.

How?: The research introduces a self-instructed in-context learning framework utilizing reinforcement learning to refine prompt generation, enabling better alignment with original prompts. Derived prompts are crafted to form contextual environments to enhance learning capability. This approach addresses semantic inconsistencies in prompt refinement and reduces discrepancies by using responses from LLMs integrated with derived prompts for a cohesive contextual demonstration.

Results: Extensive experiments show that the method improves derived prompt reliability and significantly enhances the capability of LLMs, including models like GPT-4, to produce more effective responses.

The Compressor-Retriever Architecture for Language Model OS

Why?: The research is important because it aims to transform LLMs from simple chatbots into general-purpose agents capable of interacting with the real world, which involves managing life-long context and statefulness.

How?: The research introduces a compressor-retriever architecture to manage life-long context for LLMs. This model-agnostic architecture compresses and retrieves context using the base model's forward function, maintaining end-to-end differentiability. Unlike retrieval-augmented generation, this approach focuses on exclusive usage of the base model, ensuring scalability and adaptability to long-context tasks without requiring extensive modifications to existing model structures.

Imitating Language via Scalable Inverse Reinforcement Learning

Why?: The research explores the use of inverse reinforcement learning (IRL) to enhance the imitation learning aspect of language model training, offering potential improvements in diversity and task performance.

How?: The research investigates the application of inverse reinforcement learning (IRL) for fine-tuning large language models. It reformulates inverse soft-Q-learning as a temporal difference regularized extension of maximum likelihood estimation (MLE), establishing a new connection between MLE and IRL. This approach seeks to balance the complexity of the model with improved performance and diversity in language generation. The study emphasizes extracting rewards and optimizing sequences directly, which could improve the sequential structure of autoregressive generation.

Prompt Compression with Context-Aware Sentence Encoding for Fast and Improved LLM Inference

Why?: The research proposes a more efficient prompt compression method.

How?: The research introduces context-aware prompt compression (CPC), which uses a context-aware sentence encoder providing relevance scores for sentences related to a given question. A new dataset is created containing questions, positive sentences (relevant) and negative sentences (irrelevant), which are used to train the encoder under a contrastive setup aimed at learning sentence representations.

Results: The method considerably outperforms previous prompt compression techniques on benchmark datasets and achieves up to 10.93 times faster inference compared to token-level compression.

Balancing Performance and Efficiency: A Multimodal Large Language Model Pruning Method based Image Text Interaction

Why?: High computational costs of multimodal LLMs restrict their practical application.

How?: The research introduces a dynamic pruning algorithm for visual tokens in multimodal large language models. Firstly, the visual and CLS token similarity curve is analyzed to identify an inflection point, which helps determine a segmentation point for pruning visual tokens. Next, the concatenated visual and textual tokens are pruned again in the LLM layer. This is achieved by filtering out tokens with low text correlation, balancing efficiency and performance by leveraging interactions between visual and textual features.

Results: The proposed method reduces the token quantity by an average of 22% of the original token quantity.

AI Strategies & tools that will skyrocket your Marketing ROI by 50% 🚀

You don’t realize it yet, but AI has massive potential for you as a marketer.

This free 3-hour Masterclass on AI & ChatGPT (worth $399) will help you become a master of 20+ AI tools & prompting techniques. Join it now for $0

This is for you if you work in any vertical of marketing– writing, designer, campaign managing, influencer marketing, growth marketing, etc.

Ready to shock your team with a 10x boost in revenue & campaign performance? 🚀

Get it now for absolutely free! 🎁

You will join 1 Million+ people who have taken this masterclass to learn how to:

Create 100+ content pieces for reels,blogs, from one single long form video
Put data tracking & reporting for your campaigns on autopilot
Do predictive analysis and optimize your marketing campaigns for better results
Personalize customer experiences by leveraging the power of AI

You’ll wish you knew about this FREE AI masterclass sooner (Btw, it’s rated at 9.8/10 ⭐)

LLMOps & GPU level optimization

ISO: Overlap of Computation and Communication within Sequence For LLM Inference

Why?: LLM inference efficiency improvement is crucial due to the resource-intensive nature of transformer models and multi-GPU tensor parallelism, which can underutilize computing resources.

How?: The research introduces a novel strategy for overlapping computation and communication at the sequence level during LLM inference. This method enhances the overlap degree and reduces application constraints, compared to existing techniques that overlap matrix computations and interleave micro-batches.

Results: The proposed technique decreased time consumption during the prefill stage by 35% on 4090 GPU and by roughly 15% on A800 GPU.

Accelerating Large Language Model Training with Hybrid GPU-based Compression

Why?: The research addresses the overhead in large language model (LLM) training due to data-intensive communication routines. Improving training efficiency is critical as models grow larger, requiring optimized communication strategies to handle massive data scales without sacrificing accuracy.

How?: The research investigates using compression with MPI collectives in distributed LLM training with 3D parallelism and ZeRO optimizations. It tests basic compression across all communication collectives, followed by a refined approach where compression varies by the type of data. Gradients receive aggressive compression due to their low-rank structure, while activations, optimizer states, and model parameters use milder compression to maintain precision.

Results: The naive compression scheme yields a 22.5% increase in TFLOPS per GPU and a 23.6% increase in samples per second for GPT-NeoX-20B training. Using hybrind appraoch team achieved 17.3% increase in TFLOPS per GPU and a 12.7% increase in samples per second while reaching baseline loss convergence.

Foundations of Large Language Model Compression -- Part 1: Weight Quantization

Why?: The research addresses the challenge of deploying LLMs on resource-constrained devices, reducing computational costs, and minimizing the environmental impact of AI.

How?: The paper introduces CVXQ, a novel quantization framework derived from convex optimization principles. It allows for post-training quantization of LLMs with hundreds of billions of parameters to any target size. The method leverages convex optimization to efficiently scale and optimize model weights, ensuring performance is preserved while reducing the model's size significantly. A reference implementation of CVXQ is available for public use.

FlashFlex: Accommodating Large Language Model Training over Heterogeneous Environment

Github: https://github.com/Relaxed-System-Lab/FlashFlex

Why?: This research showcase heterogeneous GPUs based environment which allows more flexible and efficient utilization of computational resources beyond traditional homogeneous setups.

How?: The research introduces FlashFlex, a system that supports asymmetric partitioning of training computations across data, pipeline, and tensor model parallelism. It formalizes allocation as a constrained optimization problem and uses a hierarchical graph partitioning algorithm to efficiently distribute tasks across heterogeneous GPUs. This approach adaptively allocates training workloads to fully leverage available computational resources.

Results: FlashFlex achieves comparable training MFU for LLMs ranging from 7B to 30B parameters on heterogeneous GPUs to state-of-the-art systems using homogeneous high-performance GPUs, with minimal performance gap as low as 0.30%.

Duplex: A Device for Large Language Models with Mixture of Experts, Grouped Query Attention, and Continuous Batching

Why?: The research tries to reduce computation of MoE and attention layers by optimizing resource allocation for high and low arithmetic operations.

How?: The research introduces Duplex, a device that combines xPU tailored for high-Op/B processes and Logic-PIM designed for low-Op/B operations. The system dynamically selects the appropriate processor for each layer of LLMs based on their arithmetic intensity. Logic-PIM enhances data transmission by adding through-silicon vias (TSVs) for high-bandwidth communication, enabling efficient handling of variable Op/B operations found in LLM's MoE and attention layers.

LuWu: An End-to-End In-Network Out-of-Core Optimizer for 100B-Scale Model-in-Network Data-Parallel Training on Distributed GPUs

Why?: During LLM training, current GPU cluster setups face challenges with memory and bandwidth, which hinder efficient scaling.

How?: The research introduces LuWu, an innovative in-network optimizer that allows 100B-scale model training by offloading optimizer states and parameters from GPU workers onto an in-network optimizer node. It also shifts collective communication from GPU-NCCL to SmartNIC-SmartSwitch co-optimization, reducing interference and CPU overhead. This setup retains the communication pattern of model-sharded data parallelism while ensuring efficient resource utilization.

Results: LuWu achieves a 3.98x improvement over state-of-the-art systems when training a 175B model on an 8-worker cluster.

Creative ways to use LLMs!!

Physical Rule-Guided Convolutional Neural Network

This paper introduces Physics-Guided CNNs (PGCNNs), incorporating dynamic, trainable, and automated LLM-generated physical rules into CNN architecture. These rules are integrated as custom layers, addressing challenges such as limited data and low confidence scores. The architecture enhances the CNN model by enabling it to utilize domain-specific knowledge dynamically during training and inference.

Here's Charlie! Realising the Semantic Web vision of Agents in the age of LLMs - Paper develops semi-autonomous web agents that consult users only if lacking context or confidence. A demonstration is provided using Notation3 rules to ensure safety in belief, data sharing, and usage. LLMs are integrated for natural language interaction between users and agents, enabling user training on preferences and serendipitous agent dialogues.

Multi-language Unit Test Generation using LLMs - Let’s ask LLMs to generate compilable and high-coverage unit tests!

Creating a Gen-AI based Track and Trace Assistant MVP (SuperTracy) for PostNL - LLMs in customer service and logistics to improve communication in parcel tracking. LLMs system autonomously manage user inquiries and improve knowledge handling regarding parcel journeys.

Hypothesizing Missing Causal Variables with LLMs

This paper explores enhancing the scientific discovery process by leveraging LLMs to hypothesize missing variables in causal relationships, a pivotal task for advancing scientific knowledge efficiently. Researchers used partial causal graphs with missing variables and tasked LLMs with hypothesizing about these missing elements.

SmileyLlama: Modifying Large Language Models for Directed Chemical Space Exploration - The researchers modified the Llama LLM, an open-source model, using supervised fine-tuning (SFT) and direct preference optimization (DPO). These methods adapted the model to handle chemical SMILES string data, enabling it to generate molecules based on desired properties. This involved training the LLM to respond to specific prompts and provide functional outputs for chemistry and materials applications.

LLMs for robotics & VLLMs

Mamba as a motion encoder for robotic imitation learning

Why?: The research introduces Mamba as an effective tool for robotic imitation learning, aiming to enhance robots' dexterity and adaptability through improved contextual information capture.

How?: The study proposes the use of Mamba, a novel architecture akin to an encoder, for robotic imitation learning. Mamba operates similarly to an autoencoder by reducing the dimensionality of the state space while preserving essential temporal dynamics for effective motion prediction. It compresses sequential data into state variables, facilitating superior performance in practical tasks. The research includes experimental evaluations on tasks such as cup placing and case loading.

Results: Experimental results indicate that Mamba, despite higher estimation errors, achieves superior success rates compared to Transformers in task execution.

Large Language Models Can Understanding Depth from Monocular Images

Why?: Monocular depth estimation is vital for computer vision, with applications in robotics, autonomous driving, and augmented reality. Leveraging large language models' capabilities can enhance depth estimation, providing efficient solutions across these domains.

How?: The research introduces LLM-MDE, a framework using LLMs for depth estimation from monocular images. It applies cross-modal reprogramming to align vision data with text, and an adaptive prompt estimation module that generates automatic prompts for the LLM based on image input. Both strategies integrate with a pretrained LLM to improve depth estimation tasks with minimal supervision. The framework is validated through experiments on real-world monocular depth estimation datasets.

Survey papers

Contemporary Model Compression on Large Language Models Inference - The survey explores contemporary model compression techniques to make LLMs memory efficient and to achieve faster computation.

LLM-Assisted Visual Analytics: Opportunities and Challenges - This is a comprehensive survey of current directions in integrating LLMs into Visual Analytics (VA) systems.

Large Language Model-Based Agents for Software Engineering: A Survey - This paper collects and analyzing 106 papers related to LLM-based agents in SE.

Let’s make LLMs safe!!

Alignment-Aware Model Extraction Attacks on Large Language Models - The researchers developed a novel attack algorithm called Locality Reinforced Distillation (LoRD). LoRD leverages a policy-gradient-style training task that uses the responses from victim models to guide preference crafting for the local model. Theoretical analysis ensures that LoRD's convergence aligns with the LLM's training alignments, and it reduces query complexity while countering watermark protections. Extensive experiments were conducted on various state-of-the-art LLMs to evaluate the effectiveness of the algorithm.

Well, that escalated quickly: The Single-Turn Crescendo Attack (STCA) - Research exposes vulnerabilities in LLMs by highlighting how adversarial inputs can provoke inappropriate outputs, emphasizing the need for stronger safeguards in responsible AI. The research introduces STCA, which combines the escalating techniques of previous multi-turn attacks into a single potent prompt. This method condenses escalation into one interaction, effectively bypassing LLM moderation filters designed to block harmful responses. The approach was developed to showcase the weaknesses in existing filter mechanisms of LLMs, which typically monitor dialogues over several interactions.

SafeEmbodAI: a Safety Framework for Mobile Robots in Embodied AI Systems - The research introduced SafeEmbodAI, a framework composed of secure prompting, state management, and safety validation mechanisms to better integrate mobile robots in embodied AI systems. It aims to secure and assist LLMs in processing multi-modal data and validating robotic behaviors through comprehensive safety checks. This framework's effectiveness is tested through a newly designed metric focusing on mission-oriented exploration in simulated environments.

Membership Inference Attacks Against In-Context Learning - The researchers developed the first membership inference attack specifically for ICL, using only generated texts. They crafted four attack strategies optimized for different restrictive scenarios and conducted experiments on four major LLMs. The study also evaluates a hybrid attack strategy that combines the strengths of the four individual strategies. Furthermore, they examined three defence mechanisms focusing on data handling, instructional adjustments, and output modifications to mitigate privacy risks. The attack strategies achieved a 95% accuracy advantage in most cases.

LLMs related research papers published between 26th august to 1st September

Mon, 16 Sep 2024 11:00:00 +0000

🔑 takeaway from today’s newsletter

From Ambiguous to Accurate: LLMs Now Tackle Vague Questions with Sharp Precision!
Guardians of the Algorithm: New Shields Protect AI from Sneaky Prompt Hacks and Data Poisoning!
AI Gets a Creative Spark: Mixing Logic with LLMs for Stories You've Never Dreamed Of!
Overconfidence Overruled: Entropic Steering Makes AI Agents Explore More and Guess Less!
Cracking the Cultural Code: Can LLMs Master Multilingual Chats and Hidden Contexts?

Core research improving LLMs!

Empirical influence functions to understand the logic of fine-tuning

Why?: Improving and interpreting the learning process in neural networks is crucial for enhancing their performance.

How?: The researchers investigated how individual training samples influence the output of neural networks during fine-tuning. They measured empirical influences and evaluated these influences against criteria such as decreasing influence with semantic distance, sparseness, noise invariance, transitive causality, and logical consistency. The investigation was conducted on both simple convolutional networks and a modern LLM. Additionally, they explored the impact of prompts in potentially mitigating the shortcomings found in these models when they fail to meet the given criteria.

Results: The study found that popular models, including simple convolutional networks and modern LLMs, fail to meet the desiderata for influences. However, prompting showed potential in partially rescuing this failure, offering a practical way to quantify how well neural networks learn from fine-tuning stimuli.

A Closer Look at Logical Reasoning with LLMs: The Choice of Tool Matters

Why?: Understanding how different symbolic solvers impact LLMs logical reasoning could help improve their performance on complex tasks.

How?: The researchers integrated LLMs with three different symbolic solvers: Z3, Pyke, and Prover9. They then evaluated the combined systems performance on three logical reasoning datasets: ProofWriter, PrOntoQA, and FOLIO. The integration involved designing interfaces for LLMs to communicate effectively with the solvers. Extensive experimentation measured the impact of each solver on the LLMs ability to solve logical reasoning tasks. Performance was assessed based on accuracy and the number of questions each solver executed successfully.

Results: Z3's overall accuracy performance slightly surpasses Prover9, but Prover9 executed more questions successfully. Pyke's performance was significantly inferior to both Z3 and Prover9.

Controlling Large Language Model Agents with Entropic Activation Steering

Why?: This research tackles the problem of overconfidence in LLM agents, which is critical for improving their decision-making and explorative behaviors.

How?: Experiments were conducted in controlled sequential decision-making tasks to observe LLM behavior. Researchers identified that LLM agents display overconfidence due to a collapse in the entropy of the action distribution when sampling. Traditional token-level sampling methods were found insufficient for promoting exploration. Therefore, an activation steering method called Entropic Activation Steering (EAST) was introduced. EAST works by calculating a steering vector as an entropy-weighted combination of representations and then manipulating the LLM's activations during the forward pass to increase uncertainty and exploration in the actions.

Results: EAST was able to reliably increase the entropy in LLM agent actions, leading to enhanced explorative behavior. It also allows for better control and interpretation of how LLM agents represent uncertainty.

Let’s make LLMs safe!!

On Overcoming Miscalibrated Conversational Priors in LLM-based Chatbots

LLMs often stumble when faced with under-specified queries—questions that lack sufficient detail. This leads to poor or irrelevant responses. Researchers analyzed public chat logs and found this issue to be widespread. To address it, they modeled the problem using Partially Observed Decision Processes (PODPs). Think of PODPs as a way to make optimal decisions when you don't have all the information—a bit like navigating a foggy road with limited visibility.

By framing the chatbot's decision-making process as a PODP, they could derive improved policies that guide the LLM to ask clarifying questions or provide more useful answers despite the ambiguity. They then recalibrated LLMs using learned control messages based on these policies, effectively teaching the models to handle vagueness better.

Exploring Vulnerabilities and Protections in Large Language Models: A Survey

LLMs are vulnerable to prompt hacking and adversarial attacks, which can manipulate them into producing harmful or misleading content. For instance:

Prompt Injection Attacks: Crafting inputs that trick the model into revealing confidential information or behaving undesirably.
Jailbreaking Attacks: Bypassing the model's safety protocols to generate prohibited content.
Data Poisoning and Backdoor Attacks: Injecting malicious data during training so the model behaves incorrectly when triggered.

A comprehensive survey mapped out these vulnerabilities and highlighted mitigation strategies. Defenses include:

Input Sanitization: Filtering or rephrasing user inputs to remove malicious patterns.
Robust Training: Exposing models to adversarial examples during training to make them resilient.
Monitoring and Detection: Implementing systems to detect unusual model behaviors indicative of an attack.

LIDAO: Towards Limited Interventions for Debiasing (Large) Language Models

Bias in LLM outputs can perpetuate stereotypes and unfairness. Traditional debiasing methods often degrade the model's language fluency. Enter LIDAO (Limited Interventions for Debiasing AI Outputs), a framework based on information theory. Instead of overhauling the entire model, LIDAO makes minimal adjustments to reduce bias.

Imagine tuning a radio to eliminate static without changing the station. LIDAO fine-tunes the model's outputs to maintain clarity and expressiveness while ensuring fairness. Tests on models ranging from 0.7 to 7 billion parameters showed that LIDAO effectively debiased outputs even in adversarial scenarios designed to provoke biased responses.

Enhancing Text Authenticity: A Novel Hybrid Approach for AI-Generated Text Detection

With AI-generated content proliferating, distinguishing it from human-written text is crucial to combat misinformation. Researchers developed a hybrid detection method combining:

Traditional TF-IDF Techniques: Analyzing word importance based on frequency.
Advanced Machine Learning Models: Utilizing classifiers like Bayesian models, Stochastic Gradient Descent (SGD), Categorical Gradient Boosting (CatBoost), and multiple instances of DeBERTa-v3-large models.

By merging classic text analysis with deep learning, this approach enhances detection accuracy. Extensive testing confirmed its superiority over existing methods, aiding efforts to maintain content authenticity.

The Best of Both Worlds: Toward an Honest and Helpful Large Language Model

LLMs sometimes provide confident answers even when unsure, which can mislead users. To promote honesty without sacrificing helpfulness, researchers introduced HoneSet, a dataset of 930 queries across six categories designed to evaluate model honesty.

They proposed two methods:

Training-Free Approach: Using curiosity-driven prompting to encourage the model to express uncertainty naturally.
Fine-Tuning Approach: A two-stage training process where the model first learns to distinguish honest from dishonest responses, then focuses on being helpful while maintaining honesty.

These methods teach LLMs that it's acceptable to admit uncertainty—much like a knowledgeable person saying, "I'm not sure, but I can find out." Experiments on nine different LLMs showed improved trustworthiness in their responses.

Large Language Model Confidence Estimation via Black-Box Access

Knowing how confident an LLM is in its response helps users assess reliability. Researchers developed a framework to estimate this confidence using:

Engineered Features: Extracting interpretable data from the model's outputs, such as response length, specificity, and use of hedging words.
Logistic Regression Model: Training a statistical model to predict confidence levels based on these features.

By treating the LLM as a black box, they could apply this method without altering the model's architecture. Tests using benchmark datasets like TriviaQA, SQuAD, CoQA, and Natural Questions demonstrated that this approach effectively indicates when the model's answers can be trusted.

Creative ways to use LLMs!!

Guiding and Diversifying LLM-Based Story Generation via Answer Set Programming

LLMs excel at generating coherent text but often lack diversity in storytelling. To tackle this, this paper introduced a hybrid approach that combines LLMs with Answer Set Programming (ASP), a form of symbolic logic programming. By using ASP to generate abstract story structures, the LLMs are guided to produce narratives with greater diversity. This method results in stories that are more varied and creative compared to those generated by LLMs alone, highlighting the benefits of integrating symbolic methods for guiding content generation.

Wav2Prompt: End-to-End Speech Prompt Generation and Tuning For LLM in Zero and Few-shot Learning

Bridging the gap between speech processing and language models, Wav2Prompt presents an end-to-end approach for integrating spoken input with LLMs. It trains on speech data to learn continuous speech representations, which serve as prompts for the LLM. This method aligns speech and text using a continuous integrate-and-fire mechanism, allowing the model to handle tasks like speech translation and spoken-query-based QA effectively. Notably, Wav2Prompt outperforms traditional cascaded models in few-shot learning scenarios, demonstrating significant improvements in tasks such as English-French speech translation.

Domain-specific ReAct for physics-integrated iterative modeling: A case study of LLM agents for gas path analysis of gas turbines

Exploring the capabilities of LLMs in specialized domains, researchers developed a domain-specific ReAct framework for physics-integrated iterative modeling, focusing on gas path analysis of gas turbines. By setting up a dual-agent tool-calling process, the LLMs integrate reasoning with expert knowledge and industry tools. While smaller models struggled, larger LLMs showed promising abilities in handling complex, multi-component problems when fine-tuned appropriately. This underscores the potential of large-scale LLMs in specialized engineering tasks, provided they are equipped with domain knowledge and proper prompt design.

An LLM-based Recommender System Environment

Training recommender systems often suffers from limited online data and evaluation challenges. An innovative solution is the creation of an LLM-based synthetic environment that simulates human behavior for training and evaluating recommender systems using reinforcement learning (RL). This framework allows for more comprehensive testing and development of recommendation algorithms by providing a controlled environment that mimics user interactions, as demonstrated in experiments with movie and book recommendations.

Enhancing Presentation Slide Generation by LLMs with a Multi-Staged End-to-End Approach

Generating presentation slides from lengthy documents can be labor-intensive. A multi-staged end-to-end model combines LLMs with Vision-Language Models (VLMs) to automate this process. The LLM first extracts and summarizes key content, while the VLM identifies and incorporates relevant visual elements. Through iterative refinement, the model produces cohesive and visually appealing presentations, outperforming state-of-the-art prompting methods in both automated metrics and human evaluations.

A Practice-Friendly Two-Stage LLM-Enhanced Paradigm in Sequential Recommendation

In sequential recommendation systems (SRS), incorporating LLMs can be inefficient, especially with sparse textual data. The TSLRec paradigm addresses this with a two-stage approach. Initially, it performs user-level supervised fine-tuning to inject collaborative filtering information using a pre-trained SRS model. Then, it enhances item representations by generating embeddings that merge this collaborative information with the LLM's inference capabilities. This method has been validated on benchmark datasets, showing improved efficiency and effectiveness in recommendations.

Supporting Self-Reflection at Scale with Large Language Models: Insights from Randomized Field Experiments in Classrooms

Self-reflection enhances learning, but scaling personalized reflection activities is challenging. Randomized field experiments in undergraduate courses demonstrated that LLM-guided reflection can significantly improve student confidence and exam performance. By providing tailored reflection prompts, LLMs help students consolidate knowledge more effectively than traditional methods like reviewing lecture slides, indicating a valuable role for LLMs in educational settings.

Large Language Models for Relevance Judgment in Product Search

Accurate relevance judgment is essential for effective product search. Researchers explored techniques for fine-tuning LLMs to automate the assessment of query-item pairs (QIPs). By optimizing hyperparameters and experimenting with different attribute concatenation methods and prompting strategies, they achieved relevance annotations comparable to human evaluators. This advancement has immediate applications in improving product search engines by automating a traditionally labor-intensive process.

LLMs for robotics & VLLMs

Evaluating Uncertainty-based Failure Detection for Closed-Loop LLM Planners

Why?: The research is important as it seeks to enhance the reliability of LLM-based closed-loop planning in robotic manipulation tasks by detecting and addressing planning failures.

How?: The authors introduce KnowLoop, a framework that incorporates an uncertainty-based failure detection mechanism in closed-loop LLM-based planning. The framework quantifies uncertainty using three methods: token probability, entropy, and self-explained confidence. These methods are evaluated using specific prompting strategies. A custom dataset with various manipulation tasks and an LLM-based robot system is used to test these metrics. The effectiveness of the metrics is measured by filtering out uncertain predictions and actively seeking human help when thresholds are exceeded.

Results: The research demonstrates that token probability and entropy are more reflective metrics compared to self-explained confidence. By setting an appropriate threshold to filter out uncertain predictions, the accuracy of failure detection can be significantly enhanced, thus boosting the effectiveness of closed-loop planning and the overall success rate of tasks.

LLMs evaluations

Prompt Chaining or Stepwise Prompt? Refinement in Text Summarization

Why?: To determine the most effective iterative refinement approach in text summarization for improving LLM performance.

How?: The researchers designed two strategies, Prompt Chaining and Stepwise Prompt, for iterative refinement in text summarization. They implemented Prompt Chaining by using three distinct prompts to sequentially draft, critique, and refine the text. For the Stepwise Prompt method, these three phases were integrated within a single prompt to simulate the refinement process. The effectiveness of each method was evaluated through extensive experiments comparing the quality of the generated summaries.

Results: Experimental results show that the prompt chaining method can produce a more favorable outcome. The stepwise prompt might produce a simulated refinement process, according to various experiments. These insights could be extrapolated to other applications, contributing to the broader development of LLMs.

Beyond Metrics: Evaluating LLMs' Effectiveness in Culturally Nuanced, Low-Resource Real-World Scenarios

Why?: Evaluating LLMs in multilingual and culturally nuanced scenarios is crucial for effective real-world applications.

How?: Researchers conducted a performance evaluation of seven LLMs (Mistral-7b, Mixtral-8x7b, GPT-3.5-Turbo, Llama-2-70b, Gemma-7b, GPT-4, and GPT-4-Turbo) on sentiment analysis tasks using a dataset comprising code-mixed WhatsApp chats with Swahili, English, and Sheng. They employed both quantitative metrics like F1 score and qualitative assessments by analyzing the explanations provided by each LLM for its predictions. The study focused on linguistic and contextual nuances, decision-making transparency, and cultural understanding, comparing the models' effectiveness in these areas.

An Evaluation Benchmark for Autoformalization in Lean4

Why?: The research is important because autoformalization can vastly improve the efficiency and accuracy of formalizing mathematical proofs, thereby advancing scientific research and development.

How?: The research introduced a new evaluation benchmark specifically designed for Lean4, a mathematical programming language. The benchmark was then applied to test the autoformalization capabilities of state-of-the-art LLMs, including GPT-3.5, GPT-4, and Gemini Pro. The evaluation involved analyzing the performance of these LLMs on various mathematical tasks, particularly focusing on more complex areas of mathematics to gauge their autoformalization potential comprehensively.

Results: The comprehensive analysis revealed that, despite recent advancements, these LLMs still exhibit limitations in autoformalization, particularly in more complex areas of mathematics. These findings underscore the need for further development in LLMs to fully harness their potential in scientific research and development.

Are Large Vision Language Models up to the Challenge of Chart Comprehension and Reasoning? An Extensive Investigation into the Capabilities and Limitations of LVLMs

Why?: This research is important because it explores the capabilities and limitations of LVLMs in chart comprehension and reasoning, a nuanced area that demands both vision-language reasoning and understanding of data visualizations.

How?: The researchers conducted a comprehensive evaluation of LVLMs, including models such as GPT-4V and Gemini, focusing on four major chart reasoning tasks: chart question answering, chart summarization, fact-checking with charts, and general chart comprehension. The evaluation included both quantitative assessments and qualitative analysis across a diverse range of chart types. The qualitative evaluation was aimed at identifying specific strengths and weaknesses of LVLMs. Techniques used involved assessing the fluency and accuracy of generated text, and identifying issues such as hallucinations, factual errors, and data bias.

Results: The findings reveal that LVLMs demonstrate impressive abilities in generating fluent texts covering high-level data insights while also encountering common problems like hallucinations, factual errors, and data bias. The study highlights the key strengths and limitations of chart comprehension tasks, offering insights for future research.

Summary of LLMs related research papers published on 30th May, 2024

Mon, 22 Jul 2024 15:45:33 +0000

Core research improving LLMs!

Two Optimizers Are Better Than One: LLM Catalyst Empowers Gradient-Based Optimization for Prompt Tuning

Why?: The research is important as it explores the integration of gradient-based optimization with LLM-based optimization to enhance the efficacy of solving complex non-convex optimization problems in prompt tuning.

How?: The research proposes a collaborative optimization approach that combines gradient-based and LLM-based optimizers in an interleaved manner. The team begins by using a gradient-based optimizer to make locally optimal updates at each step, akin to a diligent doer. Alongside, they introduce an LLM-based optimizer that acts as a high-level instructor by inferring solutions from natural language instructions. Task descriptions and optimization trajectories recorded during the gradient-based optimization process are utilized to instruct the LLMs. The inferred results from these LLMs serve as restarting points for subsequent stages of gradient optimization. This iterative process leverages both the rigorous local updates of gradient-based methods and the high-level deductive reasoning capabilities of LLMs.

Results: The combined optimization method consistently yields improvements over competitive baseline prompt tuning methods, demonstrating the synergistic effect of conventional gradient-based optimization and the inference ability of LLMs.

Visual Perception by Large Language Model's Weights

Why?: This research addresses the high computational costs involved in multimodal LLMs by proposing a more efficient method for integrating visual information.

How?: The researchers introduced VLoRA, a new approach leveraging parameter space alignment instead of the conventional input space alignment. They utilized a vision encoder to extract visual features from images, which were then converted into perceptual weights. These perceptual weights were subsequently merged with the LLM's weights. This approach eliminated the need for visual tokens in the input sequence, thereby reducing its length and enhancing computational efficiency. The primary innovation was the perceptual weights generator that transformed visual features into low-rank perceptual weights, aligning with the structure similar to LoRA (Low-Rank Adaptation). The experimental validation was conducted on various benchmarks for multimodal LLMs to establish the effectiveness of this method.

S3D: A Simple and Cost-Effective Self-Speculative Decoding Scheme for Low-Memory GPUs

Why?: Improving the efficiency of speculative decoding in low-memory GPUs is crucial for enabling faster LLM inference on less powerful hardware.

How?: The researchers proposed an approach called Skippy Simultaneous Speculative Decoding (S3D), which leverages simultaneous multi-token decoding combined with mid-layer skipping. This method aims at minimizing the memory overhead while still providing substantial speedup for LLM inference. Multi-token decoding enables the generation of multiple tokens in one step, thus speeding up the decoding process, while mid-layer skipping reduces the computational load by skipping unnecessary layers during the inference. The researchers designed S3D with minimal architectural changes and without requiring extensive new training data. They utilized the memory efficiency provided by S3D to create a high-performing yet smaller SD model based on the Phi-3 architecture and compared its performance against recent open-source SD systems, particularly focusing on the quantized EAGLE model, analyzing speed and memory usage in half-precision mode.

Results: When compared against recent effective open-source SD systems, the proposed method has achieved one of the top performance-memory ratios while requiring minimal architecture changes and training data. The smaller yet more effective SD model based on Phi-3 was found to be 1.4 to 2 times faster than the quantized EAGLE model and operates in half-precision while using less VRAM.

Towards Hierarchical Multi-Agent Workflows for Zero-Shot Prompt Optimization

Why?: The research addresses the critical dependency of LLM outputs on prompt design, enhancing their effectiveness without human intervention or task-specific training.

How?: The researchers proposed a novel approach named Hierarchical Multi-Agent Workflow (HMAW) that leverages multiple LLMs in a hierarchical structure to autonomously design and optimize prompts. HMAW divides the process into two main steps. First, one or more LLMs collaboratively construct an optimal prompt by refining instructions and wording through multiple iterations. Second, the final prompt generated by the hierarchy is used by another LLM to answer the user's query. This method is designed to be task-agnostic, meaning it does not require tuning to specific tasks nor does it involve pre-training or task-specific data. The efficacy of HMAW was evaluated through both quantitative and qualitative experiments across various benchmarks, demonstrating its ability to boost LLM performance without significant overhead or complexity.

InstructionCP: A fast approach to transfer Large Language Models into target language

Why?: This research addresses the challenge of adapting English-focused LLMs to other languages while maintaining their conversational and content filtering abilities.

How?: The researchers propose a method called Instruction Continual Pre-training (InsCP), which incorporates instruction tags into the continual pre-training (CP) process. This approach aims to prevent the degradation of conversational abilities and content filtering effectiveness when transferring LLMs to new languages. The process starts by collecting 0.1 billion tokens of high-quality instruction-following data in the target language. These tokens contain specific instruction tags that guide the model in maintaining its conversational proficiency. The InsCP method then integrates these tokens into a modified CP regimen, ensuring that language adaptation is achieved without compromising other essential functionalities of the LLM. The efficacy of InsCP is validated through empirical evaluations on benchmarks for language alignment, reliability, and knowledge retention, ensuring that the model performs comparably to its original version while effectively responding in the target language.

Results: Empirical evaluations confirm that InsCP retains conversational and Reinforcement Learning from Human Feedback (RLHF) abilities. The method requires only 0.1 billion tokens of high-quality instruction-following data, thereby reducing resource consumption.

The Fine-Tuning Paradox: Boosting Translation Quality Without Sacrificing LLM Abilities

Why?: To improve translation quality while preserving desirable abilities in large language models (LLMs).

How?: The research involves an extensive evaluation of the translation capabilities of the LLaMA and Falcon families of models, which range from 7 billion to 65 billion parameters. The authors fine-tuned these models on parallel data to observe its impact on various desirable LLM behaviors, such as steerability, inherent document-level translation, and the ability to produce less literal translations. During this process, they assessed how these abilities degraded or improved post fine-tuning. To address the degradation of certain abilities, the researchers incorporated monolingual data as part of the fine-tuning process. This strategy aimed to balance between enhancing translation quality and maintaining other beneficial LLM properties.

Results: The study found that while fine-tuning improved general translation quality, it led to a decline in formality steering, technical translation through few-shot examples, and document-level translation abilities. However, models produced less literal translations. By including monolingual data in the fine-tuning process, these abilities were better preserved while still enhancing overall translation quality.

Why?: The research aims to improve Retrieval Augmented Generation (RAG) by addressing the limitations of relying solely on similarity between queries and documents, which can lead to performance degradation in knowledge-intensive tasks.

How?: The research introduces MetRag, a multi-layered thought framework that enhances Retrieval Augmented Generation. MetRag incorporates three layers of thoughts: similarity-oriented, utility-oriented, and compactness-oriented thoughts.

1. Similarity-Oriented Thought: This conventional approach retrieves documents based on similarity metrics to the given query.

2. Utility-Oriented Thought: A small-scale utility model, supervised by an LLM, evaluates the usefulness of the retrieved documents, adding a layer of relevance beyond mere similarity.

3. Compactness-Oriented Thought: To deal with the often large set of retrieved documents, an LLM is utilized as a task-adaptive summarizer.

This step aims to capture commonalities and characteristics within the document set, making the information more compact and manageable.multi-layered thoughts are then combined in the final stage where an LLM generates the knowledge-augmented response. This comprehensive approach aims to tackle the shortcomings of existing RAG methods by integrating not just similarity and relevance, but also conciseness and coherence in the generation process.

Parrot: Efficient Serving of LLM-based Applications with Semantic Variable

Why?: The research aims to enhance the efficiency of LLM-based applications by addressing limitations in current public LLM services, which lead to sub-optimal end-to-end performance.

How?: The research introduces Parrot, a service system designed to improve the end-to-end performance of LLM-based applications. Parrot employs a novel concept called 'Semantic Variable,' which serves as a unified abstraction to expose application-level knowledge to public LLM services. Semantic Variables annotate input/output variables in the prompts of requests and create data pipelines when linking multiple LLM requests. This allows for a more integrated workflow and helps uncover correlations across multiple LLM requests through data flow analysis. By performing such an analysis, Parrot can identify optimization opportunities that would not be apparent when handling individual LLM requests separately.

Results: Extensive evaluations demonstrate that Parrot can achieve up to an order-of-magnitude improvement for popular and practical use cases of LLM applications.

KNOW: A Real-World Ontology for Knowledge Capture with Large Language Models

Why?: The research is important as it aims to create a structured ontology that enhances LLMs' ability to handle real-world generative AI tasks, especially for applications like personal AI assistants.

How?: To create the KNOW ontology, the researchers focused on a pragmatic approach that started with established human universals, specifically spacetime (places and events) and social structures (people, groups, organizations). The ontology is designed to capture the everyday knowledge that LLMs can use to augment various applications. The team compared and contrasted KNOW with previous projects like Schema.org and Cyc, noting how LLMs already internalize much of the commonsense knowledge that older projects painstakingly gathered over decades. They emphasize simplicity and usability by making software libraries available in 12 popular programming languages. These libraries enable developers to leverage the ontology concepts directly in their software engineering efforts, promoting better AI interoperability and enhancing the developer experience. Key criteria for defining concepts in the ontology include universality and utility, ensuring that only the most pragmatically valuable concepts are included.

Is In-Context Learning Sufficient for Instruction Following in LLMs?

Why?: Understanding how effectively in-context learning (ICL) aligns with long-context language models (LLMs) for following instructions is important for enhancing their performance in real-world tasks.

How?: The researchers began by evaluating the performance of LLMs utilizing URIAL, a method that leverages three in-context examples to align base LLMs, and compared its performance to instruction fine-tuning on standard benchmarks such as MT-Bench and AlpacaEval 2.0 (LC). They observed that adding more ICL demonstrations did not systematically improve instruction following performance for long-context LLMs. To improve this, they introduced a greedy selection approach for ICL examples, which aimed at optimizing the choice of examples to enhance performance. Additionally, the researchers conducted ablation studies to better understand why there was a performance gap between ICL alignment and instruction fine-tuning, specifically looking into aspects of ICL that are unique to the instruction tuning context.

Results: While effective, ICL alignment with URIAL underperforms compared to instruction fine-tuning, especially with more capable base LMs. Adding more ICL demonstrations does not systematically improve instruction following performance. A greedy selection approach for ICL examples improves performance but does not bridge the gap to instruction fine-tuning. Ablation studies reveal specific reasons behind the remaining gap and aspects of ICL unique to the instruction tuning setting.

One Token Can Help! Learning Scalable and Pluggable Virtual Tokens for Retrieval-Augmented Large Language Models

Why?: The research addresses how to improve retrieval-augmented generation (RAG) in LLMs without compromising their general generation abilities.

How?: The research proposes learning scalable and pluggable virtual tokens tailored for retrieval-augmented generation. Instead of fine-tuning the entire LLM, which can affect its general capabilities, the approach optimizes only the embeddings of these virtual tokens. This ensures that LLMs can benefit from updated and factual information retrieved without altering their inherent parameters. The method involves multiple training strategies to ensure the virtual tokens are scalable and can integrate seamlessly with various tasks, thus maintaining the general flexibility and functionality of the original model. The training process emphasizes improving embeddings while keeping the primary LLM parameters intact to preserve overall model performance while enhancing retrieval-augmented capabilities.

Let’s make LLMs safe!!

Phantom: General Trigger Attacks on Retrieval Augmented Language Generation

Why?: This research is significant because it uncovers new security vulnerabilities in Retrieval Augmented Generation (RAG) systems, which can be exploited to compromise the integrity and safety of chatbot applications that rely on LLMs.

How?: The research was conducted by designing a framework called Phantom, which executes a two-step attack on RAG systems. First, the researchers created a poisoned document designed to be retrieved by the RAG system. This poisoned document included a specific sequence of words, acting as a backdoor trigger, that ensured it would be among the top-k results for certain queries. During the second step, this adversarial document, once retrieved, included an adversarial string that could trigger various harmful behaviors in the LLM generator. These behaviors included denial of service, reputation damage, privacy violations, and inducing harmful actions in the chatbot. The attacks were validated on multiple LLM architectures, including Gemma, Vicuna, and Llama, demonstrating the broad applicability and effectiveness of the attack method across different systems.

Results: The research successfully demonstrated the Phantom attack framework across multiple LLM architectures, validating its effectiveness and broad applicability. The attacks were able to compromise the RAG system by retrieving malicious documents that triggered adversarial behaviors in the LLM generator, confirming the new security risks posed by RAG systems.

Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characters

Why?: The research addresses the vulnerability of Large Language Models (LLMs) to specific prompts designed to bypass moderation guardrails, which can lead to harmful behaviors. Current benchmarks often overlook these moderation triggers, complicating the evaluation of jailbreak strategies.

How?: The research proposes JAMBench as a new benchmark to evaluate the moderation guardrails of LLMs by including 160 manually crafted instructions across four risk categories with varying levels of severity. The authors also introduce a new jailbreak method called JAM (Jailbreak Against Moderation). The JAM method involves using tailored jailbreak prefixes to bypass input-level filters and employing a fine-tuned shadow model that is functionally similar to the guardrail model. This shadow model generates cipher characters designed to circumvent output-level filters. The study conducts extensive experiments on four different LLMs to evaluate the efficacy of JAM against existing guardrails.

Results: JAM achieves higher jailbreak success ( x19.88) and lower filtered-out rates ( x1/6) compared to baseline methods during extensive experiments on four different LLMs.

Creative ways to use LLMs!!

ParSEL: Parameterized Shape Editing with Language - Focuses on enabling precise 3D shape edits through natural language instructions, democratizing 3D content creation. The system effectively translates language inputs into geometric editing tasks, outperforming traditional methods in control and precision.

Large Language Models Can Self-Improve At Web Agent Tasks - Examines the self-improvement capabilities of LLMs in navigating and performing complex web agent tasks, using synthetic training data for fine-tuning. The method shows significant enhancements in performance and robustness.

PostDoc: Generating Poster from a Long Multimodal Document Using Deep Submodular Optimization - Develops a method to automatically transform long multimodal documents into concise and visually appealing posters. Demonstrates significant improvements in content relevance and design aesthetics, enhancing educational and professional presentations.

GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoning - Combines Graph Neural Networks with LLMs to improve question-answering over Knowledge Graphs. Shows state-of-the-art performance on benchmark tests, significantly enhancing factual question-answering capabilities.

LLMs in robotics!!

Nadine: An LLM-driven Intelligent Social Robot with Affective Capabilities and Human-like Memory - Enhances human-robot interactions by integrating advanced LLMs into the Nadine social robot platform, giving it human-like memory and emotional appraisal capabilities. The robot system uses multimodal inputs and episodic memory to simulate emotional states and provide natural interaction experiences.

Robo-Instruct: Simulator-Augmented Instruction Alignment For Finetuning CodeLLMs - Introduces Robo-Instruct, which enhances the performance of open-weight LLMs in robot programming by integrating simulator-based verification and instruction alignment. This method successfully brings smaller models up to or beyond the capability of larger proprietary models like GPT-3.5-Turbo.

InterPreT: Interactive Predicate Learning from Language Feedback for Generalizable Task Planning - Develops a framework that teaches robots to perform long-horizon planning by learning symbolic predicates from human language feedback. The system translates learned predicates and operators into PDDL for robust task planning in diverse settings, achieving high success rates in real-world robot manipulation scenarios.

Summary of LLMs related research papers published on 29th May, 2024

LLMs Research — Fri, 12 Jul 2024 20:28:11 +0000

Core research improving LLMs!

Preference Learning Algorithms Do Not Learn Preference Rankings

Why?: Understanding the limitations of preference learning algorithms is crucial for improving how LLMs are trained to align with human preferences.

How?: The researchers conducted a detailed analysis of the ranking accuracy of state-of-the-art preference-tuned models. They evaluated models trained with popular preference learning algorithms like Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO). To measure performance, they assessed how often these models assigned higher likelihoods to more preferred outputs compared to less preferred ones, using established preference datasets. Additionally, they derived the idealized ranking accuracy that would be achieved if a preference-tuned LLM optimized the DPO or RLHF objectives perfectly. An efficient formula was introduced to quantify the difficulty of learning from given preference datapoints, and experiments were conducted to explore the relationship between ranking accuracy and the empirically popular win rate metric when the model closely matches the reference model used in learning objectives. This provided insights into the alignment gap and differences between on-policy (e.g., RLHF) and off-policy (e.g., DPO) learning algorithms.

Conveyor: Efficient Tool-aware LLM Serving with Tool Partial Execution

Why?: Integrating LLMs with external tool invocations has significantly increased the complexity and latency of LLM serving workloads.

How?: The researchers introduced Conveyor, an LLM serving system designed to optimize the handling of requests that involve external tools. Conveyor employs a novel interface specifically for tool developers, allowing them to expose partial execution opportunities. This enables portion of tool operations to be executed concurrently alongside LLM decoding. Additionally, the system includes a request scheduler that efficiently manages these partial tool executions. By doing so, Conveyor aims to reduce the latency ordinarily associated with handling LLM requests that require external tool interactions.

Self-Exploring Language Models: Active Preference Elicitation for Online Alignment

Why?: The research focuses on enhancing the alignment of LLMs to better follow human intentions, which is crucial for their practical utility and safety.

How?: The authors propose a novel approach called Self-Exploring Language Models (SELM) to improve the alignment of LLMs through online preference elicitation. The method involves a bilevel optimization objective that is optimistically biased towards responses that are potentially high-reward but out-of-distribution. The inner-level optimization problem is solved using a reparameterized reward function, eliminating the need for a separate reward model. This iterative method allows the LLM to explore more diverse response spaces efficiently. Experiments were conducted using the Zephyr-7B-SFT and Llama-3-8B-Instruct models, and the SELM approach was tested on various benchmarks, including MT-Bench and AlpacaEval 2.0.

Results: The experimental results indicate that SELM significantly improves performance on instruction-following benchmarks such as MT-Bench and AlpacaEval 2.0, as well as various standard academic benchmarks in different settings. This demonstrates that SELM's active exploration leads to better alignment and enhanced performance of LLMs.

Normative Modules: A Generative Agent Architecture for Learning Norms that Supports Multi-Agent Cooperation

Why?: Fostering cooperation between agents and humans in environments with social norms is a critical challenge.

How?: The research introduces the concept of 'Normative Modules,' an architectural enhancement for generative agents. These modules allow agents to recognize and adapt to normative structures within an environment, focusing particularly on the problem of equilibrium selection in cooperative scenarios. To test this, the researchers designed a new type of environment that supports institutions. Agents equipped with the normative module learn via peer interactions to determine which among several candidate institutions is treated as authoritative by a group. This learning process helps the agents to better coordinate their sanctioning behavior, which in turn influences primary behaviors within a social setting. The study makes use of two key evaluation criteria: (i) the ability of the agent to ignore non-authoritative institutions and (ii) the ability to identify authoritative institutions among multiple candidates. These evaluations are essential to measure the stability and effectiveness of cooperation induced by the normative module.

Results: The study demonstrates that agents equipped with normative modules achieve more stable cooperative outcomes compared to baseline agents lacking such modules. This showcases the module's effectiveness in enabling agents to navigate and adapt to social norms, thereby fostering higher average welfare in multi-agent interactions.

MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Series - project page

Why?: The research aims to improve the transparency and performance of LLMs by creating an open-sourced, highly capable bilingual LLM.

How?: The researchers designed and trained a new bilingual language model called MAP-Neo with 7 billion parameters. The training was conducted on 4.5 trillion high-quality tokens. Crucially, they open-sourced not only the model's weights but all the details required to reproduce the model, including the pre-training corpus, data cleaning pipeline, intermediate checkpoints, and the training and evaluation frameworks. This approach allows for complete transparency and offers the research community the tools needed to study, improve, and innovate upon the model.

Results: The MAP-Neo model showed performance comparable to the state-of-the-art LLMs and provided a fully transparent and reproducible framework, enhancing the open research community's capability to study, innovate, and facilitate further improvements.

Nearest Neighbor Speculative Decoding for LLM Generation and Attribution

Why?: The research addresses the limitations of LLMs in hallucination and lack of attribution for their outputs.

How?: The authors introduce Nearest Neighbor Speculative Decoding (NEST), a semi-parametric language modeling approach. Unlike traditional LLMs, NEST performs token-level retrieval at each inference step and computes a semi-parametric mixture distribution to identify promising text span continuations from a non-parametric data store. The approach uses an approximate speculative decoding mechanism to either accept a retrieved text prefix or generate a new token, enhancing text fluency and providing source attribution. This approach refines the output of the base LM, outperforming kNN-LM in both quality and speed. The method was applied to Llama-2-Chat 70B for evaluation.

Results: NEST significantly enhances the generation quality and attribution rate of the base LM across a variety of knowledge-intensive tasks, surpassing the conventional kNN-LM method and performing competitively with in-context retrieval augmentation. Additionally, NEST substantially improves the generation speed, achieving a 1.8x speedup in inference time when applied to Llama-2-Chat 70B.

VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videos

Why?: There is a growing need for improving the understanding of long-form videos, as current methods often miss important details and operate inefficiently.

How?: The research introduces VideoTree, a hierarchical framework designed to dynamically extract and organize query-related information from long videos for better LLM reasoning. The process involves three stages: 1. Adaptive Frame Selection: Instead of using all frames, VideoTree uses an iterative clustering mechanism based on visual features. It then scores these clusters based on their relevance to the given query, ensuring that only significant frames are selected for captioning. 2. Hierarchical Tree Organization: The selected frames are organized into a tree structure where nodes represent clusters of varying relevance and granularity. This tree structure allows finer details to be encoded where needed most, according to the query's requirements. 3. Tree Traversal for Answer Generation: The hierarchical tree's keyframes are traversed, and their captions are sent to the LLM to generate an accurate answer. This ensures that only the most relevant information is processed, optimizing both accuracy and efficiency.

MEMoE: Enhancing Model Editing with Mixture of Experts Adaptors

Why?: Enhancing how LLMs can be efficiently and effectively edited improves their adaptability and reliability for various tasks without degrading their overall performance.

How?: The research introduces MEMoE, a novel model editing adapter that uses a Mixture of Experts (MoE) architecture paired with a knowledge anchor routing strategy. The MoE architecture facilitates the updating of the model's knowledge through a bypass mechanism, which ensures that the original parameters of the LLMs remain unchanged, thus preserving their general capabilities. The knowledge anchor routing component is crucial as it routes inputs that demand similar knowledge to the same expert within the MoE framework, which enhances the generalization of the new, updated knowledge while maintaining local specificity. This method was rigorously tested through different tasks, specifically batch editing and sequential batch editing, to compare its performance against existing model editing techniques.

Results: Experimental results show the superiority of our approach over both batch editing and sequential batch editing tasks, exhibiting exceptional overall performance alongside outstanding balance between generalization and locality.

Evaluating the External and Parametric Knowledge Fusion of Large Language Models

Why?: Integrating both external and parametric knowledge into LLMs can overcome limitations like outdated static memory, potentially improving their performance.

How?: The authors propose a four-scenario framework to deconstruct knowledge fusion within LLMs. They developed a systematic pipeline for creating datasets and infusing knowledge, which facilitates controlled experiments. This setup allowed for comprehensive testing of LLMs' ability to blend external and parametric knowledge. The investigation involves assessing the various fusion scenarios to identify strengths and weaknesses in LLMs' knowledge integration processes. Specific methods for enhancing parametric knowledge were applied to test if it would subsequently boost the overall capability of the model in these fusion scenarios.

Results: The study demonstrates that enhancing parametric knowledge within LLMs can significantly improve their capability for knowledge integration. However, it also highlights challenges around memorizing and accurately eliciting parametric knowledge and delineating its boundaries. These insights are intended to guide future work in harmonizing external and parametric knowledge in LLMs.

Towards Faithful Chain-of-Thought: Large Language Models are Bridging Reasoners

Why?: LLMs often generate unfaithful chain-of-thought (CoT) reasoning, impacting their reliability in tasks requiring complex reasoning.

How?: The researchers first analyzed CoT faithfulness at the level of individual reasoning steps and identified two reasoning paradigms: centralized and distributed reasoning, noting their relationship with faithfulness. They conducted a joint analysis examining the causal relevance among context, CoT, and the answer, discovering that the model can recall correct information from context that was missing in the CoT, leading to faithfulness issues. To address this, they proposed an inferential bridging method that employs an attribution method to recall pertinent information as hints during CoT generation and uses semantic consistency and attribution scores to filter out noisy CoTs.

Language Generation with Strictly Proper Scoring Rules

Why?: Improving the effectiveness of language generation models by exploring alternative scoring rules beyond the commonly used logarithmic score.

How?: The researchers propose using strictly proper scoring rules other than the logarithmic score, specifically the Brier score and the Spherical score, for training language generation models. Instead of relying solely on maximum likelihood estimation (MLE) with log-likelihood loss, they adapt these non-local scoring rules to the context of language generation. The adaptation involves training the models with these alternative loss functions without adjusting other hyperparameters. They evaluate the performance of these models using large language models like LLaMA-7B and LLaMA-13B to determine if the alternative loss functions improve language generation capabilities.

Results: Experimental results indicate that simply substituting the loss function, without adjusting other hyperparameters, can yield substantial improvements in the model's generation capabilities.

Compressing Large Language Models using Low Rank and Low Precision Decomposition

Why?: Large Language Models are immense in size, making them difficult to deploy on memory-constrained edge devices. Reducing their size without significant losses in performance is critical for broader and more efficient applications.

How?: The researchers introduced a post-training LLM compression algorithm named CALDERA, which leverages the low-rank structure of weight matrices. The algorithm approximates a weight matrix 𝐖 using a decomposition of the form 𝐖≈𝐐 + 𝐋𝐑, where 𝐋 and 𝐑 are low-rank matrices, and 𝐐, 𝐋, and 𝐑 are quantized to low precision. The model is compressed by substituting each layer with its 𝐐 + 𝐋𝐑 decomposition. The decomposition is achieved by setting up an optimization problem: min_𝐐,𝐋,𝐑‖(𝐐 + 𝐋𝐑 - 𝐖)𝐗^⊤‖_ F^2, with 𝐗 as calibration data. Constraints ensure that 𝐐, 𝐋, and 𝐑 are in low-precision formats. Additionally, 𝐋 and 𝐑 can be adapted for enhancing zero-shot performance. The research also provides theoretical upper bounds on the approximation error using rank-constrained regression framework and explores the tradeoff between compression ratios and model performance.

Results: Results illustrate that compressing LlaMa-2 7B/70B and LlaMa-3 8B models obtained using CALDERA outperforms existing post-training LLM compression techniques in the regime of less than 2.5 bits per parameter.

LLMs evaluations

MathChat: Benchmarking Mathematical Reasoning and Instruction Following in Multi-Turn Interactions - Introduces MathChat, a benchmark for testing LLMs in multi-turn mathematical problem solving. Highlights the significant improvements in model performance after fine-tuning with the MathChat sync dataset, particularly in multi-turn settings.

MASSIVE Multilingual Abstract Meaning Representation: A Dataset and Baselines for Hallucination Detection - Describes the creation of the MASSIVE-AMR dataset for studying multilingual abstract meaning representation and hallucination detection in AMR tasks. Shows that while LLMs are promising in structured parsing tasks, issues like linguistic diversity and accuracy remain challenges.

Are You Sure? Rank Them Again: Repeated Ranking For Better Preference Datasets - Proposes a Repeat Ranking method to enhance consistency in ranking outputs, improving the quality of datasets for training LLMs with reinforcement learning. Demonstrates that repeated ranking yields better performance and more reliable training data compared to standard methods.

LLMs achieve adult human performance on higher-order theory of mind tasks - Evaluates the ability of LLMs like GPT-4 and Flan-PaLM to handle complex theory of mind tasks against adult human benchmarks. Finds that these models reach or exceed adult human performance, underscoring the potential of LLMs in advanced cognitive processing.

Let’s make LLMs safe!!

Stress-Testing Capability Elicitation With Password-Locked Models

Why?: Assessing the safety of large language models (LLMs) by fully eliciting their capabilities is critical for AI developers.

How?: The researchers created password-locked models, which are LLMs fine-tuned to hide certain capabilities unless a specific password is present in the input prompt. The purpose is to see if sophisticated elicitation methods, particularly fine-tuning, can reveal these hidden capabilities. They conducted experiments to test various elicitation techniques. First, they provided a few high-quality demonstrations to the models to see if these could fully reveal the password-locked capabilities. They also tested the efficacy of fine-tuning with the password as well as with different passwords. Further, they evaluated reinforcement learning methods to elicit capabilities when demonstrations were not available.

Results: The study found that a few high-quality demonstrations are often sufficient to fully elicit password-locked capabilities. More surprisingly, fine-tuning can also elicit other capabilities that have been locked using the same or different passwords. Furthermore, reinforcement learning methods proved effective in eliciting capabilities when demonstrations were unavailable. Therefore, fine-tuning is an effective but potentially unreliable method for eliciting hidden capabilities, especially when models' hidden capabilities exceed those of human demonstrators.

One-Shot Safety Alignment for Large Language Models via Optimal Dualization

Why?: Addressing safety concerns in LLM alignment with human preferences is critical for enhancing their usability and reliability.

How?: The researchers propose a novel method to simplify the constrained alignment problem in LLMs by transforming it into an unconstrained alignment problem using a dualization approach. Instead of relying on computationally expensive and unstable Lagrangian-based primal-dual policy optimization methods, the authors pre-optimize a smooth and convex dual function that has a closed form. This eliminates the need for repeated primal-dual policy iterations. They develop two practical algorithms based on this strategy: MoCAN for model-based scenarios and PeCAN for preference-based scenarios. Both algorithms leverage the dualization technique to reduce computational burden and improve training stability.

Results: A broad range of experiments demonstrate the effectiveness of the proposed MoCAN and PeCAN algorithms in improving computational efficiency and training stability for safety alignment in LLMs.

Are Large Language Models Chameleons?

Why?: Understanding the biases and variability in LLM responses is crucial to ensure robust and fair applications of these models in modeling decisions or collective behaviors.

How?: The researchers conducted over one million simulations where LLMs were asked to answer subjective questions. They compared these answers to real data obtained from the European Social Survey (ESS). To measure the differences and biases between LLMs and survey data, they employed methods such as calculating weighted means and introduced a new measure inspired by the Jaccard similarity index. This approach allowed them to quantitatively analyze and highlight the cultural, age, and gender biases in the models' responses. Additionally, the study emphasized the importance of evaluating the robustness and variability of prompts used for LLMs.

Results: The study found significant cultural, age, and gender biases in LLMs' responses compared to real-world survey data. These findings underline the necessity of careful prompt analysis and bias mitigation to ensure LLMs' outputs are reliable and trustworthy for modeling individual or collective behaviors.

Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF

Why?: To address the challenge of incorporating uncertainty estimation in the reward function for reinforcement learning (RL) from human feedback (RLHF), making it suitable for large language models (LLMs).

How?: The researchers introduced Value-Incentivized Preference Optimization (VPO), a unified approach for both online and offline RLHF. VPO works by regularizing the maximum likelihood estimate of the reward function using the corresponding value function. This regularization is modulated by a sign indicating whether optimism or pessimism is chosen. The core of VPO is its implicit reward modeling, which ensures a simpler RLHF pipeline akin to direct preference optimization. Theoretical guarantees are provided for VPO in both online and offline scenarios, aligning with rates of standard RL methods. The method was validated through empirical experiments in text summarization and dialog tasks, demonstrating its practical effectiveness.

Results: The method showed practicality and effectiveness in empirical experiments, specifically in text summarization and dialog tasks.

Creative ways to use LLMs!!

Unlocking the Potential of Large Language Models for Clinical Text Anonymization: A Comparative Study - Examines LLMs for anonymizing clinical texts, introducing six new evaluation metrics to ensure privacy while maintaining data utility.

Reasoning3D – Grounding and Reasoning in 3D: Fine-Grained Zero-Shot Open-Vocabulary 3D Reasoning Part Segmentation via Large Vision-Language Models - Introduces Reasoning3D, a method using LLMs for 3D object segmentation based on textual queries, with rapid deployment in various applications.

Towards Next-Generation Urban Decision Support Systems through AI-Powered Generation of Scientific Ontology using Large Language Models – A Case in Optimizing Intermodal Freight Transportation - Details an AI workflow using the ChatGPT API for generating scientific ontologies, enhancing decision-making in urban and environmental management.

Learning from Litigation: Graphs and LLMs for Retrieval and Reasoning in eDiscovery - Explores a hybrid approach combining graph-based and LLM-based methods for improving eDiscovery processes in legal contexts.

Can Graph Learning Improve Task Planning? - Examines the integration of Graph Neural Networks with LLMs for enhancing task planning, demonstrating improvements in decision-making on complex tasks.

Kestrel: Point Grounding Multimodal LLM for Part-Aware 3D Vision-Language Understanding - Introduces Kestrel, a multimodal LLM enhancing 3D vision-language tasks by focusing on part-level understanding and grounded language comprehension.

Grasp as You Say: Language-guided Dexterous Grasp Generation - Introduces DexGYSNet, a dataset enabling robots to perform dexterous grasping from human language commands, enhancing human-robot interaction.

Validates the effectiveness of the DexGYSGrasp framework through extensive experiments, proving its capability in generating intent-aligned and high-quality dexterous grasps.

Summary of LLMs related research papers published on 27th May, 2024

Sat, 22 Jun 2024 19:04:53 +0000

Core research improving LLMs!

Generative Query Reformulation Using Ensemble Prompting, Document Fusion, and Relevance Feedback

Why?: Paper improves the query reformulation

How?: The authors propose two ensemble-based prompting techniques, GenQREnsemble and GenQRFusion, which generate multiple sets of keywords to enhance retrieval performance. They also introduce post-retrieval variants that incorporate relevance feedback from various sources, including an oracle simulating a human user and a 'critic' LLM. The research assesses the impact of ensemble query reformulations, feedback documents, domain-specific instructions, filtered reformulations, and fluent reformulations.

Results: An ensemble of query reformulations led to an improvement in retrieval effectiveness by up to 18% on nDCG@10 in pre-retrieval settings and 9% on post-retrieval settings on multiple benchmarks, outperforming all previously reported state-of-the-art results.

Matryoshka Multimodal Models

Why?: Large Multimodal Models (LMMs) can face inefficiency in handling dense visual scenarios like high-resolution images and videos. The existing token pruning/merging methods lack the flexibility to adjust information density versus efficiency. Therefore, there is a need to develop a more flexible model for representing visual content in LMMs.

How?: The Matryoshka Multimodal Models (M3) proposed in this research learn to represent visual content as nested sets of visual tokens across multiple coarse-to-fine granularities. This allows explicit control over visual granularity per instance during inference, enabling adjustment of the number of tokens based on content complexity. The study analyzes the granularity required for existing datasets and explores the trade-off between performance and visual token length at the sample level.

NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models

Why?: The research is important because it introduces the NV-Embed model with enhanced techniques for training LLMs to serve as versatile embedding models, outperforming existing models in text embedding tasks.

How?: The research introduces the NV-Embed model with a latent attention layer for pooled embeddings, removes the causal attention mask of LLMs during contrastive training, and implements a two-stage contrastive instruction-tuning method, utilizing curated hard negatives and a blend of retrieval and non-retrieval datasets for training.

Various Lengths, Constant Speed: Efficient Language Modeling with Lightning Attention

Why?: Efficient language modeling with constant speed for various sequence lengths is crucial for improving model training and deployment.

How?: The research introduces Lightning Attention, which splits attention calculation into intra-blocks and inter-blocks using different strategies. Intra-blocks use conventional attention computation, while inter-blocks utilize linear attention kernel tricks to eliminate the need for cumsum. Moreover, a tiling technique is applied during forward and backward procedures to maximize GPU hardware utilization. A new architecture called TransNormerLLM (TNL) is also introduced to enhance accuracy. Rigorous testing on diverse datasets with different model sizes and sequence lengths is conducted.

CLAQ: Pushing the Limits of Low-Bit Post-Training Quantization for LLMs

Why?: Parameter quantization for LLMs is essential to reduce memory costs and enhance computational efficiency.

How?: The research introduces a Column-Level Adaptive weight Quantization (CLAQ) framework that utilizes K-Means clustering, outlier-guided adaptive precision search strategy, and dynamic outlier reservation scheme for LLM quantization. These strategies enable dynamic generation of quantization centroids, assign varying bit-widths to different columns, and retain some parameters in their original float point precision. Experiments were conducted on various mainstream open source LLMs to validate the effectiveness of the proposed methods.

Autoformalizing Euclidean Geometry

Why?: Autoformalization is important for translating informal math into formal theorems and proofs that are machine-verifiable.

How?: The research introduces a neuro-symbolic framework that combines domain knowledge, SMT solvers, and LLMs for autoformalizing Euclidean geometry. The framework leverages theorem provers to automatically fill in diagrammatic information from informal proofs, making the formalization process easier for LLMs. Automatic semantic evaluation is also provided for autoformalized theorem statements. A benchmark called LeanEuclid is constructed with problems from Euclid's Elements and the UniGeo dataset formalized in the Lean proof assistant. Experiments with GPT-4 and GPT-4V demonstrate the capabilities and limitations of state-of-the-art LLMs in autoformalizing geometry problems.

Tokenization Matters! Degrading Large Language Models through Challenging Their Tokenization

Why?: LLMs have limitations in accurately understanding inputs and generating responses due to tokenization. Improving tokenization can enhance LLMs capabilities.

How?: The research constructs an adversarial dataset, ADT (Adversarial Dataset for Tokenizer), drawing on various LLMs vocabularies to challenge LLMs tokenization. It comprises ADT-Human and ADT-Auto subsets. The study evaluates the effectiveness of ADT on leading LLMs like GPT-4o, Llama-3, Qwen2.5-max, etc., thereby degrading their performance. Automatic data generation proves efficient and applicable to diverse LLMs, highlighting tokenization flaws.

Position: Foundation Agents as the Paradigm Shift for Decision Making

Why?: Decision making demands intricate interplay between perception, memory, and reasoning to discern optimal policies. Conventional approaches face challenges such as low sample efficiency and poor generalization. Foundation agents can offer rapid adaptation to diverse tasks, making them crucial for transforming the learning paradigm.

How?: The research proposes the construction of foundation agents by formulating their fundamental characteristics and challenges inspired by the success of large language models. The roadmap includes steps from data collection to pretraining and adaptation, aligning knowledge and values with LLMs. Critical research questions are identified, and trends for foundation agents in real-world use cases are outlined.

Exploring the LLM Journey from Cognition to Expression with Linear Representations

Why?: To investigate the evolution and interaction of cognitive and expressive capabilities in LLMs and understand the development patterns of these abilities.

How?: The research examines Baichuan-7B and Baichuan-33B, bilingual LLMs, to define and explore cognitive and expressive capabilities using linear representations in three phases: Pretraining, Supervised Fine-Tuning (SFT), and Reinforcement Learning from Human Feedback (RLHF). It involves statistical analyses to establish correlations between cognitive and expressive capabilities, assesses how cognitive capacity may restrict expressive potential, delves into the theoretical aspects of these developmental trajectories, evaluates optimization strategies like few-shot learning and repeated sampling, and investigates the connection between the hidden space and output space.

Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?

Why?: It is important to determine if large language models can effectively express their intrinsic uncertainty in natural language to improve their trustworthiness and reliability in providing answers.

How?: The research formalized faithful response uncertainty by measuring the gap between the model's confidence in its answers and the decisiveness with which they are conveyed. This metric penalizes both excessive and insufficient hedging, indicating how well the uncertainty is reflected. Various aligned LLMs were evaluated on knowledge-intensive question answering tasks to assess their ability to communicate uncertainty.

Match, Compare, or Select? An Investigation of Large Language Models for Entity Matching

Why?: Entity matching is crucial for entity resolution, and utilizing LLMs for this task has shown promise. However, existing LLM-based approaches often overlook global consistency between records. This research aims to investigate different methodologies to enhance LLM-based entity matching.

How?: The research compares three main strategies - matching, comparing, and selecting - for LLM-based entity matching. It explores how these strategies incorporate interactions among records from various perspectives. A compositional entity matching (ComEM) framework is designed, leveraging a mix of strategies and LLMs to capitalize on their respective strengths. This framework aims to improve both effectiveness and efficiency of entity matching.

Results: Experimental results showcase significant performance enhancements and cost reductions in LLM-based entity matching by using the ComEM framework.

AutoCV: Empowering Reasoning with Automated Process Labeling via Confidence Variation

Why?: The research is important as it aims to enhance the reasoning capabilities of large language models by automatically annotating the reasoning steps, which can improve the accuracy of these models in selecting the correct answers.

How?: The approach involves training a verification model on final answer correctness to generate automatic process annotations by assigning confidence scores to each reasoning step. It detects relative changes in these confidence scores to automatically annotate the reasoning process, eliminating the need for manual annotations and reducing computational costs.

Results: Substantial improvements in accuracy across five datasets in mathematics and common sense reasoning.

THREAD: Thinking Deeper with Recursive Spawning

Why?: LLMs often struggle with understanding complex and lengthy contexts. The research introduces the THREAD framework to address this challenge by allowing the model to dynamically spawn new threads based on the context, enabling adaptive problem-solving.

How?: The research conducted involves proposing the THREAD framework that treats model generation as a thread of execution. This thread can either run to completion or spawn new threads dynamically, offloading work to child threads. The model decomposes complex tasks or questions into simpler sub-problems for separate child threads to solve. THREAD is implemented using a few-shot learning approach and tested on various benchmarks for agent tasks and question answering.

Results: THREAD achieves state-of-the-art performance with GPT-4 and GPT-3.5 on different benchmarks, including ALFWorld, TextCraft, and WebShop. It also outperforms existing frameworks by 10% to 50% absolute points with smaller models like Llama-3-8b and CodeLlama-7b.

Mechanistic Interpretability of Binary and Ternary Transformers

Why?: Can binary and ternary transformer networks offer a more interpretable alternative in LLMs while maintaining efficiency.

How?: The research compares how binary and ternary transformer networks performs compared to full-precision transformer networks.

Let’s make LLMs safe!!

Cross-Modal Safety Alignment: Is textual unlearning all you need?

Why?: Integrating new modalities into LLMs increases the vulnerability to adversarial attacks, bypassing traditional safety training methods.

How?: The research investigates whether unlearning solely in the textual domain is sufficient for cross-modality safety alignment. The study evaluates this approach across six datasets to analyze its effectiveness in reducing the Attack Success Rate (ASR) for text-based and vision-text-based attacks while maintaining utility. The experiments compare unlearning in VLMs with and without a multi-modal dataset to understand its impact on safety alignment.

Exploring Backdoor Attacks against Large Language Model-based Decision Making

The research proposes a framework for Backdoor Attacks against LLM-enabled Decision-making systems (BALD) by introducing attacks during the fine-tuning phase using word injection, scenario manipulation, and knowledge injection mechanisms. These attacks target different components in the LLM-based decision-making pipeline. Extensive experiments are conducted with three popular LLMs (GPT-3.5, LLaMA2, PaLM2) on two datasets (HighwayEnv, nuScenes) to demonstrate the effectiveness and stealthiness of the proposed backdoor triggers and mechanisms.

The Uncanny Valley: Exploring Adversarial Robustness from a Flatness Perspective

Why?: Understanding the relationship between flatness of the loss surface and adversarial robustness is crucial for improving the robustness of machine learning models, including LLMs.

How?: The research empirically analyzes the relation between adversarial examples and relative flatness with respect to the parameters of one layer. It observes a peculiar property of adversarial examples during iterative first-order white-box attacks. The study is conducted across various model architectures and datasets, extending the results to large language models. The theoretical connection between relative flatness and adversarial robustness is established by bounding the third derivative of the loss surface.

Can We Trust LLMs? Mitigate Overconfidence Bias in LLMs through Knowledge Transfer

Paper introduces a knowledge transfer (KT) method using detailed reasoning paths to transfer knowledge from larger LLMs to smaller ones. This helps fine-tune smaller models for producing accurate predictions with calibrated confidence levels. Experimental evaluation includes multiple-choice questions and sentiment analysis on various datasets to compare the KT method with vanilla and question-answer pair (QA) fine-tuning methods.

Safe LoRA: the Silver Lining of Reducing Safety Risks when Fine-tuning Large Language Models

Why?: Fine-tuning LLMs requires significant hardware resources, which can be impractical for typical users. As fine-tuning can increase safety risks to LLMs, there is a need to develop methods to reduce these risks while maintaining performance.

How?: The research proposes Safe LoRA, a simple one-liner patch to the LoRA fine-tuning method. Safe LoRA introduces the projection of LoRA weights from selected layers to a safety-aligned subspace, reducing safety risks in LLM fine-tuning while maintaining utility. It is a training-free and data-free approach, requiring only the knowledge of weights from the base and aligned LLMs. Extensive experiments were conducted to demonstrate the effectiveness of Safe LoRA.

Creative ways to use LLMs!!

Video Enriched Retrieval Augmented Generation Using Aligned Video Captions - Utilize visual captions from videos to add more context in LLMs input

HEART-felt Narratives: Tracing Empathy and Narrative Style in Personal Stories with LLMs
Understand the relationship between narrative style and empathy in personal stories

How?: The research empirically examines the relationship between style and empathy using LLMs and large-scale crowdsourcing studies. It introduces the HEART taxonomy to delineate narrative style elements influencing empathy, evaluates LLMs in extracting these elements, and collects a dataset of empathy judgments via crowdsourcing.

Reason3D: Searching and Reasoning 3D Segmentation via Large Language Model - A novel LLM specifically designed for comprehensive 3D understanding.

How?: Reason3D takes point cloud data and text prompts as input to produce textual responses and segmentation masks. It uses a hierarchical mask decoder for locating small objects in expansive scenes, generating coarse estimates followed by detailed segmentation. The approach enhances the precision of object identification and segmentation.

The Peripatetic Hater: Predicting Movement Among Hate Subreddits - LLMs used to understand how users move between different hate subreddits can help in predicting radicalization and identifying the associated risks on social media platforms.

QUB-Cirdan at 'Discharge Me!': Zero shot discharge letter generation by open-source LLM - LLMs to automate the generation of critical sections of patient discharge letters in clinics.

How?: The approach utilized the Llama3 8B quantized model with a zero-shot method combined with Retrieval-Augmented Generation (RAG) to generate the 'Brief Hospital Course' and 'Discharge Instructions' sections. A curated template-based approach was developed to ensure reliability and consistency, and RAG was integrated for word count prediction. Several unsuccessful experiments were also conducted to provide insights for the competition.

LLM-Assisted Static Analysis for Detecting Security Vulnerabilities - LLMs based static analysis to identify security vulnerabilities

How?: The researchers propose IRIS, a method that integrates LLMs with static analysis to conduct whole-repository reasoning for identifying security vulnerabilities. They create a new dataset, CWE-Bench-Java, containing 120 manually validated security vulnerabilities in real-world Java projects. The projects are complex, averaging 300,000 lines of code and reaching up to 7 million. IRIS utilizes GPT-4 to detect 69 out of 120 vulnerabilities, outperforming the state-of-the-art static analysis tool that only identifies 27. Moreover, IRIS substantially decreases false alarms by over 80% in the best scenario.

LLM-Optic: Unveiling the Capabilities of Large Language Models for Universal Visual Grounding - Visual grounding links text queries with specific regions in an image. However, existing models struggle with complex queries.

How?: First, LLM-Optic utilizes LLMs as a text grounder to interpret complex queries and identify intended objects accurately. Then, a pre-trained visual grounding model generates candidate bounding boxes based on the refined query. Subsequently, LLM-Optic annotates these bounding boxes with numerical marks to link text with image regions. Finally, a Large Multimodal Model acts as a Visual Grounder to select objects that best match the original query.

A Large Language Model-based multi-agent manufacturing system for intelligent shopfloor

The research proposes a LLM-based multi-agent manufacturing system that defines different agents with specific roles like Machine Server Agent, Bid Inviter Agent, Bidder Agent, Thinking Agent, and Decision Agent. The LLMs support the Thinking Agent and Decision Agent in analyzing shopfloor conditions intelligently and choosing suitable machines rather than following predefined rules. The system involves negotiation steps among agents like BAs and BIA to finalize the distribution of orders and connect manufacturing resources, with Machine Server Agents coordinating physical shopfloor operations.

TIE: Revolutionizing Text-based Image Editing for Complex-Prompt Following and High-Fidelity Editing - Interpreting complex prompts and maintaining image consistency during editing are crucial in the image generation.

How?: The researchers design a Chain-of-Thought (CoT) process involving instruction decomposition, region localization, and detailed description. They fine-tune the LISA model, a lightweight multimodal LLM, using the CoT process and the mask of the edited image. By enhancing the diffusion models with prompt knowledge and image mask, they enable superior image generation with improved prompt understanding.

Laurel: Generating Dafny Assertions Using Large Language Models

Why?: Automating the generation of helper assertions in Dafny programs can reduce the burden on proof engineers and improve the efficiency of program verification.

How?: The tool, Laurel, uses LLMs to automatically generate helper assertions by designing two domain-specific prompting techniques. First, it helps the LLM determine the location of the missing assertion by analyzing the verifier's error message and inserting an assertion placeholder. Second, it provides example assertions from the same codebase based on a new lemma similarity metric. The techniques were evaluated on a dataset of helper assertions extracted from real-world Dafny codebases.

CHESS: Contextual Harnessing for Efficient SQL Synthesis - convert user context to SQL queries

Paper utilizes hierarchical retrieval methods, model-generated keywords, locality-sensitive hashing indexing, vector databases, and an adaptive schema pruning technique based on problem complexity and contextual size. The approach is tested on both proprietary models like GPT-4 and open-source models such as Llama-3-70B, demonstrating its effectiveness through ablation studies.

LLMs for robotics & VLLMs

Enhanced Robot Arm at the Edge with NLP and Vision Systems - LLMs and vision systems were used together to interpret and execute complex commands conveyed through natural language to improve the accessibility and adaptability of assistive robotic systems.

LARM: Large Auto-Regressive Model for Long-Horizon Embodied Intelligence - The research introduces the Large Auto-Regressive Model (LARM) that utilizes text and multi-view images to predict subsequent actions in an auto-regressive manner. It incorporates a novel data format called auto-regressive node transmission structure and a corresponding dataset. LARM is trained using a two-phase training approach to harvest enchanted equipment in Minecraft, demonstrating more complex decision-making chains than previous methods. The training also enhances the speed of LARM by 6.8 times.

MotionLLM: Multimodal Motion-Language Learning with Large Language Models - single-human, multi-human motion generation, and motion captioning through fine-tuning of pre-trained LLMs, showcasing scalability and flexibility.

How?: MotionLLM encodes and quantizes motions into discrete tokens understandable by LLMs, creating a unified vocabulary of motion and text tokens. By training LLMs with adapters using only 1-3% of their parameters, the framework achieves comparable results in single-human motion generation compared to diffusion models and transformer-based models. It can be extended to multi-human motion generation through autoregressive generation of single-human motions.

Summary of LLMs related research papers published on 25th May, 2024

Sat, 15 Jun 2024 00:59:37 +0000

Core research improving LLMs!

SLoPe: Double-Pruned Sparse Plus Lazy Low-Rank Adapter Pretraining of LLMs

Why?: The paper addresses the challenge of enhancing the accuracy of sparse LLMs while also speeding up their pretraining and inference processes and decreasing their memory footprint.

How?: The approach involves a new method termed SLoPe, which includes sparse plus lazy low-rank adapter pretraining. Key strategies are the addition of low-rank adapters in the final 1% of pretraining iterations, a double-pruned backward pass using N:M sparsity, and efficiency tests on large-scale LLMs (OPT-33B and OPT-66B).

Results: SLoPe achieved training and inference acceleration up to 1.14x and 1.34x respectively for OPT-33B and OPT-66B, alongside reduced memory usage by up to 0.77x and 0.51x for training and inference, respectively.

Mechanism Design for LLM Fine-tuning with Multiple Reward Models

Why?: This research is crucial as it develops a structured mechanism to incentivize agents to report their preferences truthfully during the fine-tuning of large language models, essential for effective and equitable model training.

How?: The paper treats the incentive issue as a multi-parameter mechanism design challenge, where training and payment rules are structured to fulfill specific objectives. It introduces an 'affine maximizer payment scheme' aligned with 'social welfare maximizing training rules', ensuring compliance with dominant-strategy incentive compatibility and individual rationality. It also shows adaptability of this scheme in real-world scenarios where reporting biases exist.

A statistical framework for weak-to-strong generalization

Why?: This research addresses the significant challenge of using less capable human feedback to train and align more capable LLMs without degrading their performance. It tackles the issue of weak-to-strong generalization, which is crucial for advancing LLM capabilities beyond current human limits.

How?: The approach conceptualizes the problem as a transfer learning challenge, aiming to transfer latent knowledge from weaker models to enhance a stronger, pre-trained model. By identifying issues with simple fine-tuning methods and proposing a refinement-based strategy, the researchers develop a method that effectively incorporates feedback while maintaining the superior capabilities of LLMs. This involves systematic adjustments using a structured refinement process rather than straightforward fine-tuning.

Results: The practical applicability of the refined approach was demonstrated through successful alignment tasks on three LLMs, showing that it can maintain the advanced capabilities of LLMs while effectively utilizing human-level feedback.

Accelerating Inference of Retrieval-Augmented Generation via Sparse Context Selection

Why?: To enhance the efficiency and response quality of LLMs augmented with retrieval capabilities by reducing latency and computational overhead caused by processing large volumes of retrieved documents.

How?: The proposed Sparse RAG model introduces a mechanism for parallel encoding of retrieved documents to eliminate the latency generated by long-range attention mechanisms. It further employs a selection strategy where LLMs use special control tokens to identify and focus only on the most relevant documents during the auto-regressive generation phase. This not only curtails the amount of data processed but also helps in generating more relevant and focused responses.

Results: Evaluation on two datasets demonstrated that Sparse RAG achieves an optimal balance between generation quality and computational efficiency, suggesting its effectiveness and generalizability across different types of generation tasks.

Prompt Optimization with EASE? Efficient Ordering-aware Automated Selection of Exemplars

Why?: The research addresses the need for effective prompt optimization in LLMs, focusing on the automated, efficient selection and ordering of exemplars to enhance in-context learning without the need for fine-tuning or increasing computational demands at test-time.

How?: The study introduces a novel method named EASE that uses hidden embeddings from a pre-trained language model to represent ordered sets of exemplars. EASE employs a neural bandit algorithm to dynamically optimize the selection and sequencing of exemplars based on their potential impact on task performance. It considers the often overlooked influence of exemplar order within the prompt and enables efficient selection of optimal exemplar sets for all test instances of a task, thus eliminating extra computational needs during test-time. Additionally, the method is designed to concurrently optimize both the exemplars and the instructions in the prompt.

Results: Through extensive empirical evaluations (including on novel tasks), EASE demonstrated superior performance over existing methods, providing insights into the impact of exemplar selection on in-context learning effectiveness.

Keypoint-based Progressive Chain-of-Thought Distillation for LLMs

Why?: The research addresses the inadequacies in traditional chain-of-thought distillation methods for LLMs, particularly in accurately mimicking critical reasoning steps and managing the learning sequence which typically does not align with human cognitive patterns.

How?: The study introduces a novel framework named KPOD which focuses on two main enhancements. First, a token weighting module is developed, utilizing mask learning to prioritize and accurately replicate keystone tokens during distillation. This component helps overcome the challenge where tokens of varying importance are treated equally, which can lead to errors in logical reasoning in the student models. Second, the research proposes an 'in-rationale progressive distillation' strategy. This strategy involves initially training the smaller, student model to generate the simpler, final steps of a rationale. As training progresses, the scope expands to gradually include the entire set of reasoning steps. This mimics cognitive development by starting with easier tasks before advancing to more complex problems. To effectively implement this strategy, a weighted token generation loss is introduced to gauge the difficulty of each step in the rationale. Additionally, a value function is created to dynamically schedule the distillation process, taking into account both the difficulty of reasoning steps and the diversity of questions. The approach, unlike previous methods, aligns more closely with natural learning progressions and is designed to enhance the transfer of reasoning abilities more effectively.

Results: Extensive experiments on four reasoning benchmarks illustrate that KPOD outperforms previous methods by a large margin.

SPP: Sparsity-Preserved Parameter-Efficient Fine-Tuning for Large Language Models

Why?: This research is important because it tackles the challenge of efficiently fine-tuning and deploying large language models (LLMs) that have been reduced in size through pruning without sacrificing their performance.

How?: The paper introduces SPP, a Sparsity-Preserved Parameter-efficient fine-tuning method that uses lightweight learnable column and row matrices. This method optimizes sparse LLM weights by maintaining the structural and sparsity integrity of pruned models through element-wise multiplication and residual addition. This allows the models to retain their sparsity patterns and ratios throughout training and merging processes. The effectiveness of SPP was demonstrated on the LLaMA and LLaMA-2 model families using various sparsity patterns (unstructured and N:M), particularly focusing on high sparsity ratios like 75%.

Results: The application of SPP significantly improved the models' performance across different sparsity patterns, especially in models with higher sparsity ratios, proving it as a viable solution for efficient fine-tuning of sparse LLMs.

LLMs evaluations

Comparative Analysis of Open-Source Language Models in Summarizing Medical Text Data

Why?: The study addresses a significant gap by methodically evaluating the performance of LLMs specifically for the summarization of medical text data, an area that is under-researched but critical for enhancing digital health applications.

How?: The research involves the use of open-source LLMs, such as Llama2 and Mistral, to perform summarization tasks on medical text data. GPT-4 is utilized as an assessor to evaluate the summaries generated by these models. The approach includes both quantitative metrics and qualitative assessments by subject-matter experts to ensure a thorough evaluation of performance.

Survey papers

Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models

Let’s make LLMs safe!!

Devil's Advocate: Anticipatory Reflection for LLM Agents

Why?: Improves the reliability and efficiency of LLMs

How?: The method incorporates three phases of introspective interventions:

Anticipatory reflection to assess potential failures and solutions before actions,
Post-action alignment to evaluate and adjust ongoing actions according to task objectives, and
A comprehensive review post-completion to refine future strategies.
This innovative approach was implemented in a zero-shot setup on a platform called WebArena, dedicated to simulating practical tasks in web environments.

Results: The experiments demonstrated that this introspective-driven methodology significantly enhances the performance of LLMs by reducing the number of trials and necessary revisions, enabling more effective navigation through unexpected challenges and improving the overall efficiency in completing tasks compared to existing zero-shot methods.

ConStat: Performance-Based Contamination Detection in Large Language Models

Why?: The paper addresses the critical issue of data contamination in benchmarking LLMs, which can lead to misleading performance metrics and unreliable model comparisons.

How?: Introducing a novel definition of contamination, the study defines it as performance that artificially inflates and does not generalize across different scenarios like rephrased samples, synthetic samples, or varied benchmarks. They developed 'ConStat', a statistical method that detects and quantifies contamination by assessing performance discrepancies between a primary and a reference benchmark relative to a set of reference models.

Results: ConStat successfully detected significant levels of contamination in several well-known models, thereby demonstrating its effectiveness in identifying and quantifying performance inflation in LLM benchmarks.

FastQuery: Communication-efficient Embedding Table Query for Private LLM Inference

Why?: The research addresses the privacy issues associated with user queries in LLMs by focusing on the computational and communication inefficiencies of using homomorphic encryption for private embedding table queries.

How?: The researchers developed FastQuery, a framework designed to optimize private embedding table queries for LLMs. FastQuery employs a communication-aware embedding table quantization algorithm and a one-hot-aware dense packing algorithm to reduce both the computation and communication costs.

Results: FastQuery achieves more than 4.3 times, 2.7 times, 1.3 times latency reduction, and more than 75.7 times, 60.2 times, 20.2 times communication reduction compared to existing HE-based frameworks on both LLAMA-7B and LLAMA-30B models.

No Two Devils Alike: Unveiling Distinct Mechanisms of Fine-tuning Attacks

Why?: Understanding the mechanisms of attacks on LLMs safety alignment is crucial to improve defenses against malicious manipulations.

How?: The research breaks down the safeguarding process into three stages: recognizing harmful instructions, initiating a refusing response, and completing the refusal. Techniques such as logit lens and activation patching are employed to analyze the influence of different attack strategies on these stages. The 'logit cohort' technique is aimed at observing the prediction confidence of a model on various outputs, while 'activation cohort' involves modifying certain neurons to observe changes in behavior. Moreover, cross-model probing is utilized to examine shifts in model representations before and after attacks. Two primary attack types were investigated: Explicit Harmful Attack (EHA) and Identity-Shifting Attack (ISA). Analysis of how these attacks affect each safeguarding stage provided insights into the divergent ways these attacks compromise model safety.

Uncovering LLM-Generated Code: A Zero-Shot Synthetic Code Detector via Code Rewriting

Why?: Detection of code generated by LLMs

How?: The research introduces a novel approach to detect LLM-generated code using a zero-shot mechanism based on code rewriting. Their hypothesis suggests that synthetic code differs less from its rewritten variants compared to human-written code. The authors developed a self-supervised contrastive learning model to train on code samples and their rewritten forms. The contrastive learning framework essentially learns to distinguish whether two code snippets are similar (original and rewritten versions of a human-written code) or dissimilar (original and rewritten versions of synthetic code). To generate rewritten code versions, the researchers utilized another LLM trained specifically to alter code while maintaining functional equivalence. The assessment of the model's efficacy was conducted using two synthetic code detection benchmarks: the APPS benchmark and the MBPP benchmark.

Creative ways to use LLMs!!

LLMs for User Interest Exploration in Large-scale Recommendation Systems

Proposed system integrates LLMs with traditional recommendation techniques using a hierarchical approach. At the high level, 'interest clusters' are defined with controllable granularity. These clusters use language descriptions to encapsulate various user interests. A fine-tuned LLM interprets these clusters to generate descriptions of potentially novel interests, which then guide the recommendation process at the lower level. Here, traditional machine learning models, like a transformer-based recommender system, are constrained to suggest only items that align with the newly identified interest clusters.

M³GPT: An Advanced Multimodal, Multitask Framework for Motion Comprehension and Generation

Why?: Generating human motions

How?: The M³ GPT framework integrates multiple modalities—text, music, motion/dance—using a novel approach called discrete vector quantization. This technique allows the model to handle different input types within a single vocabulary, facilitating smoother and more efficient processing. The model generates motions directly in the raw motion space rather than using a discrete tokenizer, which helps in retaining more information and generating more detailed motion dynamics. A key aspect of the framework is its multitask capacity, where text is used as a connecting thread between different motion tasks. This not only simplifies the modeling process but also enhances the model’s learning efficiency through cross-modal reinforcement. The framework was trained and tested on several datasets, involving complex tasks such as zero-shot motion generation, to demonstrate its robustness and versatility.

GeneAgent: Self-verification Language Agent for Gene Set Knowledge Discovery using Domain Databases

GeneAgent, a novel LLM designed to reduce the hallucinations in gene set knowledge discovery. GeneAgent incorporates a self-verification mechanism allowing it to autonomously interact with various biological databases to verify and incorporate relevant data into its outputs.

C3LLM: Conditional Multimodal Content Generation Using Large Language Models

C3LLM integrates three different tasks (video-to-audio, audio-to-text, text-to-audio) with LLMs.

Finetuning Large Language Model for Personalized Ranking

Why?: Recommendation system

How?: The research introduced Direct Multi-Preference Optimization (DMPO), a method designed to make LLMs more suitable for recommendation tasks by addressing the differences in data nature between LLM pre-training and recommendation demands. DMPO functions by optimizing LLMs to increase the likelihood of selecting relevant (positive) items while reducing the chances of picking irrelevant (negative) ones. The researchers fine-tuned an existing LLM using DMPO by simultaneously maximizing the probability of positive sample outcomes and minimizing the probabilities of various negative samples.

COLT: Towards Completeness-Oriented Tool Retrieval for Large Language Models

Why?: Integrating external tools with LLMs

How?: The COLT model involves a two-stage process: initial PLM-based models are fine-tuned for semantic understanding between user queries and tools, followed by a dual-view graph collaborative learning framework that constructs bipartite graphs linking queries, scenes, and tools for capturing diverse tool relationships and reducing redundancy.

Semantic Importance-Aware Communications with Semantic Correction Using Large Language Models

Why?: Improving semantic communications between agents and humans or agents and agents by enhancing understanding and minimizing semantic loss in transmitting visual data.

How?: The paper introduces a method for understanding-level semantic communications (ULSC), employing an image caption neural network (ICNN) to convert visual data into natural language descriptions, which are then refined using a pre-trained large language model (LLM) for semantic importance quantification and error correction. It involves adaptive strategies for minimizing semantic loss while managing transmission delays and also uses LLM at the receiving end for further error corrections.

Streaming Long Video Understanding with Large Language Models

Why?: Efficient and precise long video understanding is crucial for enhancing multimedia content accessibility and interaction but is challenged by high computational costs and data redundancy.

How?: The proposed VideoStreaming system addresses long video understanding through two main techniques: Memory-Propagated Streaming Encoding and Adaptive Memory Selection. Initially, long videos are segmented into manageable short clips. Each clip is then sequentially encoded while integrating a propagated memory from previously processed clips, which helps in retaining historical content relevance and reducing redundancy. This encoding process continues clip-by-clip, cumulatively updating the memory that captures essential video content elements. After encoding, the Adaptive Memory Selection mechanism comes into play, which dynamically selects a fixed number of relevant memories based on the specific questions posed about the video. This allows the system to focus only on pertinent information, thereby optimizing processing time and memory usage while ensuring that the answers generated by the LLM are contextually appropriate and detailed. This disentangled approach of video extraction and memory utilization enables the LLM to generate responses without reprocessing the entire video, allowing for efficient querying and reduced computational demands.

LLMs related research papers published on 23rd May, 2024

Sat, 25 May 2024 15:00:00 +0000

Core research improving LLMs!

Revisiting MoE and Dense Speed-Accuracy Comparisons for LLM Training - project page

💡Why?: The research paper addresses the problem of accurately measuring and comparing the performance of Mixture-of-Experts (MoE) models and dense models. Prior work has used FLOPs or activated parameters as a measure of model complexity, but this does not take into account the communication overhead in sparse layers, leading to a larger actual training budget for MoE. This setting favors MoE and does not give an accurate representation of its performance compared to dense models.

💻How?: To solve this problem, the research paper proposes to use step time as a more accurate measure of model complexity, and to determine the total compute budget under the Chinchilla compute-optimal settings. This ensures a fair comparison between MoE and dense models. Additionally, the paper introduces a 3D sharding method for running MoE efficiently on modern accelerators, keeping the dense-to-MoE step time increase within a healthy range.

OAC: Output-adaptive Calibration for Accurate Post-training Quantization

The research paper proposes a solution to reduce the computational cost of LLMs by introducing a technique called Output-adaptive Calibration (OAC), which incorporates the model output in the calibration process. This is done by formulating the quantization error based on the distortion of the output cross-entropy loss. OAC also utilizes output-adaptive Hessians, which are approximated for each layer under reasonable assumptions to reduce the computational complexity. These Hessians are used to update the weight matrices and detect the most salient weights, in order to maintain the model output. This approach outperforms existing methods such as SpQR and BiLLM, especially at extreme low-precision quantization (2-bit and binary).

📊Results: The research paper achieved significant performance improvements compared to state-of-the-art baselines such as SpQR and BiLLM, particularly at extreme low-precision quantization. This demonstrates the effectiveness of the proposed OAC technique in reducing the memory footprint, latency, and energy required for inference, without sacrificing accuracy.

RE-Adapt: Reverse Engineered Adaptation of Large Language Models

💡Why?: The research paper addresses the problem of fine-tuning LLMs on new domains without degrading any pre-existing instruction-tuning. This is important because fine-tuning is often necessary to adapt the model to a specific domain, but it can also lead to overfitting and degrade the performance on the original task.

💻How?: The research paper proposes a solution called RE-Adapt, which stands for Reverse-Engineering Adaptation. This approach involves reverse engineering an adapter, which isolates the knowledge that an instruction-tuned model has learned beyond its corresponding pretrained base model. This adapter is then used to fine-tune the base model on a new domain and readapt it to instruction following. This process requires no additional data or training, making it efficient and effective.

📊Results: The research paper demonstrates that RE-Adapt and a low-rank variant called LoRE-Adapt outperform other methods of fine-tuning on multiple popular LLMs and datasets. This improvement is seen even when the models are used in conjunction with RAG, which is a more challenging task. This shows the effectiveness of the proposed approach in maintaining the performance of the model on the original task while improving its performance on

Linking In-context Learning in Transformers to Human Episodic Memory

The research paper examines the relationship between attention heads in transformer models and human episodic memory. The researchers focus on induction heads, which contribute to in-context learning, and compare them to the contextual maintenance and retrieval (CMR) model of human episodic memory. By analyzing pre-trained LLMs, the researchers demonstrate that CMR-like heads often emerge in intermediate model layers and exhibit similar behavioural and functional characteristics to human memory biases.

Mallows-DPO: Fine-Tune Your LLM with Preference Dispersions

💡Why?: The research paper addresses the problem of improving reinforcement learning with human feedback (RLHF) through Direct Preference Optimization (DPO) and its limitations in characterizing the diversity of human preferences.

💻How?: The research paper proposes a new approach called Mallows-DPO, inspired by Mallows' theory of preference ranking. This approach introduces a dispersion index to capture the diversity of human preferences towards prompts. It works by incorporating this dispersion index into existing DPO models, thus enhancing their performance in various benchmark tasks such as synthetic bandit selection, controllable generations, and dialogues.

PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression

💡Why?: LLMs compression technique

💻How?: The research paper proposes a representation-agnostic framework called PV-Tuning, which improves upon existing fine-tuning strategies and provides convergence guarantees in restricted cases. It works by fine-tuning the compressed parameters over a limited amount of calibration data, but instead of using straight-through estimators (STE) which may not perform well in this setting, PV-Tuning uses a more optimized approach.

📊Results: The research paper achieved better performance in terms of accuracy-vs-bit-width trade-off compared to existing techniques for highly-performant language models such as Llama and Mistral. Specifically, using PV-Tuning, the research paper achieved the first Pareto-optimal quantization for Llama 2 family models at 2 bits per parameter.

HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models - GitHub

💡Why?: The research paper addresses the problem of efficiently and effectively integrating a large amount of new experiences into LLMs after pre-training.

💻How?: The research paper proposes a retrieval framework called HippoRAG, inspired by the hippocampal indexing theory of human long-term memory. It works by synergistically orchestrating LLMs, knowledge graphs, and the Personalized PageRank algorithm to mimic the different roles of the neocortex and hippocampus in human memory. This allows for deeper and more efficient knowledge integration over new experiences.

📊Results: The research paper shows that HippoRAG outperforms existing retrieval-augmented generation (RAG) methods on multi-hop question answering tasks, achieving up to a 20% improvement. It also performs comparably or better than iterative retrieval methods while being significantly cheaper and faster. Additionally, integrating HippoRAG into existing methods leads to further substantial gains.

WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models - project page 🔥🔥🔥

💡Why?: The research paper addresses the problem of updating knowledge in LLMs to ensure accurate responses and facilitate lifelong model editing. Specifically, the paper looks at the challenge of where to store and access this updated knowledge, as well as how to maintain reliability, generalization, and locality in the editing process.

💻How?: The research paper proposes a solution called WISE (Where to Insert updated knowledge for lifelong model Editing), which involves a dual parametric memory scheme. This includes a main memory for pretrained knowledge and a side memory for edited knowledge. The paper also suggests using a router to decide which memory to access for a given query. Additionally, the paper introduces a knowledge-sharding mechanism to handle continual editing, where different sets of edits are stored in separate subspaces of parameters and then merged into a shared memory without conflicts.

📊Results: The research paper reports significant performance improvements over previous model editing methods, as demonstrated through extensive experiments on various LLM architectures such as GPT, LLaMA, and Mistral.

SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models

💡Why?: Existing post-training quantization (PTQ) methods are not ideal for accurately quantizing LLMs to low bit-widths (below 4 bits), which leads to a decrease in performance and requires substantial computation and memory resources.

💻How?: The research paper proposes a Salience-Driven Mixed-Precision Quantization scheme, called SliM-LLM which utilizes the salience distribution of weights to determine the optimal bit-width and quantizers for accurate LLM quantization. It also aligns the bit-width partition to groups for compact memory usage and fast integer inference. The proposed SliM-LLM mainly relies on two techniques: (1) Salience-Determined Bit Allocation, which uses the clustering characteristics of salience distribution to allocate bit-widths for each group, and (2) Salience-Weighted Quantizer Calibration, which optimizes the quantizer parameters by considering the element-wise salience within the group. This helps balance the maintenance of salient information and minimization of errors.

Implicit In-context Learning 🔥
GitHub

💡Why?: In-context Learning (ICL) is a powerful approach that allows LLMs to adapt to new tasks during inference by providing demonstration examples. However, it incurs significant computational and memory costs and is susceptible to the selection and order of demonstration examples.

💻How?: The research paper proposes a new paradigm called Implicit In-context Learning (I2CL) to address the limitations of traditional ICL. I2CL absorbs the demonstration examples within the activation space instead of prefixing them to the test queries. It first generates a condensed vector representation, called a context vector, from the demonstration examples. Then, during inference, it integrates the context vector by injecting a linear combination of the context vector and query activations into the model's residual streams. This allows I2CL to achieve few-shot performance with zero-shot cost and also exhibits robustness against the variation of demonstration examples. Additionally, I2CL facilitates a novel representation of "task-ids", which enhances task similarity detection and enables effective transfer learning.

Multi-turn Reinforcement Learning from Preference Human Feedback

This research paper proposes novel methods for Reinforcement Learning (RL) from preference feedback between two full multi-turn conversations. In the tabular setting, it presents a mirror-descent-based policy optimization algorithm that is specifically designed for the general multi-turn preference-based RL problem. It works by continuously updating the policy based on the feedback received from the human, and it has been proven to converge to Nash equilibrium.

📊Results: The research paper shows that a deep RL variant of their algorithm outperforms existing RLHF baselines in their newly created environment, Education Dialogue, where a teacher agent guides a student in learning a random topic. This indicates a significant performance improvement in aligning LLMs with human preferences. Additionally, the paper also demonstrates that their algorithm can achieve the same performance as a reward-based RL baseline in an environment with explicit rewards, despite solely relying on a weaker preference signal.

PerLLM: Personalized Inference Scheduling with Edge-Cloud Collaboration for Diverse LLM Services

💡Why?: The research paper addresses the issue of processing LLMs services in real-time on bandwidth-constrained cloud servers, which has become difficult due to the rapid growth in the number of LLM users.

💻How?: The research paper proposes PerLLM, a personalized inference scheduling framework with edge-cloud collaboration to improve the efficiency of processing diverse LLM services. It integrates the upper confidence bound algorithm based on the constraint satisfaction mechanism to optimize service scheduling and resource allocation within the edge-cloud infrastructure. This allows for meeting processing time requirements while minimizing energy costs.

📊Results: PerLLM achieved significant performance improvements compared to other methods. It achieved 2.2x, 2.1x, and 1.6x higher throughput and reduced energy costs by more than 50% in experimental results from different model deployments.

Base of RoPE Bounds Context Length

💡Why?: Long context capability in LLMs.

💻How?: The research paper proposes a novel property of long-term decay in order to better understand the role of Rotary position embedding (RoPE) in LLMs. It suggests that there is an absolute lower bound for the base value of RoPE in order to obtain certain context length capability in LLMs. This can help shed light on future long context training and improve the overall performance of LLMs.

Top-Down Partitioning for Efficient List-Wise Ranking

💡Why?: Paper try to overcomes the limited document ranking capacity in LLMs, which hinders their application for list-wise re-ranking.

💻How?: The paper proposes a novel algorithm that partitions the ranking to a specific depth (k) and processes documents in a top-down manner. This algorithm is inherently parallelizable, as it uses a pivot element that allows for concurrent comparison of documents at any depth. This reduces the number of expected inference calls by 33% when ranking at depth 100, while maintaining the performance of previous approaches.

LoRA-Ensemble: Efficient Uncertainty Modelling for Self-attention Networks

The paper proposes a parameter-efficient deep ensemble method called LoRA-Ensemble, which is based on Low-Rank Adaptation (LoRA). It works by training a single pre-trained self-attention network with shared weights for all ensemble members, while also training member-specific low-rank matrices for attention projections. This approach reduces the computational cost and memory requirements of traditional explicit ensemble methods.

Mitigating Quantization Errors Due to Activation Spikes in GLU-Based LLMs - GitHub

💡Why?: The research paper addresses the challenge of activation quantization in GLU variants, which are commonly used in feed-forward networks of modern large language models (LLMs). These activation quantization errors, caused by excessive magnitudes of activation, significantly degrade the performance of quantized LLMs.

💻How?: The research paper proposes two empirical methods, Quantization-free Module (QFeM) and Quantization-free Prefix (QFeP), to isolate the activation spikes during quantization. These methods work by identifying and removing the activation spikes, which are dedicated to a couple of tokens rather than being shared across a sequence. This helps to reduce the local quantization errors and improve the performance of the quantized LLMs.

📊Results: The research paper has achieved significant performance improvements in the activation quantization of various modern LLMs with GLU variants, including LLaMA-2/3, Mistral, Mixtral, SOLAR, and Gemma. These methods have shown to be more effective than current techniques like SmoothQuant, which fail to control the activation spikes. The code for these methods is available on GitHub.

EdgeShard: Efficient LLM Inference via Collaborative Edge Computing

Paper proposes to use collaborative edge computing, where the LLM model is partitioned into shards and deployed on distributed devices. This allows for more efficient and secure LLM inference by bringing the computation closer to the data sources. An adaptive joint device selection and model partition problem is formulated, and an efficient dynamic programming algorithm is designed to optimize the inference latency and throughput.

📊Results: EdgeShard achieves up to 50% latency reduction and 2x throughput improvement over baseline methods.

MiniCache: KV Cache Compression in Depth Dimension for Large Language Models 🔥

💡Why?: The research paper tries to reduce latency in autoregressive generation by compressing the Key-Value (KV) cache, which stores previously generated tokens.

💻How?: The research paper proposes a solution called MiniCache, which compresses the KV cache across layers from a depth perspective. This is done by disentangling the states into magnitude and direction components, and then interpolating the directions while preserving their lengths. Additionally, a token retention strategy is introduced to keep highly distinct state pairs unmerged. This approach is training-free and can be applied to existing KV cache compression strategies such as quantization and sparsity.

Let’s make LLMs safe!!

This research paper Representation noising effectively prevents harmful fine-tuning on LLMs tries to mitigate the dual-use risk in releasing open-source LLMs, where bad actors can easily fine-tune these models for harmful purposes. It proposes a defence mechanism called Representation Noising (RepNoise) which removes information about harmful representations in the LLM, making it difficult for attackers to recover them during fine-tuning. This defence is effective even when attackers have access to the weights and the defender no longer has any control. RepNoise also has the ability to generalize across different subsets of harm, without degrading the overall capability of the LLM or hindering its ability to train on harmless tasks.

The research paper Impact of Non-Standard Unicode Characters on Security and Comprehension in Large Language Models analyze fifteen distinct LLMs to expose their vulnerabilities. This analysis involves a standardized test with 38 queries and three key metrics: jailbreaks, hallucinations, and comprehension errors. The models are then assessed based on the total occurrences of these metrics. The paper also investigates the impact of non-standard Unicode characters on language models and their safeguarding mechanisms, such as Reinforcement Learning Human Feedback (RLHF). The study suggests that incorporating non-standard Unicode text in training data can enhance the capabilities of language models.

The research paper MoGU: A Framework for Enhancing Safety of Open-Sourced LLMs While Preserving Their Usability proposes the MoGU framework as a solution to enhance the safety of LLMs while preserving their usability. The framework transforms the base LLM into two variants: the usable LLM and the safe LLM, and uses dynamic routing to balance their contribution. This means that when faced with malicious instructions, the router will assign a higher weight to the safe LLM to ensure harmless responses, while for benign instructions, the router prioritizes the usable LLM to generate helpful responses.

Creative ways to use LLMs!!

Research paper In-context Time Series Predictor proposes a new method that reformulates TSF tasks as input tokens, by constructing a series of (lookback, future) pairs within the tokens. This aligns more closely with the in-context mechanisms of LLMs and is more parameter-efficient, without the need for pre-trained LLM parameters.

This method also addresses issues such as overfitting in existing Transformer-based TSF models.

AnalogCoder: Analog Circuit Design via Training-Free Code Generation presents AnalogCoder which is a training-free LLM agent for designing analog circuits through Python code generation. It works by incorporating a feedback-enhanced flow with tailored domain-specific prompts, which allows for automated and self-correcting design of analog circuits with a high success rate.

Additionally, AnalogCoder also proposes a circuit tool library to archive successful designs as reusable modular sub-circuits, making it easier to create composite circuits.

FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Models is an open source platform that supports multiple financially specialized AI agents powered by LLMs.

FinRobot consists of four layers: Financial AI Agents, Financial LLM Algorithms, LLMOps and DataOps, and Multi-source LLM Foundation Models. These layers work together to break down complex financial problems, configure appropriate model application strategies, produce accurate models, and integrate various LLMs for direct access. FinRobot aims to democratize access to specialized LLM-based toolchains and promote wider adoption of AI in financial analysis.

Large language models can be zero-shot anomaly detectors for time series? uses LLMs for time series anomaly detection. Paper proposes a framework called sigllm for time series anomaly detection. It includes a time-series-to-text conversion module and end-to-end pipelines that prompt language models to perform the detection task. The framework leverages the flexibility of LLMs to handle time series data and the ability to identify anomalies in the input sequence. This is achieved by using two paradigms - a prompt-based detection method that directly asks the LLM to indicate anomalies in the input, and a forecasting method that uses the LLMs forecasting capability to guide the anomaly detection process.

AGILE: A Novel Framework of LLM Agents proposes a new framework called AGILE (AGent that Interacts and Learns from Environments) that utilizes LLMs, memory, tools, and interactions with experts to enable agents to perform complex conversational tasks. The key idea is to formulate the construction of such an LLM agent as a reinforcement learning problem, where the LLM serves as the policy model. This means that the agent learns and improves its performance through trial and error, using labeled data of actions and the PPO algorithm for fine-tuning. Additionally, the agent is equipped with the ability to reflect, utilize tools, and consult with experts, allowing it to continuously improve its performance and handle challenging questions.

MultiCast: Zero-Shot Multivariate Time Series Forecasting Using LLMs tries to predict multivariate time series using a proposed solution called MultiCast, which is a zero-shot approach that uses large language models (LLMs) to handle multivariate time series data. This is achieved through three novel token multiplexing solutions that reduce dimensionality while retaining key repetitive patterns. Additionally, a quantization scheme is introduced to allow LLMs to better learn these patterns while minimizing the number of tokens used.

LLMs for robotics & VLLMs

Towards Efficient LLM Grounding for Embodied Multi-Agent Collaboration - project page

💡Why?: The paper addresses the challenge of grounding the reasoning ability of LLMs for embodied tasks, specifically in the context of multi-agent collaboration. This problem is caused by the complexity of the physical world and the need for effective coordination between agents.

💻How?: The paper proposes a framework called Reinforced Advantage feedback (ReAd) to address this problem. This framework involves using critic regression to learn a sequential advantage function from LLM-planned data, and then treating the LLM planner as an optimizer to generate actions that maximize the advantage function. This allows the LLM to have the foresight to determine whether an action will contribute to accomplishing the final task. The paper provides theoretical analysis and extends advantage-weighted regression in reinforcement learning to multi-agent systems.

Agentic Skill Discovery propose a novel skill discovery approach for robots using LLMs.

The framework generates task proposals based on the scene description and robot's configurations and then uses reinforcement learning to develop corresponding policies. The reliability and trustworthiness of learned behaviors are ensured by an independent vision-language model. This approach allows for incremental skill acquisition, starting from zero skills, and leads to the emergence and expansion of a diverse and reliable skill library for the robot.

CityGPT: Towards Urban IoT Learning, Analysis and Interaction with Multi-Agent System proposes a framework called CityGPT, which utilizes three agents to facilitate the learning and analysis of IoT time series data.

The requirement agent allows users to input natural language queries, which are then decomposed into temporal and spatial analysis processes. These processes are completed by corresponding data analysis agents, and the spatiotemporal fusion agent visualizes the results and provides textual descriptions based on user demands. The framework is agentized and facilitated by a large language model (LLM) to increase data comprehensibility.

Calibrated Self-Rewarding Vision Language Models

💡Why?: The research paper addresses the issue of hallucination in Large Vision-Language Models (LVLMs). This refers to the phenomenon where the generated text responses from the model appear linguistically plausible but contradict the input image, indicating a misalignment between image and text pairs.

💻How?: The research paper proposes the Calibrated Self-Rewarding (CSR) approach to address this issue. This approach enables the model to self-improve by iteratively generating candidate responses, evaluating the reward for each response, and curating preference data for fine-tuning. The reward modeling incorporates visual constraints into the self-rewarding process to place greater emphasis on visual input, thus reducing hallucinations.

📊Results: The research paper achieved a substantial improvement of 7.62% over existing methods across ten benchmarks and tasks.

LLMs evaluations

This research paper Dissociation of Faithful and Unfaithful Reasoning in LLMs tries to benchmark Chain of Thought output. It analyzes LLM error recovery behaviours. Through this analysis, the researchers identified factors that influence LLM recovery behavior, such as the difficulty of the error and the amount of evidence for the correct answer in the context. The paper also distinguishes between faithful and unfaithful error recoveries, suggesting that there are distinct mechanisms driving each type of recovery.

This research paper A Nurse is Blue and Elephant is Rugby: Cross Domain Alignment in Large Language Models Reveal Human-like Patterns tries to evaluate the conceptualization and reasoning abilities of LLMs by adapting a cross-domain alignment task from cognitive science. To do that, paper conducts a behavioural study in which several LLMs are prompted with a cross-domain mapping task and their responses are analyzed and compared at both the population and individual levels. This task is designed to investigate how well the models represent abstract and concrete concepts and how they reason about these concepts through their mappings between categories.

This research paper Can LLMs Solve longer Math Word Problems Better? evaluates the ability of LLMs to solve long and complex Math Word Problems (MWPs), which is crucial for their applications in real-world scenarios. Paper published an extended grade-school math (E-GSM), a collection of MWPs with lengthy narratives, and two novel metrics to assess the efficacy and resilience of LLMs in solving these problems. For proprietary LLMs, a new instructional prompt is proposed to mitigate the influence of long context, while for open-source LLMs, a new data augmentation task is developed to improve their Context Length Generalizability (CoLeG).

This research paper Subtle Biases Need Subtler Measures: Dual Metrics for Evaluating Representative and Affinity Bias in Large Language Models evaluates the impact of bias in LLMs. Paper introduces two novel metrics - the Representative Bias Score (RBS) and the Affinity Bias Score (ABS) - to measure the biases within LLMs. These metrics are then applied in the Creativity-Oriented Generation Suite (CoGS), a collection of open-ended tasks designed to detect these biases. The CoGS uses customized rubrics to analyze the outputs of LLMs and identify any biases present.

Survey papers

This survey paper A Comprehensive Overview of Large Language Models (LLMs) for Cyber Defences: Opportunities and Directions discusses several approaches to identify anomalies of cyber threats, enhance incident response, and automate routine security operations. This is achieved by training LLMs on massive textual datasets, which allows them to encode context and provide powerful comprehension for downstream tasks.

LLMs Research

LLM Research Highlights: March 1-15, 2025 [ Part 2/2 ]

🔑 takeaway from today’s newsletter

Reasoning and context length improvement of large language models

Reasoning

Chain of Draft: Thinking Faster by Writing Less

LADDER: Self-Improving LLMs Through Recursive Problem Decomposition

LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL

Context length improvement

Explore Past Highlights

LLM Research Highlights: March 1-15, 2025

🔑 takeaway from today’s newsletter

LLM Research Highlights: March 1-15, 2025

Core research

Instruction Tuning

Cache management

Quantization

RSQ: Learning from Important Tokens Leads to Better Quantized LLMs

Unlearning

Breakthrough papers improving LLMs performance

Highlights

Papers imprving performance of LLMs published between February 16th-28th 2025

Survey paper

Optimize global IT operations with our World at Work Guide

LLMs architecture improvement

Creative way of using LLMs

Highlights

(Feb 15-28 2025) Innovative applications using LLMs

Creative ways to use LLMs!!

The AI That Called 📞 2,739 Strangers (And Got Them Talking)

Stanford's code-editing model analyzed 1.4M student submissions

Optimize global IT operations with our World at Work Guide

ArtInsight: Making Children's Art Accessible to All

Engineering Diagrams That Speak Your Language

BayesGenie: Precision Image Editing Through Language

Discovering Hidden Data in Vast Data Lakes

A detailed survey paper on LLMs in mobile apps

Research papers improving performance of LLMs [3/3]

What’s in it today?

Papers improving performance of LLMs published in 2025

Scaling Embedding Layers in Language Models

Sponsored: Special offer for you claim $50 free credits!

The Differences Between Direct Alignment Algorithms are a Blur

Ignore the KL Penalty! Boosting Exploration on Critical Tokens to Enhance RL Fine-Tuning

LIMO: Less is More for Reasoning

Research papers improving performance of LLMs [2/3]

What’s in it today?

[2] Jan 16th-Feb 15th 2025

ReLearn: Unlearning via Learning for Large Language Models

Sponsored: Special offer for you claim $50 free credits!

Better Embeddings with Coupled Adam

TransMLA: Multi-head Latent Attention Is All You Need

LASP-2: Rethinking Sequence Parallelism for Linear Attention and Its Hybrid

Research papers improving performance of LLMs [1/3]

Highlights

[1] Jan 16th-Feb 15th 2025

DarwinLM: Evolutionary Structured Pruning of Large Language Models

Sponsored: Special offer for you claim $50 free credits!

InfiniteHiP: Extending Language Model Context Up to 3 Million Tokens on a Single GPU

Explaining Context Length Scaling and Bounds for Language Models

LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens

DeepSeek-R1 Special Edition

📑 IN THIS ISSUE

DeepSeek-R1 Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

Introduction

Market reaction

Technical Analysis

Stage 1. Supervised Fine-Tuning (SFT)

Stage 2: Reward Model Development

Stage 3: Reinforcement Learning

Implementation Details

Evaluation Framework

Reasoning Quality Assessment

Challenges and Solutions

There’s a reason 400,000 professionals read this daily.

Exploring New LLM Applications: Insights from January 1–15, 2025 Research Papers

Fun & engaging podcast using NotebookLM

Podcast: Application of LLMs (Jan 1-15th, 2025)

Transforming healthcare with LLMs

Stay up-to-date with AI