From alphaxiv.org
0 0
View recent discussion. Abstract: What makes an interaction with the LLM more preferable for the user? While it is intuitive to assume that information accuracy in the LLM's responses would be one of the influential variables, recent studies have found that inaccurate LLM's responses could still...
on Apr 29
From alphaxiv.org
Rethinking Early Stopping: Refine, Then Calibrate | alphaXiv
0 0
View 4 comments: This wasn't intuitive to me even after reading the paper...
on Apr 8
From alphaxiv.org
Deep Model Merging: The Sister of Neural Network Interpretability -- A Survey | alphaXiv
0 0
View 3 comments: What about compute, some of the newer ones start requiring memory and compute that is not trivial any more
on Apr 2
From alphaxiv.org
0 0
View recent discussion. Abstract: Language models (LMs) tend to show human-like preferences on a number of syntactic phenomena, but the extent to which these are attributable to direct exposure to the phenomena or more general properties of language is unclear. We explore this with the English...
on Mar 31
From alphaxiv.org
LLM Alignment for the Arabs: A Homogenous Culture or Diverse Ones? | alphaXiv
0 0
View recent discussion. Abstract: Large language models (LLMs) have the potential of being useful tools that can automate tasks and assist humans. However, these models are more fluent in English and more aligned with Western cultures, norms, and values. Arabic-specific LLMs are being developed...
on Mar 25
From alphaxiv.org
ECLeKTic: a Novel Challenge Set for Evaluation of Cross-Lingual Knowledge Transfer | alphaXiv
0 0
View recent discussion. Abstract: To achieve equitable performance across languages, multilingual large language models (LLMs) must be able to abstract knowledge beyond the language in which it was acquired. However, the current literature lacks reliable ways to measure LLMs' capability of...
on Mar 21
From alphaxiv.org
0 0
View recent discussion. Abstract: The veracity of a factoid is largely independent of the language it is written in. However, language models are inconsistent in their ability to answer the same factual question across languages. This raises questions about how LLMs represent a given fact across...
on Mar 21
From alphaxiv.org
AdaMerging: Adaptive Model Merging for Multi-Task Learning | alphaXiv
0 0
View recent discussion. Abstract: Multi-task learning (MTL) aims to empower a model to tackle multiple tasks simultaneously. A recent development known as task arithmetic has revealed that several models, each fine-tuned for distinct tasks, can be directly merged into a single model to execute...
on Feb 21
From alphaxiv.org
0 1
View recent discussion. Abstract: This paper introduces the counter-intuitive generalization results of overfitting pre-trained large language models (LLMs) on very small datasets. In the setting of open-ended text generation, it is well-documented that LLMs tend to generate repetitive and dull...
on Jan 18
From alphaxiv.org
Truth is Universal: Robust Detection of Lies in LLMs | alphaXiv
0 2
View 2 comments: Calling it a lie is a strong name, which I understand why it is used for buzz. Lie has intention behind it.But I wonder if there is a way to check for alternative theories. For example, is it the "rel...
on Jan 16
From alphaxiv.org
Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models | alphaXiv
0 0
View 3 comments: Delete the space?
on Dec 14
From alphaxiv.org
Faux Polyglot: A Study on Information Disparity in Multilingual Large Language Models | alphaXiv
0 0
View 2 comments: The text is quite small in the pictures and the quality not great (consider saving in .pdf latex eats it well, regarding size, look at the regular text size to know what is +- expected size of texts)I...
on Dec 4
From alphaxiv.org
Discuss | GPT or BERT: why not both? | alphaXiv
0 0
View recent discussion. Abstract: We present a simple way to merge masked language modeling with causal language modeling. This hybrid training objective results in a model that combines the strengths of both modeling paradigms within a single transformer stack: GPT-BERT can be transparently...
on Nov 19
From alphaxiv.org
0 0
View recent discussion. Abstract: Language models are typically trained on large corpora of text in their default orthographic form. However, this is not the only option; representing data as streams of phonemes can offer unique advantages, from deeper insights into phonological language...
on Nov 19
From alphaxiv.org
0 0
View recent discussion. Abstract: While current large language models have achieved a remarkable success, their data efficiency remains a challenge to overcome. Recently it has been suggested that child-directed speech (CDS) can improve training data efficiency of modern language models based on...
on Nov 19
From alphaxiv.org
Parameter Efficient Multi-task Model Fusion with Partial Linearization | alphaXiv
0 0
Large pre-trained models have enabled significant advances in machine learning and served as foundation components. Model fusion methods, such as task arithmetic, have been proven to be powerful and scalable to incorporate fine-tuned weights from different tasks into a multi-task model....
on Nov 8
From alphaxiv.org
To Code, or Not To Code? Exploring Impact of Code in Pre-training | alphaXiv
0 0
Including code in the pre-training data mixture, even for models not specifically designed for code, has become a common practice in LLMs pre-training. While there has been anecdotal consensus among practitioners that code data plays a vital role in general LLMs' performance, there is...
on Oct 5
From alphaxiv.org
The Impact of Large Language Models in Academia: from Writing to Speaking | alphaXiv
0 0
Large language models (LLMs) are increasingly impacting human society, particularly in textual information. Based on more than 30,000 papers and 1,000 presentations from machine learning conferences, we examined and compared the words used in writing and speaking, representing the first...
on Sep 23
From alphaxiv.org
Do These LLM Benchmarks Agree? Fixing Benchmark Evaluation with BenchBench | alphaXiv
0 0
Recent advancements in Language Models (LMs) have catalyzed the creation of multiple benchmarks, designed to assess these models' general capabilities. A crucial task, however, is assessing the validity of the benchmarks themselves. This is most commonly done via Benchmark Agreement Testing...
on Sep 17
From alphaxiv.org
The Future of Open Human Feedback | alphaXiv
0 0
Human feedback on conversations with language language models (LLMs) is central to how these systems learn about the world, improve their capabilities, and are steered toward desirable and safe behaviors. However, this feedback is mostly collected by frontier AI labs and kept behind closed...
on Sep 7
From alphaxiv.org
The Unbearable Slowness of Being | alphaXiv
0 0
This article is about the neural conundrum behind the slowness of human behavior. The information throughput of a human being is about 10 bits/s. In comparison, our sensory systems gather data at an enormous rate, no less than 1 gigabits/s. The stark contrast between these numbers remains...
on Aug 27
From alphaxiv.org
Large Language Monkeys: Scaling Inference Compute with Repeated Sampling | alphaXiv
0 0
Scaling the amount of compute used to train language models has dramatically improved their capabilities. However, when it comes to inference, we often limit the amount of compute to only one attempt per problem. Here, we explore inference compute as another axis for scaling by increasing the...
on Aug 1