From openreview.net
Unified Lookup Tables: Privacy-Preserving Foundation Models
0 1
Transformers, despite their success in a variety of sequence modeling tasks, have a significant limitation: they are inherently data-greedy, which can lead to overfitting when the data are scarce....
on Sun, 6PM
From openreview.net
Relating Hopfield Networks to Episodic Control
0 0
Neural Episodic Control is a powerful reinforcement learning framework that employs a differentiable dictionary to store non-parametric memories. It was inspired by episodic memory on the...
on Dec 12
From openreview.net
Visual Autoregressive Modeling: Scalable Image Generation via...
0 0
We present Visual AutoRegressive modeling (VAR), a new generation paradigm that redefines the autoregressive learning on images as coarse-to-fine "next-scale prediction" or "next-resolution...
on Dec 12
From openreview.net
ACM TheWebConf 2025 Workshop MORE
0 0
Welcome to the OpenReview homepage for ACM TheWebConf 2025 Workshop MORE
on Dec 11
From openreview.net
MonkeySee: Space-time-resolved reconstructions of natural images...
0 0
In this paper, we reconstruct naturalistic images directly from macaque brain signals using a convolutional neural network (CNN) based decoder. We investigate the ability of this CNN-based decoding...
on Nov 19
From openreview.net
A Path Towards Autonomous Machine Intelligence
0 0
How could machines learn as efficiently as humans and animals? How could machines learn to reason and plan? How could machines learn representations of percepts and action plans at multiple...
on Nov 14
From openreview.net
N$\mathsf{L}^2$PS: A Natural Language to LEAN Proofs System
0 0
The inference capabilities of large language models (LLMs) are rapidly advancing, nearing the limits of current benchmarks. Notably, models like Llama3 have shown substantial improvements on MATH...
on Nov 11
From openreview.net
0 0
Welcome to the OpenReview homepage for NeurIPS 2024 Workshop MATH-AI
on Oct 20
From openreview.net
AGaLiTe: Approximate Gated Linear Transformers for Online...
0 0
In this paper we investigate transformer architectures designed for partially observable online reinforcement learning. The self-attention mechanism in the transformer architecture is capable of...
on Oct 18
From openreview.net
Learned feature representations are biased by complexity, learning...
0 0
Representation learning, and interpreting learned representations, are key areas of focus in machine learning and neuroscience. Both fields generally use representations as a means to understand or...
on Sep 24
From openreview.net
Language Modeling Is Compression
0 1
It has long been established that predictive models can be transformed into lossless compressors and vice versa. Incidentally, in recent years, the machine learning community has focused on...
on Sep 4
From openreview.net
Projected Language Models: A Large Model Pre-Segmented Into Smaller...
0 0
Large language models are versatile tools but are not suitable for small inference budgets. Small models have more efficient inference but their lower capacity means that their performance can be...
on Jul 23
From openreview.net
Gradual Optimization Learning for Conformational Energy Minimization
0 0
Molecular conformation optimization is crucial to computer-aided drug discovery and materials design. Traditional energy minimization techniques rely on iterative optimization methods that use...
on Jul 23
From openreview.net
Don’t Label Twice: Quantity Beats Quality when Comparing Binary...
0 0
We study how to best spend a budget of noisy labels to compare the accuracy of two binary classifiers. It’s common practice to collect and aggregate multiple noisy labels for a given data point...
on Jul 18
From openreview.net
Position: Enforced Amnesia as a Way to Mitigate the Potential Risk...
0 0
Science fiction has explored the possibility of a conscious self-aware mind being locked in silent suffering for prolonged periods of time. Unfortunately, we still do not have a reliable test for...
on Jul 15
From openreview.net
SceneCraft: An LLM Agent for Synthesizing 3D Scenes as Blender Code
0 0
This paper introduces SceneCraft, a Large Language Model (LLM) Agent converting text descriptions into Blender-executable Python scripts which render complex scenes with up to a hundred 3D assets....
on Jun 26
From openreview.net
VerityMath: Advancing Mathematical Reasoning by Self-Verification...
0 0
Large Language Models (LLMs), combined with program-based solving techniques, are increasingly demonstrating proficiency in mathematical reasoning. For example, closed-source models such as OpenAI...
on Jun 26
From openreview.net
PutnamBench: A Multilingual Competition-Mathematics Benchmark for...
0 0
We present PutnamBench, a new benchmark for evaluating the ability of neural theorem-provers to solve competition mathematics problems. PutnamBench consists of formalizations of problems sourced...
on Jun 26
From openreview.net
Lean4trace: Data augmentation for neural theorem proving in Lean
0 0
Integrating large language models as proof assistants with theorem provers has shown great promise. However, one of the major challenges in this field is the scarcity of training data. To address...
on Jun 26
From openreview.net
More Details, Please: Improving Autoformalization with More...
0 0
The formalization of mathematical theorems and their proofs is a time-consuming and tedious process which, despite recent advances in the reasoning capabilities of AI systems, remains a challenging...
on Jun 26
From openreview.net
Logic-LM: Empowering Large Language Models with Symbolic Solvers...
0 0
Large Language Models (LLMs) have shown human-like reasoning abilities but still struggle with complex logical problems. This paper introduces a novel framework, Logic-LM, which integrates LLMs...
on May 28
From openreview.net
SpaRC and SpaRP: Spatial Reasoning Characterization and Path...
0 0
Spatial reasoning is a crucial component of both biological and artificial intelligence. In this work, we present a comprehensive study of the capability of current state-of-the-art large language...
on May 27
From openreview.net
Vision Transformers Need Registers
0 0
Transformers have recently emerged as a powerful tool for learning visual representations. In this paper, we identify and characterize artifacts in feature maps of both supervised and...
on May 11
From openreview.net
Modeling Boundedly Rational Agents with Latent Inference Budgets
0 0
We study the problem of modeling a population of agents pursuing unknown goals subject to unknown computational constraints. In standard models of bounded rationality, sub-optimal decision-making...
on May 9
From openreview.net
0 0
Welcome to the OpenReview homepage for ICLR 2024 Workshop AfricaNLP
on May 2
From openreview.net
MC Layer Normalization for calibrated uncertainty in Deep Learning
0 0
Efficiently estimating the uncertainty of neural network predictions has become an increasingly important challenge as machine learning models are adopted for high-stakes industrial applications...
on Mar 13
From openreview.net
LIFT: Efficient Layer-wise Fine-tuning for Large Model Models
0 0
Fine-tuning is widely applied in language language processing to adapt the model for downstream tasks. However, as model sizes grow rapidly, fine-tuning the full model is computationally expensive....
on Mar 10
From openreview.net
Pooling Image Datasets with Multiple Covariate Shift and Imbalance
0 0
Small sample sizes are common in many disciplines, which necessitates pooling roughly similar datasets across multiple sites/institutions to study weak but relevant associations between images...
on Mar 9
From openreview.net
MetaGPT: Meta Programming for Multi-Agent Collaborative Framework
0 0
Recently, remarkable progress has been made on automated problem solving through societies of agents based on large language models (LLMs). Previous LLM-based multi-agent systems can already solve...
on Mar 1
From openreview.net
Fine-Tuning Language Models with Just Forward Passes
0 0
Fine-tuning language models (LMs) has yielded success on diverse downstream tasks, but as LMs grow in size, backpropagation requires a prohibitively large amount of memory. Zeroth-order (ZO)...
on Feb 9
From openreview.net
AI for Whom? Shedding Critical Light on AI for Social Good
0 0
In recent years, AI for Social Good (AI4SG) projects have grown in scope and popularity, covering a variety of topics from climate change to education and being the subject of numerous workshops...
on Feb 8
From openreview.net
Detecting Backdoors with Meta-Models
0 0
It is widely known that it is possible to implant backdoors into neural networks, by which an attacker can choose an input to produce a particular undesirable output (e.g.\ misclassify an...
on Jan 26
From openreview.net
Simultaneous linear connectivity of neural networks modulo permutation
0 0
The usual parameterization of neural networks exhibits permutation symmetry, where reordering neurons in each layer does not change the underlying function computed by a network. These symmetries...
on Jan 23
From openreview.net
DINOv2: Learning Robust Visual Features without Supervision
0 0
The recent breakthroughs in natural language processing for model pretraining on large quantities of data have opened the way for similar foundation models in computer vision. These models could...
on Jan 22
From openreview.net
Stabilizing Backpropagation Through Time to Learn Complex Physics
0 0
Of all the vector fields surrounding the minima of recurrent learning setups, the gradient field with its exploding and vanishing updates appears a poor choice for optimization, offering little...
on Jan 19
From openreview.net
Symmetric Basis Convolutions for Learning Lagrangian Fluid Mechanics
0 0
Learning physical simulations has been an essential and central aspect of many recent research efforts in machine learning, particularly for Navier-Stokes-based fluid mechanics. Classic numerical...
on Jan 19
From openreview.net
Turing Complete Transformers: Two Transformers Are More Powerful...
0 0
This paper presents Find+Replace transformers, a family of multi-transformer architectures that can provably do things no single transformer can, and which outperforms GPT-4 on several challenging...
on Jan 9
From openreview.net
Are Emergent Abilities of Large Language Models a Mirage?
0 0
Recent work claims that large language models display \textit{emergent abilities}, abilities not present in smaller-scale models that are present in larger-scale models. What makes emergent...
on Jan 2
From openreview.net
Efficient Estimation of Word Representations in Vector Space
0 0
We propose two novel model architectures for computing continuous vector representations of words from very large data sets. The quality of these representations is measured in a word similarity...
on Dec 18, 2023
From openreview.net
Learning better with Dale’s Law: A Spectral Perspective
0 0
Most recurrent neural networks (RNNs) do not include a fundamental constraint of real neural circuits: Dale's Law, which implies that neurons must be excitatory (E) or inhibitory (I). Dale's Law is...
on Dec 12, 2023
From openreview.net
Meta-learning families of plasticity rules in recurrent spiking...
0 0
There is substantial experimental evidence that learning and memory-related behaviours rely on local synaptic changes, but the search for distinct plasticity rules has been driven by human...
on Dec 11, 2023
From openreview.net
DeepLearningIndaba 2023 Conference
0 0
Welcome to the OpenReview homepage for DeepLearningIndaba 2023 Conference
on Dec 10, 2023
From openreview.net
Guided Sketch-Based Program Induction by Search Gradients
0 0
Many tasks can be easily solved using machine learning techniques. However, some tasks cannot readily be solved using statistical models, requiring a symbolic approach instead. Program induction is...
on Dec 7, 2023
From openreview.net
How to Turn Your Knowledge Graph Embeddings into Generative Models
0 0
Some of the most successful knowledge graph embedding (KGE) models for link prediction – CP, RESCAL, TuckER, ComplEx – can be interpreted as energy-based models. Under this perspective they are not...
on Dec 5, 2023