NeurIPS'25 · Conference paper
Dense Associative Memory with Epanechnikov Energy
Exact recall, plus new memories that emerge
- Spotlight (top 3%)
Abstract
We propose a novel energy function for Dense Associative Memory (DenseAM) networks, the log-sum-ReLU (LSR), inspired by optimal kernel density estimation. Unlike the common log-sum-exponential (LSE) function, LSR is based on the Epanechnikov kernel and enables exact memory retrieval with exponential capacity without requiring exponential separation functions. Moreover, it introduces abundant additional \emph{emergent} local minima while preserving perfect pattern recovery -- a characteristic previously unseen in DenseAM literature. Empirical results show that LSR energy has significantly more local minima (memories) that have comparable log-likelihood to LSE-based models. Analysis of LSR's emergent memories on image datasets reveals a degree of creativity and novelty, hinting at this method's potential for both large-scale memory storage and generative tasks.
More
Cite
@inproceedings{
hoover2025dense,
title={Dense Associative Memory with Epanechnikov Energy},
author={Benjamin Hoover and Zhaoyang Shi and Krishnakumar Balasubramanian and Dmitry Krotov and Parikshit Ram},
booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems},
year={2025},
url={https://openreview.net/forum?id=ZbQ5Zq3zA3}
}
Benjamin Hoover
Zhaoyang Shi
Krishnakumar Balasubramanian
Dmitry Krotov
Parikshit Ram