Adam
Technology & Computing abbreviation · Artificial Intelligence
| Stands for | Adaptive moment estimation |
|---|---|
| Categories | Technology & Computing |
Adam stands for Adaptive moment estimation, an artificial intelligence abbreviation.
The most widely used optimizer for training neural networks, which adapts the learning rate for each parameter using running averages of recent gradients and their squares. It usually converges faster than plain stochastic gradient descent with little tuning.
Example: Train with Adam at a learning rate of 3e-4, then decay it over the final epochs.
Usage notes
Introduced by Diederik Kingma and Jimmy Ba in a 2014 paper that became one of the most cited in machine learning. AdamW, which decouples weight decay from the gradient update, is the standard choice for training transformers. Written as a name, "Adam", rather than in capitals.
Related abbreviations
More artificial intelligence abbreviations
- AGI – Artificial general intelligence
- AI – Artificial intelligence
- AIGC – AI-generated content
- ANI – Artificial narrow intelligence
- ANN – Approximate nearest neighbor (search)
- ASI – Artificial superintelligence
- ASL – AI Safety Level
- ASR – Automatic speech recognition
- AUC – Area under the curve
- AutoML – Automated machine learning
- BERT – Bidirectional Encoder Representations from Transformers
- BLEU – Bilingual evaluation understudy