BPE
Technology & Computing abbreviation · Artificial Intelligence
| Stands for | Byte-pair encoding |
|---|---|
| Categories | Technology & Computing |
BPE stands for Byte-pair encoding, an artificial intelligence abbreviation.
A method that builds a vocabulary of subword units by repeatedly merging the most frequent pairs of characters or bytes, so a model can represent any word with a limited vocabulary. GPT models and many other LLMs tokenize text with BPE variants.
Example: With BPE, "unhappiness" might be split into "un", "happi" and "ness" tokens.
Usage notes
Originally a 1994 data-compression algorithm by Philip Gage, adapted for neural machine translation by Rico Sennrich and colleagues in 2015. Tokenization explains quirks such as models struggling to count the letters in a word. Libraries such as tiktoken and Hugging Face Tokenizers implement it.
Related abbreviations
More artificial intelligence abbreviations
- ASR – Automatic speech recognition
- AUC – Area under the curve
- AutoML – Automated machine learning
- BERT – Bidirectional Encoder Representations from Transformers
- BLEU – Bilingual evaluation understudy
- BoW – Bag of words
- CLIP – Contrastive Language-Image Pre-training
- CNN – Convolutional neural network
- CoT – Chain of thought
- CUDA – Compute Unified Device Architecture
- CV – Computer vision
- DL – Deep learning