References
- The case for 4-bit precision: k-bit Inference Scaling Laws (2212.09720)
- GPT-2: Language Models are Unsupervised Multitask Learners (Radford et al., 2019)
- Language Models are Few-Shot Learners (2005.14165)
- Scaling Laws for Neural Language Models (2001.08361)
- LLM.int8() (2208.07339)
- GPTQ (2210.17323)
- AWQ (2306.00978)
- SmoothQuant (2211.10438)
- QLoRA (NF4) (2305.14314)
- QuIP (2307.13304) / QuIP# (2402.04396)
- AQLM (2401.06118)
- Scaling Laws for Precision (2411.04330)
- Efficiently Scaling Transformer Inference (2211.05102)
- Chinchilla (2203.15556)
- FP8 Formats for Deep Learning (2209.05433)
- KIVI (2402.02750) / KVQuant (2401.18079)