#low-memory inference
Low-Memory Inference: 1 AI articles covering low-memory inference news, analysis, and research
Articles
180B Parameters, 32GB of RAM: Pushing MoE Inference to the LimitNEWβ10
Run a 180B-parameter MoE model on a single 32GB machine using Transformers and GGUF. Learn how conditional computation and quantization make local inference pos...
