#llama-cpp

2 posts

AI Model Quantization Explained: How Smaller Models Run on Less Hardware

FP16, BF16, INT8 and 4-bit compared with a real laptop experiment: how much memory each format saves, how much faster it runs, and what it does to answers.

#quantization#local-ai #llama-cpp#gguf

Oct 13, 202616m read

Build a Local AI Coding Assistant: From Setup to Working Application

A 204-line Python app that searches your codebase, answers questions and suggests changes with a local model, and never runs or applies code. Tested, mistakes included.

#local-ai#coding-assistant#rag #llama-cpp

Oct 13, 202621m read

Read