Is BNB quantization supported?
MichaelKarpe
Registered Posts: 4 ✭✭
BNB quantization is not mentionned in Running Hugging Face models — Dataiku DSS 13 documentation.
For instance unsloth/Llama-3.3-70B-Instruct-bnb-4bit · Hugging Face.
What 4-bit quantized model do you recommand for Llama 3.3 otherwise?