Is BNB quantization supported?

MichaelKarpe
MichaelKarpe Registered Posts: 4 ✭✭

BNB quantization is not mentionned in Running Hugging Face models — Dataiku DSS 13 documentation.

For instance unsloth/Llama-3.3-70B-Instruct-bnb-4bit · Hugging Face.

What 4-bit quantized model do you recommand for Llama 3.3 otherwise?

Setup Info
    Tags
      Help me…