TheBloke

Open-source work

TheBloke is the public handle of a Hugging Face account whose profile lists quantisation and fine tuning as interests. The account's repositories, created between April 2023 and January 2024, publish quantized versions of open language models so they can run on local hardware.¹⁵⁶

Quantization formats

The earliest repositories, for models such as Koala 7B, used the GPTQ and GGML formats, and the Koala GGML card notes that some files were quantized with an older version of llama.cpp to stay compatible with an earlier commit. Later repositories adopted the GGUF format alongside GPTQ and AWQ: the Llama 2 7B GGUF repository, for example, offers files from Q2_K to Q8_0 and lists llama.cpp and LM Studio among the clients that can load them.⁷³²

Among the last repositories created, in January 2024, were GGUF, GPTQ and AWQ versions of Argilla's CapybaraHermes 2.5 Mistral 7B.⁶⁸

Work and contributions

• llama.cpp Open-source code

  Authored the May 16, 2023 commit "convert.py: Support models which are stored in a single pytorch_model.bin" (#1469). The repository is MIT-licensed.⁴⁹

• Quantized model repositories on Hugging Face Open weights

  Publishes GGUF, GPTQ and AWQ quantizations of open models under the TheBloke account. The Llama-2-7B-GGUF card says the repository contains GGUF files for Meta's Llama 2 7B under the llama2 license, and lists llama.cpp, LM Studio and other clients that support the format. The koala-7B-GGML card says its original quantisation methods were quantized using an older version of llama.cpp so that they stay compatible with llama.cpp as of commit 2d5db48.¹²³

Timeline

1. Apr 7, 2023Created the first TheBloke repositories on Hugging Face, including koala-7B-GPTQ.⁵⁷
2. Sep 4, 2023Created the Llama-2-7B-GGUF repository of GGUF quantizations of Meta's Llama 2 7B.⁵²
3. Jan 31, 2024Created GGUF, GPTQ and AWQ repositories for CapybaraHermes 2.5 Mistral 7B, among the account's last uploads.⁶⁸

Documented roles

The sources do not confirm a current role; these are the roles they document.

• Contributor (convert.py, 2023), llama.cpp project⁴

Related people

• Younes BelkadaLanguage models · Inference and serving
• Song HanInference and serving · Language models
• Furu WeiLanguage models · Inference and serving
• Krithik PuthalathLanguage models · Inference and serving
• Eric HartfordLanguage models · Inference and serving
• Arjun DesaiLanguage models · Inference and serving
• BartowskiInference and serving · Language models
• Waseem AlShikhLanguage models · Inference and serving
• Oleksiy OstapenkoLanguage models · Inference and serving
• Azalia MirhoseiniLanguage models · Inference and serving