LLMs on Apple Silicon

Options
tgb417
tgb417 Dataiku DSS Core Designer, Dataiku DSS & SQL, Dataiku DSS ML Practitioner, Dataiku DSS Core Concepts, Neuron 2020, Neuron, Registered, Dataiku Frontrunner Awards 2021 Finalist, Neuron 2021, Neuron 2022, Frontrunner 2022 Finalist, Frontrunner 2022 Winner, Dataiku Frontrunner Awards 2021 Participant, Frontrunner 2022 Participant, Neuron 2023 Posts: 1,595 Neuron

Given that LLMs can be installed on Apple Silicon Based Computers that can have significant GPU access able RAM. It is possible to run LLMs and Diffusion Models. One can use a variety of approaches. For example using instructions like this:

https://github.com/domschl/HuggingFaceGuidedTourForMac

And other methods.

Has anyone worked out how to connect these models to DSS as part of the LLM mesh?


Operating system used: Mac OS

Setup Info
    Tags
      Help me…