AnythingLLM & Bootstrap Studio 8

[mark.logan2010]

Hi everyone,

I recently purchased Bootstrap Studio 8, and I’d really like to use its Assistant feature with a local AI model instead of relying on a cloud-based AI service.

I’m currently running AnythingLLM locally on an iMac (Intel, 32GB RAM, macOS Monterey), and I’ve been experimenting with local models like Qwen and Gemma.

The problem is, I cannot seem to get Bootstrap Studio 8 to connect to AnythingLLM. I’ve tried several tutorials and instructions I’ve found online, but no matter what I try, Bootstrap Studio never connects to the local model.

I’m assuming I’m missing something simple, probably involving the local API endpoint or configuration in AnythingLLM. I’ve tried various localhost URLs, but nothing seems to work.

Has anyone successfully connected Bootstrap Studio 8 to AnythingLLM and a locally running model?

If so, I’d really appreciate step-by-step instructions, including:

• How to configure the local model in AnythingLLM
• What API/server settings need to be enabled
• The correct localhost or 127.0.0.1 URL and port to use
• What settings need to be entered in Bootstrap Studio 8
• Whether I need an API key or any additional configuration (i believe I do on this, not certain)
• Any other settings that are easy to overlook

I’d love to get this working so I can use a local AI model with Bootstrap Studio rather than sending my code and design work to a cloud service.

Thanks in advance! Any help would be greatly appreciated.

[mark.logan2010]

UPDATE: I found a legacy version of Ollama (ollama 0.1.32) for my iMac. Followed this tutorial:

Step 1: Prep Your Ollama Model

Bootstrap Studio needs a powerful model to successfully generate and replace code blocks without formatting errors.

1. Open your Mac’s Terminal app.

2. Download a high-performing coding model like Llama 3.1 (8B) or Qwen 2.5 Coder by running:

   ollama run llama3.1
   

3. Once the download finishes, you can close the Terminal. Ensure the Ollama menu bar app is still running. [1, 2, 3, 4]

Step 2: Configure Ollama in Bootstrap Studio (see screenshot below)

1. Open Bootstrap Studio on your Mac.

2. Navigate to the AI Assistant panel (usually located on the right side or under the Window/View menus).

3. Open the AI Settings / Providers menu.

4. Look for the model provider dropdown list and select Ollama.

5. Set the BASE URL to Ollama’s local address

   This one worked for me: http://localhost:11434/api

6. Type in the exact name of the model you downloaded (e.g., llama3.1) and save the configuration. [1, 3, 4, 5]

   

NOTE: i was able to connect, but I havent tried it out as of yet! will update.

[mark.logan2010]

UPDATE - next morning!
Typed in prompts with a multitude of models. Guess what? NO DICE!
In the Assistant window, it does its thing (thinks), spits out the HTML/CSS, etc, but it doesn’t implement over to the design window.

Blank, white screen. RUDE! LOL!!

If anyone has suggestions / tips regarding a local model that will work, please let me know. - I’m disappointed.

[catkin]

Try experimenting with the Provider version, you need a model that can make tool calls, and uses the Open AI API.

If you are running an AI model without a GPU, the reliability of the model successfully being able to make tool calls decreases and it can take a long time for the model to work.

[mark.logan2010]

Thank you catkin! Much appreciated.

Do you have an OpenAI model you rely on?

[catkin]

I am fluent in using the program, and have years of experience in web design, so do not rely on AI for anything but writing backend code. I also do not have a computer powerful enough for a local model so I am simply speaking from my experience when trying the feature so I could support others. I do not remember the model I tried.

[mark.logan2010]

Thank you very much!