i would suggest using other APIs such as groq on the llama3-70b parameter model. Also… because groq is quite fast…
Thank you for the suggestion! If others would like to see groq added, write below. We will investigate how easy it would be to integrate.
The Groq API reference is here : https://console.groq.com/docs/quickstart
Since they do allow free API access i think it would be a good addition as their LPUs produce the fastest inferences worldwide.