I have an ask for the AI community: can someone create a standard for interacting with LLMs.
There is a whole category of new apps and features ready to be made on top of LLMs. But to get these in the hands of users is hard. Apps will need to have subscriptions, ask the user to add their OpenAI API key or include and ship their own LLMs. It’s time to add a fourth option: create a standardized API that allows interacting with LLMs that cloud and local services can implement to put the user in control.
There are new LLM models and tuned variations popping up all the time, and most of these models end up implementing some part of the Chat Completions endpoint of the OpenAI Rest API. The recommendation is for users to use the OpenAI SDK and override the endpoint (even Google is doing this now!). The OpenAI API is tailored to the needs of OpenAI, which means it’s set up to interact with their cloud-based LLM mode