Connect to OpenAI-compatible API endpoints from Word

GPT for Word can connect to any LLM service that provides an OpenAI-compatible API endpoint. You can connect to two main types of services: cloud-based platforms, which provide access to models over the internet with no software installation or setup required on your part, and local servers, which you run on a local machine, such as your own computer or another computer on a local network. Once connected, all models available through the service become available to use in GPT for Word.

Here are some popular services that provide an OpenAI-compatible API endpoint:

• Cloud-based platforms:

  • Anyscale

  • Fireworks AI

  • Together AI

• Local servers:

  • LM Studio

  • LocalAI

  • Ollama

Prerequisites

• You have an endpoint URL and, if needed, an API key for accessing the service.

• The endpoint must support the following OpenAI-compatible API resource paths:

  • /v1/models
  • /v1/completions

info

Depending on the model you use, the model settings defined in GPT for Word (temperature (creativity), top-p, frequency penalty, and presence penalty) may not work as expected. If you get bad results, try tweaking the settings.

To use models from an OpenAI-compatible API endpoint:

1. Set up the endpoint.

2. Select a model available through the endpoint.

Set up an API endpoint

1. Sign in to the GPT for Work dashboard with your Microsoft account.

2. In the sidebar, select API endpoints.

3. Click OpenAI-compatible endpoints.

   tip

   If you already have OpenAI-compatible API endpoints set up, click Add another OpenAI compatible endpoint.

4. Define the endpoint settings:

   • Endpoint URL: Enter the URL of the service that provides the endpoint.

   • Display name (optional): Enter a short name that is prefixed to the model names in the model switcher. By default, the model names are prefixed with "custom/".

   • API key (optional): If required, enter the API key for accessing the service.

   

5. Click Check to validate the settings and click Save.

   

   tip

   If you get an error saying that the URL is invalid, make sure that the endpoint supports:

   • Cross-origin resource sharing (CORS). See the LM Studio setup instructions for macOS or Windows.

   • HTTPS (needed with Safari and with Microsoft Office for Mac). See the LM Studio setup instructions for macOS.

You have set up the OpenAI-compatible API endpoint. You can now use all models available through the endpoint in GPT for Word.

tip

To set up another OpenAI-compatible API endpoint, click Add another OpenAI compatible endpoint and repeat the previous two steps.

Select a model from the API endpoint

1. Open a Microsoft Word document.

2. On the ribbon, select Home > GPT for Excel Word.

   

   info

   If GPT for Excel Word is not visible on the ribbon, or if it's grayed out, select Home > Add-ins > My Add-ins > GPT for Excel Word. Learn more.

3. Open the model switcher. You can find the models under Use your API key or endpoint.

   

   note

   The available models depend on the models provided by the service.

4. Select a model.

GPT for Word now uses the selected model to generate responses.

What's next

• To try out an OpenAI-compatible LLM service with GPT for Word, set up LM Studio on macOS or Windows.