How to Use Google Gemini AI APIs in Google Docs: A Guide

Introduction to Google Gemini AI API integration

In this blog post, we will explore the process of integrating Google Gemini generative AI into Google Docs using Apps Script. I will walk you through a step-by-step guide on how to marry the Google Gemini generative AI with Google Docs using the Gemini APIs.

There are two approaches for using Gemini AI in Google Docs or sheets:

1. You could use the built-in integration of Gemini with Google applications. This is the simplest method and requires no additional setup. As long as you have a Google Workspace account with access to Gemini (certain tiers required), the potential of using artificial intelligence to automate the creation of docs or sheets is easy and powerful. If you have no programming experience, I recommend this approach. However, this integration requires a paid subscription to Google Workspaces.

2. The second approach is to call the Gemini APIs directly from the Google Apps script. This approach allows you to integrate the Gemini AI functionality into the Google applications without requiring a Workspaces subscription. There is, however, a charge for using the Gemini APIs, but the cost of the API is relatively inexpensive compared to a Workspaces subscription. The additional advantage of calling the APIs directly is that you can use any of the functionality of the Gemini API and are not limited to the built-in integration capabilities.

What I'll cover in this post

To integrate the Gemini AI APIs into your Google doc will require a few steps. This blog post will take you through the following step-by-step:

1. Creating an empty Google Doc

2. Create the new Google apps script by copying in the code

3. Enabling the Vertex AI (Gemini) & Apps Scripts APIs

4. Setting the Project ID, API location & model

5. Setting up OAuth credentials for the Vertex AI API

6. Adding scopes to the Apps Script

7. Copying the Apps Script code to the doc & implement the Gemini API integration

8. Running and testing the script call to the Gemini API for generateContent

9. Resolving any errors if they occur

By following this tutorial, you can harness the power of AI to improve productivity and efficiency in your document workflow. Let's start and unlock the potential of AI in Google Docs.

Introduction to Google Gemini AI

Google Gemini AI is a new AI model developed by Google designed to be more powerful and flexible than previous AI models.  While Google has a history of developing AI models, such as BERT and MUM, Gemini is expected to be a significant step forward. It is still under development, but it is expected to be able to perform a wide range of tasks, including generating text, translating languages, and creating different kinds of creative content.

Introduction to the Gemini API

The Google Gemini APIs allow developers to access and use the power of Gemini AI. They are the bridge between the complex AI model and the applications that people use. These APIs can be used to build various applications, including chatbots, virtual assistants, and creative tools. The APIs are designed to be easy to use, even for developers who are not experts in AI and can be integrated into existing applications. Using the Gemini APIs, developers can leverage the latest advancements in AI to create innovative new products and services, opening up new possibilities in various industries.

Introduction to Google Apps Script

Google Apps Script is a cloud-based JavaScript platform that lets you integrate with and automate tasks across Google products. It allows users to automate tasks within various Google applications, including Google Docs. This Javascript language enables users to customize and extend the functionality of Google Docs by writing small snippets of code. With Google Apps Script, users can automate repetitive tasks or build complete applications within Google Docs. Whether sending personalized emails, integrating with other Google services, or creating an AI-powered action items generator, Google Apps Script provides an efficient way to streamline workflows and boost productivity within Google Docs.

Prerequisites for these Step-by-Step instructions

There are a few requirements for you to implement ChatGPT for Google Docs. They are the following:

1. Google Vertex AI: We will use the Google Gemini API through the Google Vertex AI API. Vertex AI is the production environment for using Google generative AI. It supports many more functions than we will use in this example, but gives us production-level access to the Gemini APIs vs. a prototyping and experimentation environment provided by Vertex AI Studio (PKA Google AI Studio).

2. A Google Cloud account: the Google Cloud account required by Vertex AI provides you with the Google Cloud Project with billing enabled. It uses Identity and Access Management (IAM) for authentication. To get that access, you will need a Google Cloud account. To create this account, refer to the instructions for Creating a cloud billing account. Any usage of the Gemini API will be billed to this account. The pricing of the Gemini APIs depends on which generative Language model you will be using. The pricing for the models and use of Vertex AI can be found at the link to Vertex AI pricing. Create the Google billing account before proceeding with the rest of the steps.

3. Access to Google Docs: Since the call to the API is from a document, you will require access to Google Docs. As long as you have a Google Gmail account, you can access Google Docs.

4. Familiarity with programming is helpful: The Gemini API will be called from the Google Apps Script, which is programmed in Javascript.  Even without knowing how to program, you should be able to follow all the steps in this guide, but if you run into problems, some programming familiarity would help determine the issue and how to fix it.

A step-by-step guide to implementing Gemini APIs in Google Docs

Now that we have a basic understanding of the Gemini API and the prerequisites let's dive into the step-by-step tutorial on calling the Gemini API from a Google doc.

For this tutorial, the Apps Script retrieves all the text in the document and calls the Gemini AI API to generate a response. I wanted to use a real example of a task that you might use Gemini AI for, so I have used an example of calling Gemini AI to create a set of action items from the text in the document. The prompt in the API call asks Gemini to review the text it retrieves and generate a set of action items.

The Apps script code shown in this example can be used as a base for using Gemini in a Google Doc for other purposes. You will need to change the prompt instructions in the API call and possibly some of the parameters to generate your desired output, but many of the steps to integrate the Gemini API into Google Docs should be usable.

Step 1: Create the new Google doc

1) On your Google account, at the top right should be a menu that, when you click, opens the item to create "Google Docs".

2) Choose Start a new "Blank document" which creates a new Untitled document

3) Rename the document to something more meaningful using the "rename" as shown below:

Step 2: Create the Google Apps Script

1) On the top menu in the Doc, choose "Extensions" -> "Apps Script"

2) This opens the Apps Script editor in a new browser tab, allowing you to create the Javascript code that calls the Vertex AI API. It creates an empty "Untitled" project with an initial Apps Script function called "myFunction". We will replace this default function with code that has the Vertex AI API integration.

3)  Next, you want to copy the Apps Script code I provide at the below GitHub URL. The entire code can be copied from the below URL to the editor window, replacing the empty function.

• Example Apps Script code for calling the Google Gemini AI API

You should end up with a project similar to the one below:

The code will need some small changes that I will guide you through later in the post.

4) Rename the project to something more useful. Click on the "Untitled project" name to open a window for you to enter the name. I renamed it "Call Gemini API."

5) Be sure to save the new project so you don't lose your work by clicking on the icon that resembles a floppy disk.

One item to note is that when you click the save button, Apps Script will syntax-check the code to ensure there are no syntactic errors. If you have entered the code incorrectly, as shown in the example below, you may get an error reported to the UI. If you encounter an error, check that all the code is copied correctly.

There are a few things to note in the Apps Script code:

• The default model (LLM - Large Language Model) used in the text generation is "gemini-1.5-pro-001", since it gave me the best results for the action items. You can change this to use other models of your choice. I suggest starting with Gemini-1.5-pro, and once you have everything set up and working, you can change it to other models for performing different functions.

• The API endpoint that is being called is the Vertex AI API. The API allows you to use Gemini as well as other language models. Depending on which language model you choose will determine what input or modality is allowed (text, image, audio, etc), what parameters are necessary, and the type of response wanted, such as generateContent or streamGenerateContent. The code provided uses the "generateContent" response using text input. If you change the code to use a different modality or response, you will need to refer to the documentation for Gemini API on Vertex AI. I have not provided the link since Google often changes and restructures its generative AI information. Search the Google Cloud console for information on this API.

• The code has error handling to ensure the text input is found. g minutes are found. There is limited logging of errors to the Google apps script log. If you encounter errors, refer to the details in the error resolution section later in this blog post.

• The script has a function called "onOpen" that creates a Google Doc menu item to allow you to execute the script. You will see this explained later.

Step 3: Enabling the Vertex AI & Apps Scripts APIs for this project

Now that you have the base for the apps script code, we will discuss the steps required to enable this script and the API in your Google Cloud account.

1) If you have created your Google Cloud account as noted in the prerequisites, you will automatically have a project created for you. You can use this project to enable and run the Gemini AI API. In another browser tab, open the Google Cloud console and navigate to the project you want to use to run the API. You can find your project on the Google Cloud console, as shown below. In this example, I will use the default project created for me, called "My First Project".

2) To call the Vertex Gemini AI API from the Google Apps Script in the doc, you must enable the API in the project. On the console, open the left-hand navigation panel and choose "APIs & Services" -> "Enabled APIs & Services".

3) On this APIs & Services UI you will see that no APIs are yet enabled for this new project. We need to add the Gemini API to the list of APIs enabled to run in this project.

Click the "ENABLE APIS AND SERVICES" button to open the API search UI.

4) In the search box, enter "Vertex AI API" and press enter. This will then find and list the Vertex AI API we will use to call Gemini, as shown below. Gemini models are accessed through the Vertex AI platform. The Vertex AI API provides the interface to interact with various AI models and services, including Gemini.

5) Click on the "Vertex AI API" to open the UI to enable this API, as shown below. On this UI, you can also read additional information about the API to better understand its capabilities and features. Then, click on the "Enable" button on this screen.

6) This will enable the API for the current project and present you with its details. You know it is enabled by the status indicating "Enabled." Make sure the project you want to use is listed on the console's top navigation bar. The API is enabled for a specific project, so you must ensure the "enablement" you just set is for the correct project.

7) Additionally, you will want to enable the "Apps Script API". It is not strictly needed to support the apps script we will use, but it is a best practice and ensures that the code should still work if Google Cloud makes changes in the future. In the search box in the "API Library" -> "ENABLE APIS AND SERVICES", search for "Apps Script API":

Using similar steps for 5 and 6 above to enable the "Apps Script API" for this project.

At this point, you should have 2 APIs enabled for the project. We can continue to the additional setup.

Step 4: Setting the Project ID, API location & model in the Apps script code

1) To be able to run the API, you need to specify the project that you will be using to run the Apps script code. In the Google Cloud console, on the left navigation bar, choose "Cloud overview" -> "Dashboard".

2) In the dashboard, you can view the details of your projects. You will need the

"Project ID" specified on this UI for the project. This project ID will be used in the Apps script code to specify which project will call the API and has already been enabled.

3) From the above UI listing the project details, copy/paste the Project ID into the line of Apps Script code that sets the value of the const "projectId" as shown below:

Note: The project ID listed above is no longer valid. This is not normally something that should be exposed outside of your organization.

4) The code also specifies a "location" const that indicates the geographic location of the physical cloud computing resources. Google Cloud has different geographic regions that can be used for cloud computing resources. In this enablement, we need to specify which region will be used to run the Vertex AI API. Therefore, we must define the region (location) used in the API call. You can find the available locations for the Vertex AI in the documentation on the Vertex AI locations page.

Note: Google constantly evolves the AI platform, and the documentation may change. If the above link does not work on your Google Cloud Console dashboard, search for" Vertex AI locations."

5) From the Vertex AI locations documentation, ensure the value used in the Apps Script code for the const "location" is valid. As shown below, the example code uses "us-central1".

6) The last part of the API URL is the model setting. You can see that the model used in this example is the Gemini 1.5 pro model. You can change this value to any of the valid Gemini models and versions currently available. Check the Gemini documentation to ensure you are using a current valid model.

7) Now that the three values have been set, let's look at the URL const used for the Vertex API call to Gemini. You can see where the three values for location, projectId, and modelId will be replaced in the const for the Vertex API URL. Make sure this const adheres to this format for the URL, or you may get an error on the call.

You should also note that when calling the Gemini API we specify the wanted response. In this example, we use the "generateContent" response. This function in the API allows you to give the AI model instructions on what kind of text you want, and it generates that text for you based on its knowledge and understanding. The input type such as text image, audio files and the response provided varies by Gemini model and version. If you want to change the type of input used and the response, check the Vertex AI documentation for the latest capabilities.

Step 5: Setting up OAuth credentials for the API

PLEASE NOTE: This step details how to create an OAuth credential for the API call. The Google Cloud project you created automatically creates an OAuth 2 credential not shown in the Google console. You can use this default OAuth credential. If you want to use the default, skip this step and proceed to step 6.

If the default OAuth credential does not work, follow the remaining instructions in Step 5 and then proceed to Step 6.

You must authenticate the caller to call the API from the Apps Script code. In some cases, you will use an API key for authentication. The Vertex AI API requires that you use OAuth 2.0 authentication. The line of code in the API call that uses the OAuth token is the following:

 "Authorization": "Bearer " + ScriptApp.getOAuthToken()

When you request an OAuth token using this code, you use the OAuth 2.0 authorization flow. This flow involves the user granting permission to your script to access specific Google services (scopes), and the token is then generated based on this authorization.

1) you must create the OAuth credentials for this code to work. We will explicitly create an OAuth client ID in the Google Cloud project and configure it according to the instructions for the Vertex AI API we are using.

In the Cloud Console, go to "APIs & Services" -> "Credentials". Then "Create Credentials"

-> "OAuth client ID":

2) On this create OAuth client ID screen, choose "configure consent screen".

3) Choose "Internal" as the User Type on the OAuth consent screen. Then click "Create":

4) Fill in the required information on the first screen of the consent, and then click "save and continue".

5) On the scopes step, choose "Add or remove scopes":

Then scroll down to the field "Manually add scope" and enter the following scopes:

https://www.googleapis.com/auth/cloud-platform,   https://www.googleapis.com/auth/script.external_request,   https://www.googleapis.com/auth/documents.currentonly 

Click "Add to table" and then "Update". You should have the UI that looks like the following:

Once they are added, click "Save and continue" at the bottom of the UI. This should take you to the summary page. Check that the information is correct and click "back to dashboard".

6) If you are not seeing the "Create credentials" page, then go to "APIs & Services" -> "Credentials". Then "Create Credentials" -> "OAuth client ID":

Enter "Web application" in the above UI for the application type. Give a name for the OAuth 2.0 client.

In the "Authorized redirect URIs" field, enter the following

https://script.google.com/macros/d/YOUR_SCRIPT_ID/usercallback

replace "YOUR_SCRIPT_ID" with your actual Apps Script ID. The script ID can be found on the Apps Script editor under the UI "Project settings" on the left nav bar.

Copy/paste the Script ID to the line of code above, and then click "Create" to create the OAuth 2.0 credential.

You should see a confirmation UI that confirms the OAuth client has been created.

Step 6: Adding scopes to the Apps Script

1) In Google Apps Script, scopes are like keys. They give your script permission to access specific Google services or parts of your Google account. We need to list the scopes for the particular apps script in the "appsscript.json" file. This file is found in the apps script editor, as shown below. If you do not see this file, you must go to "Project settings" and check the box "Show appsscript.json manifest file in editor".

2) Open the appsscript.json manifest file, add the following lines of code to specify the scopes, and click "save."

  "oauthScopes": [
    "https://www.googleapis.com/auth/cloud-platform",
    "https://www.googleapis.com/auth/script.external_request",
    "https://www.googleapis.com/auth/documents.currentonly"
  ],

Your appsscript.json file should look similar to the following:

These scopes allow you to make an external request to the Vertex AI API, access the cloud account, and access the current Google document from which you are running the apps script code from.

Step 7: Running and testing the script call to Gemini

Now that you have the Apps Script project created, the OAuth credentials set, the APIs enabled, and the scopes defined, you can test the script to see if it is set up properly. The Apps Script you created is a container-bound project. This means the Apps Script project is attached or bound to the Google Doc you initially created. In other words, the project is embedded within the docs file.

1)  Return to the Google Doc and refresh the tab on the browser or reopen the doc. Once the document is refreshed, you will see the "AI Tools" menu option with the subitem "Generate Action Items". This menu item was created using the "onOpen" function in the script.

2)  Copy some example meeting minutes into the document. Don't worry about the content since this is a test to ensure the apps script call to Gemini AI is working.

3)  Once the document is ready, choose "AI Tools" -> "Generate Action Items" on the doc menu. This will execute the script in the "Call Gemini API" project. The first time you execute the script, you must provide authorization for the script to run on this Google account.

4) It will ask for you to authorize the script to run. You will see a UI with the following request. Choose "OK".

5) It will then ask you to choose the account to authorize. I chose the default account, which is where I am creating this document and project.

6) It will then ask you to enter your password to verify that you authorize the access. Enter your password and then click "Next"

7) Next, it will ask you to confirm the specific access you are requesting. It wants confirmation of accessing your Google document and account and connecting to an external service. You should check that the access it is requesting is what you expect, and if it is correct, then choose "Allow."

This first time, it will not execute the script; it only provides access authorization. Run it again by choosing "AI Tools" -> "Generate Action Items" on the doc menu. A message will be displayed in the document's UI when the script runs, as shown below.

4)  If the Gemini API request is successful, a list of action items will be generated and inserted into the document after any existing text.  I have included a screenshot of a partial example below:

If no text is inserted, check for error messages indicating possible problems. If there are errors, check the steps above to ensure all actions have been implemented properly. Errors will be logged to the Apps Script execution log. Refer to the section Error Resolution for a description of how to find the execution log and a list of possible errors and how to resolve them.

Simplifying Tasks and Boosting Productivity with AI

This is just one example of the types of calls and output that can be generated. You can use this base set of steps and code to build on for allowing other use cases

As AI technology advances, the accuracy of generated output will improve. This automation process can become even more efficient by providing additional training on your business processes or templates.

As I stated in this blog post introduction, there is built-in Gemini integration for Google apps such as Gmail, Docs, and Sheets. However, if the built-in integration doesn't provide you with the necessary function, you can add the capability by making calls to the Gemini API using apps script. Explore the other capabilities in the Gemini models. Have fun adding advanced generative AI features to your Google applications to improve your productivity further and simplify your work or life.

Error resolution

There are a few possible problems you may encounter when running the generator.

Open the log window to view error messages sent to the Apps Script execution log.

Accessing the Apps Script execution logs:

In the Apps Script editor on the left, click "Executions"

This will open the Execution log window and allow you to view all logs from the script running on this document. Click on any of the rows to see the details of the logs. Any errors handled by the script code or unexpected errors will have their details logged here.

Possible error conditions and resolutions:

• Truncation of the text: If the generated text looks truncated, it’s likely due to the number of tokens specified in the request. The script provided limits it to 800 tokens; if your list of action items is extensive, you must increase this limit. To make this change, increase the number of the parameter “maxOutputTokens” in the line of code for the payload, as shown below.

• If the document does not have any meeting minutes as input, you will see the following message in the log:

or this text inserted into the document

• If you see the following 401 error, it means you have invalid OAuth 2.0 credentials. Check that you have correctly created the OAuth credentials according to step 5 and that the URI callback is correct.

• If you see the following error, there is an issue with the scopes you have set for the script. Check that all the steps are correctly implemented as described in Step 6.

• If you see an error 400, as shown below, you may have incorrect configuration parameters in the API request. Check the Gemini API documentation to ensure you have correctly formatted the input to the API payload.

• Error 404 indicates that you have specified an incorrect URL for the API call. This could be caused by incorrect values for "projectId," "modelId," and "location" consts. Check that the values are correct for the specific model you are using and that you have specified the correct apps script project.