Google Drive

This notebook covers how to retrieve documents from Google Drive.

Prerequisites

Create a Google Cloud project or use an existing project
Enable the Google Drive API
Authorize credentials for desktop app
pip install -U google-api-python-client google-auth-httplib2 google-auth-oauthlib

Retrieve the Google Docs

By default, the GoogleDriveRetriever expects the credentials.json file to be ~/.credentials/credentials.json, but this is configurable using the GOOGLE_ACCOUNT_FILE environment variable. The location of token.json uses the same directory (or use the parameter token_path). Note that token.json will be created automatically the first time you use the retriever. GoogleDriveRetriever can retrieve a selection of files with some requests. By default, If you use a folder_id, all the files inside this folder can be retrieved to Document. You can obtain your folder and document id from the URL:

Folder: drive.google.com/drive/u/0/folders/1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5 -> folder id is "1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5"
Document: docs.google.com/document/d/1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw/edit -> document id is "1bfaMQ18_i56204VaQDVeAFpqEijJTgvurupdEDiaUQw"

The special value root is for your personal home.

from langchain_googledrive.retrievers import GoogleDriveRetriever

folder_id = "root"
# folder_id='1yucgL9WGgWZdM1TOuKkeghlPizuzMYb5'

retriever = GoogleDriveRetriever(
    num_results=2,
)

By default, all files with these MIME types can be converted to Document.

text/text
text/plain
text/html
text/csv
text/markdown
image/png
image/jpeg
application/epub+zip
application/pdf
application/rtf
application/vnd.google-apps.document (GDoc)
application/vnd.google-apps.presentation (GSlide)
application/vnd.google-apps.spreadsheet (GSheet)
application/vnd.google.colaboratory (Notebook colab)
application/vnd.openxmlformats-officedocument.presentationml.presentation (PPTX)
application/vnd.openxmlformats-officedocument.wordprocessingml.document (DOCX)

It’s possible to update or customize this. See the documentation of GoogleDriveRetriever. But, the corresponding packages must be installed.

%pip install -qU  unstructured

retriever.invoke("machine learning")

You can customize the criteria to select the files. A set of predefined filter are proposed:

Template	Description
`gdrive-all-in-folder`	Return all compatible files from a `folder_id`
`gdrive-query`	Search `query` in all drives
`gdrive-by-name`	Search file with name `query`
`gdrive-query-in-folder`	Search `query` in `folder_id` (and sub-folders in `_recursive=true`)
`gdrive-mime-type`	Search a specific `mime_type`
`gdrive-mime-type-in-folder`	Search a specific `mime_type` in `folder_id`
`gdrive-query-with-mime-type`	Search `query` with a specific `mime_type`
`gdrive-query-with-mime-type-and-folder`	Search `query` with a specific `mime_type` and in `folder_id`

retriever = GoogleDriveRetriever(
    template="gdrive-query",  # Search everywhere
    num_results=2,  # But take only 2 documents
)
for doc in retriever.invoke("machine learning"):
    print("---")
    print(doc.page_content.strip()[:60] + "...")

Else, you can customize the prompt with a specialized PromptTemplate

from langchain_core.prompts import PromptTemplate

retriever = GoogleDriveRetriever(
    template=PromptTemplate(
        input_variables=["query"],
        # See https://developers.google.com/drive/api/guides/search-files
        template="(fullText contains '{query}') "
        "and mimeType='application/vnd.google-apps.document' "
        "and modifiedTime > '2000-01-01T00:00:00' "
        "and trashed=false",
    ),
    num_results=2,
    # See https://developers.google.com/drive/api/v3/reference/files/list
    includeItemsFromAllDrives=False,
    supportsAllDrives=False,
)
for doc in retriever.invoke("machine learning"):
    print(f"{doc.metadata['name']}:")
    print("---")
    print(doc.page_content.strip()[:60] + "...")

Use Google Drive ‘description’ metadata

Each Google Drive has a description field in metadata (see the details of a file). Use the snippets mode to return the description of selected files.

retriever = GoogleDriveRetriever(
    template="gdrive-mime-type-in-folder",
    folder_id=folder_id,
    mime_type="application/vnd.google-apps.document",  # Only Google Docs
    num_results=2,
    mode="snippets",
    includeItemsFromAllDrives=False,
    supportsAllDrives=False,
)
retriever.invoke("machine learning")

快速开始

核心组件

进阶使用

在生产环境使用

Prerequisites

Retrieve the Google Docs

Use Google Drive ‘description’ metadata

​Prerequisites

​Retrieve the Google Docs

​Use Google Drive ‘description’ metadata

Prerequisites

Retrieve the Google Docs

Use Google Drive ‘description’ metadata