Setting Up Google Content Providers - ericfitz/tmi GitHub Wiki
This page is an operator walkthrough for configuring TMI's two Google content providers:
-
google_drive— service-account access to Google Drive files explicitly shared with TMI's bot identity. -
google_workspace— delegated, per-user access to a user's own Drive files via OAuth + the Google Picker.
The two providers solve different problems and use different parts of the Google Cloud Console. The first three steps are shared (Cloud project setup); after that the two diverge. You can run either one alone, or both at the same time — the registry is wired so the per-user google_workspace source wins for picker-attached docs and google_drive covers everything else.
For the architectural background of the delegated model (token storage, refresh, link/unlink endpoints, lifecycle), see the Delegated Content Providers page. This page is the operator setup guide for Google specifically.
| Provider | Auth model | Who can read the doc | When to use |
|---|---|---|---|
google_drive (service-account) |
One bot/service account, no end-user OAuth | Only docs explicitly shared with the SA's email | Shared corporate drives where ops can grant the SA access; works without users having to link Google. |
google_workspace (delegated) |
Per-user OAuth + Google Picker SDK in the UI | The end-user's own Drive access (drive.file scope: only files they pick) |
Per-user content where each TMI user attaches their own Drive docs. Recommended for production. |
You almost never want both pointed at the same documents. Pick one, or run both with disjoint document sets.
Open the Google Cloud Console and either pick an existing project or create a new one. Note the project ID (and the project number — you'll need that for google_workspace).
Enable the Google Drive API for the project: APIs & Services → Library → search Google Drive API → Enable.
If you plan to enable google_workspace, also enable the Google Picker API in the same library.
-
IAM & Admin→Service Accounts→Create service account. - Give it a name (e.g.
tmi-content-gdrive). Note the generated email — it will look liketmi-content-gdrive@<project>.iam.gserviceaccount.com. This is theservice_account_emailvalue. - Skip the "Grant access" step — Drive permissions are per-document, not project-IAM.
- After creation, click into the SA →
Keys→Add Key→Create new key→ JSON. A.jsonfile downloads. This is thecredentials_file. - Move the JSON file somewhere TMI can read it. Treat it as a secret. Never commit it to source control.
The SA can only read Drive files explicitly shared with its email address.
-
Per-document: in Drive, right-click the file →
Share→ paste the SA email → setViewer(TMI only reads). -
Per-shared-drive: in Drive →
Shared drives→ pick a drive →Manage members→ add the SA email asViewerorContent Manager.
The service-mode google_drive provider can advertise the in-browser Google Picker so users can browse the SA's accessible files instead of pasting URLs. This is purely a client UX upgrade — it requires no new server credentials, no per-user OAuth, and no encryption key. If you skip this step, clients fall back to the URL-paste flow against the same SA.
The Picker SDK runs entirely in the browser and needs three values, all of which Google designs as public/browser-safe — they identify your application but do not authorize anything on their own. The user OAuth handshake happens client-side via Google Identity Services with PKCE; no client_secret is involved or surfaced.
-
Web OAuth Client ID —
APIs & Services→Credentials→Create Credentials→OAuth client ID→ Web application.-
Authorized JavaScript origins: add the origin(s) that host your TMI front-end (e.g.
http://localhost:4200for local dev,https://app.example.comfor production). The SDK rejects browser requests from any origin not on this list. - Authorized redirect URIs: leave empty. The Picker uses Google Identity Services with PKCE and does not need a redirect target.
- After creation, copy the Client ID (looks like
1234567890-abc.apps.googleusercontent.com). - Discard the Client Secret. It is generated alongside the client id but is not used by this flow and must not be put anywhere TMI or its config files can see — it is not browser-safe and the PKCE flow proves possession without it.
-
Authorized JavaScript origins: add the origin(s) that host your TMI front-end (e.g.
-
Picker Developer Key —
APIs & Services→Credentials→Create Credentials→ API key. Edit the new key:-
Application restrictions:
HTTP referrers→ add the same front-end origin(s) with a/*suffix (e.g.http://localhost:4200/*,https://app.example.com/*). -
API restrictions:
Restrict key→ select Google Picker API only. - Make sure the Google Picker API is enabled for the project (
APIs & Services→Library→Google Picker API→Enable). - Copy the key (looks like
AIzaSy...).
-
Application restrictions:
-
GCP Project Number —
Cloud Console→IAM & Admin→Settings. Copy the Project number (numeric, e.g.675196260523). This is not the project id string (e.g.tmi-server).
These three values become picker_config.client_id, picker_config.developer_key, and picker_config.app_id on the google_drive entry of the /config response. The server emits picker_config only when all three are set — partial configuration is treated as unconfigured and the field is omitted.
YAML (under content_sources):
content_sources:
google_drive:
enabled: true
service_account_email: "tmi-content-gdrive@<project>.iam.gserviceaccount.com"
credentials_file: "/etc/tmi/google-drive-credentials.json"
# Optional — enables the in-browser Google Picker UX (Step 2c).
# All three required to advertise picker_config; partial config is ignored.
browser_oauth_client_id: "<Web OAuth client id from step 2c.1>"
picker_developer_key: "<Picker API key from step 2c.2>"
picker_app_id: "<GCP project number from step 2c.3>"Or environment variables:
| Variable | Purpose |
|---|---|
TMI_CONTENT_SOURCE_GOOGLE_DRIVE_ENABLED |
true to enable. |
TMI_CONTENT_SOURCE_GOOGLE_DRIVE_SERVICE_ACCOUNT_EMAIL |
The SA email from step 2a. Surfaced to users in the share-with-the-bot remediation message. |
TMI_CONTENT_SOURCE_GOOGLE_DRIVE_CREDENTIALS_FILE |
Absolute path to the SA JSON key. |
TMI_CONTENT_SOURCE_GOOGLE_DRIVE_BROWSER_OAUTH_CLIENT_ID |
(Optional) Web OAuth client id from step 2c.1. Surfaced to the browser via /config. Browser-safe. |
TMI_CONTENT_SOURCE_GOOGLE_DRIVE_PICKER_DEVELOPER_KEY |
(Optional) Picker API key from step 2c.2. Surfaced to the browser via /config. Browser-safe. |
TMI_CONTENT_SOURCE_GOOGLE_DRIVE_PICKER_APP_ID |
(Optional) GCP project number from step 2c.3. Surfaced to the browser via /config. Browser-safe. |
That's all google_drive needs. There is no OAuth provider entry, no encryption key, and no per-user link flow — even with the picker enabled, the picker UX uses the user's own browser session against Google and the actual file fetch still happens via the service account.
This is more involved. It uses per-user OAuth tokens, encrypted at rest, plus the Google Picker SDK in the browser.
-
APIs & Services→OAuth consent screen→ configure the app.- For an internal Google Workspace org, choose Internal.
- For external testing, choose External and add yourself as a test user.
- Add the OAuth scope
https://www.googleapis.com/auth/drive.file. TMI deliberately uses the narrowdrive.filescope so it can only read files the user explicitly picks. -
APIs & Services→Credentials→Create Credentials→OAuth client ID→ Web application. - Add an Authorized redirect URI that matches TMI's content callback. For local dev that is
http://localhost:8080/oauth2/content_callback. For production, use the public URL of your TMI deployment with the same path. - After creation, copy the Client ID and Client Secret.
The browser-side Google Picker SDK needs two non-secret values:
-
Picker Developer Key — in Cloud Console:
Credentials→Create Credentials→ API key. Restrict it to the Google Picker API and to your TMI front-end's HTTP referrers (e.g.https://app.example.com/*). -
Picker App ID — your GCP project number (not the project ID). Find it on
Cloud Console→Home→Project info→Project number.
Make sure the Google Picker API is enabled (Step 1 covers this).
Per-user tokens are stored AES-256-GCM-encrypted at rest. You need a 32-byte hex-encoded key (64 hex characters):
openssl rand -hex 32Set it via:
content_token_encryption_key: "<64 hex chars>"or env: TMI_CONTENT_TOKEN_ENCRYPTION_KEY=<64 hex chars>.
If google_workspace is enabled and this key is missing, the server refuses to start. Rotating this key invalidates all stored content tokens — every user will have to re-link their Google account.
Under content_oauth.providers, add a google_workspace entry:
content_oauth:
callback_url: "https://tmi.example.com/oauth2/content_callback"
allowed_client_callbacks:
- "https://app.example.com/*"
providers:
google_workspace:
enabled: true
name: "Google"
icon: "/static/provider-logos/signin/google.svg"
client_id: "<from step 3a>"
client_secret: "<from step 3a>"
auth_url: "https://accounts.google.com/o/oauth2/v2/auth"
token_url: "https://oauth2.googleapis.com/token"
userinfo_url: "https://openidconnect.googleapis.com/v1/userinfo"
revocation_url: "https://oauth2.googleapis.com/revoke"
required_scopes:
- "https://www.googleapis.com/auth/drive.file"
- "openid"
- "email"
extra_authorize_params:
access_type: "offline" # required to receive a refresh token
prompt: "consent" # forces refresh-token issuance on every (re)linkNotes:
-
access_type=offlineis required. Without it, Google does not return a refresh token, and users will have to re-link every hour when their access token expires. -
prompt=consentis recommended — Google only re-issues a refresh token if the user goes through the consent screen, so without this, re-linking does not produce a new refresh token. - The
revocation_urlis optional but recommended. If set, TMI calls Google's RFC 7009 endpoint when a user unlinks (DELETE /me/content_tokens/google_workspace).
Under content_sources:
content_sources:
google_workspace:
enabled: true
picker_developer_key: "<API key from step 3b>"
picker_app_id: "<GCP project number from step 3b>"Or environment variables:
| Variable | Purpose |
|---|---|
TMI_CONTENT_SOURCE_GOOGLE_WORKSPACE_ENABLED |
true to enable. |
TMI_CONTENT_SOURCE_GOOGLE_WORKSPACE_PICKER_DEVELOPER_KEY |
Picker API key from step 3b. Surfaced to the browser. |
TMI_CONTENT_SOURCE_GOOGLE_WORKSPACE_PICKER_APP_ID |
GCP project number from step 3b. Surfaced to the browser. |
Startup will fail fast if enabled=true but either picker value is empty, or the OAuth provider entry above is missing or disabled. The error message in the log identifies which check tripped.
Start the server (e.g. make dev-up) and watch the log for:
-
Content source enabled: google_drive— service-account source registered. -
Content source enabled: google_workspace (delegated, drive.file scope)— delegated source registered.
If you configured the optional browser picker bootstrap for google_drive (Step 2c), confirm the round-trip:
curl -s http://localhost:8080/config | jq '.content_providers[] | select(.id=="google_drive")'Expect a picker_config object with client_id, developer_key, and app_id. If picker_config is absent, at least one of the three fields is empty — the server emits the object only when all three are set.
For google_workspace, exercise the link flow end-to-end:
POST /me/content_tokens/google_workspace/authorize
body: {"client_callback": "https://app.example.com/content-linked"}
→ {"authorization_url": "https://accounts.google.com/...", "expires_at": "..."}
The browser navigates the user to authorization_url, they grant consent, Google redirects to /oauth2/content_callback, TMI stores the encrypted token, and the user's browser is redirected to the client_callback URL with status=success.
After a successful link:
-
GET /me/content_tokenslistsgoogle_workspacefor the calling user. -
DELETE /me/content_tokens/google_workspacerevokes the token at Google (ifrevocation_urlis configured) and removes the row.
If both google_drive and google_workspace are enabled, the registry is wired so that the delegated google_workspace source is matched before google_drive for the same Drive URLs. In practice:
- A document attached via the Google Picker (carrying a
picker_registrationpayload tied to the user's linked token) is fetched viagoogle_workspaceusing the user's own credentials. - A document with a plain Drive URL and no picker registration falls through to
google_driveand is fetched with the service account, which can only see it if the doc has been shared with the SA email.
This is by design: it lets a deployment serve both per-user content (delegated) and shared corporate content (service account) at the same time without ambiguity.
The browser-side Google Picker integration lives in tmi-ux (the web client), not in the TMI server. The TMI server's role is to advertise the bootstrap values via /config and (for delegated) to broker the OAuth flow; the SDK initialization and picker UX are client-side.
-
google_drive(service-mode picker) — consumescontent_providers[id=google_drive].picker_configfrom/config. Tracked in tmi-ux issue #671. Server side shipped in tmi#387. -
google_workspace(delegated picker) — consumespicker_developer_key/picker_app_idplus thepicker_registrationpayload sent to TMI on document attach. Tracked in tmi-ux issue #626.
The TMI server side is complete for both; the operator-facing setup on this page is what the deployment needs in any case.
For local development without a real Google account, the dev config (config-development.yml) ships a devstub OAuth provider that runs against scripts/oauth-client-callback-stub.py. You can exercise the entire delegated link flow against the stub before plugging in real Google credentials. See Delegated Content Providers for the architecture; Authentication for the OAuth callback stub.
| Symptom | Likely cause |
|---|---|
Server refuses to start: content_sources.google_workspace.enabled=true requires content-token encryption key and OAuth provider configuration
|
TMI_CONTENT_TOKEN_ENCRYPTION_KEY is unset, or content_oauth.providers.google_workspace.enabled=true is missing. |
Server refuses to start: requires content_oauth.providers.google_workspace.enabled=true
|
The OAuth provider entry exists but is disabled, or the entry is missing entirely. Add it and re-start. |
Server refuses to start: requires picker_developer_key and picker_app_id
|
content_sources.google_workspace.picker_developer_key or picker_app_id is empty. Fill in both. |
Users land on the callback with error=invalid_state
|
The OAuth state TTL is 10 minutes. The user took too long, or Redis was reset between authorize and callback. Restart the link flow. |
Users land on the callback with error=client_callback_not_allowed
|
The client_callback URL the UI supplied is not in content_oauth.allowed_client_callbacks. Add it (supports trailing-* wildcards). |
| Users have to re-link every hour | Refresh token is not being issued. Confirm access_type=offline is in extra_authorize_params, and that the user went through a consent prompt (prompt=consent on the next link). |
google_drive returns "file not found" for a document the user can see in their browser |
The file is not shared with the service-account email. Share it with the SA, or attach it via picker (delegated path) instead. |
| Users see "account needs reconnecting" after a long idle | Stored row is in status=failed_refresh — Google rejected the refresh token. The user reconnects via the link flow. |
- Delegated Content Providers — architecture, token lifecycle, link/unlink API, infra-level troubleshooting.
-
Configuration Reference — full inventory of
content_sources.*andcontent_oauth.*keys. -
Authentication — user-auth OAuth (a different subsystem under
oauth.providers.*; do not conflate withcontent_oauth.providers.*). - Tracking issue: ericfitz/tmi#249 — Confluence and OneDrive content providers (Google Workspace shipped under sub-project 4).