Skip to content

feat(#3996): guard image-output requests across Google surfaces - #4020

Open
aheritier wants to merge 1 commit into
gemini-output-capabilityfrom
gemini-image-request-guard
Open

feat(#3996): guard image-output requests across Google surfaces#4020
aheritier wants to merge 1 commit into
gemini-output-capabilityfrom
gemini-image-request-guard

Conversation

@aheritier

@aheritier aheritier commented Aug 22, 2026

Copy link
Copy Markdown
Collaborator

What and why

Request TEXT+IMAGE for eligible ordinary Google calls through the Docker models gateway, direct Gemini API, and Vertex. Reject custom function tools and structured output before dispatch; allow Google Search, Maps, and code-execution built-ins. Title generation and compaction bypass the guard and omit image response modalities. Utility behavior is unchanged. Vertex coverage is deterministic guard/predicate coverage; live Vertex validation remains deferred.

Part of #3996. Review this PR against its immediate parent, #4019, rather than the aggregate stack against main.

Commit inventory

Head: ac6390e97f8c0f8cb20f890ac59e555dccdcc0d2; parent SHA: 51559ff510bf01ea053e3923c329c80ba724dca5.

Validation

Build, test compilation, owning-package tests and the named fixture passed at this PR head.

Exact deterministic fixture command:

go test -v -count=1 ./pkg/model/provider/gemini -run 'TestCheckImageOutputRequestCompatibility|TestImageOutputRequestIncompatibleError|TestCreateChatCompletionStream_ImageOutputGuard|TestCreateChatCompletionStream_ImageResponseModalities|TestWantsImageResponseModalities'

Matched top-level tests: pkg/model/provider/gemini: 7.

Deterministic scope: the named local fixture exercises this PR boundary with disposable configuration/stores and fake or loopback providers as applicable. Every listed package ran nonzero matching top-level tests.

Deferred/live scope: No direct/gateway live reproduction was rerun; live Vertex remains deferred. The final stack head passed build, lint, full tests, an uncached full suite, focused race tests and documentation checks in disposable environments. Remote CI is tracked by the checks below; no new paid-provider or active-database validation was run.

@aheritier aheritier added area/docs Documentation changes area/providers/gemini Google Gemini provider support area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection area/tui For features/issues/fixes related to the TUI kind/feat PR adds a new feature (maps to feat:). Use on PRs only. labels Aug 22, 2026
@aheritier
aheritier force-pushed the gemini-image-request-guard branch 2 times, most recently from ee1c735 to 70bd556 Compare August 25, 2026 07:58
@aheritier
aheritier force-pushed the gemini-image-request-guard branch 2 times, most recently from 65cfdea to 890abfa Compare August 26, 2026 11:07
@aheritier
aheritier force-pushed the gemini-image-request-guard branch 2 times, most recently from 48f364f to 890abfa Compare September 1, 2026 16:18
@aheritier
aheritier force-pushed the gemini-image-request-guard branch from 890abfa to d1eeba5 Compare September 1, 2026 16:29
aheritier

This comment was marked as resolved.

@aheritier
aheritier force-pushed the gemini-image-request-guard branch from 8ba3b99 to 4f67ed5 Compare September 7, 2026 16:13
@aheritier
aheritier force-pushed the gemini-image-request-guard branch from 4f67ed5 to 0279445 Compare September 7, 2026 16:47
@aheritier
aheritier force-pushed the gemini-image-request-guard branch from 0279445 to c126514 Compare September 8, 2026 06:17
@aheritier
aheritier force-pushed the gemini-image-request-guard branch from c126514 to 2b211fa Compare September 8, 2026 07:57
@aheritier
aheritier force-pushed the gemini-image-request-guard branch from 2b211fa to c2b2abd Compare September 8, 2026 17:31
Request TEXT+IMAGE on supported Google surfaces when an explicit override
or models.dev enables image output. Before ordinary dispatch, reject
custom function tools and structured output while allowing server-side
built-ins. Title generation and compaction omit image response modalities
and bypass the guard; omission does not explicitly force a TEXT modality.

Use catalogue tool_call metadata to distinguish unsupported tools from an
image-output combination conflict. Keep unknown support conservative and
make the diagnostic accurate for either override or catalogue resolution.
Remove dead ToolConfig when no custom tools remain and keep the TUI inline
error visible without a misleading footer toast. Cover provider, runtime,
and TUI paths, explicit false, unknown support and no-dispatch failures.
@aheritier
aheritier force-pushed the gemini-image-request-guard branch from c2b2abd to ac6390e Compare September 8, 2026 20:33
@aheritier aheritier changed the title feat(#3996): reject incompatible image-output gateway requests feat(#3996): guard image-output requests across Google surfaces Sep 8, 2026
@aheritier
aheritier marked this pull request as ready for review September 8, 2026 20:38
@aheritier
aheritier requested a review from a team as a code owner September 8, 2026 20:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/docs Documentation changes area/providers/gemini Google Gemini provider support area/runtime Runtime engine, agent loop execution, tool dispatch, loop detection area/tui For features/issues/fixes related to the TUI kind/feat PR adds a new feature (maps to feat:). Use on PRs only.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant