language_models: Honor images capability for custom OpenAI models (#54223)

## Summary

Users who add custom OpenAI models under
`language_models.openai.available_models` can set `capabilities.images:
true` to declare that the endpoint accepts image inputs. Today, that
setting is silently ignored: the Agent panel's image-attach button stays
disabled regardless, and the only workaround is to switch to a built-in
OpenAI model, attach the image, and switch back.

Root cause: `Model::Custom` does not carry a `supports_images` field,
and the OpenAI provider's `supports_images()` for the `Custom` arm
hardcodes `false`.

## Changes

1. `crates/settings_content/src/language_model.rs`: add `images: bool`
to `OpenAiModelCapabilities` with `#[serde(default)]` so existing
settings.json files keep working unchanged.
2. `crates/open_ai/src/open_ai.rs`: add `supports_images: bool` to
`Model::Custom` with a matching serde default.
3. `crates/language_models/src/provider/open_ai.rs`: pass
`model.capabilities.images` into the `Model::Custom` variant in
`provided_models`, and return the stored value from `supports_images()`
for `Custom`.

Existing `Model::Custom { .. }` match sites (`completion.rs:829`,
various in `open_ai.rs`) all use `..` so they continue to compile
without change.

## Testing

- `cargo check -p settings_content -p open_ai -p language_models`:
clean.
- I was not able to complete `./script/clippy` locally: the build
stalled on the first-time `webrtc-sys` download for livekit-rust-sdks
(TLS close_notify failure on docs.rs mirror). Happy to rerun once CI has
cached artifacts.
- Manually verified the capability plumbing by tracing: settings.json ->
`OpenAiModelCapabilities.images` -> `Model::Custom { supports_images }`
-> `supports_images()` -> `Thread::prompt_capabilities` ->
`SessionCapabilities.supports_images()` -> `build_add_context_menu` gate
in `thread_view.rs`.

## Related Issues

Closes #50752

Release Notes:

- Fixed custom OpenAI models ignoring the `capabilities.images` setting
in `language_models.openai.available_models`.

This contribution was developed with AI assistance (Codex).

---------

Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com>
This commit is contained in:
Matt Van Horn 2026-04-24 03:36:46 -07:00 committed by GitHub
parent c5a2807492
commit adab7b8871
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 14 additions and 5 deletions

View file

@ -182,6 +182,7 @@ impl LanguageModelProvider for OpenAiLanguageModelProvider {
max_completion_tokens: model.max_completion_tokens,
reasoning_effort: model.reasoning_effort,
supports_chat_completions: model.capabilities.chat_completions,
supports_images: model.capabilities.images,
},
);
}
@ -328,11 +329,10 @@ impl LanguageModel for OpenAiLanguageModel {
| Model::FivePointFourPro
| Model::O1
| Model::O3 => true,
Model::ThreePointFiveTurbo
| Model::Four
| Model::FourTurbo
| Model::O3Mini
| Model::Custom { .. } => false,
Model::ThreePointFiveTurbo | Model::Four | Model::FourTurbo | Model::O3Mini => false,
Model::Custom {
supports_images, ..
} => *supports_images,
}
}

View file

@ -106,6 +106,8 @@ pub enum Model {
reasoning_effort: Option<ReasoningEffort>,
#[serde(default = "default_supports_chat_completions")]
supports_chat_completions: bool,
#[serde(default = "default_supports_images")]
supports_images: bool,
},
}
@ -113,6 +115,10 @@ const fn default_supports_chat_completions() -> bool {
true
}
const fn default_supports_images() -> bool {
true
}
impl Model {
pub fn default_fast() -> Self {
Self::FiveMini

View file

@ -257,12 +257,15 @@ pub struct OpenAiCompatibleSettingsContent {
pub struct OpenAiModelCapabilities {
#[serde(default = "default_true")]
pub chat_completions: bool,
#[serde(default = "default_true")]
pub images: bool,
}
impl Default for OpenAiModelCapabilities {
fn default() -> Self {
Self {
chat_completions: default_true(),
images: default_true(),
}
}
}