vndangkhoa/zed
1
0
Fork 0
mirror of https://github.com/zed-industries/zed.git synced 2026-06-01 03:14:56 +07:00
Code Issues Projects Releases Packages Wiki Activity Actions 6
zed/crates
History
Ben Brandt 2eafa6e6aa
language_models: Remove unused language model token counting (#54177) 
Drop the `count_tokens` API and related implementations across
providers, and remove the unused `tiktoken-rs` dependency.

I was going to update the dependency becuase they finally released a fix
we needed. But then I realized we only used this api in one place, the
Rules library. And for most models it would have been wildly incorrect
becuase we use tiktoken, i.e. OpenAI tokenizers, for almost every model,
which is going to give incorrect results.

Given that, I just removed these because the difference in how we get
these has caused plenty of confusion in the past.

Self-Review Checklist:

- [x] I've reviewed my own diff for quality, security, and reliability
- [x] Unsafe blocks (if any) have justifying comments
- [x] The content is consistent with the [UI/UX
checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist)
- [x] Tests cover the new/changed behavior
- [x] Performance impact has been considered and is acceptable

Release Notes:

- N/A
2026-04-22 13:39:48 +00:00
..
acp_thread Simplify draft threads (#53940) 2026-04-15 10:08:01 +00:00
acp_tools markdown: Refactor code block copy button visibility to use enum (#52817) 2026-04-01 09:05:03 +00:00
action_log action_log: Fix race condition when committing changes (#53884) 2026-04-16 12:54:17 +00:00
activity_indicator gpui: Improve Anchored to support center position (#47154) 2026-04-21 09:01:42 +00:00
agent Fix agent default model not picking up after authentication resolve (#54397) 2026-04-21 10:49:47 +00:00
agent_servers acp: Register ACP sessions before load replay (#54431) 2026-04-22 10:29:34 +02:00
agent_settings agent: Respect favorite model settings and sync UI changes back to settings (#54318) 2026-04-20 22:40:00 +05:30
agent_ui language_models: Remove unused language model token counting (#54177) 2026-04-22 13:39:48 +00:00
ai_onboarding Simplify parallel agents onboarding (#53854) 2026-04-15 21:51:15 -03:00
anthropic language_models: Remove unused language model token counting (#54177) 2026-04-22 13:39:48 +00:00
askpass
assets
audio audio: Remove rodio audio backend (#51945) 2026-03-24 00:58:09 +01:00
auto_update auto_update: Improve Linux rsync hinting (#50637) 2026-03-18 21:31:20 -04:00
auto_update_helper Try to fix auto-updates when Explorer.exe holds Zed.exe (#50332) 2026-03-03 17:47:35 +01:00
auto_update_ui Simplify parallel agents onboarding (#53854) 2026-04-15 21:51:15 -03:00
aws_http_client
bedrock Add Claude Opus 4.7 BYOK (#54077) 2026-04-17 10:51:32 -04:00
breadcrumbs editor: Replace BreadcrumbText with HighlightedText (#51083) 2026-03-13 08:00:22 +00:00
buffer_diff action_log: Fix race condition when committing changes (#53884) 2026-04-16 12:54:17 +00:00
call Move the worktree picker to the title bar + make it always visible (#54183) 2026-04-20 09:29:51 +00:00
channel Add the ability to reorder favorited collab channels (#52649) 2026-03-30 13:07:13 +00:00
cli cli: Make zed --wait --diff not wait until window close (#54367) 2026-04-20 18:03:26 -07:00
client client: Make account_too_young() gating not apply to business orgs (#54059) 2026-04-16 19:30:27 +02:00
clock
cloud_api_client language_models: Refactor deps and extract cloud (#53270) 2026-04-07 12:28:19 -03:00
cloud_api_types collab_ui: Disable Collab panel based on organization configuration (#53567) 2026-04-10 10:15:28 -04:00
cloud_llm_client language_models: Remove unused language model token counting (#54177) 2026-04-22 13:39:48 +00:00
codestral language_model: Decouple from Zed-specific implementation details (#52913) 2026-04-02 17:06:57 -03:00
collab Clarify error message when attempting to delete channel with active participants (#54146) 2026-04-21 01:14:06 -07:00
collab_ui gpui: Improve Anchored to support center position (#47154) 2026-04-21 09:01:42 +00:00
collections Add VecMap::entry_ref (#52601) 2026-03-27 18:06:56 -07:00
command_palette fuzzy_nucleo: Add strings module and route several pickers through it (#54123) 2026-04-20 14:41:29 +02:00
command_palette_hooks
component
component_preview Simplify parallel agents onboarding (#53854) 2026-04-15 21:51:15 -03:00
context_server terminal: Use system shell for non-terminal uses (like spinning up external agents) (#51741) 2026-04-21 08:06:40 +00:00
copilot Update futures to 0.3.32 (#52910) 2026-04-05 23:05:35 +01:00
copilot_chat copilot: Wire up reasoning tokens for GPT models (#53313) 2026-04-20 14:33:18 +02:00
copilot_ui Strong app state (#52602) 2026-03-27 14:56:46 -07:00
crashes Redact string panics (#51248) 2026-03-11 03:08:39 +00:00
credentials_provider language_model: Decouple from Zed-specific implementation details (#52913) 2026-04-02 17:06:57 -03:00
csv_preview Performance tweaks (#54321) 2026-04-21 07:25:22 +00:00
dap Bump Rust version to 1.94 (#51086) 2026-03-30 09:06:59 +00:00
dap_adapters debugger: Fall back to cached adapter binaries when offline (#50928) 2026-03-16 16:29:24 +00:00
db agent: Add ability to import threads from other channels (#54002) 2026-04-15 23:25:44 +00:00
debug_adapter_extension
debugger_tools Fix cmd-e on macOS to behave more like it should (#54451) 2026-04-21 19:54:02 +00:00
debugger_ui Fix debugger start ignoring save property (#53353) 2026-04-21 19:25:21 +00:00
deepseek docs: Update mentions to "assistant panel" (#53514) 2026-04-09 10:42:21 -03:00
denoise
dev_container dev_container: Resolve compose service build args in Dockerfile expansion (#54270) 2026-04-21 16:44:44 -07:00
diagnostics Fix info colors used for hint diagnostics (#54046) 2026-04-20 08:53:15 +00:00
docs_preprocessor Remove text thread and slash command crates (#52757) 2026-03-31 17:55:05 +02:00
edit_prediction ep: Don't throw away model version in reject events (#54159) 2026-04-20 10:35:02 -04:00
edit_prediction_cli Add interleaved_reasoning option to openai compatible models (#54016) 2026-04-22 10:40:37 +00:00
edit_prediction_context Performance tweaks (#54321) 2026-04-21 07:25:22 +00:00
edit_prediction_metrics ep: Change kept_rate definition to a more intuitive one (#54306) 2026-04-20 14:03:26 +03:00
edit_prediction_types edit_prediction: Disable training data collection based on organization configuration (#53639) 2026-04-10 13:27:04 -04:00
edit_prediction_ui gpui: Improve Anchored to support center position (#47154) 2026-04-21 09:01:42 +00:00
editor editor: Don't render editor behind sticky scroll header (#54465) 2026-04-22 11:22:14 +01:00
encoding_selector Require multibuffer excerpts to be ordered and nonoverlapping (#52364) 2026-04-01 17:25:32 +00:00
env_var language_models: Refactor deps and extract cloud (#53270) 2026-04-07 12:28:19 -03:00
etw_tracing
eval_cli language_model: Decouple from Zed-specific implementation details (#52913) 2026-04-02 17:06:57 -03:00
eval_utils
explorer_command_injector
extension extension_host: Fix Windows manifest paths when uploading extensions to WSL remote (#50653) 2026-04-21 17:32:54 -04:00
extension_api extension_api: Expose preferred_line_length in LanguageSettings (#52175) 2026-03-24 19:11:30 -04:00
extension_cli extension_host: Fix Windows manifest paths when uploading extensions to WSL remote (#50653) 2026-04-21 17:32:54 -04:00
extension_host extension_host: Fix Windows manifest paths when uploading extensions to WSL remote (#50653) 2026-04-21 17:32:54 -04:00
extensions_ui gpui: Improve Anchored to support center position (#47154) 2026-04-21 09:01:42 +00:00
feature_flags Remove AgentV2FeatureFlag (#54430) 2026-04-21 09:43:44 -07:00
feature_flags_macros Feature flag overrides (#54206) 2026-04-18 06:34:19 +00:00
feedback feedback: Add action for listing installed extensions (#48664) 2026-04-16 07:45:38 +00:00
file_finder gpui: Improve Anchored to support center position (#47154) 2026-04-21 09:01:42 +00:00
file_icons
fs worktree: Fix .git modified events not being correctly filtered out (#54329) 2026-04-20 15:49:25 +00:00
fs_benchmarks
fuzzy fuzzy: Fix crash with Unicode chars whose lowercase expands to multiple codepoints (#52989) 2026-04-03 11:00:13 +00:00
fuzzy_nucleo fuzzy_nucleo: Fix out of range panic (#54371) 2026-04-21 11:01:20 +02:00
git git_panel: Load template commit message (#42827) 2026-04-16 10:14:52 -07:00
git_graph git_graph: Fix height realignment issues (#54429) 2026-04-21 15:46:25 -04:00
git_hosting_providers
git_ui git: Improve diff performance (#54435) 2026-04-22 02:42:11 +00:00
go_to_line Require multibuffer excerpts to be ordered and nonoverlapping (#52364) 2026-04-01 17:25:32 +00:00
google_ai language_models: Remove unused language model token counting (#54177) 2026-04-22 13:39:48 +00:00
gpui gpui: Improve Anchored to support center position (#47154) 2026-04-21 09:01:42 +00:00
gpui_linux Try to recover even harder from linux GPU errors (#54349) 2026-04-21 09:55:14 +02:00
gpui_macos Performance tweaks (#54321) 2026-04-21 07:25:22 +00:00
gpui_macros gpui: Make property tests deterministic by default (#54004) 2026-04-16 10:37:05 -04:00
gpui_platform agent: Fix cargo test -p agent on windows (#52311) 2026-03-24 11:05:12 +00:00
gpui_shared_string Performance tweaks (#54321) 2026-04-21 07:25:22 +00:00
gpui_tokio gpui_tokio: Use shutdown_background to avoid WASI panic on exit (#53904) 2026-04-14 18:14:31 +02:00
gpui_util
gpui_web gpui_wgpu: Add surface lifecycle methods for mobile platforms (#50815) 2026-03-31 13:16:53 +00:00
gpui_wgpu Try to recover even harder from linux GPU errors (#54349) 2026-04-21 09:55:14 +02:00
gpui_windows Fix broken is_maximized check on Windows (#54436) 2026-04-21 13:35:13 -04:00
grammars languages: Fix nested object methods missing from outline panel (#50754) 2026-04-21 00:36:39 +02:00
html_to_markdown
http_client http_client: Fix GitHub download unpack failures on some filesystems (#53286) 2026-04-07 11:19:21 +05:30
http_client_tls
icons editor: Introduce Bookmarks (#54174) 2026-04-17 13:54:43 +02:00
image_viewer gpui: Implement pinch event support for X11 and Windows (#51354) 2026-03-28 18:41:33 -04:00
input_latency_ui gpui: Record input-to-frame latency histogram (#53822) 2026-04-16 17:38:58 +00:00
inspector_ui editor: Introduce Bookmarks (#54174) 2026-04-17 13:54:43 +02:00
install_cli
journal Implement worktree interactions for the sidebar (#51421) 2026-03-12 15:34:09 -07:00
json_schema_store Feature flag overrides (#54206) 2026-04-18 06:34:19 +00:00
keymap_editor gpui: Improve Anchored to support center position (#47154) 2026-04-21 09:01:42 +00:00
language language: Fix slow Tree-sitter parsing (#52674) 2026-04-21 16:53:49 -07:00
language_core editor: Fix soft-wrap in auto-height editors (#54051) 2026-04-18 00:55:44 +02:00
language_extension language_core: Introduce fallback highlights (#52575) 2026-04-02 12:52:42 +00:00
language_model language_models: Remove unused language model token counting (#54177) 2026-04-22 13:39:48 +00:00
language_model_core gemini: Fix tool schema issues when type field is array (#53834) 2026-04-15 10:11:44 +02:00
language_models language_models: Remove unused language model token counting (#54177) 2026-04-22 13:39:48 +00:00
language_models_cloud language_models: Remove unused language model token counting (#54177) 2026-04-22 13:39:48 +00:00
language_onboarding Remove some outdated comments (#52432) 2026-03-25 12:07:34 -06:00
language_selector Require multibuffer excerpts to be ordered and nonoverlapping (#52364) 2026-04-01 17:25:32 +00:00
language_tools Fix cmd-e on macOS to behave more like it should (#54451) 2026-04-21 19:54:02 +00:00
languages languages: Fix nested object methods missing from outline panel (#50754) 2026-04-21 00:36:39 +02:00
line_ending_selector Require multibuffer excerpts to be ordered and nonoverlapping (#52364) 2026-04-01 17:25:32 +00:00
livekit_api
livekit_client audio: Remove rodio audio backend (#51945) 2026-03-24 00:58:09 +01:00
lmstudio ai: Add LMStudio API URL & API key support (#48309) 2026-03-06 16:49:55 +00:00
lsp lsp: Display readable gopls version in UI (#49835) 2026-03-17 08:36:37 +01:00
markdown Revert "markdown: Apply theme syntax color to inline code in rendered markdown" (#54504) 2026-04-22 13:38:18 +00:00
markdown_preview markdown: Add "Copy Link" to right-click context menu (#53758) 2026-04-21 18:26:45 -07:00
media
menu
migrator Handle new profile form in migrate_settings and migrate_language_setting (#53681) 2026-04-22 15:43:53 +05:30
miniprofiler_ui theme: Split out theme_settings crate (#52569) 2026-03-27 14:41:25 +01:00
mistral docs: Update mentions to "assistant panel" (#53514) 2026-04-09 10:42:21 -03:00
multi_buffer git: Improve diff performance (#54435) 2026-04-22 02:42:11 +00:00
nc
net
node_runtime acp: Use npm --prefix for registry npx agents (#53560) 2026-04-14 11:10:31 +00:00
notifications Simplify parallel agents onboarding (#53854) 2026-04-15 21:51:15 -03:00
ollama Prefer Ollama configured model context length over model architecture context length (#53543) 2026-04-15 02:21:56 -04:00
onboarding onboarding: Ensure scrollbar is shown at the proper edge (#54392) 2026-04-21 09:27:12 -03:00
open_ai language_models: Remove unused language model token counting (#54177) 2026-04-22 13:39:48 +00:00
open_path_prompt theme: Split out theme_settings crate (#52569) 2026-03-27 14:41:25 +01:00
open_router language_models: Refactor deps and extract cloud (#53270) 2026-04-07 12:28:19 -03:00
opencode Add Claude Opus 4.7 BYOK (#54077) 2026-04-17 10:51:32 -04:00
outline Require multibuffer excerpts to be ordered and nonoverlapping (#52364) 2026-04-01 17:25:32 +00:00
outline_panel gpui: Improve Anchored to support center position (#47154) 2026-04-21 09:01:42 +00:00
panel ui: Refactor the Button component icon methods (#51496) 2026-03-13 14:13:12 -03:00
paths Fix Node.js language servers failing with --user-data-dir on Windows (#50767) 2026-04-21 07:57:35 +02:00
picker gpui: Improve Anchored to support center position (#47154) 2026-04-21 09:01:42 +00:00
platform_title_bar Remove Agent V2 feature flag (#52792) 2026-04-08 00:34:54 +00:00
prettier Fix FormatSelections to only format selected ranges, not entire document (#51593) 2026-03-31 12:48:22 +03:00
project Fix blank git diff view on Windows when working in a subfolder (#52234) 2026-04-21 18:24:49 -04:00
project_benchmarks
project_panel project_panel: Allow New File from an empty hidden-root project (#53947) 2026-04-22 10:05:30 +00:00
project_symbols Fix project symbol picker UTF-8 highlight panic (#53485) 2026-04-09 16:38:07 +00:00
prompt_store zed_agent: Pick rules file in order described in docs (#52495) 2026-03-26 13:48:18 +00:00
proto agent: Support remote archive thread git operations (#54001) 2026-04-15 19:55:37 +00:00
recent_projects recent_projects: Fix remote projects dialog not showing vertical scrollbar for overflow content (#54425) 2026-04-21 20:39:06 -03:00
refineable
release_channel agent: Add ability to import threads from other channels (#54002) 2026-04-15 23:25:44 +00:00
remote remote: Reuse existing SSH ControlMaster sessions (#51604) 2026-04-21 07:44:24 +00:00
remote_connection agent: Fix find_or_create repository to support remote (#54074) 2026-04-16 16:36:09 -07:00
remote_server repl: Unify kernel searching in remote and wsl (#53049) 2026-04-21 09:24:09 +02:00
repl Bump jupyter-websocket-client to 1.1.0 (#54442) 2026-04-21 23:39:53 +00:00
reqwest_client Remove unreferenced dev dependencies (#51093) 2026-03-09 13:22:12 +01:00
rope Swap arrayvec crate for heapless to use LenT optimization (#47101) 2026-03-21 14:05:30 +01:00
rpc agent_ui: Unify draft and background threads into retained threads (#53737) 2026-04-13 09:07:09 +00:00
rules_library language_models: Remove unused language model token counting (#54177) 2026-04-22 13:39:48 +00:00
scheduler Remove Executor::close() (#50970) 2026-03-07 04:11:45 +00:00
schema_generator theme: Split out theme_settings crate (#52569) 2026-03-27 14:41:25 +01:00
search Fix cmd-e on macOS to behave more like it should (#54451) 2026-04-21 19:54:02 +00:00
session Fix shared database test isolation (#51809) 2026-03-18 12:05:57 -07:00
settings Fix rules files not loading and config file rescan clearing tokens (#53659) 2026-04-21 08:30:31 +00:00
settings_content Add interleaved_reasoning option to openai compatible models (#54016) 2026-04-22 10:40:37 +00:00
settings_json Remove some outdated comments (#52432) 2026-03-25 12:07:34 -06:00
settings_macros
settings_profile_selector Settings profile base option (#52456) 2026-04-02 00:44:53 +00:00
settings_ui Remove AgentV2FeatureFlag (#54430) 2026-04-21 09:43:44 -07:00
shell_command_parser Reject shell substitutions in terminal tool commands (#51689) 2026-03-16 23:49:34 -04:00
sidebar sidebar: Add scrollbar and max-height to the thread switcher (#54471) 2026-04-21 23:16:56 -03:00
snippet
snippet_provider
snippets_ui
sqlez sqlez: Open named in-memory databases as SQLite URIs (#50967) 2026-03-10 20:25:12 +00:00
sqlez_macros
streaming_diff
sum_tree Swap arrayvec crate for heapless to use LenT optimization (#47101) 2026-03-21 14:05:30 +01:00
svg_preview gpui: Fix BGRA conversion for SVG rendering (#52641) 2026-03-29 14:43:05 +00:00
syntax_theme Extract syntax_theme crate (#52798) 2026-03-31 08:06:37 -06:00
system_specs
tab_switcher fuzzy_nucleo: Add strings module and route several pickers through it (#54123) 2026-04-20 14:41:29 +02:00
task Don't save buffers by default when running tasks (#52976) 2026-04-02 13:09:29 +00:00
tasks_ui tasks_ui: Fix previously used task tooltip (#53104) 2026-04-07 06:14:00 +00:00
telemetry
telemetry_events
terminal Fix terminal path detection inside parentheses (#52222) 2026-04-21 08:11:30 +00:00
terminal_view Fix cmd-e on macOS to behave more like it should (#54451) 2026-04-21 19:54:02 +00:00
text Require multibuffer excerpts to be ordered and nonoverlapping (#52364) 2026-04-01 17:25:32 +00:00
theme theme: Associate .mlx extensions with OCaml (#54197) 2026-04-20 12:51:10 +02:00
theme_extension theme: Split out theme_settings crate (#52569) 2026-03-27 14:41:25 +01:00
theme_importer theme: Split out theme_settings crate (#52569) 2026-03-27 14:41:25 +01:00
theme_selector theme_selector: Preserve selected theme after empty filter (#52461) 2026-04-07 23:41:34 +00:00
theme_settings Restrict mouse wheel zoom for certain editors (#53598) 2026-04-10 09:20:38 +00:00
time_format
title_bar title_bar: Fix worktree label showing main when opening a worktree directly (#54406) 2026-04-21 09:14:10 -03:00
toolchain_selector Require multibuffer excerpts to be ordered and nonoverlapping (#52364) 2026-04-01 17:25:32 +00:00
ui sidebar: Add scrollbar and max-height to the thread switcher (#54471) 2026-04-21 23:16:56 -03:00
ui_input agent_ui: Mask API key input in Add LLM provider modal (#50379) 2026-03-13 19:37:18 -03:00
ui_macros ui: Make UI independent from settings crate (#52578) 2026-03-27 15:31:49 +00:00
ui_prompt theme: Split out theme_settings crate (#52569) 2026-03-27 14:41:25 +01:00
util extension_host: Fix Windows manifest paths when uploading extensions to WSL remote (#50653) 2026-04-21 17:32:54 -04:00
util_macros
vercel docs: Update mentions to "assistant panel" (#53514) 2026-04-09 10:42:21 -03:00
vim vim: Ensure vgl does not select newline in helix mode (#54238) 2026-04-22 13:08:32 +00:00
vim_mode_setting workspace: Only suppress auto-save on focus transfer for the Vim/Helix command palette (#51949) 2026-03-19 12:53:26 -04:00
watch Remove unreferenced dev dependencies (#51093) 2026-03-09 13:22:12 +01:00
web_search
web_search_providers language_models: Refactor deps and extract cloud (#53270) 2026-04-07 12:28:19 -03:00
which_key theme: Split out theme_settings crate (#52569) 2026-03-27 14:41:25 +01:00
workspace Close the empty project before adding a project to a window (#54450) 2026-04-21 13:09:34 -07:00
worktree git: Revert skipping of events for the .git directory itself (#54443) 2026-04-21 18:54:31 +00:00
worktree_benchmarks
x_ai language_models: Remove unused language model token counting (#54177) 2026-04-22 13:39:48 +00:00
zed workspace: Fix recent-projects cleanup wiping active workspaces (#54224) 2026-04-21 16:44:18 +00:00
zed_actions Fix cmd-e on macOS to behave more like it should (#54451) 2026-04-21 19:54:02 +00:00
zed_credentials_provider language_model: Decouple from Zed-specific implementation details (#52913) 2026-04-02 17:06:57 -03:00
zed_env_vars language_model: Decouple from Zed-specific implementation details (#52913) 2026-04-02 17:06:57 -03:00
zeta_prompt Fix missing V0331SeedModelPy references (#54500) 2026-04-22 12:48:01 +00:00
zlog zlog: Fix log file rotation never truncating original file (#50571) 2026-03-03 08:37:58 +00:00
zlog_settings
ztracing
ztracing_macro