2 commits

Author	SHA1	Message	Date
Nathan Sobo	3ce0cd11ec	Extract language_core and grammars crates from language (#52238) ... This extracts a `language_core` crate from the existing `language` crate, and creates a `grammars` data crate. The goal is to separate tree-sitter grammar infrastructure, language configuration, and LSP adapter types from the heavier buffer/editor integration layer in `language`. ## Motivation The `language` crate pulls in `text`, `theme`, `settings`, `rpc`, `task`, `fs`, `clock`, `sum_tree`, and `fuzzy` — all of which are needed for buffer integration (`Buffer`, `SyntaxMap`, `Outline`, `DiagnosticSet`) but not for grammar parsing or language configuration. Extracting the core types lets downstream consumers depend on `language_core` without pulling in the full integration stack. ## Dependency graph after extraction ``` language_core ← gpui, lsp, tree-sitter, util, collections grammars ← language_core, rust_embed, tree-sitter-{rust,python,...} language ← language_core, text, theme, settings, rpc, task, fs, ... languages ← language, grammars ``` ## What moved to `language_core` - `Grammar`, `GrammarId`, and all query config/builder types - `LanguageConfig`, `LanguageMatcher`, bracket/comment/indent config types - `HighlightMap`, `HighlightId` (theme-dependent free functions `highlight_style` and `highlight_name` stay in `language`) - `LanguageName`, `LanguageId` - `LanguageQueries`, `QUERY_FILENAME_PREFIXES` - `CodeLabel`, `CodeLabelBuilder`, `Symbol` - `Diagnostic`, `DiagnosticSourceKind` - `Toolchain`, `ToolchainScope`, `ToolchainList`, `ToolchainMetadata` - `ManifestName` - `SoftWrap` - LSP data types: `BinaryStatus`, `ServerHealth`, `LanguageServerStatusUpdate`, `PromptResponseContext`, `ToLspPosition` ## What stays in `language` - `Buffer`, `BufferSnapshot`, `SyntaxMap`, `Outline`, `DiagnosticSet`, `LanguageScope` - `LspAdapter`, `CachedLspAdapter`, `LspAdapterDelegate` (reference `Arc<Language>` and `WorktreeId`) - `ToolchainLister`, `LanguageToolchainStore` (reference `task` and `settings` types) - `ManifestQuery`, `ManifestProvider`, `ManifestDelegate` (reference `WorktreeId`) - Parser/query cursor pools, `PLAIN_TEXT`, point conversion functions ## What the `grammars` crate provides - Embedded `.scm` query files and `config.toml` files for all built-in languages (via `rust_embed`) - `load_queries(name)`, `load_config(name)`, `load_config_for_feature(name, grammars_loaded)`, and `get_file(path)` functions - `native_grammars()` for tree-sitter grammar registration (behind `load-grammars` feature) ## Pre-cleanup (also in this PR) - Removed unused `Option<&Buffer>` from `LspAdapter::process_diagnostics` - Removed unused `&App` from `LspAdapter::retain_old_diagnostic` - Removed `fs: &dyn Fs` from `ToolchainLister` trait methods (`PythonToolchainProvider` captures `fs` at construction time instead) - Moved `Diagnostic`/`DiagnosticSourceKind` out of `buffer.rs` into their own module ## Backward compatibility The `language` crate re-exports everything from `language_core`, so existing `use language::Grammar` (etc.) continues to work unchanged. The only downstream change required is importing `CodeLabelExt` where `.fallback_for_completion()` is called on the now-foreign `CodeLabel` type. Release Notes: - N/A --------- Co-authored-by: Agus Zubiaga <agus@zed.dev> Co-authored-by: Tom Houlé <tom@tomhoule.com>	2026-03-25 23:41:09 +00:00
Finn Evers	e63bd1ab32	language: Improve highlight map resolution (#52183) ... This PR refactors the highlight map capture name resolution to be faster and more predictable. Speficically, - it changes the capture name matching to explicit prefix matching (e.g., `function.call.whatever.jsx` will now be matched by only `function`, `function.call`, `function.call.whatever` and `function.call.whatever.jsx`). This matches the behavior VSCode has - resolving highlights is now much more efficient, as we now look up captures in a BTreeMap as opposed to searching in a Vector for these. This substantially improves the performance for resolving capture names against themes. With the benchmark added here for creating the HighlightMap, we see quite some improvements: ``` Running benches/highlight_map.rs (target/release/deps/highlight_map-f99da68650aac85b) HighlightMap::new/small_captures/small_theme time: [161.90 ns 162.70 ns 163.55 ns] change: [-39.027% -38.352% -37.742%] (p = 0.00 < 0.05) Performance has improved. Found 3 outliers among 100 measurements (3.00%) 3 (3.00%) high mild HighlightMap::new/small_captures/large_theme time: [231.37 ns 233.02 ns 234.70 ns] change: [-91.570% -91.516% -91.464%] (p = 0.00 < 0.05) Performance has improved. HighlightMap::new/large_captures/small_theme time: [991.82 ns 994.94 ns 998.50 ns] change: [-50.670% -50.443% -50.220%] (p = 0.00 < 0.05) Performance has improved. Found 5 outliers among 100 measurements (5.00%) 5 (5.00%) high mild HighlightMap::new/large_captures/large_theme time: [1.6528 µs 1.6650 µs 1.6784 µs] change: [-91.684% -91.637% -91.593%] (p = 0.00 < 0.05) Performance has improved. Found 1 outliers among 100 measurements (1.00%) 1 (1.00%) low mild ``` For large themes and many capture names, the revised approach is much faster. With that in place, we can also add better fallbacks whenever we change tokens, since e.g. a change from `@variable` to `@preproc` would previously cause tokens to not be highlighted at all, whereas now we can add fallbacks for such cases more efficiently. I'll add this later on to this PR. ## Self-Review Checklist <!-- Check before requesting review: --> - [X] I've reviewed my own diff for quality, security, and reliability - [ ] Unsafe blocks (if any) have justifying comments - [ ] The content is consistent with the [UI/UX checklist](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist) - [X] Tests cover the new/changed behavior - [X] Performance impact has been considered and is acceptable Release Notes: - Improved resolution speed of theme highlight capture names. This might change highlighting in some rare edge cases, but should overall make highlighting more predicatable. Theme captures will now follow a strict prefix matching, so e.g. function.call.decorator.jsx` will now be matched by only `function`, `function.call`, `function.call.decorator` and `function.call.decorator.jsx` with the most specific capture always taking precedence. --------- Co-authored-by: Piotr Osiewicz <24362066+osiewicz@users.noreply.github.com> Co-authored-by: Gaauwe Rombouts <mail@grombouts.nl>	2026-03-25 17:01:17 +01:00