Colour the editor view with relocated semantic tokens

The results document only ever got TextMate colouring, because a language
server will not serve semantic tokens for a `colored-refs` URI: there is no
file and no compilation behind it. So identifiers looked the way a C# file
does before the server has analysed it.

Instead of tokenizing anything ourselves, fetch the tokens for each *source*
file and move them:

- vscode.provideDocumentSemanticTokensLegend / provideDocumentSemanticTokens
  give the legend and the delta-encoded tokens for a real file. Neither is
  listed by getCommands(), so a test calls them to prove they exist rather
  than looking them up.
- Decode to absolute positions, keep the tokens on displayed lines, and shift
  each column by CODE_INDENT minus the stripped leading whitespace — the same
  mapping that places the highlight ranges. Re-encode with the source legend,
  so the type and modifier numbers stay meaningful.
- The legend is only known after a server answers, but
  registerDocumentSemanticTokensProvider wants it up front, so registration
  is deferred to the first search and redone if a later legend differs.
- Runs after the results are on screen and only changes colours, never text,
  so nothing waits on it. coloredReferences.semanticTokens turns it off;
  results over 40 files skip it.

Verified against DotRush: every relocated token covers exactly the text it
covered in the source file, and `Profiler` comes back typed as `class`.

README: drops the claim that semantic tokens would fix the panel's colours.
They supply the classification, never the colours, and webviews are not given
theme token colours — so only the editor view can be theme-exact. Feeding the
panel real token types is now roadmap item 5.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
max
2026-09-07 17:21:59 +02:00
co-authored by Claude Opus 5
parent 864f4c2c5f
commit 2a1a2a329f
6 changed files with 396 additions and 6 deletions
+26 -6
View File
@@ -3,7 +3,9 @@
Find All References results shown the way Visual Studio does it — in two views.
**Editor view** (default) writes the results into a read-only virtual document in the *same language as your
source file*, so your theme's grammar colors every line for free. File headers show the containing project
source file*, so your theme colors every line for free — including identifiers, because the semantic tokens
are fetched from the real source files and relocated into the results (see *Semantic tokens* below). File
headers show the containing project
(`.csproj`/`.fsproj`/`.vbproj`) and reference count; line numbers are shown in the gutter. Read references are
highlighted like search matches; **writes are bold** on the theme's stronger write-occurrence background, the
same colour the editor itself uses for a write.
@@ -34,12 +36,29 @@ The **All / Reads / Writes** buttons filter by kind, and the summary line counts
the text filter, so typing `write` narrows to writes as well. See *Known limitations* for where the kind comes
from.
## Semantic tokens
VS Code colors code twice: a TextMate grammar handles keywords, strings and comments from the text alone, then
the language server's *semantic tokens* recolor identifiers by what they actually are — a class, a method, a
parameter. That second pass is why a C# file settles into its final colors a moment after you open it.
Language servers will not serve semantic tokens for the results document: its `colored-refs` URI has no file
and no compilation behind it. So the tokens are fetched for each *source* file instead and relocated into
results-document coordinates — the same line and column mapping the extension already uses to place the
highlights. The colouring you get is the server's own answer, moved, rather than a guess.
This happens after the results are on screen and only ever changes colors, never text, so nothing waits on it.
Set `coloredReferences.semanticTokens` to `false` for TextMate-only coloring. Results spanning more than 40
files skip it.
## Settings
- `coloredReferences.view` — which view `Find All References (Colored)` opens: `document` (default) or `panel`
- `coloredReferences.panelLocation` — where the panel opens: `bottom` (default, docked next to Terminal /
Problems and draggable to a side bar), `beside` (editor group to the side), or `below` (editor group
underneath, so the table is wide and short)
- `coloredReferences.semanticTokens` — color the editor view with semantic tokens fetched from the source
files (default `true`)
- `coloredReferences.openBeside` — open results beside the current editor (default `true`)
- `coloredReferences.showProject` — show the containing project in file headers (default `true`)
- `coloredReferences.reuseTab` — reuse one results tab/panel instead of opening a new one per search (default `true`)
@@ -67,11 +86,10 @@ the expected results stay deterministic.
## Known limitations
- Coloring in the editor view is TextMate-only (no semantic tokens), since language servers only serve semantic
tokens for real files. Types and identifiers therefore look like they do in a freshly opened file before the
server has analyzed it.
- The panel's code column cannot use your theme's token colors: webviews are not given them as CSS variables.
It approximates the stock Dark+/Light+ hues instead. Semantic tokens (roadmap 2) would replace this.
It classifies code with a small tokenizer of its own and paints it with approximated Dark+/Light+ hues, so
it will not match your theme exactly. Semantic tokens do not fix this — they supply the *classification*
("this is a class"), never the colors — so only the editor view can be theme-exact.
- Read/write kind is not part of the reference request, so it is derived two ways. Every reference is first
classified from the surrounding text (assignment and compound-assignment operators, `++`/`--`, `ref`/`out`
arguments), then `textDocument/documentHighlight` is asked per file and its `Read`/`Write` kinds override
@@ -86,6 +104,8 @@ the expected results stay deterministic.
1. ~~Webview panel with resizable, sortable columns~~ — done; missing: virtualized rendering for very large
result sets, and remembering column layout per workspace rather than per panel
2. Semantic token overlay via `vscode.provideDocumentSemanticTokens`
2. ~~Semantic token overlay~~ — done for the editor view; the panel still uses its own tokenizer
3. Filter by project / exclude tests
4. ~~Read/write kind~~ — done in both views
5. Feed the panel's code column the real token types instead of its regex tokenizer (colors would still be
the approximated palette, but the classification would be right)