feat(doc-symbols): Dart/Flutter, Go, Rust, R providers (all ecosystems) #18

Merged

raisondetredev merged 2 commits from refs/pull/18/head into main

2026-06-25 22:23:47 +02:00

raisondetredev commented

2026-06-25 21:33:17 +02:00

(Migrated from codeberg.org)

Phase 2/3 of "Link to docs" (#17 was Phase 1 / Python): the remaining ecosystem providers, each registered + verified against live sources via dry-run.

Ecosystem	Source	Symbols (live)
Dart/Flutter	dartdoc `index.json` (api.dart.dev core + api.flutter.dev)	~84,083
Go	stdlib `api/go1*.txt` promise files → `pkg.go.dev#anchor`	~81,518
R	official pkgdown `search.json` (tidyverse) — never rdocumentation.org/DataCamp	~661
Rust	curated std/prelude (rustdoc's index is a hashed/version-fragile blob)	~65

Each is a DocSymbolProvider registered in kDocSymbolProviders; adding an ecosystem stays a curator-only change.
Dart: kind derived from the href; shared dart: core dedups to api.dart.dev. Go: Type.Method anchor for methods, full import-path folded into search_blob. R: /reference/ topics only + a hard guard (+ test) against the commercial hosts. Rust: official doc.rust-lang.org/std URLs (spot-checked 200); comprehensive crate coverage (docs.rs rustdoc JSON) is a later expansion.
Parser unit tests per provider + live dry-runs.

App/infra unchanged. To go live: re-run the seed (defaults to all registered ecosystems):
dart run halkyon_curator:seed_doc_symbols --supabase-url https://<ref>.supabase.co (--dry-run first).

Update — CRAN + popularity (commit db9ce54). Adds a popularity weight (download/usage count) to every row, and a new RCranProvider that indexes all of CRAN (~24k packages from src/contrib/PACKAGES) linked to official cran.r-project.org/package= pages, weighted by cranlogs monthly downloads; pkgdown topics inherit their package's count. Live dry-run: 24,755 R rows. Pairs with infra project-halkyon/halkyon-infra#66 (popularity column + ranking) and app project-halkyon/halkyon-learn#687 (exact-match hint).

Phase 2/3 of **"Link to docs"** (#17 was Phase 1 / Python): the remaining ecosystem providers, each registered + **verified against live sources** via dry-run. | Ecosystem | Source | Symbols (live) | |---|---|---| | **Dart/Flutter** | dartdoc `index.json` (api.dart.dev core + api.flutter.dev) | ~84,083 | | **Go** | stdlib `api/go1*.txt` promise files → `pkg.go.dev#anchor` | ~81,518 | | **R** | official pkgdown `search.json` (tidyverse) — **never** rdocumentation.org/DataCamp | ~661 | | **Rust** | curated std/prelude (rustdoc's index is a hashed/version-fragile blob) | ~65 | - Each is a `DocSymbolProvider` registered in `kDocSymbolProviders`; adding an ecosystem stays a **curator-only** change. - Dart: kind derived from the href; shared `dart:` core dedups to api.dart.dev. Go: `Type.Method` anchor for methods, full import-path folded into search_blob. R: `/reference/` topics only + a hard guard (+ test) against the commercial hosts. Rust: official `doc.rust-lang.org/std` URLs (spot-checked 200); comprehensive crate coverage (docs.rs rustdoc JSON) is a later expansion. - Parser unit tests per provider + live dry-runs. **App/infra unchanged.** To go live: re-run the seed (defaults to all registered ecosystems): `dart run halkyon_curator:seed_doc_symbols --supabase-url https://<ref>.supabase.co` (`--dry-run` first). --- **Update — CRAN + popularity (commit db9ce54).** Adds a `popularity` weight (download/usage count) to every row, and a new `RCranProvider` that indexes all of CRAN (~24k packages from `src/contrib/PACKAGES`) linked to official `cran.r-project.org/package=` pages, weighted by cranlogs monthly downloads; pkgdown topics inherit their package's count. Live dry-run: 24,755 R rows. Pairs with infra project-halkyon/halkyon-infra#66 (popularity column + ranking) and app project-halkyon/halkyon-learn#687 (exact-match hint).