Documentation

Wikidata mapping proposal

Decision: store only GeoNames ID → Wikidata QID mappings. The consuming API Worker fetches Wikidata/Wikipedia metadata on demand. This is not implemented: there is no mapping import, mapping table, or mapping API today. GeoNames remains the canonical location source. Import inputs, location search, the current Location contract, FTS, ranking, and search cache keys stay unchanged.

See the implementation guide for the schema, mapping/update procedure, storage estimates, and operational acceptance criteria.

What is already in GeoNames

The global import parses the 19-column allCountries.txt record. It retains the GeoNames ID, name, ASCII name, alternate-name values, country code, admin1/admin2 codes, feature class/code, latitude and longitude (stored as microdegrees), population, and timezone. The importer derives normalized name and alias columns for search. It discards alternate country codes (cc2), admin3/admin4, elevation, DEM, and the source modification date; it does not create a Wikipedia-link column. The dump’s comma-separated aliases are convenience values, not language-tagged links. GeoNames countryInfo.txt contributes country reference data, but it has no country website field.

The public service Location currently returns ID, names, country/admin codes and joined names, feature class/code and joined label/description, coordinates, population, timezone, and aliases. Normalized columns remain internal. Neither a Wikidata QID, Wikipedia sitelink, image nor official website is exposed. See the GeoNames allCountries field definition, current importer import-global.ts, database schema, and service type.

Identity evidence and recommendation

Wikidata property P1566 is an external identifier, described as an identifier in the GeoNames geographical database. It is the appropriate crosswalk key, not a fuzzy name or coordinate match. Live entity JSON inspected on 2026-09-30 showed Bengaluru Q1355 has P1566 1277333 and New Delhi Q987 has P1566 1261481; each observed claim was normal rank. These two live examples agree with their GeoNames records. In this sample: two expected matches, zero conflicting P1566 values on those two entity claims, zero observed mapping conflicts. This is a two-entity spot check, not a global conflict rate or coverage estimate. Global coverage, identifier collisions, and redirect/deletion effects remain unmeasured.

First run a bounded P1566 coverage/collision pilot against the current 13,472,195 IDs before approving a global mapping import. Two examples do not establish completeness or safe update behavior. Keep unmapped IDs valid, do not guess by name/coordinates, and report ambiguous cases instead of selecting a convenient item.

P1566 may have multiple statements; its property page calls expected completeness “always incomplete” and stability “sometimes changes.” A future mapping importer must evaluate entity redirects, deleted/missing entities, claim rank, and every P1566 value. Resolve redirects to current QIDs; retain original QIDs and redirect provenance in extraction reports, not metadata columns in the published mapping. Exclude deprecated claims. If any preferred-rank claim exists, consider only that rank; otherwise consider normal-rank claims. Quarantine multiple distinct active values, conflicting QIDs for one GeoNames ID, or unresolved redirects in review reports. Never silently pick the first statement.

Mapping storage and consumer boundary

The published mapping contains exactly two fields: geoname_id and a non-null qid. Store only approved, unambiguous mappings; absence means unmapped. Do not persist labels, aliases, Wikipedia titles, article extracts, images, official websites, or entity metadata in the GeoNames store. Keep source revisions, retrieval times, redirects, rejected candidates, and collision reasons in extraction/publication reports outside the serving table.

-- Proposed schema only; no migration or binding exists.
CREATE TABLE geonames_wikidata_mapping (
  geoname_id INTEGER PRIMARY KEY,
  qid TEXT NOT NULL
);

Multiple GeoNames IDs may map to one QID, so QID is not a unique key. A GeoNames ID with conflicting candidate QIDs remains unpublished until reviewed. Versioned publication and rollback apply to the mapping artifact, not to copied entity metadata. Storage placement needs a measured pilot and approval; no separate D1/R2 resources are provisioned or required by this decision.

The GeoNames service’s responsibility stops at canonical records and the proposed identifier crosswalk. A future mapping lookup contract requires a separate implementation change. The consuming API Worker resolves the QID for a selected GeoNames ID, then calls Wikibase for labels, aliases, sitelinks, and requested claims. When Wikipedia content is needed, it uses the actual language-specific sitelink to call that wiki’s API. Do not infer an article title from the GeoNames name.

The consumer owns language selection, optional metadata caching, request limits, and failure handling. Any such cache is separate from the GeoNames search cache. Missing mappings, missing articles, deleted entities, or unavailable upstream APIs leave canonical GeoNames results usable; they must not fail search. No external metadata requests or mapping joins are added to search, FTS, or default get.

For images, the consumer must consult the Commons file page/API for current author, license, attribution, and restrictions before display. P18 is a filename, not a reuse grant; P856 is an optional website statement, not a country TLD or inferred URL. Neither is stored in the mapping. See Wikidata P18, P856, and Commons reuse guidance.

Capacity decision

The recorded remote GeoNames D1 baseline is 5,074,595,840 bytes (about 4.73 GiB). Cloudflare documents a 10 GB maximum per paid D1 database. Using a conservative decimal 10,000,000,000-byte planning ceiling leaves 4,925,404,160 bytes (4.59 GiB). These are baseline/planning figures, not a fresh remote measurement or a post-enrichment measurement.

The earlier 512–1,024-byte estimate included copied metadata and no longer applies. Mapping-only storage has an integer key and QID text; actual SQLite page/index overhead must be measured on representative rows. Coverage is unknown. Measure active mapping size, staging and rollback copies, review reports, and concurrent GeoNames growth before choosing placement. Do not claim the mapping fits or exceeds D1 based on the superseded metadata estimate.

Proposed implementation sequence (not run)

Export only P1566 claims from an approved Wikidata snapshot/API; normalize entity redirects, statement ranks, missing/deleted entities, and multiple values into explicit outcomes.
Intersect identifiers with the existing GeoNames ID set; produce matched, unmatched, duplicate/collision, ambiguous, and invalid reports. Do not use label/coordinate fallback.
Review a small pilot including the Bengaluru and New Delhi examples, ambiguous candidates, and current/deleted records. Measure mapping-only storage and choose placement.
Stage an idempotent mapping artifact with only the two serving fields. Reconcile removed/changed P1566 claims as well as additions; retain provenance in reports. Validate counts, unique GeoNames keys, QID resolution, and rejected candidates before publication.
Publish only after explicit approval and successful validation; retain the previous mapping version for rollback. No refresh schedule is implemented.
Separately implement the mapping lookup and consuming API Worker’s on-demand Wikidata/Wikipedia fetches. Verify that missing mappings and upstream failures preserve canonical location responses. No metadata backfill is part of the mapping import.

For a bounded pilot, a Wikidata Query Service VALUES query for explicit IDs avoids a global scan: PREFIX wdt: <http://www.wikidata.org/prop/direct/> SELECT ?item ?id WHERE { VALUES ?id { "1277333" "1261481" } ?item wdt:P1566 ?id }. Audit matched entities with Wikibase wbgetentities, requesting props=info|claims, to inspect full P1566 ranks/values and redirects. The truthy SPARQL projection alone is not complete audit evidence. Consumer-time metadata requests may separately use props=labels|aliases|sitelinks|claims and a requested language. These procedures are proposed, not run.

No command in this repository currently imports Wikidata. Any future extraction, staging, reconciliation, or publish commands are PROPOSED UNIMPLEMENTED and must be designed against an approved source and actual storage choice, not copied from the GeoNames import commands.