Add offline live spell checking (Hunspell via spellbook)

Spelling is now checked live as you type, fully offline, using the
pure-Rust `spellbook` crate to read Hunspell .aff/.dic dictionaries (no C
libhunspell — the binary stays self-contained; ldd unchanged).

- Canadian (en-CA, default) and British (en-GB) English dictionaries are
  compiled in; more languages are auto-discovered from system Hunspell
  folders and ~/.config/md-manuscript/dictionaries/. A "Spelling" row in
  the top bar toggles live checking and picks the dictionary.
- Misspellings are underlined in red; right-clicking a word opens a menu
  of suggested corrections. A results-panel and one-click fixes work too.
- Checks run on a background thread, debounced ~400ms after the last edit.
  Prose is extracted with pulldown-cmark so code spans, code blocks and
  link targets are skipped; ALL-CAPS initialisms are ignored.
- LanguageTool still layers on top: when its results are fresh they own
  the underlines (spelling + grammar); once you edit, the offline checker
  resumes. The editor underlines, issues panel and fix-apply logic are
  now shared between the two sources.

Bundled dictionaries are SCOWL-derived under a permissive license (kept
in dictionaries/<lang>/license). Adds spell-check unit tests (tokenizer,
code-block skipping, offset mapping, en-CA/en-GB spelling); 40 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N9kRuP7JvXoUGdNNeg5ZSs
This commit is contained in:
landon
2026-08-14 06:25:08 -05:00
parent 691d1a0515
commit 19dddf30c7
14 changed files with 101168 additions and 69 deletions
Generated
+11
View File
@@ -1858,6 +1858,7 @@ dependencies = [
"rfd", "rfd",
"serde", "serde",
"serde_json", "serde_json",
"spellbook",
"ureq", "ureq",
"zip", "zip",
] ]
@@ -3036,6 +3037,16 @@ dependencies = [
"serde", "serde",
] ]
[[package]]
name = "spellbook"
version = "0.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0d204abcbdf8e88729306a8d0ca01a79d8a49969fe1f84696909d6dbc4321c1a"
dependencies = [
"foldhash",
"hashbrown 0.17.1",
]
[[package]] [[package]]
name = "spirv" name = "spirv"
version = "0.3.0+sdk-1.3.268.0" version = "0.3.0+sdk-1.3.268.0"
+3
View File
@@ -18,6 +18,9 @@ rfd = { version = "0.15", default-features = false, features = ["xdg-portal", "a
# backend links statically (no new runtime .so deps), so both a plain-HTTP local # backend links statically (no new runtime .so deps), so both a plain-HTTP local
# server and an HTTPS domain work. # server and an HTTPS domain work.
ureq = { version = "2", default-features = false, features = ["tls"] } ureq = { version = "2", default-features = false, features = ["tls"] }
# Pure-Rust reader of Hunspell .aff/.dic dictionaries for offline spell
# checking, so the binary stays self-contained (no C libhunspell to link).
spellbook = "0.4.2"
[profile.release] [profile.release]
opt-level = 2 opt-level = 2
+30 -1
View File
@@ -110,9 +110,38 @@ The status bar shows a live **word count** for the current file, plus the net
words added (or removed) to it since the current session started. The count words added (or removed) to it since the current session started. The count
updates as you type, including unsaved edits. updates as you type, including unsaved edits.
## Spelling (offline)
Spelling is checked **live as you type**, entirely offline — no server, no
network. Misspelled words get a red underline; **right-click** one for a menu of
suggested corrections (click a suggestion to apply it). It uses Hunspell
dictionaries read by the pure-Rust [`spellbook`](https://crates.io/crates/spellbook)
crate, so the binary stays self-contained.
* **Two dictionaries are built in** — Canadian English (`en-CA`, the default)
and British English (`en-GB`). Pick one from the **Dictionary** dropdown on
the *Spelling* row of the top bar; the choice is remembered.
* **More languages** are discovered automatically from the usual Hunspell
folders (`/usr/share/hunspell`, `/usr/share/myspell`, …) and from a per-user
folder, `~/.config/md-manuscript/dictionaries/`. Drop a matching
`xx_YY.aff` + `xx_YY.dic` pair in there (e.g. from your distro's
`hunspell-de-de` package) and it appears in the dropdown.
* **Untick “Check as I type”** on the Spelling row to turn the underlines off.
* Code spans, fenced/indented code blocks and link targets are skipped, and
ALL-CAPS initialisms (ODT, HTTP) are left alone, to cut false positives.
When you run a **LanguageTool** check (below), its richer spelling-and-grammar
results take over the underlines until you next edit the text — at which point
the live offline checker resumes. In other words, LanguageTool is used when it's
available and current; the offline checker is the always-on default.
The bundled dictionaries live under `dictionaries/` and are derived from
[SCOWL](http://wordlist.sourceforge.net/) under a permissive license (kept
alongside them in each `license` file).
## Grammar & spelling (LanguageTool) ## Grammar & spelling (LanguageTool)
The editor can check the current file against a The editor can *additionally* check the current file against a
[LanguageTool](https://languagetool.org/) server — typically a **local [LanguageTool](https://languagetool.org/) server — typically a **local
instance**, so your manuscript never leaves your machine. instance**, so your manuscript never leaves your machine.
+205
View File
@@ -0,0 +1,205 @@
SET UTF-8
TRY esianrtolcdugmphbyfvkwzESIANRTOLCDUGMPHBYFVKWZ'
ICONV 1
ICONV '
NOSUGGEST !
# ordinal numbers
COMPOUNDMIN 1
# only in compounds: 1th, 2th, 3th
ONLYINCOMPOUND c
# compound rules:
# 1. [0-9]*1[0-9]th (10th, 11th, 12th, 56714th, etc.)
# 2. [0-9]*[02-9](1st|2nd|3rd|[4-9]th) (21st, 22nd, 123rd, 1234th, etc.)
COMPOUNDRULE 2
COMPOUNDRULE n*1t
COMPOUNDRULE n*mp
WORDCHARS 0123456789
PFX A Y 1
PFX A 0 re .
PFX I Y 1
PFX I 0 in .
PFX U Y 1
PFX U 0 un .
PFX C Y 1
PFX C 0 de .
PFX E Y 1
PFX E 0 dis .
PFX F Y 1
PFX F 0 con .
PFX K Y 1
PFX K 0 pro .
SFX V N 2
SFX V e ive e
SFX V 0 ive [^e]
SFX N Y 3
SFX N e ion e
SFX N y ication y
SFX N 0 en [^ey]
SFX X Y 3
SFX X e ions e
SFX X y ications y
SFX X 0 ens [^ey]
SFX H N 2
SFX H y ieth y
SFX H 0 th [^y]
SFX Y Y 1
SFX Y 0 ly .
SFX G Y 2
SFX G e ing e
SFX G 0 ing [^e]
SFX J Y 2
SFX J e ings e
SFX J 0 ings [^e]
SFX D Y 4
SFX D 0 d e
SFX D y ied [^aeiou]y
SFX D 0 ed [^ey]
SFX D 0 ed [aeiou]y
SFX T N 4
SFX T 0 st e
SFX T y iest [^aeiou]y
SFX T 0 est [aeiou]y
SFX T 0 est [^ey]
SFX R Y 4
SFX R 0 r e
SFX R y ier [^aeiou]y
SFX R 0 er [aeiou]y
SFX R 0 er [^ey]
SFX Z Y 4
SFX Z 0 rs e
SFX Z y iers [^aeiou]y
SFX Z 0 ers [aeiou]y
SFX Z 0 ers [^ey]
SFX S Y 4
SFX S y ies [^aeiou]y
SFX S 0 s [aeiou]y
SFX S 0 es [sxzh]
SFX S 0 s [^sxzhy]
SFX P Y 3
SFX P y iness [^aeiou]y
SFX P 0 ness [aeiou]y
SFX P 0 ness [^y]
SFX M Y 1
SFX M 0 's .
SFX B Y 3
SFX B 0 able [^aeiou]
SFX B 0 able ee
SFX B e able [^aeiou]e
SFX L Y 1
SFX L 0 ment .
REP 90
REP a ei
REP ei a
REP a ey
REP ey a
REP ai ie
REP ie ai
REP alot a_lot
REP are air
REP are ear
REP are eir
REP air are
REP air ere
REP ere air
REP ere ear
REP ere eir
REP ear are
REP ear air
REP ear ere
REP eir are
REP eir ere
REP ch te
REP te ch
REP ch ti
REP ti ch
REP ch tu
REP tu ch
REP ch s
REP s ch
REP ch k
REP k ch
REP f ph
REP ph f
REP gh f
REP f gh
REP i igh
REP igh i
REP i uy
REP uy i
REP i ee
REP ee i
REP j di
REP di j
REP j gg
REP gg j
REP j ge
REP ge j
REP s ti
REP ti s
REP s ci
REP ci s
REP k cc
REP cc k
REP k qu
REP qu k
REP kw qu
REP o eau
REP eau o
REP o ew
REP ew o
REP oo ew
REP ew oo
REP ew ui
REP ui ew
REP oo ui
REP ui oo
REP ew u
REP u ew
REP oo u
REP u oo
REP u oe
REP oe u
REP u ieu
REP ieu u
REP ue ew
REP ew ue
REP uff ough
REP oo ieu
REP ieu oo
REP ier ear
REP ear ier
REP ear air
REP air ear
REP w qu
REP qu w
REP z ss
REP ss z
REP shun tion
REP shun sion
REP shun cion
REP size cise
File diff suppressed because it is too large Load Diff
+347
View File
@@ -0,0 +1,347 @@
en_CA Hunspell Dictionary
Version 2020.12.07
Mon Dec 7 20:14:35 2020 -0500 [5ef55f9]
http://wordlist.sourceforge.net
README file for English Hunspell dictionaries derived from SCOWL.
These dictionaries are created using the speller/make-hunspell-dict
script in SCOWL.
The following dictionaries are available:
en_US (American)
en_CA (Canadian)
en_GB-ise (British with "ise" spelling)
en_GB-ize (British with "ize" spelling)
en_AU (Australian)
en_US-large
en_CA-large
en_GB-large (with both "ise" and "ize" spelling)
en_AU-large
The normal (non-large) dictionaries correspond to SCOWL size 60 and,
to encourage consistent spelling, generally only include one spelling
variant for a word. The large dictionaries correspond to SCOWL size
70 and may include multiple spelling for a word when both variants are
considered almost equal. The larger dictionaries however (1) have not
been as carefully checked for errors as the normal dictionaries and
thus may contain misspelled or invalid words; and (2) contain
uncommon, yet valid, words that might cause problems as they are
likely to be misspellings of more common words (for example, "ort" and
"calender").
To get an idea of the difference in size, here are 25 random words
only found in the large dictionary for American English:
Bermejo Freyr's Guenevere Hatshepsut Nottinghamshire arrestment
crassitudes crural dogwatches errorless fetial flaxseeds godroon
incretion jalapeño's kelpie kishkes neuroglias pietisms pullulation
stemwinder stenoses syce thalassic zees
The en_US, en_CA and en_AU are the official dictionaries for Hunspell.
The en_GB and large dictionaries are made available on an experimental
basis. If you find them useful please send me a quick email at
kevina@gnu.org.
If none of these dictionaries suite you (for example, maybe you want
the normal dictionary that also includes common variants) additional
dictionaries can be generated at http://app.aspell.net/create or by
modifying speller/make-hunspell-dict in SCOWL. Please do let me know
if you end up publishing a customized dictionary.
If a word is not found in the dictionary or a word is there you think
shouldn't be, you can lookup the word up at http://app.aspell.net/lookup
to help determine why that is.
General comments on these list can be sent directly to me at
kevina@gnu.org or to the wordlist-devel mailing lists
(https://lists.sourceforge.net/lists/listinfo/wordlist-devel). If you
have specific issues with any of these dictionaries please file a bug
report at https://github.com/kevina/wordlist/issues.
IMPORTANT CHANGES INTRODUCED In 2016.11.20:
New Australian dictionaries thanks to the work of Benjamin Titze
(btitze@protonmail.ch).
IMPORTANT CHANGES INTRODUCED IN 2016.04.24:
The dictionaries are now in UTF-8 format instead of ISO-8859-1. This
was required to handle smart quotes correctly.
IMPORTANT CHANGES INTRODUCED IN 2016.01.19:
"SET UTF8" was changes to "SET UTF-8" in the affix file as some
versions of Hunspell do not recognize "UTF8".
ADDITIONAL NOTES:
The NOSUGGEST flag was added to certain taboo words. While I made an
honest attempt to flag the strongest taboo words with the NOSUGGEST
flag, I MAKE NO GUARANTEE THAT I FLAGGED EVERY POSSIBLE TABOO WORD.
The list was originally derived from Németh László, however I removed
some words which, while being considered taboo by some dictionaries,
are not really considered swear words in today's society.
COPYRIGHT, SOURCES, and CREDITS:
The English dictionaries come directly from SCOWL
and is thus under the same copyright of SCOWL. The affix file is
a heavily modified version of the original english.aff file which was
released as part of Geoff Kuenning's Ispell and as such is covered by
his BSD license. Part of SCOWL is also based on Ispell thus the
Ispell copyright is included with the SCOWL copyright.
The collective work is Copyright 2000-2018 by Kevin Atkinson as well
as any of the copyrights mentioned below:
Copyright 2000-2018 by Kevin Atkinson
Permission to use, copy, modify, distribute and sell these word
lists, the associated scripts, the output created from the scripts,
and its documentation for any purpose is hereby granted without fee,
provided that the above copyright notice appears in all copies and
that both that copyright notice and this permission notice appear in
supporting documentation. Kevin Atkinson makes no representations
about the suitability of this array for any purpose. It is provided
"as is" without express or implied warranty.
Alan Beale <biljir@pobox.com> also deserves special credit as he has,
in addition to providing the 12Dicts package and being a major
contributor to the ENABLE word list, given me an incredible amount of
feedback and created a number of special lists (those found in the
Supplement) in order to help improve the overall quality of SCOWL.
The 10 level includes the 1000 most common English words (according to
the Moby (TM) Words II [MWords] package), a subset of the 1000 most
common words on the Internet (again, according to Moby Words II), and
frequently class 16 from Brian Kelk's "UK English Wordlist
with Frequency Classification".
The MWords package was explicitly placed in the public domain:
The Moby lexicon project is complete and has
been place into the public domain. Use, sell,
rework, excerpt and use in any way on any platform.
Placing this material on internal or public servers is
also encouraged. The compiler is not aware of any
export restrictions so freely distribute world-wide.
You can verify the public domain status by contacting
Grady Ward
3449 Martha Ct.
Arcata, CA 95521-4884
grady@netcom.com
grady@northcoast.com
The "UK English Wordlist With Frequency Classification" is also in the
Public Domain:
Date: Sat, 08 Jul 2000 20:27:21 +0100
From: Brian Kelk <Brian.Kelk@cl.cam.ac.uk>
> I was wondering what the copyright status of your "UK English
> Wordlist With Frequency Classification" word list as it seems to
> be lacking any copyright notice.
There were many many sources in total, but any text marked
"copyright" was avoided. Locally-written documentation was one
source. An earlier version of the list resided in a filespace called
PUBLIC on the University mainframe, because it was considered public
domain.
Date: Tue, 11 Jul 2000 19:31:34 +0100
> So are you saying your word list is also in the public domain?
That is the intention.
The 20 level includes frequency classes 7-15 from Brian's word list.
The 35 level includes frequency classes 2-6 and words appearing in at
least 11 of 12 dictionaries as indicated in the 12Dicts package. All
words from the 12Dicts package have had likely inflections added via
my inflection database.
The 12Dicts package and Supplement is in the Public Domain.
The WordNet database, which was used in the creation of the
Inflections database, is under the following copyright:
This software and database is being provided to you, the LICENSEE,
by Princeton University under the following license. By obtaining,
using and/or copying this software and database, you agree that you
have read, understood, and will comply with these terms and
conditions.:
Permission to use, copy, modify and distribute this software and
database and its documentation for any purpose and without fee or
royalty is hereby granted, provided that you agree to comply with
the following copyright notice and statements, including the
disclaimer, and that the same appear on ALL copies of the software,
database and documentation, including modifications that you make
for internal use or for distribution.
WordNet 1.6 Copyright 1997 by Princeton University. All rights
reserved.
THIS SOFTWARE AND DATABASE IS PROVIDED "AS IS" AND PRINCETON
UNIVERSITY MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PRINCETON
UNIVERSITY MAKES NO REPRESENTATIONS OR WARRANTIES OF MERCHANT-
ABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE
LICENSED SOFTWARE, DATABASE OR DOCUMENTATION WILL NOT INFRINGE ANY
THIRD PARTY PATENTS, COPYRIGHTS, TRADEMARKS OR OTHER RIGHTS.
The name of Princeton University or Princeton may not be used in
advertising or publicity pertaining to distribution of the software
and/or database. Title to copyright in this software, database and
any associated documentation shall at all times remain with
Princeton University and LICENSEE agrees to preserve same.
The 40 level includes words from Alan's 3esl list found in version 4.0
of his 12dicts package. Like his other stuff the 3esl list is also in the
public domain.
The 50 level includes Brian's frequency class 1, words appearing
in at least 5 of 12 of the dictionaries as indicated in the 12Dicts
package, and uppercase words in at least 4 of the previous 12
dictionaries. A decent number of proper names is also included: The
top 1000 male, female, and Last names from the 1990 Census report; a
list of names sent to me by Alan Beale; and a few names that I added
myself. Finally a small list of abbreviations not commonly found in
other word lists is included.
The name files form the Census report is a government document which I
don't think can be copyrighted.
The file special-jargon.50 uses common.lst and word.lst from the
"Unofficial Jargon File Word Lists" which is derived from "The Jargon
File". All of which is in the Public Domain. This file also contain
a few extra UNIX terms which are found in the file "unix-terms" in the
special/ directory.
The 55 level includes words from Alan's 2of4brif list found in version
4.0 of his 12dicts package. Like his other stuff the 2of4brif is also
in the public domain.
The 60 level includes all words appearing in at least 2 of the 12
dictionaries as indicated by the 12Dicts package.
The 70 level includes Brian's frequency class 0 and the 74,550 common
dictionary words from the MWords package. The common dictionary words,
like those from the 12Dicts package, have had all likely inflections
added. The 70 level also included the 5desk list from version 4.0 of
the 12Dics package which is in the public domain.
The 80 level includes the ENABLE word list, all the lists in the
ENABLE supplement package (except for ABLE), the "UK Advanced Cryptics
Dictionary" (UKACD), the list of signature words from the YAWL package,
and the 10,196 places list from the MWords package.
The ENABLE package, mainted by M\Cooper <thegrendel@theriver.com>,
is in the Public Domain:
The ENABLE master word list, WORD.LST, is herewith formally released
into the Public Domain. Anyone is free to use it or distribute it in
any manner they see fit. No fee or registration is required for its
use nor are "contributions" solicited (if you feel you absolutely
must contribute something for your own peace of mind, the authors of
the ENABLE list ask that you make a donation on their behalf to your
favorite charity). This word list is our gift to the Scrabble
community, as an alternate to "official" word lists. Game designers
may feel free to incorporate the WORD.LST into their games. Please
mention the source and credit us as originators of the list. Note
that if you, as a game designer, use the WORD.LST in your product,
you may still copyright and protect your product, but you may *not*
legally copyright or in any way restrict redistribution of the
WORD.LST portion of your product. This *may* under law restrict your
rights to restrict your users' rights, but that is only fair.
UKACD, by J Ross Beresford <ross@bryson.demon.co.uk>, is under the
following copyright:
Copyright (c) J Ross Beresford 1993-1999. All Rights Reserved.
The following restriction is placed on the use of this publication:
if The UK Advanced Cryptics Dictionary is used in a software package
or redistributed in any form, the copyright notice must be
prominently displayed and the text of this document must be included
verbatim.
There are no other restrictions: I would like to see the list
distributed as widely as possible.
The 95 level includes the 354,984 single words, 256,772 compound
words, 4,946 female names and the 3,897 male names, and 21,986 names
from the MWords package, ABLE.LST from the ENABLE Supplement, and some
additional words found in my part-of-speech database that were not
found anywhere else.
Accent information was taken from UKACD.
The VarCon package was used to create the American, British, Canadian,
and Australian word list. It is under the following copyright:
Copyright 2000-2016 by Kevin Atkinson
Permission to use, copy, modify, distribute and sell this array, the
associated software, and its documentation for any purpose is hereby
granted without fee, provided that the above copyright notice appears
in all copies and that both that copyright notice and this permission
notice appear in supporting documentation. Kevin Atkinson makes no
representations about the suitability of this array for any
purpose. It is provided "as is" without express or implied warranty.
Copyright 2016 by Benjamin Titze
Permission to use, copy, modify, distribute and sell this array, the
associated software, and its documentation for any purpose is hereby
granted without fee, provided that the above copyright notice appears
in all copies and that both that copyright notice and this permission
notice appear in supporting documentation. Benjamin Titze makes no
representations about the suitability of this array for any
purpose. It is provided "as is" without express or implied warranty.
Since the original words lists come from the Ispell distribution:
Copyright 1993, Geoff Kuenning, Granada Hills, CA
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. All modifications to the source code must be clearly marked as
such. Binary redistributions based on modified source code
must be clearly marked as modified versions in the documentation
and/or other materials provided with the distribution.
(clause 4 removed with permission from Geoff Kuenning)
5. The name of Geoff Kuenning may not be used to endorse or promote
products derived from this software without specific prior
written permission.
THIS SOFTWARE IS PROVIDED BY GEOFF KUENNING AND CONTRIBUTORS ``AS IS'' AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL GEOFF KUENNING OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.
Build Date: Mon Dec 7 20:19:28 EST 2020
Wordlist Command: mk-list --accents=strip en_CA 60
+205
View File
@@ -0,0 +1,205 @@
SET UTF-8
TRY esianrtolcdugmphbyfvkwzESIANRTOLCDUGMPHBYFVKWZ'
ICONV 1
ICONV '
NOSUGGEST !
# ordinal numbers
COMPOUNDMIN 1
# only in compounds: 1th, 2th, 3th
ONLYINCOMPOUND c
# compound rules:
# 1. [0-9]*1[0-9]th (10th, 11th, 12th, 56714th, etc.)
# 2. [0-9]*[02-9](1st|2nd|3rd|[4-9]th) (21st, 22nd, 123rd, 1234th, etc.)
COMPOUNDRULE 2
COMPOUNDRULE n*1t
COMPOUNDRULE n*mp
WORDCHARS 0123456789
PFX A Y 1
PFX A 0 re .
PFX I Y 1
PFX I 0 in .
PFX U Y 1
PFX U 0 un .
PFX C Y 1
PFX C 0 de .
PFX E Y 1
PFX E 0 dis .
PFX F Y 1
PFX F 0 con .
PFX K Y 1
PFX K 0 pro .
SFX V N 2
SFX V e ive e
SFX V 0 ive [^e]
SFX N Y 3
SFX N e ion e
SFX N y ication y
SFX N 0 en [^ey]
SFX X Y 3
SFX X e ions e
SFX X y ications y
SFX X 0 ens [^ey]
SFX H N 2
SFX H y ieth y
SFX H 0 th [^y]
SFX Y Y 1
SFX Y 0 ly .
SFX G Y 2
SFX G e ing e
SFX G 0 ing [^e]
SFX J Y 2
SFX J e ings e
SFX J 0 ings [^e]
SFX D Y 4
SFX D 0 d e
SFX D y ied [^aeiou]y
SFX D 0 ed [^ey]
SFX D 0 ed [aeiou]y
SFX T N 4
SFX T 0 st e
SFX T y iest [^aeiou]y
SFX T 0 est [aeiou]y
SFX T 0 est [^ey]
SFX R Y 4
SFX R 0 r e
SFX R y ier [^aeiou]y
SFX R 0 er [aeiou]y
SFX R 0 er [^ey]
SFX Z Y 4
SFX Z 0 rs e
SFX Z y iers [^aeiou]y
SFX Z 0 ers [aeiou]y
SFX Z 0 ers [^ey]
SFX S Y 4
SFX S y ies [^aeiou]y
SFX S 0 s [aeiou]y
SFX S 0 es [sxzh]
SFX S 0 s [^sxzhy]
SFX P Y 3
SFX P y iness [^aeiou]y
SFX P 0 ness [aeiou]y
SFX P 0 ness [^y]
SFX M Y 1
SFX M 0 's .
SFX B Y 3
SFX B 0 able [^aeiou]
SFX B 0 able ee
SFX B e able [^aeiou]e
SFX L Y 1
SFX L 0 ment .
REP 90
REP a ei
REP ei a
REP a ey
REP ey a
REP ai ie
REP ie ai
REP alot a_lot
REP are air
REP are ear
REP are eir
REP air are
REP air ere
REP ere air
REP ere ear
REP ere eir
REP ear are
REP ear air
REP ear ere
REP eir are
REP eir ere
REP ch te
REP te ch
REP ch ti
REP ti ch
REP ch tu
REP tu ch
REP ch s
REP s ch
REP ch k
REP k ch
REP f ph
REP ph f
REP gh f
REP f gh
REP i igh
REP igh i
REP i uy
REP uy i
REP i ee
REP ee i
REP j di
REP di j
REP j gg
REP gg j
REP j ge
REP ge j
REP s ti
REP ti s
REP s ci
REP ci s
REP k cc
REP cc k
REP k qu
REP qu k
REP kw qu
REP o eau
REP eau o
REP o ew
REP ew o
REP oo ew
REP ew oo
REP ew ui
REP ui ew
REP oo ui
REP ui oo
REP ew u
REP u ew
REP oo u
REP u oo
REP u oe
REP oe u
REP u ieu
REP ieu u
REP ue ew
REP ew ue
REP uff ough
REP oo ieu
REP ieu oo
REP ier ear
REP ear ier
REP ear air
REP air ear
REP w qu
REP qu w
REP z ss
REP ss z
REP shun tion
REP shun sion
REP shun cion
REP size cise
File diff suppressed because it is too large Load Diff
+347
View File
@@ -0,0 +1,347 @@
en_GB-ise Hunspell Dictionary
Version 2020.12.07
Mon Dec 7 20:14:35 2020 -0500 [5ef55f9]
http://wordlist.sourceforge.net
README file for English Hunspell dictionaries derived from SCOWL.
These dictionaries are created using the speller/make-hunspell-dict
script in SCOWL.
The following dictionaries are available:
en_US (American)
en_CA (Canadian)
en_GB-ise (British with "ise" spelling)
en_GB-ize (British with "ize" spelling)
en_AU (Australian)
en_US-large
en_CA-large
en_GB-large (with both "ise" and "ize" spelling)
en_AU-large
The normal (non-large) dictionaries correspond to SCOWL size 60 and,
to encourage consistent spelling, generally only include one spelling
variant for a word. The large dictionaries correspond to SCOWL size
70 and may include multiple spelling for a word when both variants are
considered almost equal. The larger dictionaries however (1) have not
been as carefully checked for errors as the normal dictionaries and
thus may contain misspelled or invalid words; and (2) contain
uncommon, yet valid, words that might cause problems as they are
likely to be misspellings of more common words (for example, "ort" and
"calender").
To get an idea of the difference in size, here are 25 random words
only found in the large dictionary for American English:
Bermejo Freyr's Guenevere Hatshepsut Nottinghamshire arrestment
crassitudes crural dogwatches errorless fetial flaxseeds godroon
incretion jalapeño's kelpie kishkes neuroglias pietisms pullulation
stemwinder stenoses syce thalassic zees
The en_US, en_CA and en_AU are the official dictionaries for Hunspell.
The en_GB and large dictionaries are made available on an experimental
basis. If you find them useful please send me a quick email at
kevina@gnu.org.
If none of these dictionaries suite you (for example, maybe you want
the normal dictionary that also includes common variants) additional
dictionaries can be generated at http://app.aspell.net/create or by
modifying speller/make-hunspell-dict in SCOWL. Please do let me know
if you end up publishing a customized dictionary.
If a word is not found in the dictionary or a word is there you think
shouldn't be, you can lookup the word up at http://app.aspell.net/lookup
to help determine why that is.
General comments on these list can be sent directly to me at
kevina@gnu.org or to the wordlist-devel mailing lists
(https://lists.sourceforge.net/lists/listinfo/wordlist-devel). If you
have specific issues with any of these dictionaries please file a bug
report at https://github.com/kevina/wordlist/issues.
IMPORTANT CHANGES INTRODUCED In 2016.11.20:
New Australian dictionaries thanks to the work of Benjamin Titze
(btitze@protonmail.ch).
IMPORTANT CHANGES INTRODUCED IN 2016.04.24:
The dictionaries are now in UTF-8 format instead of ISO-8859-1. This
was required to handle smart quotes correctly.
IMPORTANT CHANGES INTRODUCED IN 2016.01.19:
"SET UTF8" was changes to "SET UTF-8" in the affix file as some
versions of Hunspell do not recognize "UTF8".
ADDITIONAL NOTES:
The NOSUGGEST flag was added to certain taboo words. While I made an
honest attempt to flag the strongest taboo words with the NOSUGGEST
flag, I MAKE NO GUARANTEE THAT I FLAGGED EVERY POSSIBLE TABOO WORD.
The list was originally derived from Németh László, however I removed
some words which, while being considered taboo by some dictionaries,
are not really considered swear words in today's society.
COPYRIGHT, SOURCES, and CREDITS:
The English dictionaries come directly from SCOWL
and is thus under the same copyright of SCOWL. The affix file is
a heavily modified version of the original english.aff file which was
released as part of Geoff Kuenning's Ispell and as such is covered by
his BSD license. Part of SCOWL is also based on Ispell thus the
Ispell copyright is included with the SCOWL copyright.
The collective work is Copyright 2000-2018 by Kevin Atkinson as well
as any of the copyrights mentioned below:
Copyright 2000-2018 by Kevin Atkinson
Permission to use, copy, modify, distribute and sell these word
lists, the associated scripts, the output created from the scripts,
and its documentation for any purpose is hereby granted without fee,
provided that the above copyright notice appears in all copies and
that both that copyright notice and this permission notice appear in
supporting documentation. Kevin Atkinson makes no representations
about the suitability of this array for any purpose. It is provided
"as is" without express or implied warranty.
Alan Beale <biljir@pobox.com> also deserves special credit as he has,
in addition to providing the 12Dicts package and being a major
contributor to the ENABLE word list, given me an incredible amount of
feedback and created a number of special lists (those found in the
Supplement) in order to help improve the overall quality of SCOWL.
The 10 level includes the 1000 most common English words (according to
the Moby (TM) Words II [MWords] package), a subset of the 1000 most
common words on the Internet (again, according to Moby Words II), and
frequently class 16 from Brian Kelk's "UK English Wordlist
with Frequency Classification".
The MWords package was explicitly placed in the public domain:
The Moby lexicon project is complete and has
been place into the public domain. Use, sell,
rework, excerpt and use in any way on any platform.
Placing this material on internal or public servers is
also encouraged. The compiler is not aware of any
export restrictions so freely distribute world-wide.
You can verify the public domain status by contacting
Grady Ward
3449 Martha Ct.
Arcata, CA 95521-4884
grady@netcom.com
grady@northcoast.com
The "UK English Wordlist With Frequency Classification" is also in the
Public Domain:
Date: Sat, 08 Jul 2000 20:27:21 +0100
From: Brian Kelk <Brian.Kelk@cl.cam.ac.uk>
> I was wondering what the copyright status of your "UK English
> Wordlist With Frequency Classification" word list as it seems to
> be lacking any copyright notice.
There were many many sources in total, but any text marked
"copyright" was avoided. Locally-written documentation was one
source. An earlier version of the list resided in a filespace called
PUBLIC on the University mainframe, because it was considered public
domain.
Date: Tue, 11 Jul 2000 19:31:34 +0100
> So are you saying your word list is also in the public domain?
That is the intention.
The 20 level includes frequency classes 7-15 from Brian's word list.
The 35 level includes frequency classes 2-6 and words appearing in at
least 11 of 12 dictionaries as indicated in the 12Dicts package. All
words from the 12Dicts package have had likely inflections added via
my inflection database.
The 12Dicts package and Supplement is in the Public Domain.
The WordNet database, which was used in the creation of the
Inflections database, is under the following copyright:
This software and database is being provided to you, the LICENSEE,
by Princeton University under the following license. By obtaining,
using and/or copying this software and database, you agree that you
have read, understood, and will comply with these terms and
conditions.:
Permission to use, copy, modify and distribute this software and
database and its documentation for any purpose and without fee or
royalty is hereby granted, provided that you agree to comply with
the following copyright notice and statements, including the
disclaimer, and that the same appear on ALL copies of the software,
database and documentation, including modifications that you make
for internal use or for distribution.
WordNet 1.6 Copyright 1997 by Princeton University. All rights
reserved.
THIS SOFTWARE AND DATABASE IS PROVIDED "AS IS" AND PRINCETON
UNIVERSITY MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PRINCETON
UNIVERSITY MAKES NO REPRESENTATIONS OR WARRANTIES OF MERCHANT-
ABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF THE
LICENSED SOFTWARE, DATABASE OR DOCUMENTATION WILL NOT INFRINGE ANY
THIRD PARTY PATENTS, COPYRIGHTS, TRADEMARKS OR OTHER RIGHTS.
The name of Princeton University or Princeton may not be used in
advertising or publicity pertaining to distribution of the software
and/or database. Title to copyright in this software, database and
any associated documentation shall at all times remain with
Princeton University and LICENSEE agrees to preserve same.
The 40 level includes words from Alan's 3esl list found in version 4.0
of his 12dicts package. Like his other stuff the 3esl list is also in the
public domain.
The 50 level includes Brian's frequency class 1, words appearing
in at least 5 of 12 of the dictionaries as indicated in the 12Dicts
package, and uppercase words in at least 4 of the previous 12
dictionaries. A decent number of proper names is also included: The
top 1000 male, female, and Last names from the 1990 Census report; a
list of names sent to me by Alan Beale; and a few names that I added
myself. Finally a small list of abbreviations not commonly found in
other word lists is included.
The name files form the Census report is a government document which I
don't think can be copyrighted.
The file special-jargon.50 uses common.lst and word.lst from the
"Unofficial Jargon File Word Lists" which is derived from "The Jargon
File". All of which is in the Public Domain. This file also contain
a few extra UNIX terms which are found in the file "unix-terms" in the
special/ directory.
The 55 level includes words from Alan's 2of4brif list found in version
4.0 of his 12dicts package. Like his other stuff the 2of4brif is also
in the public domain.
The 60 level includes all words appearing in at least 2 of the 12
dictionaries as indicated by the 12Dicts package.
The 70 level includes Brian's frequency class 0 and the 74,550 common
dictionary words from the MWords package. The common dictionary words,
like those from the 12Dicts package, have had all likely inflections
added. The 70 level also included the 5desk list from version 4.0 of
the 12Dics package which is in the public domain.
The 80 level includes the ENABLE word list, all the lists in the
ENABLE supplement package (except for ABLE), the "UK Advanced Cryptics
Dictionary" (UKACD), the list of signature words from the YAWL package,
and the 10,196 places list from the MWords package.
The ENABLE package, mainted by M\Cooper <thegrendel@theriver.com>,
is in the Public Domain:
The ENABLE master word list, WORD.LST, is herewith formally released
into the Public Domain. Anyone is free to use it or distribute it in
any manner they see fit. No fee or registration is required for its
use nor are "contributions" solicited (if you feel you absolutely
must contribute something for your own peace of mind, the authors of
the ENABLE list ask that you make a donation on their behalf to your
favorite charity). This word list is our gift to the Scrabble
community, as an alternate to "official" word lists. Game designers
may feel free to incorporate the WORD.LST into their games. Please
mention the source and credit us as originators of the list. Note
that if you, as a game designer, use the WORD.LST in your product,
you may still copyright and protect your product, but you may *not*
legally copyright or in any way restrict redistribution of the
WORD.LST portion of your product. This *may* under law restrict your
rights to restrict your users' rights, but that is only fair.
UKACD, by J Ross Beresford <ross@bryson.demon.co.uk>, is under the
following copyright:
Copyright (c) J Ross Beresford 1993-1999. All Rights Reserved.
The following restriction is placed on the use of this publication:
if The UK Advanced Cryptics Dictionary is used in a software package
or redistributed in any form, the copyright notice must be
prominently displayed and the text of this document must be included
verbatim.
There are no other restrictions: I would like to see the list
distributed as widely as possible.
The 95 level includes the 354,984 single words, 256,772 compound
words, 4,946 female names and the 3,897 male names, and 21,986 names
from the MWords package, ABLE.LST from the ENABLE Supplement, and some
additional words found in my part-of-speech database that were not
found anywhere else.
Accent information was taken from UKACD.
The VarCon package was used to create the American, British, Canadian,
and Australian word list. It is under the following copyright:
Copyright 2000-2016 by Kevin Atkinson
Permission to use, copy, modify, distribute and sell this array, the
associated software, and its documentation for any purpose is hereby
granted without fee, provided that the above copyright notice appears
in all copies and that both that copyright notice and this permission
notice appear in supporting documentation. Kevin Atkinson makes no
representations about the suitability of this array for any
purpose. It is provided "as is" without express or implied warranty.
Copyright 2016 by Benjamin Titze
Permission to use, copy, modify, distribute and sell this array, the
associated software, and its documentation for any purpose is hereby
granted without fee, provided that the above copyright notice appears
in all copies and that both that copyright notice and this permission
notice appear in supporting documentation. Benjamin Titze makes no
representations about the suitability of this array for any
purpose. It is provided "as is" without express or implied warranty.
Since the original words lists come from the Ispell distribution:
Copyright 1993, Geoff Kuenning, Granada Hills, CA
All rights reserved.
Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
1. Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
2. Redistributions in binary form must reproduce the above copyright
notice, this list of conditions and the following disclaimer in the
documentation and/or other materials provided with the distribution.
3. All modifications to the source code must be clearly marked as
such. Binary redistributions based on modified source code
must be clearly marked as modified versions in the documentation
and/or other materials provided with the distribution.
(clause 4 removed with permission from Geoff Kuenning)
5. The name of Geoff Kuenning may not be used to endorse or promote
products derived from this software without specific prior
written permission.
THIS SOFTWARE IS PROVIDED BY GEOFF KUENNING AND CONTRIBUTORS ``AS IS'' AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
ARE DISCLAIMED. IN NO EVENT SHALL GEOFF KUENNING OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
SUCH DAMAGE.
Build Date: Mon Dec 7 20:19:30 EST 2020
Wordlist Command: mk-list --accents=strip en_GB-ise 60
+422 -58
View File
@@ -5,12 +5,40 @@ use crate::order;
use eframe::egui; use eframe::egui;
use std::collections::HashMap; use std::collections::HashMap;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use std::sync::Arc;
use std::time::{Duration, Instant};
/// Result of a background grammar check: the matches, or an error message. /// Result of a background grammar check: the matches, or an error message.
type LtCheckResult = Result<Vec<crate::langtool::Match>, String>; type LtCheckResult = Result<Vec<crate::langtool::Match>, String>;
/// Channel payload from a background check: the text that was checked, paired /// Channel payload from a background check: the text that was checked, paired
/// with its result (so the app can confirm the buffer hasn't changed since). /// with its result (so the app can confirm the buffer hasn't changed since).
type LtCheckMsg = (String, LtCheckResult); type LtCheckMsg = (String, LtCheckResult);
/// Channel payload from a background spell check: the checked text and the
/// spelling matches found in it.
type SpellCheckMsg = (String, Vec<crate::langtool::Match>);
/// Which check currently owns the editor's underlines and the issues panel.
#[derive(Clone, Copy, PartialEq, Eq)]
enum IssueSource {
/// Fresh LanguageTool results for the current buffer (spelling + grammar).
LanguageTool,
/// Live offline spell-check results (spelling only).
Spell,
/// Nothing current to show.
None,
}
/// An owned snapshot of one issue for the results panel, decoupled from `self`
/// so the panel can render without holding a borrow across its click handling.
struct IssueItem {
/// Index of this item in the source match list (for applying a fix).
idx: usize,
/// The offending text.
snippet: String,
message: String,
spelling: bool,
replacements: Vec<String>,
}
pub struct App { pub struct App {
config: Config, config: Config,
@@ -86,6 +114,25 @@ pub struct App {
find_focus: Option<bool>, find_focus: Option<bool>,
/// Request the editor to scroll the active match into view next frame. /// Request the editor to scroll the active match into view next frame.
find_scroll: bool, find_scroll: bool,
/// The loaded offline spell-check dictionary (shared with the worker thread).
spell_dict: Option<Arc<spellbook::Dictionary>>,
/// Dictionaries available to choose from (bundled + discovered on disk).
spell_dicts: Vec<crate::spell::DictEntry>,
/// Live spelling matches and the buffer text they were computed against.
spell_matches: Vec<crate::langtool::Match>,
spell_checked_text: String,
/// Set when the buffer changed and a re-check is owed.
spell_dirty: bool,
/// When the buffer was last edited, for debouncing the live check. `None`
/// means "check as soon as possible" (e.g. right after opening a file).
spell_last_edit: Option<Instant>,
/// In-flight background spell check, if any.
spell_rx: Option<std::sync::mpsc::Receiver<SpellCheckMsg>>,
/// One-line status for the spell checker (dictionary name, count, or error).
spell_status: String,
/// Index (into the currently displayed matches) of the word a right-click
/// suggestion menu is open for, if any.
spell_menu: Option<usize>,
} }
impl App { impl App {
@@ -131,7 +178,17 @@ impl App {
find_needs_refresh: false, find_needs_refresh: false,
find_focus: None, find_focus: None,
find_scroll: false, find_scroll: false,
spell_dict: None,
spell_dicts: crate::spell::available(),
spell_matches: Vec::new(),
spell_checked_text: String::new(),
spell_dirty: true,
spell_last_edit: None,
spell_rx: None,
spell_status: String::new(),
spell_menu: None,
}; };
app.load_spell_dict();
app.open_workspace(); app.open_workspace();
app app
} }
@@ -295,6 +352,12 @@ impl App {
self.find_matches.clear(); self.find_matches.clear();
self.find_active = 0; self.find_active = 0;
self.find_needs_refresh = true; self.find_needs_refresh = true;
// Re-run the live spell check on the newly loaded buffer immediately.
self.spell_matches.clear();
self.spell_checked_text.clear();
self.spell_dirty = true;
self.spell_last_edit = None;
self.spell_menu = None;
if let Some(name) = self.files.get(idx) { if let Some(name) = self.files.get(idx) {
let path = self.path_for(name); let path = self.path_for(name);
self.buffer = std::fs::read_to_string(&path).unwrap_or_default(); self.buffer = std::fs::read_to_string(&path).unwrap_or_default();
@@ -643,34 +706,223 @@ impl App {
} }
} }
/// Apply replacement `rep_idx` of match `match_idx` to the buffer, then shift /// Apply a LanguageTool suggestion to the buffer, then shift the remaining
/// the remaining matches so their highlights stay valid. /// matches so their highlights stay valid.
fn apply_replacement(&mut self, match_idx: usize, rep_idx: usize) { fn apply_replacement(&mut self, match_idx: usize, rep_idx: usize) {
// Only safe while the buffer still matches what was checked. // Only safe while the buffer still matches what was checked.
if self.buffer != self.lt_checked_text { if self.buffer != self.lt_checked_text {
return; return;
} }
let Some(m) = self.lt_matches.get(match_idx).cloned() else { if splice_fix(&mut self.buffer, &mut self.lt_matches, match_idx, rep_idx) {
return;
};
let Some(replacement) = m.replacements.get(rep_idx).cloned() else {
return;
};
if m.end > self.buffer.len() || !self.buffer.is_char_boundary(m.start) {
return;
}
self.buffer.replace_range(m.start..m.end, &replacement);
self.dirty = true; self.dirty = true;
self.find_needs_refresh = true;
self.lt_checked_text = self.buffer.clone(); self.lt_checked_text = self.buffer.clone();
// The applied match overlaps its own region, so it is dropped here too.
remap_matches(&mut self.lt_matches, m.start, m.end, replacement.len());
self.lt_status = match self.lt_matches.len() { self.lt_status = match self.lt_matches.len() {
0 => "No issues remaining".to_string(), 0 => "No issues remaining".to_string(),
1 => "1 issue".to_string(), 1 => "1 issue".to_string(),
n => format!("{n} issues"), n => format!("{n} issues"),
}; };
} }
}
/// Apply a spell-check suggestion to the buffer, keeping the remaining
/// misspelling underlines aligned.
fn apply_spell_fix(&mut self, match_idx: usize, rep_idx: usize) {
if self.buffer != self.spell_checked_text {
return;
}
if splice_fix(&mut self.buffer, &mut self.spell_matches, match_idx, rep_idx) {
self.dirty = true;
self.find_needs_refresh = true;
// Keep these matches valid without forcing a full re-check.
self.spell_checked_text = self.buffer.clone();
self.spell_status = match self.spell_matches.len() {
0 => "No spelling issues".to_string(),
1 => "1 spelling issue".to_string(),
n => format!("{n} spelling issues"),
};
}
}
/// Apply suggestion `rep_idx` of the match at `match_idx` from whichever
/// source currently owns the issues (LanguageTool or the spell checker).
fn apply_current_fix(&mut self, match_idx: usize, rep_idx: usize) {
match self.issue_source() {
IssueSource::LanguageTool => self.apply_replacement(match_idx, rep_idx),
IssueSource::Spell => self.apply_spell_fix(match_idx, rep_idx),
IssueSource::None => {}
}
}
// ---- Spell check (offline) ---------------------------------------------
/// (Re)load the dictionary named by `config.spell_language`, falling back to
/// the default and then to whatever is available. Resets the live results so
/// the buffer is re-checked against the new dictionary.
fn load_spell_dict(&mut self) {
let want = self.config.spell_language.clone();
let entry = self
.spell_dicts
.iter()
.find(|d| d.id == want)
.or_else(|| {
self.spell_dicts
.iter()
.find(|d| d.id == crate::spell::DEFAULT_LANGUAGE)
})
.or_else(|| self.spell_dicts.first())
.cloned();
match entry {
Some(e) => match e.load() {
Ok(dict) => {
self.spell_dict = Some(Arc::new(dict));
self.spell_status = format!("Dictionary: {}", e.id);
}
Err(err) => {
self.spell_dict = None;
self.spell_status = err;
}
},
None => {
self.spell_dict = None;
self.spell_status = "No spelling dictionary available".to_string();
}
}
self.spell_matches.clear();
self.spell_checked_text.clear();
self.spell_dirty = true;
self.spell_last_edit = None;
}
/// Switch the active dictionary and persist the choice.
fn set_spell_language(&mut self, id: &str) {
if self.config.spell_language == id {
return;
}
self.config.spell_language = id.to_string();
self.config.save();
self.load_spell_dict();
}
/// True when LanguageTool results describe the current buffer exactly (so
/// they, rather than the live spell checker, own the underlines).
fn lt_is_current(&self) -> bool {
!self.lt_checked_text.is_empty() && self.buffer == self.lt_checked_text
}
/// Which check currently owns the issues shown in the editor and panel.
fn issue_source(&self) -> IssueSource {
if self.lt_is_current() {
IssueSource::LanguageTool
} else if self.config.spell_check
&& self.spell_dict.is_some()
&& self.buffer == self.spell_checked_text
{
IssueSource::Spell
} else {
IssueSource::None
}
}
/// The matches currently owning the editor's underlines (may be empty).
fn current_matches(&self) -> &[crate::langtool::Match] {
match self.issue_source() {
IssueSource::LanguageTool => &self.lt_matches,
IssueSource::Spell => &self.spell_matches,
IssueSource::None => &[],
}
}
/// Index of the current match covering byte offset `byte`, if any.
fn match_index_at(&self, byte: usize) -> Option<usize> {
self.current_matches()
.iter()
.position(|m| byte >= m.start && byte < m.end)
}
/// An owned snapshot of the current issues for the results panel.
fn issue_items(&self) -> (IssueSource, Vec<IssueItem>) {
let source = self.issue_source();
let (matches, checked) = match source {
IssueSource::LanguageTool => (&self.lt_matches, self.lt_checked_text.as_str()),
IssueSource::Spell => (&self.spell_matches, self.spell_checked_text.as_str()),
IssueSource::None => return (source, Vec::new()),
};
let items = matches
.iter()
.enumerate()
.map(|(idx, m)| IssueItem {
idx,
snippet: checked.get(m.start..m.end).unwrap_or("").to_string(),
message: m.message.clone(),
spelling: m.spelling,
replacements: m.replacements.clone(),
})
.collect();
(source, items)
}
/// The status line for the spelling checker given a match count.
fn spell_count_status(n: usize) -> String {
match n {
0 => "No spelling issues".to_string(),
1 => "1 spelling issue".to_string(),
n => format!("{n} spelling issues"),
}
}
/// Pick up a finished background spell check.
fn poll_spell(&mut self) {
let received = self.spell_rx.as_ref().and_then(|rx| rx.try_recv().ok());
if let Some((text, matches)) = received {
self.spell_rx = None;
if self.spell_dict.is_some() {
self.spell_status = Self::spell_count_status(matches.len());
}
self.spell_matches = matches;
self.spell_checked_text = text;
}
}
/// Start a background spell check of the current buffer if one is due:
/// enabled, a dictionary loaded, a file open, the buffer changed, no check
/// already running, and the debounce interval elapsed since the last edit.
fn maybe_start_spell_check(&mut self, ctx: &egui::Context) {
if !self.config.spell_check || self.selected.is_none() || !self.spell_dirty {
return;
}
// LanguageTool results already cover this exact buffer (spelling too), so
// don't spend effort on a spell pass that wouldn't be shown.
if self.lt_is_current() {
return;
}
let Some(dict) = self.spell_dict.clone() else {
return;
};
if self.spell_rx.is_some() {
return;
}
// Debounce so we don't re-check on every keystroke.
const DEBOUNCE: Duration = Duration::from_millis(400);
if let Some(edited) = self.spell_last_edit {
let elapsed = edited.elapsed();
if elapsed < DEBOUNCE {
ctx.request_repaint_after(DEBOUNCE - elapsed);
return;
}
}
self.spell_dirty = false;
let text = self.buffer.clone();
let (tx, rx) = std::sync::mpsc::channel();
self.spell_rx = Some(rx);
let ctx = ctx.clone();
std::thread::spawn(move || {
let matches = crate::spell::check(&dict, &text);
let _ = tx.send((text, matches));
ctx.request_repaint();
});
}
// ---- Find / replace ---------------------------------------------------- // ---- Find / replace ----------------------------------------------------
@@ -903,6 +1155,47 @@ impl App {
ui.label(egui::RichText::new(&self.lt_status).weak()); ui.label(egui::RichText::new(&self.lt_status).weak());
} }
}); });
ui.add_space(2.0);
ui.horizontal(|ui| {
ui.label("Spelling:").on_hover_text(
"Offline spell check (Hunspell dictionaries). Underlines misspellings \
as you type when LanguageTool results aren't current; right-click a \
word for suggestions.",
);
if ui
.checkbox(&mut self.config.spell_check, "Check as I type")
.on_hover_text("Underline misspellings live using the dictionary below")
.changed()
{
self.config.save();
if self.config.spell_check {
// Re-check the current buffer straight away.
self.spell_dirty = true;
self.spell_last_edit = None;
}
}
ui.label("Dictionary:");
let current = self.config.spell_language.clone();
let mut pick: Option<String> = None;
egui::ComboBox::from_id_salt("spell_dict")
.selected_text(current.clone())
.show_ui(ui, |ui| {
for d in &self.spell_dicts {
if ui
.selectable_label(current == d.id, &d.label)
.clicked()
{
pick = Some(d.id.clone());
}
}
});
if let Some(id) = pick {
self.set_spell_language(&id);
}
if !self.spell_status.is_empty() {
ui.label(egui::RichText::new(&self.spell_status).weak());
}
});
ui.add_space(4.0); ui.add_space(4.0);
}); });
@@ -1229,24 +1522,23 @@ impl App {
self.show_settings = now_open; self.show_settings = now_open;
} }
/// Bottom panel listing grammar/spelling issues with one-click fixes. /// Bottom panel listing the current grammar/spelling issues with one-click
/// fixes. Shows LanguageTool results when they're fresh, otherwise the live
/// spell-check results.
fn lt_panel(&mut self, ctx: &egui::Context) { fn lt_panel(&mut self, ctx: &egui::Context) {
let (source, items) = self.issue_items();
egui::TopBottomPanel::bottom("ltpanel") egui::TopBottomPanel::bottom("ltpanel")
.resizable(true) .resizable(true)
.default_height(190.0) .default_height(190.0)
.show(ctx, |ui| { .show(ctx, |ui| {
let stale = self.buffer != self.lt_checked_text; let (title, status) = match source {
IssueSource::LanguageTool => ("Grammar & spelling", self.lt_status.clone()),
_ => ("Spelling", self.spell_status.clone()),
};
ui.horizontal(|ui| { ui.horizontal(|ui| {
ui.label(egui::RichText::new("Grammar & spelling").strong()); ui.label(egui::RichText::new(title).strong());
if !self.lt_status.is_empty() { if !status.is_empty() {
ui.label(egui::RichText::new(&self.lt_status).weak()); ui.label(egui::RichText::new(status).weak());
}
if stale && !self.lt_matches.is_empty() {
ui.label(
egui::RichText::new("· edited since check — re-check to apply fixes")
.weak()
.italics(),
);
} }
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| { ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
if ui.button("Hide").clicked() { if ui.button("Hide").clicked() {
@@ -1256,13 +1548,10 @@ impl App {
}); });
ui.separator(); ui.separator();
if self.lt_matches.is_empty() { if items.is_empty() {
let busy = self.lt_rx.is_some() || self.spell_rx.is_some();
ui.label( ui.label(
egui::RichText::new(if self.lt_rx.is_some() { egui::RichText::new(if busy { "Checking…" } else { "No issues to show." })
"Checking…"
} else {
"No issues to show."
})
.weak(), .weak(),
); );
return; return;
@@ -1272,34 +1561,31 @@ impl App {
egui::ScrollArea::vertical() egui::ScrollArea::vertical()
.auto_shrink([false, false]) .auto_shrink([false, false])
.show(ui, |ui| { .show(ui, |ui| {
for (i, m) in self.lt_matches.iter().enumerate() { for item in &items {
ui.horizontal_wrapped(|ui| { ui.horizontal_wrapped(|ui| {
let (dot, col) = if m.spelling { let col = if item.spelling {
("", egui::Color32::from_rgb(0xE0, 0x40, 0x40)) egui::Color32::from_rgb(0xE0, 0x40, 0x40)
} else { } else {
("", egui::Color32::from_rgb(0x3B, 0x82, 0xF6)) egui::Color32::from_rgb(0x3B, 0x82, 0xF6)
}; };
ui.label(egui::RichText::new(dot).color(col)); ui.label(egui::RichText::new("").color(col));
if let Some(snippet) = self.lt_checked_text.get(m.start..m.end) { if !item.snippet.is_empty() {
ui.label( ui.label(
egui::RichText::new(format!("{snippet}")).strong(), egui::RichText::new(format!("{}", item.snippet)).strong(),
); );
} }
ui.label(&m.message); ui.label(&item.message);
}); });
ui.horizontal_wrapped(|ui| { ui.horizontal_wrapped(|ui| {
ui.add_space(16.0); ui.add_space(16.0);
if m.replacements.is_empty() { if item.replacements.is_empty() {
ui.label( ui.label(
egui::RichText::new("(no suggestion)").weak().italics(), egui::RichText::new("(no suggestion)").weak().italics(),
); );
} else { } else {
for (j, rep) in m.replacements.iter().take(8).enumerate() { for (j, rep) in item.replacements.iter().enumerate() {
if ui if ui.button(egui::RichText::new(rep).small()).clicked() {
.add_enabled(!stale, egui::Button::new(rep).small()) apply = Some((item.idx, j));
.clicked()
{
apply = Some((i, j));
} }
} }
} }
@@ -1309,7 +1595,7 @@ impl App {
}); });
if let Some((i, j)) = apply { if let Some((i, j)) = apply {
self.apply_replacement(i, j); self.apply_current_fix(i, j);
} }
}); });
} }
@@ -1435,16 +1721,13 @@ impl App {
// Editor text colour, brightened toward max contrast by the contrast slider. // Editor text colour, brightened toward max contrast by the contrast slider.
let text_color = editor_text_color(ui.visuals(), self.config.editor_text_contrast); let text_color = editor_text_color(ui.visuals(), self.config.editor_text_contrast);
// Underline grammar/spelling matches, but only while the buffer still // Underline whichever check currently owns the buffer's issues
// equals the text they were computed against (edits invalidate offsets). // (LanguageTool when fresh, otherwise the live spell checker).
let ranges: Vec<(usize, usize, bool)> = if self.buffer == self.lt_checked_text { let ranges: Vec<(usize, usize, bool)> = self
self.lt_matches .current_matches()
.iter() .iter()
.map(|m| (m.start, m.end, m.spelling)) .map(|m| (m.start, m.end, m.spelling))
.collect() .collect();
} else {
Vec::new()
};
// Search-match highlights (all matches shaded, the active one stronger). // Search-match highlights (all matches shaded, the active one stronger).
let find_ranges: Vec<(usize, usize)> = if self.show_find { let find_ranges: Vec<(usize, usize)> = if self.show_find {
@@ -1482,8 +1765,11 @@ impl App {
.show(ui); .show(ui);
if output.response.changed() { if output.response.changed() {
self.dirty = true; self.dirty = true;
// The buffer changed, so any search matches are now stale. // The buffer changed, so search matches and the live spell
// check both need refreshing (the latter debounced).
self.find_needs_refresh = true; self.find_needs_refresh = true;
self.spell_dirty = true;
self.spell_last_edit = Some(Instant::now());
} }
// Markdown formatting hotkeys, applied to the current selection // Markdown formatting hotkeys, applied to the current selection
// while the editor is focused (Ctrl/Cmd + B / I / E / K, and // while the editor is focused (Ctrl/Cmd + B / I / E / K, and
@@ -1524,6 +1810,55 @@ impl App {
} }
self.find_scroll = false; self.find_scroll = false;
} }
// Right-click a misspelling → a menu of suggested corrections.
if output.response.secondary_clicked() {
self.spell_menu = output.response.interact_pointer_pos().and_then(|pos| {
let local = pos - output.galley_pos;
let cursor = output.galley.cursor_from_pos(local);
let byte = char_to_byte(&self.buffer, cursor.ccursor.index);
self.match_index_at(byte)
});
}
// Snapshot the target's suggestions (owned) so the menu closure
// doesn't borrow self; compute them on demand if the background
// check hadn't produced any for this word.
let menu: Option<(usize, Vec<String>)> = self.spell_menu.and_then(|i| {
let m = self.current_matches().get(i)?;
let mut reps = m.replacements.clone();
if reps.is_empty() {
if let (Some(dict), Some(word)) =
(&self.spell_dict, self.buffer.get(m.start..m.end))
{
reps = crate::spell::suggestions(dict, word);
}
}
Some((i, reps))
});
let mut chosen: Option<(usize, usize)> = None;
output.response.context_menu(|ui| {
match &menu {
Some((i, reps)) if !reps.is_empty() => {
ui.label(egui::RichText::new("Suggestions").strong());
for (j, rep) in reps.iter().enumerate() {
if ui.button(rep).clicked() {
chosen = Some((*i, j));
ui.close_menu();
}
}
}
Some(_) => {
ui.label(egui::RichText::new("No suggestions").weak());
}
None => {
ui.label(egui::RichText::new("No spelling issue here").weak());
}
}
});
if let Some((i, j)) = chosen {
self.apply_current_fix(i, j);
self.spell_menu = None;
}
}); });
} }
@@ -1690,6 +2025,8 @@ impl eframe::App for App {
self.poll_lt(); self.poll_lt();
self.poll_settings_test(); self.poll_settings_test();
self.poll_spell();
self.maybe_start_spell_check(ctx);
self.menu_bar(ctx); self.menu_bar(ctx);
self.top_bar(ctx); self.top_bar(ctx);
@@ -2180,6 +2517,33 @@ fn resolve_chapter_title(
.unwrap_or_else(|| format!("{:0width$}.", index + 1, width = pad_width)) .unwrap_or_else(|| format!("{:0width$}.", index + 1, width = pad_width))
} }
/// Apply suggestion `rep_idx` of `matches[match_idx]` to `buffer` in place and
/// remap the remaining matches. Returns whether a replacement was made (false if
/// the indices are out of range or the match's bounds aren't valid boundaries).
fn splice_fix(
buffer: &mut String,
matches: &mut Vec<crate::langtool::Match>,
match_idx: usize,
rep_idx: usize,
) -> bool {
let Some(m) = matches.get(match_idx).cloned() else {
return false;
};
let Some(replacement) = m.replacements.get(rep_idx).cloned() else {
return false;
};
if m.end > buffer.len()
|| !buffer.is_char_boundary(m.start)
|| !buffer.is_char_boundary(m.end)
{
return false;
}
buffer.replace_range(m.start..m.end, &replacement);
// The applied match overlaps its own region, so it is dropped here too.
remap_matches(matches, m.start, m.end, replacement.len());
true
}
/// Re-position matches after the byte range `[s, e)` was replaced with `new_len` /// Re-position matches after the byte range `[s, e)` was replaced with `new_len`
/// bytes. Matches that overlapped the edited region (including the one that was /// bytes. Matches that overlapped the edited region (including the one that was
/// just applied) are dropped; matches entirely after it are shifted by the /// just applied) are dropped; matches entirely after it are shifted by the
+20
View File
@@ -45,6 +45,24 @@ pub struct Config {
/// Language passed to LanguageTool (`auto` to detect, or a code like `en-US`). /// Language passed to LanguageTool (`auto` to detect, or a code like `en-US`).
#[serde(default = "default_languagetool_language")] #[serde(default = "default_languagetool_language")]
pub languagetool_language: String, pub languagetool_language: String,
/// Whether the offline (Hunspell) live spell checker underlines misspellings
/// as you type. Used whenever LanguageTool results aren't current.
#[serde(default = "default_spell_check")]
pub spell_check: bool,
/// Id of the spell-check dictionary to use (e.g. `en-CA`, `en_GB`), matching
/// a [`crate::spell::DictEntry::id`].
#[serde(default = "default_spell_language")]
pub spell_language: String,
}
/// Live spell checking is on by default.
pub fn default_spell_check() -> bool {
true
}
/// Default dictionary: the built-in Canadian English one.
pub fn default_spell_language() -> String {
crate::spell::DEFAULT_LANGUAGE.to_string()
} }
/// Default scheme: plain HTTP, matching a local server. /// Default scheme: plain HTTP, matching a local server.
@@ -89,6 +107,8 @@ impl Default for Config {
languagetool_port: default_languagetool_port(), languagetool_port: default_languagetool_port(),
languagetool_token: String::new(), languagetool_token: String::new(),
languagetool_language: default_languagetool_language(), languagetool_language: default_languagetool_language(),
spell_check: default_spell_check(),
spell_language: default_spell_language(),
} }
} }
} }
+13 -4
View File
@@ -140,12 +140,21 @@ fn cheatsheet_body(ui: &mut egui::Ui) {
blank to export the whole file. Comments are removed everywhere.", blank to export the whole file. Comments are removed everywhere.",
); );
section(ui, "Grammar & spelling"); section(ui, "Spelling & grammar");
body( body(
ui, ui,
"Press ✓ Check (or the Grammar panel) to run the current file through your \ "Spelling is checked live and offline: misspellings are underlined in red \
LanguageTool server — configure it in Settings ▸ LanguageTool. Spelling issues \ as you type — right-click a word for correction suggestions. Choose the \
are underlined in red, grammar/style in blue.", dictionary (Canadian or British English are built in) on the Spelling row \
of the top bar, or drop more Hunspell .aff/.dic files in \
~/.config/md-manuscript/dictionaries/.",
);
note(
ui,
"For grammar and style too, press ✓ Check to run the file through a \
LanguageTool server (configure it in Settings ▸ LanguageTool). Its results \
(grammar in blue, spelling in red) take over until you edit again, then the \
offline spell check resumes.",
); );
} }
+1
View File
@@ -10,6 +10,7 @@ mod langtool;
mod odt; mod odt;
mod order; mod order;
mod preprocess; mod preprocess;
mod spell;
use eframe::egui; use eframe::egui;
+392
View File
@@ -0,0 +1,392 @@
//! Offline spell checking with [`spellbook`], a pure-Rust reader of Hunspell
//! `.aff`/`.dic` dictionaries (so the binary stays self-contained — no C
//! `libhunspell` to link against).
//!
//! Two dictionaries are compiled into the program (Canadian and British
//! English); additional Hunspell dictionaries are discovered at runtime from
//! the usual system folders and a per-user folder. The checker walks the prose
//! of a markdown buffer (skipping code spans, code blocks and link targets via
//! `pulldown-cmark`) and reports each unknown word as a spelling
//! [`Match`](crate::langtool::Match), reusing the same type LanguageTool
//! produces so the editor's underlines, results panel and one-click fixes work
//! unchanged.
use crate::langtool::Match;
use spellbook::Dictionary;
use std::collections::HashMap;
use std::path::PathBuf;
// --- Bundled dictionaries (SCOWL-derived, permissive license; see the files
// under `dictionaries/<lang>/license`) --------------------------------------
const EN_CA_AFF: &str = include_str!("../dictionaries/en-CA/index.aff");
const EN_CA_DIC: &str = include_str!("../dictionaries/en-CA/index.dic");
const EN_GB_AFF: &str = include_str!("../dictionaries/en-GB/index.aff");
const EN_GB_DIC: &str = include_str!("../dictionaries/en-GB/index.dic");
/// The language id selected by default when none is configured.
pub const DEFAULT_LANGUAGE: &str = "en-CA";
/// A dictionary the app can load, either compiled in or found on disk.
#[derive(Debug, Clone)]
pub struct DictEntry {
/// Stable identifier stored in the config (e.g. `en-CA`, `en_GB`, `de_DE`).
pub id: String,
/// Human-readable label for the picker.
pub label: String,
kind: DictKind,
}
#[derive(Debug, Clone)]
enum DictKind {
/// Compiled into the binary.
Bundled {
aff: &'static str,
dic: &'static str,
},
/// A pair of files on disk (`<stem>.aff` + `<stem>.dic`).
Files { aff: PathBuf, dic: PathBuf },
}
impl DictEntry {
/// Read and parse this dictionary into a usable [`Dictionary`].
pub fn load(&self) -> Result<Dictionary, String> {
match &self.kind {
DictKind::Bundled { aff, dic } => Dictionary::new(aff, dic)
.map_err(|e| format!("Could not parse the built-in {} dictionary: {e}", self.id)),
DictKind::Files { aff, dic } => {
let aff_text = std::fs::read_to_string(aff)
.map_err(|e| format!("Cannot read {}: {e}", aff.display()))?;
let dic_text = std::fs::read_to_string(dic)
.map_err(|e| format!("Cannot read {}: {e}", dic.display()))?;
Dictionary::new(&aff_text, &dic_text)
.map_err(|e| format!("Could not parse {}: {e}", dic.display()))
}
}
}
}
/// The list of dictionaries available to the app: the two bundled English
/// variants first, then any others discovered on disk (deduplicated by id),
/// sorted by label.
pub fn available() -> Vec<DictEntry> {
let mut entries = vec![
DictEntry {
id: "en-CA".to_string(),
label: "English (Canada) — built-in".to_string(),
kind: DictKind::Bundled {
aff: EN_CA_AFF,
dic: EN_CA_DIC,
},
},
DictEntry {
id: "en-GB".to_string(),
label: "English (UK) — built-in".to_string(),
kind: DictKind::Bundled {
aff: EN_GB_AFF,
dic: EN_GB_DIC,
},
},
];
for dir in search_dirs() {
let Ok(read) = std::fs::read_dir(&dir) else {
continue;
};
for entry in read.flatten() {
let path = entry.path();
if path.extension().and_then(|e| e.to_str()) != Some("dic") {
continue;
}
let Some(stem) = path.file_stem().and_then(|s| s.to_str()) else {
continue;
};
let aff = path.with_extension("aff");
if !aff.exists() {
continue;
}
// Don't shadow a bundled entry (or an earlier directory's copy).
if entries.iter().any(|e| e.id == stem) {
continue;
}
entries.push(DictEntry {
id: stem.to_string(),
label: format!("{stem} ({})", dir.display()),
kind: DictKind::Files { aff, dic: path },
});
}
}
entries.sort_by(|a, b| a.label.cmp(&b.label));
entries
}
/// Directories scanned for extra `.aff`/`.dic` pairs, most specific last so a
/// user copy can be preferred. Missing directories are simply skipped.
fn search_dirs() -> Vec<PathBuf> {
let mut dirs = vec![
PathBuf::from("/usr/share/hunspell"),
PathBuf::from("/usr/share/myspell"),
PathBuf::from("/usr/share/myspell/dicts"),
PathBuf::from("/usr/local/share/hunspell"),
];
if let Some(data) = dirs::data_dir() {
dirs.push(data.join("hunspell"));
}
if let Some(config) = dirs::config_dir() {
dirs.push(config.join("md-manuscript").join("dictionaries"));
}
dirs
}
/// How many distinct misspellings we compute suggestions for per check. Beyond
/// this the words are still underlined but carry no suggestions (a right-click
/// falls back to computing them on demand). Keeps a document full of unknown
/// words — names, invented terms — from stalling the background check.
const MAX_SUGGEST_WORDS: usize = 250;
/// Spell-check the prose in `text`, returning a spelling [`Match`] for every
/// unknown word (byte offsets into `text`, best suggestions first).
pub fn check(dict: &Dictionary, text: &str) -> Vec<Match> {
let mut out = Vec::new();
let mut cache: HashMap<&str, Vec<String>> = HashMap::new();
let mut suggested = 0usize;
for (offset, word) in prose_words(text) {
if should_skip(word) || dict.check(word) {
continue;
}
let replacements = if let Some(cached) = cache.get(word) {
cached.clone()
} else if suggested < MAX_SUGGEST_WORDS {
let sugg = suggestions(dict, word);
suggested += 1;
cache.insert(word, sugg.clone());
sugg
} else {
Vec::new()
};
out.push(Match {
start: offset,
end: offset + word.len(),
message: format!("“{word}” may be misspelled"),
replacements,
spelling: true,
});
}
out
}
/// Suggestions for a single word (best first, capped), for on-demand use.
pub fn suggestions(dict: &Dictionary, word: &str) -> Vec<String> {
let mut sugg = Vec::new();
dict.suggest(word, &mut sugg);
sugg.truncate(8);
sugg
}
/// Whether a token should not be checked: too short, an all-caps initialism, or
/// containing no alphabetic character.
fn should_skip(word: &str) -> bool {
let mut letters = 0usize;
let mut all_upper = true;
for ch in word.chars() {
if ch.is_alphabetic() {
letters += 1;
if !ch.is_uppercase() {
all_upper = false;
}
}
}
if letters < 2 {
return true;
}
// Skip ALL-CAPS tokens (acronyms/initialisms like ODT, HTTP, NASA).
all_upper
}
/// Extract the checkable prose words of a markdown buffer as `(byte_offset,
/// word)` pairs. Code spans, fenced/indented code blocks and link/image targets
/// are excluded (they arrive as non-`Text` events), and surrounding apostrophes
/// are trimmed so contractions like `don't` stay intact but `'quoted'` doesn't
/// keep its quotes.
fn prose_words(text: &str) -> Vec<(usize, &str)> {
use pulldown_cmark::{Event, Options, Parser, Tag, TagEnd};
let mut words = Vec::new();
let mut in_code_block = false;
let parser = Parser::new_ext(text, Options::ENABLE_STRIKETHROUGH).into_offset_iter();
for (event, range) in parser {
match event {
// Fenced/indented code-block *content* arrives as Text events, so
// suppress tokenizing while inside one. (Inline code is Event::Code.)
Event::Start(Tag::CodeBlock(_)) => in_code_block = true,
Event::End(TagEnd::CodeBlock) => in_code_block = false,
Event::Text(_) if !in_code_block => {
// Use the source slice (not the decoded Cow) so offsets are exact.
tokenize(range.start, &text[range.clone()], &mut words);
}
_ => {}
}
}
words
}
/// Split a run of prose into words, pushing `(absolute_byte_offset, word)`.
fn tokenize<'a>(base: usize, s: &'a str, out: &mut Vec<(usize, &'a str)>) {
let is_word = |c: char| c.is_alphabetic() || c == '\'' || c == '';
let mut start: Option<usize> = None;
for (idx, ch) in s.char_indices() {
match (start, is_word(ch)) {
(None, true) => start = Some(idx),
(Some(st), false) => {
push_word(base, s, st, idx, out);
start = None;
}
_ => {}
}
}
if let Some(st) = start {
push_word(base, s, st, s.len(), out);
}
}
/// Trim leading/trailing apostrophes from `s[st..en]` and, if anything remains,
/// push it with its absolute byte offset.
fn push_word<'a>(base: usize, s: &'a str, st: usize, en: usize, out: &mut Vec<(usize, &'a str)>) {
let raw = &s[st..en];
let quote = |c: char| c == '\'' || c == '';
let trimmed = raw.trim_matches(quote);
if trimmed.is_empty() {
return;
}
let lead = raw.len() - raw.trim_start_matches(quote).len();
out.push((base + st + lead, trimmed));
}
#[cfg(test)]
mod tests {
use super::*;
fn en_ca() -> Dictionary {
DictEntry {
id: "en-CA".to_string(),
label: String::new(),
kind: DictKind::Bundled {
aff: EN_CA_AFF,
dic: EN_CA_DIC,
},
}
.load()
.expect("bundled en-CA dictionary parses")
}
#[test]
fn bundled_dictionaries_are_listed_and_parse() {
let list = available();
for id in ["en-CA", "en-GB"] {
let entry = list
.iter()
.find(|d| d.id == id)
.unwrap_or_else(|| panic!("{id} should be available"));
entry
.load()
.unwrap_or_else(|e| panic!("bundled {id} should parse: {e}"));
}
}
#[test]
fn en_gb_prefers_british_spelling() {
let dict = available()
.into_iter()
.find(|d| d.id == "en-GB")
.unwrap()
.load()
.unwrap();
// "realise" is British; "color" (American) is not en-GB.
assert!(check(&dict, "realise").is_empty());
assert_eq!(check(&dict, "color").len(), 1);
}
#[test]
fn tokenizer_yields_prose_words_with_offsets() {
let text = "The cat sat.";
let mut words = Vec::new();
tokenize(0, text, &mut words);
assert_eq!(words, vec![(0, "The"), (4, "cat"), (8, "sat")]);
}
#[test]
fn tokenizer_keeps_contractions_but_trims_quotes() {
let text = "don't 'quoted'";
let mut words = Vec::new();
tokenize(0, text, &mut words);
assert_eq!(words, vec![(0, "don't"), (7, "quoted")]);
}
#[test]
fn prose_words_skip_code_spans_and_blocks() {
// Inline code and a fenced block must not be tokenized.
let text = "real word `codeword` end\n\n```\nblockword\n```\n";
let words = prose_words(text);
let found: Vec<&str> = words.iter().map(|(_, w)| *w).collect();
assert!(found.contains(&"real"));
assert!(found.contains(&"word"));
assert!(found.contains(&"end"));
assert!(!found.contains(&"codeword"));
assert!(!found.contains(&"blockword"));
}
#[test]
fn should_skip_short_and_allcaps() {
assert!(should_skip("a")); // too short
assert!(should_skip("ODT")); // acronym
assert!(should_skip("HTTP"));
assert!(!should_skip("word"));
assert!(!should_skip("The")); // initial cap is fine
}
#[test]
fn check_flags_misspellings_with_correct_offsets() {
let dict = en_ca();
let text = "The quikc brown fox.";
let matches = check(&dict, text);
assert_eq!(matches.len(), 1, "only 'quikc' is misspelled");
let m = &matches[0];
assert_eq!(&text[m.start..m.end], "quikc");
assert!(m.spelling);
// spellbook should suggest the obvious correction.
assert!(
m.replacements.iter().any(|r| r == "quick"),
"expected 'quick' among suggestions, got {:?}",
m.replacements
);
}
#[test]
fn check_respects_canadian_spelling() {
let dict = en_ca();
// "colour" is correct in en-CA; "color" is not.
assert!(check(&dict, "colour").is_empty());
assert_eq!(check(&dict, "color").len(), 1);
}
#[test]
fn check_offsets_survive_multibyte_prose() {
let dict = en_ca();
// The é is two bytes; a misspelling after it must still map to the right
// bytes (whether or not "café" itself is in the dictionary).
let text = "café qmzxk";
let matches = check(&dict, text);
let target = matches
.iter()
.find(|m| text.get(m.start..m.end) == Some("qmzxk"));
assert!(
target.is_some(),
"expected a match slicing to 'qmzxk', got {:?}",
matches
.iter()
.map(|m| &text[m.start..m.end])
.collect::<Vec<_>>()
);
}
}