diff options
| author | JSDurand <mmemmew@gmail.com> | 2023-08-04 16:36:50 +0800 |
|---|---|---|
| committer | JSDurand <mmemmew@gmail.com> | 2023-08-04 16:36:50 +0800 |
| commit | bfb85bf0cd7a75b910d7125c69d5e5eae2bd51c1 (patch) | |
| tree | 4c6b3acd208909a4539da2c9acdebb8f3a7f95a2 /grammar/src | |
| parent | 7033187abaf42772097377c0a1ffc2cd4cefdada (diff) | |
add a crate 'tokens'
This crate is responsible for the users to conveniently define their
own tokenizers. Right now the user has to tokenize the input string
manually, which is not quite what is expected from a parser, from the
view point of users.
Diffstat (limited to 'grammar/src')
0 files changed, 0 insertions, 0 deletions