aboutsummaryrefslogtreecommitdiff
path: root/README.org
diff options
context:
space:
mode:
Diffstat (limited to 'README.org')
-rw-r--r--README.org170
1 files changed, 170 insertions, 0 deletions
diff --git a/README.org b/README.org
new file mode 100644
index 0000000..ae298ed
--- /dev/null
+++ b/README.org
@@ -0,0 +1,170 @@
+#+TITLE: elel
+
+Emacs Lexical Embedding Link: a notes system whose essence is
+searching. Notes are found by their words (index), their meaning
+(semantics, second pass) and their relations (structure).
+
+* Layout
+- =src/= :: the engine, a Rust CLI that only /reads/ note files.
+- =elel.el= :: the Emacs front end, with a Gnus-like interface.
+
+* Files and notes
+A file is a container of one or more notes. Each note carries its own
+ID. Formats:
+- =.org= :: each top-level heading is a note; metadata in the property
+ drawer (=:ID: :DATE: :KIND: :PARENT: :LINKS: :CONTEXT: :SUMMARY:=),
+ tags in the heading.
+- =.txt= :: notes are separated by a form-feed line; each starts with
+ =Key: value= lines (=ID Title Date Kind Tags Parent Links Context
+ Summary=), a blank line, then the body.
+A link in a body is written =[[note:ID][TITLE]]= in every format, and
+=[[note::KIND:ID][TITLE]]= when it has a type (=KIND= is letters, digits,
+=-= or =_=). The title is for the reader; the ID is how the engine finds
+the note. An ID has no whitespace; a uuid-style ID is fine as it is,
+e.g. =[[note:<a.b@c.local>][T]]=. Typed links can also go in the =LINKS=
+property, as =supports:ID=. In org, =note= is a registered link type, so
+org follows (=C-c C-o=), completes (=C-c C-l=, which offers the title as
+the description) and exports these links itself.
+
+* Searching
+One query string. Filters are words of the form =key:value=; they are
+always ANDed. The remaining words go to the full text index (ANDed;
+="a phrase"=, =-word=, =a OR b= and =title:foo= work; Chinese is
+matched by character pairs).
+| Filter | Meaning |
+|-------------------------+-------------------------------------------------|
+| =tag:X= =-tag:X= | has / lacks the tag X |
+| =tag:*= =-tag:*= | has some tag / has no tag |
+| =tag:alg*= | has a tag that starts with =alg= |
+| =kind:X= =kind:*= | the KIND is X / there is a KIND (=-kind:= too) |
+| =file:X= =-file:X= | the path (relative to the root) contains X |
+| =from:ID= | the notes that ID links to |
+| =to:ID= | the notes that link to ID |
+| =parent:ID= | the children of ID |
+| =thread:ID= | the whole thread of ID |
+| =date:2025= | the DATE starts with 2025 |
+| =date:2025-03..2025-06= | a range; =date:..2024= and =date:2025..= work |
+| =is:inbox= | no links, parent, children or tags |
+| =like:X= =-like:X= | by meaning: X is a note ID or a text (see Embeddings) |
+Quote a value with spaces: =file:"my notes"=. The empty query lists
+every note. Without words, results are newest first; with words, by
+score.
+
+* Engine
+#+begin_src sh
+cargo build --release
+elel index [--no-embed] # scan ELEL_ROOT (default ~/notes), update the index and the embeddings
+elel search [-n N] [--hybrid] -- QUERY
+elel count -- QUERY... # the number of notes for each query
+elel hood ID [--hops N]
+elel get ID # the note, with its exact text in "raw"
+elel check # missing/duplicate IDs, dangling links
+elel tags [--like ID] # with --like: the tags that fit the note, best first
+#+end_src
+The index lives in ELEL_INDEX (default =~/.local/share/elel=) and can be
+deleted at any time.
+
+* Emacs
+#+begin_src emacs-lisp
+(add-to-list 'load-path "~/.emacs.d/claude/elel")
+(autoload 'elel "elel" nil t)
+(setq elel-root "~/notes")
+#+end_src
+The interface follows Gnus. =M-x elel= opens the views buffer (the
+group buffer); =RET= opens a view in the summary buffer, with the
+article below it. =elel-views= is a list of =(NAME QUERY . PLIST)=;
+=:limit N= caps a view and =:hidden t= keeps it out of sight until =L=.
+Views for a tag or a file are just more entries, e.g.
+=("Math" "tag:math")=.
+
+*Views buffer*: =RET= open, =L= list hidden views, =G= or =/= search,
+=u= update the index, =g= redraw, =N= new note, =C= check.
+
+*Summary buffer*: the notes are threaded by PARENT, drawn as in Gnus
+(see =elel-thread-*=). =RET= shows the note, =SPC= / =DEL= scroll it,
+=n= / =p= next / previous note, =g= show the note again (as in Gnus; to
+refresh the list, search again with =G=), =/= narrow the query, =s=
+snippets, ==== make the summary fill the frame (=C-u ==== restores both),
+=A= select the article window, =#= / =-= mark liked / disliked, =R= related, =S= suggestions, =a= add the whole thread of the note, =^= parent, =c= children,
+=o= outgoing links, =b= backlinks, =H= neighbourhood, =e= edit, =r= / =N=
+child / new note (finish with =C-c C-c=), =q= leave.
+
+*Article buffer*: the note as it is in its file, in the major mode that
+=elel-major-mode-alist= picks (org-mode for =.org=), with the minor mode
+=elel-article-mode= on top. =SPC= / =j= scroll (at the end, go to the next
+note), =DEL= / =k= back, =n= / =p= next / previous note of the summary,
+=TAB= / =S-TAB= and =RET= move to and follow links, =l= back, =^= =c= =o=
+=b= =H= =a= as in the summary, =h= go to the summary, =q= close the
+article, =e= edit, =r= child note, =g= reread. You may close the
+summary window; =n= and =p= still work. =elel-summary-window-height=
+and =elel-article-keep-summary= control the windows.
+
+=M-x elel-assign-ids= adds missing IDs.
+
+In the note files (=elel-link-mode=, on automatically under
+=elel-root=): =C-c n l= inserts a link by picking a title (an active
+region becomes the description), =C-c C-o= follows the link at point
+(org's own behaviour elsewhere), =C-c n o= opens the note around
+point.
+
+* Importing old notes
+=elel-import.el= (Emacs Lisp; it only reads the sources and never
+overwrites). Without an output directory, or with =dry-run=, it only
+reports.
+#+begin_src sh
+# notes kept as Gnus drafts (Gcc nnmaildir+notes:GROUP): one GROUP.org
+# per group, Message-IDs as IDs, In-Reply-To as PARENT
+emacs --batch -L ~/.emacs.d/claude/elel -l elel-import \
+ --eval '(elel-import-drafts "~/News/drafts/drafts" "~/notes")'
+# an org-roam v1 directory: notes are collected into few files, each named
+# by a tag chosen to cover many notes (see elel-import-roam-min-group)
+emacs --batch -L ~/.emacs.d/claude/elel -l elel-import \
+ --eval '(elel-import-roam "~/org/roam" "~/notes")'
+#+end_src
+Both are also commands (=M-x elel-import-drafts=, =M-x elel-import-roam=)
+that ask before writing.
+
+* Embeddings
+The "E" of elel. Install [[https://ollama.com][Ollama]] and run
+=ollama pull qwen3-embedding:0.6b=. Then =elel index= also embeds the
+notes (the first run takes a few minutes; later runs only embed what
+changed). If Ollama is not running, indexing still works; the notes
+stay pending and the next run catches up. =--no-embed= skips it.
+
+- *Units*: a note is cut into paragraph chunks (=src/chunk.rs=). A
+ paragraph that is only a formula joins its neighbour, small paragraphs
+ are merged, source blocks and =\begin..\end= stay whole, and the title is
+ put in front of each chunk. A note's own vector is the mean of its
+ chunks.
+- *Storage*: =/vec/= holds =f16.bin= (exact vectors),
+ =bits.bin= (one sign bit per dimension, for a first Hamming pass when
+ there are more than 2000 chunks) and =meta.json=. All of it can be
+ rebuilt. Changing the model (=ELEL_MODEL=, default
+ =qwen3-embedding:0.6b=) re-embeds everything. =ELEL_OLLAMA= is the
+ server URL.
+- *=like:X=*: notes close in meaning. X is a note ID (the whole note
+ is compared) or a text such as =like:"moduli of curves"= (each note
+ scores as its best chunk, which is shown as the snippet). Several
+ =like:= are merged by their mean; =-like:Y= pushes away from Y.
+ Other filters still apply: =like:ID tag:algebra date:2024..=.
+- *Hybrid*: =search --hybrid= with words fuses the text ranking and the
+ meaning ranking by reciprocal rank fusion. In Emacs
+ =elel-hybrid-search= (default t) does this for every search; without
+ Ollama it falls back to the text search.
+- *Tags as anchors*: =elel tags --like ID= scores each tag (the mean of
+ the other notes that carry it) against the note, to suggest tags.
+
+In the summary buffer, =#= marks a note as liked, =-= as disliked, and
+=R= shows the notes closest to the marked ones (or to the note at point).
+In the article, =R= shows the notes like this one. =S= (=elel-suggest=)
+lists the notes that the note could link to (the ones it is already
+related to are left out), with the best tags in the header; =i= there
+inserts =[[note:ID][TITLE]]= where the suggestion was asked for. After a
+new note is saved, the suggestions appear by themselves
+(=elel-suggest-after-capture=), and =i= inserts at the point where you
+typed =C-c C-c=. =elel-insert-link= lists the notes closest to the
+current note first.
+
+* Not yet done
+Anchors written as free phrases (instead of tags), a Gnus link
+adapter, an optional daemon for faster calls.