#+TITLE: elel Emacs Lexical Embedding Link: a notes system whose essence is searching. Notes are found by their words (index), their meaning (semantics, second pass) and their relations (structure). * Layout - =src/= :: the engine, a Rust CLI that only /reads/ note files. - =elel.el= :: the Emacs front end, with a Gnus-like interface. * Files and notes A file is a container of one or more notes. Each note carries its own ID. Formats: - =.org= :: each top-level heading is a note; metadata in the property drawer (=:ID: :DATE: :KIND: :PARENT: :LINKS: :CONTEXT: :SUMMARY:=), tags in the heading. - =.txt= :: notes are separated by a form-feed line; each starts with =Key: value= lines (=ID Title Date Kind Tags Parent Links Context Summary=), a blank line, then the body. A link in a body is written =[[note:ID][TITLE]]= in every format, and =[[note::KIND:ID][TITLE]]= when it has a type (=KIND= is letters, digits, =-= or =_=). The title is for the reader; the ID is how the engine finds the note. An ID has no whitespace; a uuid-style ID is fine as it is, e.g. =[[note:][T]]=. Typed links can also go in the =LINKS= property, as =supports:ID=. In org, =note= is a registered link type, so org follows (=C-c C-o=), completes (=C-c C-l=, which offers the title as the description) and exports these links itself. * Searching One query string. Filters are words of the form =key:value=; they are always ANDed. The remaining words go to the full text index (ANDed; ="a phrase"=, =-word=, =a OR b= and =title:foo= work; Chinese is matched by character pairs). | Filter | Meaning | |-------------------------+-------------------------------------------------| | =tag:X= =-tag:X= | has / lacks the tag X | | =tag:*= =-tag:*= | has some tag / has no tag | | =tag:alg*= | has a tag that starts with =alg= | | =kind:X= =kind:*= | the KIND is X / there is a KIND (=-kind:= too) | | =file:X= =-file:X= | the path (relative to the root) contains X | | =from:ID= | the notes that ID links to | | =to:ID= | the notes that link to ID | | =parent:ID= | the children of ID | | =thread:ID= | the whole thread of ID | | =date:2025= | the DATE starts with 2025 | | =date:2025-03..2025-06= | a range; =date:..2024= and =date:2025..= work | | =is:inbox= | no links, parent, children or tags | | =like:X= =-like:X= | by meaning: X is a note ID or a text (see Embeddings) | Quote a value with spaces: =file:"my notes"=. The empty query lists every note. Without words, results are newest first; with words, by score. * Engine #+begin_src sh cargo build --release elel index [--no-embed] # scan ELEL_ROOT (default ~/notes), update the index and the embeddings elel search [-n N] [--hybrid] -- QUERY elel count -- QUERY... # the number of notes for each query elel hood ID [--hops N] elel get ID # the note, with its exact text in "raw" elel check # missing/duplicate IDs, dangling links elel tags [--like ID] # with --like: the tags that fit the note, best first #+end_src The index lives in ELEL_INDEX (default =~/.local/share/elel=) and can be deleted at any time. * Emacs #+begin_src emacs-lisp (add-to-list 'load-path "~/.emacs.d/claude/elel") (autoload 'elel "elel" nil t) (setq elel-root "~/notes") #+end_src The interface follows Gnus. =M-x elel= opens the views buffer (the group buffer); =RET= opens a view in the summary buffer, with the article below it. =elel-views= is a list of =(NAME QUERY . PLIST)=; =:limit N= caps a view and =:hidden t= keeps it out of sight until =L=. Views for a tag or a file are just more entries, e.g. =("Math" "tag:math")=. *Views buffer*: =RET= open, =L= list hidden views, =G= or =/= search, =u= update the index, =g= redraw, =N= new note, =C= check. *Summary buffer*: the notes are threaded by PARENT, drawn as in Gnus (see =elel-thread-*=). =RET= shows the note, =SPC= / =DEL= scroll it, =n= / =p= next / previous note, =g= show the note again (as in Gnus; to refresh the list, search again with =G=), =/= narrow the query, =s= snippets, ==== make the summary fill the frame (=C-u ==== restores both), =A= select the article window, =#= / =-= mark liked / disliked, =R= related, =S= suggestions, =a= add the whole thread of the note, =^= parent, =c= children, =o= outgoing links, =b= backlinks, =H= neighbourhood, =e= edit, =r= / =N= child / new note (finish with =C-c C-c=), =q= leave. *Article buffer*: the note as it is in its file, in the major mode that =elel-major-mode-alist= picks (org-mode for =.org=), with the minor mode =elel-article-mode= on top. =SPC= / =j= scroll (at the end, go to the next note), =DEL= / =k= back, =n= / =p= next / previous note of the summary, =TAB= / =S-TAB= and =RET= move to and follow links, =l= back, =^= =c= =o= =b= =H= =a= as in the summary, =h= go to the summary, =q= close the article, =e= edit, =r= child note, =g= reread. You may close the summary window; =n= and =p= still work. =elel-summary-window-height= and =elel-article-keep-summary= control the windows. =M-x elel-assign-ids= adds missing IDs. In the note files (=elel-link-mode=, on automatically under =elel-root=): =C-c n l= inserts a link by picking a title (an active region becomes the description), =C-c C-o= follows the link at point (org's own behaviour elsewhere), =C-c n o= opens the note around point. * Importing old notes =elel-import.el= (Emacs Lisp; it only reads the sources and never overwrites). Without an output directory, or with =dry-run=, it only reports. #+begin_src sh # notes kept as Gnus drafts (Gcc nnmaildir+notes:GROUP): one GROUP.org # per group, Message-IDs as IDs, In-Reply-To as PARENT emacs --batch -L ~/.emacs.d/claude/elel -l elel-import \ --eval '(elel-import-drafts "~/News/drafts/drafts" "~/notes")' # an org-roam v1 directory: notes are collected into few files, each named # by a tag chosen to cover many notes (see elel-import-roam-min-group) emacs --batch -L ~/.emacs.d/claude/elel -l elel-import \ --eval '(elel-import-roam "~/org/roam" "~/notes")' #+end_src Both are also commands (=M-x elel-import-drafts=, =M-x elel-import-roam=) that ask before writing. * Embeddings The "E" of elel. Install [[https://ollama.com][Ollama]] and run =ollama pull qwen3-embedding:0.6b=. Then =elel index= also embeds the notes (the first run takes a few minutes; later runs only embed what changed). If Ollama is not running, indexing still works; the notes stay pending and the next run catches up. =--no-embed= skips it. - *Units*: a note is cut into paragraph chunks (=src/chunk.rs=). A paragraph that is only a formula joins its neighbour, small paragraphs are merged, source blocks and =\begin..\end= stay whole, and the title is put in front of each chunk. A note's own vector is the mean of its chunks. - *Storage*: =/vec/= holds =f16.bin= (exact vectors), =bits.bin= (one sign bit per dimension, for a first Hamming pass when there are more than 2000 chunks) and =meta.json=. All of it can be rebuilt. Changing the model (=ELEL_MODEL=, default =qwen3-embedding:0.6b=) re-embeds everything. =ELEL_OLLAMA= is the server URL. - *=like:X=*: notes close in meaning. X is a note ID (the whole note is compared) or a text such as =like:"moduli of curves"= (each note scores as its best chunk, which is shown as the snippet). Several =like:= are merged by their mean; =-like:Y= pushes away from Y. Other filters still apply: =like:ID tag:algebra date:2024..=. - *Hybrid*: =search --hybrid= with words fuses the text ranking and the meaning ranking by reciprocal rank fusion. In Emacs =elel-hybrid-search= (default t) does this for every search; without Ollama it falls back to the text search. - *Tags as anchors*: =elel tags --like ID= scores each tag (the mean of the other notes that carry it) against the note, to suggest tags. In the summary buffer, =#= marks a note as liked, =-= as disliked, and =R= shows the notes closest to the marked ones (or to the note at point). In the article, =R= shows the notes like this one. =S= (=elel-suggest=) lists the notes that the note could link to (the ones it is already related to are left out), with the best tags in the header; =i= there inserts =[[note:ID][TITLE]]= where the suggestion was asked for. After a new note is saved, the suggestions appear by themselves (=elel-suggest-after-capture=), and =i= inserts at the point where you typed =C-c C-c=. =elel-insert-link= lists the notes closest to the current note first. * Not yet done Anchors written as free phrases (instead of tags), a Gnus link adapter, an optional daemon for faster calls.