Add some auxiliary data.

Try to fix some minor issues.

1 files changed, 37 insertions, 0 deletions

diff --git a/DIARY b/DIARY
index 13c9c6e..afef8e2 100644
--- a/DIARY
+++ b/DIARY
@@ -1,3 +1,7 @@
+======================================================================
+                              2023-06-02
+======================================================================
+
 This is a "diary" that records my thoughts when trying to debug the
 forest manipulations.  Basically the forests are wrongly "merged", and
 produce weird clones.
@@ -23,3 +27,36 @@ the end-users, but postponed to later since that was not essential for
 my developments.  Now this seems to be quite important for me to
 properly observe the forests, so it is perhaps time to implement this
 feature first.
+
+
+
+======================================================================
+                              2023-07-22
+======================================================================
+
+Now a stable version of the package is in place, so I can start
+working on some non-essential features, like the tokenization process
+and the visulization feature.
+
+Amongst these, the tokenization process is the most important, as the
+users will not expect to work with tokens: they expect to use
+characters.  So in some sense this package is not ready for the
+end-users yet.
+
+My thought is to offer two modes: one for operating on characters and
+the other for operating on tokens directly.  This can be achieved by
+letting the data type representing terminals contain more information
+on how to match characters.
+
+My thought is to attach either a vector of "machine codes" or a single
+character to terminals.  If a terminal is attached a single character,
+it can only match that character.
+
+On the other hand, if a terminal corresponds to a vector of machine
+codes, then it means to execute this sequence of codes to determine if
+a character matches.  We can even simplify the format of machine codes
+to a sequence of ranges, and a character only matches if it is
+contained in one of the ranges.
+
+After this is finished, we can bump the version to 0.1.3 and simplify
+the build process.  Hope it all goes well.