====================================================================== 2023-06-02 ====================================================================== This is a "diary" that records my thoughts when trying to debug the forest manipulations. Basically the forests are wrongly "merged", and produce weird clones. Right now I have finished refactoring most of the codes, but still fail to perceive why the forests behave in that way. I even finished a proto-type of my procedural macros which help me reduce boiler-plates but are not necessary. Right now I think I need more refined control over the details of the forests manipulations. To be more precise, I need to be able to observe, step by step, how the chain-rule machine manipulated the forests. This means in particular to be able to produce graphs of half-way forests whenever I want. I have not yet figured out how to do this exactly. The rough plan is to compress all steps of graphs into an "incremental format" that stores not just forests information, but how they were created, and allow the users to re-create the forests step by step. Simply put, this is just another feature that I would like to have for the end-users, but postponed to later since that was not essential for my developments. Now this seems to be quite important for me to properly observe the forests, so it is perhaps time to implement this feature first. ====================================================================== 2023-07-22 ====================================================================== Now a stable version of the package is in place, so I can start working on some non-essential features, like the tokenization process and the visulization feature. Amongst these, the tokenization process is the most important, as the users will not expect to work with tokens: they expect to use characters. So in some sense this package is not ready for the end-users yet. My thought is to offer two modes: one for operating on characters and the other for operating on tokens directly. This can be achieved by letting the data type representing terminals contain more information on how to match characters. My thought is to attach either a vector of "machine codes" or a single character to terminals. If a terminal is attached a single character, it can only match that character. On the other hand, if a terminal corresponds to a vector of machine codes, then it means to execute this sequence of codes to determine if a character matches. We can even simplify the format of machine codes to a sequence of ranges, and a character only matches if it is contained in one of the ranges. After this is finished, we can bump the version to 0.1.3 and simplify the build process. Hope it all goes well.