% language=uk

\environment luametatex-style

\startcomponent luametatex-introduction

\startchapter[title=Introduction]

Around 2005 we started the \LUATEX\ project and it took about a decade to reach
a state where we could consider the experiments to have reached a stable state.
Pretty soon \LUATEX\ could be used in production, even if some of the interfaces
evolved, but \CONTEXT\ was kept in sync so that was not really a problem. In 2018
the functionality was more or less frozen. Of course we might add some features
in due time but nothing fundamental will change as we consider version 1.10 to be
reasonable feature complete. Among the reasons is that this engine is now used
outside \CONTEXT\ too which means that we cannot simply change much without
affecting other macro packages.

In reaching that state some decisions were delayed because they didn't go well
with a current stable version. This is why at the 2018 \CONTEXT\ meeting those
present agreed that we could move on with a follow up tagged \METATEX, a name we
already had in mind for a while, but as \LUA\ is an important component, it got
expanded to \LUAMETATEX. This follow up is a lightweight companion to \LUATEX\
that will be maintained alongside. More about the reasons for this follow up as
well as the philosophy behind it can be found in the document(s) describing the
development. During \LUATEX\ development I kept track of what happened in a
series of documents, parts of which were published as articles in user group
journals, but all are in the \CONTEXT\ distribution. I did the same with the
development of \LUAMETATEX.

The \LUAMETATEX\ engine is, as said, a follow up on \LUATEX. Just as we have
\CONTEXT\ \MKII\ for \PDFTEX\ and \XETEX, we have \MKIV\ for \LUATEX. For
\LUAMETATEX\ we have yet another version of \CONTEXT: \LMTX. By freezing \MKII,
and at some point freezing \MKIV, we can move on as we like, but we try to remain
downward compatible where possible, something that the user interface makes
possible. Although \LUAMETATEX\ can be used for production we can also use it for
possibly drastic experiments but without affecting \LUATEX. Because we can easily
adapt \CONTEXT\ to support both, no other macro package will be harmed when (for
instance) the interface that the engine provides change as part of an experiment
or cleanup of code. Of course, when we consider something to be useful, it can be
ported back to \LUATEX, but only when there are good reasons for doing so and
when no compatibility issues are involved.

By now the code of these two related engines differs a lot so in retrospect it
makes less sense to waste time on porting back. When considering this follow up
one consideration was that a lean and mean version with an extension mechanism is
a bit closer to original \TEX. Of course, because we also have new primitives,
this is not entirely true. The basic algorithms remain the same but code got
reshuffled and because we expose internal names of variables and such that is
reflected in the code base (like more granularity in nodes and token commands).
Delegating tasks to \LUA\ already meant that some aspects, especially system
dependent ones, no longer made sense and therefore had consequences for the
interface at the system level. In \LUAMETATEX\ more got delegated, like all file
related operations. The penalty of moving even more responsibility to \LUA\ has
been compensated by (hopefully) harmless optimization of code in the engine and
some more core functionality. In the process system dependencies have been
minimalized.

This manual started as an adaptation of the \LUATEX\ manual and therefore looks
similar. Some chapters are removed, others were added and the rest has been (and
will be further) adapted. It also discusses the (main) differences. Some of the
new primitives or functions that show up in \LUAMETATEX\ might show up in
\LUATEX\ at some point, but most will be exclusive to \LUAMETATEX, so don't take
this manual as reference for \LUATEX ! As long as we're experimenting we can
change things at will but as we keep \CONTEXT\ \LMTX\ synchronized users normally
won't notice this. Often you can find examples of usage in \CONTEXT\ related
documents and the source code so that serves a reference too.

For \CONTEXT\ users the \LUAMETATEX\ engine will become the default. As
mentioned, the \CONTEXT\ variant for this engine is tagged \LMTX. The pair can be
used in production, just as with \LUATEX\ and \MKIV. In fact, most users will
probably not really notice the difference. In some cases there will be a drop in
performance, due to more work being delegated to \LUA, but on the average
performance is much be better, due to some changes below the hood of the engine.
Memory consumption is also less. The timeline of development is roughly: from
2018 upto 2020 engine development, 2019 upto 2021 the stepwise code split between
\MKIV\ and \LMTX, while in 2021 and 2022 we will (mostly) freeze \MKIV\ and
\LMTX\ will be the default.

As this follow up is closely related to \CONTEXT\ development, and because we
expect stock \LUATEX\ to be used outside the \CONTEXT\ proper, there will be no
special mailing list nor coverage (or pollution) on the \LUATEX\ related mailing
lists. We have the \CONTEXT\ mailing lists for that. In due time the source code
will be part of the regular \CONTEXT\ distribution so that is then also the
reference implementation: if needed users can compile the binary themselves.

This manual sometimes refers to \LUATEX, especially when we talk of features
common to both engine, as well as to \LUAMETATEX, when it is more specific to the
follow up. A substantial amount of time went into the transition and more will go
in, so if you want to complain about \LUAMETATEX, don't bother me. Of course, if
you really need professional support with these engines (or \TEX\ in general),
you can always consider contacting the developers.

% And yes, I'm really fed up with receiving mails or seeing comments where there's
% this 'always need to be present' negative remark (nagging) about the program,
% documentation, development, support, etc. present, probably to put the writer on
% a higher stand, or maybe to compensate some other personal shortcoming ... who
% knows. This 'I need to make my stupid point' behaviour seems to come with the
% internet and it also seems to increase, but that doesn't mean that I want to deal
% with those unpleasant people for the sake of the larger "tex good". Therefore, I'm
% quite happy in the nearly always positive and motivating ConTeXt bubble. It's also
% why I (start) avoid(ing) certain mailing lists and don't really follow forums.

\blank[big]

Hans Hagen

\blank[2*big]

\starttabulate[|||]
\NC Version       \EQ \currentdate \NC \NR
\NC \LUAMETATEX   \EQ \cldcontext{LUATEXENGINE} %
                      \cldcontext{LUATEXVERSION} / %
                      \cldcontext{LUATEXFUNCTIONALITY}
                      \NC \NR
\NC \CONTEXT      \EQ MkIV \contextversion \NC \NR
\NC \LUATEX\ Team \EQ Hans Hagen, Hartmut Henkel, Taco Hoekwater, Luigi Scarso \NC \NR
\stoptabulate

\vfilll

{\bf remark:} \LUAMETATEX\ development is mostly done by Hans Hagen and in
adapting the macros to the new features Wolfgang Schuster, who knows the code
inside||out is a instrumental. In the initial phase Alan Braslau, who love
playing with the three languages did extensive testing and compiled for several
platforms. Later Mojca Miklavec make sure all compiles well on the buildbot
infrastructure. After the first release more users got involved in testing. Many
thanks for their patience! The development also triggered upgrading of the wiki
support infrastructure where Taco Hoekwater and Paul Mazaitis have teamed up. So,
progress all around.

{\bf remark:} When there are non|-|intrusive features that also make sense in
\LUATEX, these will be applied in the experimental branch first, so that there is
no interference with the stable release. However, given that in the meantime the
code bases differs a lot, it is unlikely that much will trickle back. This is no
real problem as there's not much demand for that anyway.

{\bf remark:} Most \CONTEXT\ users seem always willing to keep up with the latest
versions which means that \LMTX\ is tested well. We can therefore safely claim
that end of 2019 the code has become quite stable, although after that in some
areas there were substantial additions. There are no complaints about performance
(on my 2013 laptop this manual compiles at 24.5 pps with \LMTX\ versus 20.7 pps
for the \LUATEX\ manual with \MKIV). After updating some of the \CONTEXT\ code to
use recently added features by the end of 2020 I could do more than 25.5 pps but
don't expect spectacular bumps in performance (I need a new machine for that to
happen). Probably no one notices it, but memory consumption stepwise got reduced
too. And \unknown\ the binary is still below 3~MegaBytes on all platforms.

\stopchapter

\stopcomponent

% I'm not that strict with incrementing numbers, but let's occasionally bump the
% number. Once we're stable it might happen more systematically. For sure I don't
% want to end up with these 0.99999999 kind of numbers.

% We started with 2.00.0 and kept that number till November 2019, after Alan
% Braslau and I did the initial beta release at April 1, 2019. After more than a
% year working on the code base after the \CONTEXT\ 2019 meeting a state was
% reached where nothing fundamental got added for a while. When \LUATEX\ needs a
% patch, I check the \LUAMETATEX\ code base as the same patch might be needed
% there. On the other hand, we don't need a strict compatibility, so some patched
% in \LUATEX\ are not applied here.
%
% In November 2019 I started wondering if we should bump the number, just for the
% sake of showing that there's still some progress. So I decided to bump to 2.01.0
% then. Just as a reminder for myself: it was the day when I watched Jacob Collier
% perform \LUA\ (feat. MARO) live on YouTube (of course that is not about the
% language at all, but still a nice coincidence). Just for the fun of it the number
% bumped a few more times, just to catch up, so end 2019 we're at 2.03.5.
%
% Thanks to the patient \CONTEXT\ users we were able to apply the new macro scanner
% and protection mechanisms that were introduced mid 2020.
