%%! TEX program = lualatex
\ProvidesFile{saveenv.tex} [2022/12/23 v0.0.1 ]
\documentclass{l3doc}
\EnableCrossrefs
\CodelineIndex
\RecordChanges
\fvset{gobble=0,tabsize=4,frame=single,numbers=left,numbersep=3pt}
\usepackage{hyperref}
\usepackage{csquotes}

%\MakeOuterQuote{"}
\begin{document}
\GetFileInfo{\jobname.tex}
\DoNotIndex{\begin,\end,\detokenize,\ExplSyntaxOn}

\iffalse\ExplSyntaxOff\fi  % (fix some syntax highlighting issue)

\title{\textsf{saveenv} --- Save environment content verbatim
\thanks{This file describes version \fileversion, last revised \filedate.}
}
\author{user202729%
%\thanks{E-mail: (not set)}
}
\date{Released \filedate}

\maketitle


\let\backtick=`
\MakeShortVerb{`}  % something wrong with \DefineShortVerb

\begin{abstract}
Allow users to collect environment content verbatim.
\end{abstract}

\section{Motivation}

This package provides tools to create your own verbatim environments, and works in all values of \cs{endlinechar}.

\section{Comparison to existing similar packages}

\begin{itemize}
	\item \pkg{scontents} package is similar; however it does not allow programmer to programmatically retrieve the macro content, only do a limited number of actions with them (execute, typeset as verbatim code, etc.).
	\item \pkg{fancyvrb}, \pkg{xsimverb} and \pkg{verbatimcopy} provides some internal macro for defining similar environments, however they're internal and/or too specialized (writes to file etc.).
	\item \pkg{filecontentsdef} and \pkg{newverbs} provides very similar facilities (\env{filecontentsdefmacro} environment and |\Collectverbenv| command respectively); however it requires |\endlinechar=13| and does not support |\ExplSyntaxOn| environment.
	\item \pkg{verbatim} provides facilities to define custom verbatim environment that processes the content line-by-line, however the interface is more complicated.

		For comparison, the following code

\begin{verbatim}
\ExplSyntaxOn
\makeatletter
\cs_generate_variant:Nn \tl_build_gput_right:Nn {No}
\NewDocumentEnvironment{}{}{
	\@bsphack

	\endlinechar=`\^^M ~  \let\do\@makeother \dospecials
	\@makeother \^^I
	%\cctab_select:N \c_other_cctab
	% the above cannot be used, as \verbatim@ relies on
	% the environment name having letter catcode

	\catcode `\^^M \active

	\tl_build_gbegin:N \__all_data
	\def\verbatim@processline{
		\tl_build_gput_right:No \__all_data{\the\verbatim@line ^^J}
	}
	\verbatim@
}{
	\tl_build_gend:N \__all_data
	\str_gset:NV \__all_data \__all_data

	% can do something else with \__all_data here

	\@esphack
}
\makeatother
\ExplSyntaxOff
\end{verbatim}

defines an environment that saves the data similar to \env{saveenv} environment described below (with the overhead of |\tl_build_*| functions),
but inside |\ExplSyntaxOn| environment it generates a spurious space at the beginning of the string.
\end{itemize}



\section{Main environment}

\DescribeEnv{saveenv}
  Environment that saves its body.
  
  For example, the following code
\begin{verbatim}
\begin{saveenv}{\data}
123
456
\end{saveenv}
\end{verbatim}
will save the string \texttt{123\meta{newline}456\meta{newline}} globally into |\data|.

Remark: it is consistent to keep the trailing newline, as
\begin{verbatim}
\begin{saveenv}{\data}
\end{saveenv}
\end{verbatim}
will make |\data| empty, and
\begin{verbatim}
\begin{saveenv}{\data}

\end{saveenv}
\end{verbatim}
will make |\data| consist of a single \meta{newline}.

The braces around |{\data}| is optional; however, in the unlikely case if \cs{endlinechar} has the \enquote{letter} catcode, it might be absorbed and gives unexpected result.

A newline is represented as a token with charcode 10 (|^^J|) and catcode other.

\textbf{Note} that this is unusual, for comparison \pkg{filecontentsdef} stores it as an active |^^M|.

In the current implementation, it can be used in functions such as \cs{iow_now:Nn} and newline characters will appear as newline.
(the number 10 is hard coded in the implementation of |\iow_now:Nn|,
nevertheless I'm not sure if this behavior is guaranteed. Manually replacing them with |\iow_newline:| tokens and |x|-expand the result
would be guaranteed to work)

The assignment is global, and done before the macro \cs{endsaveenv} is executed.

The data is represented as an \pkg{expl3}-string, that is, a sequence of tokens with catcode 12 (for non-space characters) or 10 (for space character).

In other words, the token list is equal to its own |\detokenize|.

\DescribeEnv{saveenvghost}
Similar to above; however the content inside is still typesetted/executed, and the Sync\TeX\ information is preserved.

Note that this environment is implemented by reading the whole file from the beginning, therefore there might be some performance hit for large files and multiple usages of the environment. Use sparingly.

\DescribeEnv{saveenvkeeplast}
Similar to above; however the final newline and the space characters before \texttt{\cs{end}\{\meta{environment name}\}} are preserved.

For example, the example above with \texttt{saveenv} replaced with \texttt{saveenvkeeplast}
will save the string \texttt{123\meta{newline}456\meta{newline}} into |\data| instead.

\section{Reinsert environments}

Sometimes it's desirable to execute something (e.g. do some local assignments) after the group ends.

There are a few options:
\begin{itemize}
	\item use |\aftergroup|,
	\item close the group, execute the code and open a new one (remember to preserve the value of |\@currenvir|),
	\item use one of the environments below.
\end{itemize}

\DescribeEnv{saveenvreinsert}
Environment that takes two arguments, the \meta{str var} to be set and the code to be put after the group end.

Usage example: If the following code is executed
\begin{verbatim}
\begin{saveenvreinsert}{\data}{\myfunction {args etc.}}
123
456
\end{saveenvreinsert}
some other content
\end{verbatim}
the effect would be identical to as if |\data| is set to \texttt{123\meta{newline}456\meta{newline}},
then the following code is executed
\begin{verbatim}
\myfunction {args etc.}some other content
\end{verbatim}
If the second argument is empty, this environment is identical to the \env{saveenv} environment.

Note that in the example above |\myfunction| is executed after the group ends,
while any code in the second block of the |\NewDocumentEnvironment| definition
would be executed before the group ends (such as in the example below), thus any local assignment will not persist.

\DescribeEnv{saveenvkeeplastreinsert}
Same as above, but the trailing whitespaces (after the last newline) are preserved.

\section{Extending the environments}

All of the environments are extensible.

Follow the instruction in \url{https://tex.stackexchange.com/q/14683/250119} to define an environment based on an existing one.

For example, the following definition

\begin{verbatim}
\ExplSyntaxOn
\NewDocumentEnvironment{custom}{}{
	\saveenv \data
}{
	\endsaveenv
	\tl_show:N \data
}
\end{verbatim}
will define an environment \env{custom} that prints out the content inside the environment using |\tl_show:N|.

Note that in this case |\tl_show:N \data| is executed before the group ends.

\section{Limitation}

\begin{itemize}
	\item This package does not process the content line-by-line. Therefore:
		\begin{itemize}
			\item The memory consumption might be larger than approaches that process the content line-by-line, in case you only need to do something with the line.

				Nevertheless, in modern computers, the overhead is negligible.
			\item In case it's desired to typeset the content afterwards, it's difficult to preserve the Sync\TeX\ data.

				(although with Lua\LaTeX\ it's possible, see \pkg{rescansync} package)
		\end{itemize}
		
	\item Currently nested environments with the same name are not supported (unlike \pkg{scontents} package).

	\item Note that there must be nothing after the |\begin|\texttt{\{\meta{environment name}\}} or the |\end|\texttt{\{\meta{environment name}\}} line.

	\item Note that |\end|\texttt{\{\meta{environment name}\}} must not be in the middle of any line.

	\item The \cs{endlinechar} must not be tokenized in advance (or if it is, its catcode must be 12/other).
		This might happen when for example your environment looks ahead for optional argument.
		See \url{https://tex.stackexchange.com/q/649331/250119} for an example and one way how to fix it.

		Alternatively, you can 
		\begin{itemize}
			\item manually remove the end line token,
			\item set |\endlinechar=-1\relax| inside the environment before calling \cs{saveenv}.
		\end{itemize}
		The value of |\endlinechar| will automatically be reset after the group of the environment is closed.
\end{itemize}



\StopEventually{%
  \PrintChanges
  \PrintIndex
}
\Finale
\end{document}

