[Templates] PATCH: enhances latex filter to cope with forward references, indexes and bibliographies (take 2)
[prev] [thread] [next] [Date index for 2004/07/16]
From: Andrew Ford
Subject: [Templates] PATCH: enhances latex filter to cope with forward references, indexes and bibliographies (take 2)
Date: 09:01 on 16 Jul 2004
Subject: [Templates] PATCH: enhances latex filter to cope with forward references, indexes and bibliographies (take 2)
Date: 09:01 on 16 Jul 2004
This is a multi-part message in MIME format.
--------------060400010501010800040703
Content-Type: multipart/alternative;
boundary="------------050005010805010900010205"
--------------050005010805010900010205
Content-Type: text/plain; charset=us-ascii; format=flowed
Content-Transfer-Encoding: 7bit
This is the second version of my patch for the Template::Filters module
(against the current CVS tree) to enhance the latex filter so that it
re-runs latex to resolve forward references, and generate and format the
index and/or bibliography if the document includes them.
This version of the patch includes a check for missing table of contents
or list of tables or figures files on the first run, so a document with
a table of contents, but no forward references will have latex run at
least twice so that the TOC is generated properly (the first run just
writes TOC entries to the file doc.toc, the second run reads the file
and generates the TOC). Of course if a document includes a TOC then
once the TOC is included in the document the page numbers may change,
but the filter should notice the warning that labels have changed and
re-run latex to sort out the TOC.
The patch also modifies Makefile.PL and Template::Config.pm to do
configuration of the bibtex and makeindex paths.
I wanted to change the Template::Manual::Filters document to reflect the
changes I made, but couldn't quite work out where the source document
was. The following text replaces the last paragraph describing the
latex filter (if the patch meets with approval, then perhaps someone
could apply it and add the text to the documentation source):
The latex filter will re-run latex or pdflatex if the document
contains undefined references after the first run. It will run bibtex
if it undefined citations are reported, and will run makeindex if it
detects a ".idx" file. If bibtex or makeindex are run then latex (or
pdflatex) are run up to three more times to allow forward references
to stabilize.
Note that the latex filter runs up to eight external programs, so it
isn't very fast. But for modest documents the performance is adequate,
even for interactive applications.
I have run a "make test" and everything looks fine. I have not added
tests for forward references, index entries or citations in that I'm not
sure how to go reliably poking around in a dvi, ps or pdf file looking
for appropriate results. I have though run ttree over one of my web
document hierarchies that uses these features and they do work. I'll
have a bit more of a think about it to get some effective tests written.
I think that with this patch the latex filter will cope with most latex
documents. The main failing is that documents in the source tree cannot
be included in the main latex document (with \include{file}) unless one
adds the relevant directories to the TEXINPUTS environment variable when
ttree is run (or when the Template->process method is invoked).
Andrew
--
Andrew Ford, Director Pauntley Prints / Ford & Mason Ltd
A.Ford@xxxxxxxxxx.xx.xx South Wing Compton House
pauntley-prints.co.uk Compton Green, Redmarley Tel: +44 1531 829900
ford-mason.co.uk Gloucester GL19 3JB Fax: +44 1531 829901
refcards.com/cronolog.org Great Britain Mobile: +44 7785 258278
--------------050005010805010900010205
Content-Type: text/html; charset=us-ascii
Content-Transfer-Encoding: 7bit
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
This is the second version of my patch for the Template::Filters module
(against the
current CVS tree) to enhance the latex filter so that it re-runs latex
to resolve forward references, and generate and format the index and/or
bibliography if the document includes them. <br>
<br>
This version of the patch includes a check for missing table of
contents or list of tables or figures files on the first run, so a
document with a table of contents, but no forward references will have
latex run at least twice so that the TOC is generated properly (the
first run just writes TOC entries to the file doc.toc, the second run
reads the file and generates the TOC). Of course if a document
includes a TOC then once the TOC is included in the document the page
numbers may change, but the filter should notice the warning that
labels have changed and re-run latex to sort out the TOC.<br>
<br>
The patch also modifies Makefile.PL and
Template::Config.pm to do configuration of the bibtex and makeindex
paths.<br>
<br>
I wanted to change the Template::Manual::Filters document to reflect
the changes I made, but couldn't quite work out where the source
document was. The following text replaces the last paragraph
describing the latex filter (if the patch meets with approval, then
perhaps someone could apply it and add the text to the documentation
source):<br>
<br>
<blockquote>The latex filter will re-run latex or pdflatex if the
document<br>
contains undefined references after the first run. It will run bibtex<br>
if it undefined citations are reported, and will run makeindex if it<br>
detects a ".idx" file. If bibtex or makeindex are run then latex (or<br>
pdflatex) are run up to three more times to allow forward references<br>
to stabilize.<br>
<br>
Note that the latex filter runs up to eight external programs, so it<br>
isn't very fast. But for modest documents the performance is adequate,<br>
even for interactive applications.<br>
</blockquote>
I have run a "make test" and everything looks fine. I have not added
tests for forward references, index entries or citations in that I'm
not sure how to go reliably poking around in a dvi, ps or pdf file
looking for appropriate results. I have though run ttree over one of
my web document hierarchies that uses these features and they do work.
I'll have a bit more of a think about it to get some effective tests
written.<br>
<br>
I think that with this patch the latex filter will cope with most latex
documents. The main failing is that documents in the source tree
cannot be included in the main latex document (with \include{file})
unless one adds the relevant directories to the TEXINPUTS environment
variable when ttree is run (or when the Template->process method is
invoked).<br>
<br>
Andrew
<pre class="moz-signature" cols="72">--
Andrew Ford, Director Pauntley Prints / Ford & Mason Ltd
<a class="moz-txt-link-abbreviated" href="mailto:A.Ford@xxxxxxxxxx.xx.xx">A.Ford@xxxxxxxxxx.xx.xx</a>; South Wing Compton House
pauntley-prints.co.uk Compton Green, Redmarley Tel: +44 1531 829900
ford-mason.co.uk Gloucester GL19 3JB Fax: +44 1531 829901
refcards.com/cronolog.org Great Britain Mobile: +44 7785 258278
</pre>
</body>
</html>
--------------050005010805010900010205--
--------------060400010501010800040703
Content-Type: text/plain;
name="template-filter-latex-patch.txt"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
filename="template-filter-latex-patch.txt"
--- Makefile.PL.orig 2003-10-09 09:42:20.000000000 +0100
+++ Makefile.PL 2004-07-15 12:07:36.000000000 +0100
@@ -16,6 +16,7 @@
use vars qw( $TT_VERSION $TT_PREFIX $TT_IMAGES $TT_RUN_DBI
$TT_BUILD_DOCS $TT_SPLASH_DOCS $TT_EXAMPLES $TT_EXTRAS
$TT_LATEX_ENABLE $TT_LATEX_PATH $TT_PDFLATEX_PATH $TT_DVIPS_PATH
+ $TT_BIBTEX_PATH $TT_MAKEINDEX_PATH
$TT_XS_ENABLE $TT_XS_DEFAULT
$TT_SPLASH_THEME $TT_QUIET $TT_ACCEPT $TT_YES );
@@ -76,6 +77,8 @@
TT_LATEX install LaTeX filter (y if LaTeX found)
TT_LATEX_PATH path to latex (system dependant)
+ TT_BIBTEX_PATH path to bibtex (system dependant)
+ TT_MAKEINDEX_PATH path to makeindex (system dependant)
TT_PDFLATEX_PATH path to pdflatex ( " " " )
TT_DVIPS_PATH path to dvips ( " " " )
@@ -109,6 +112,8 @@
$TT_XS_DEFAULT = 'y';
$TT_LATEX_ENABLE = 'y';
$TT_LATEX_PATH = '';
+$TT_BIBTEX_PATH = '';
+$TT_MAKEINDEX_PATH = '';
$TT_PDFLATEX_PATH = '';
$TT_DVIPS_PATH = '';
$TT_QUIET = 'n';
@@ -134,6 +139,8 @@
$TT_IMAGES = $ttconfig{ TT_IMAGES } if $ttconfig{ TT_IMAGES };
$TT_SPLASH_THEME = $ttconfig{ TT_THEME } if $ttconfig{ TT_THEME };
$TT_LATEX_PATH = $ttconfig{ TT_LATEX_PATH } if $ttconfig{ TT_LATEX_PATH };
+$TT_BIBTEX_PATH = $ttconfig{ TT_BIBTEX_PATH } if $ttconfig{ TT_BIBTEX_PATH };
+$TT_MAKEINDEX_PATH = $ttconfig{ TT_MAKEINDEX_PATH } if $ttconfig{ TT_MAKEINDEX_PATH };
$TT_PDFLATEX_PATH = $ttconfig{ TT_PDFLATEX_PATH } if $ttconfig{ TT_PDFLATEX_PATH };
$TT_DVIPS_PATH = $ttconfig{ TT_DVIPS_PATH } if $ttconfig{ TT_DVIPS_PATH };
$TT_RUN_DBI = $ttconfig{ TT_DBI } if defined $ttconfig{ TT_DBI };
@@ -530,6 +537,8 @@
# return if $TT_ACCEPT && (! $TT_LATEX_ENABLE || $TT_LATEX_ENABLE eq 'n');
$TT_LATEX_PATH ||= find_program($ENV{PATH}, "latex") || '';
+ $TT_BIBTEX_PATH ||= find_program($ENV{PATH}, "bibtex") || '';
+ $TT_MAKEINDEX_PATH ||= find_program($ENV{PATH}, "makeindex") || '';
$TT_PDFLATEX_PATH ||= find_program($ENV{PATH}, "pdflatex") || '';
$TT_DVIPS_PATH ||= find_program($ENV{PATH}, "dvips") || '';
@@ -540,7 +549,7 @@
-------------
TT2 supports PDF, DVI and PostScript output using the latex filter,
-implemented with the programs pdflatex, latex and dvips.
+implemented with the programs pdflatex, latex, bibtex, makeindex and dvips.
Because the latex filter runs latex and pdflatex, template authors could
use this feature to include any arbitrary file in their latex input, or
@@ -549,10 +558,12 @@
your site. If you don't trust your template authors then don't enable
the latex filter.
-I found the following locations for pdflatex, latex and dvips:
- + pdflatex => $TT_PDFLATEX_PATH
- + latex => $TT_LATEX_PATH
- + dvips => $TT_DVIPS_PATH
+I found the following locations for pdflatex, latex, bibtex, makeindex and dvips:
+ + pdflatex => $TT_PDFLATEX_PATH
+ + latex => $TT_LATEX_PATH
+ + bibtex => $TT_BIBTEX_PATH
+ + makeindex => $TT_MAKEINDEX_PATH
+ + dvips => $TT_DVIPS_PATH
EOF
@@ -567,9 +578,11 @@
$TT_LATEX_ENABLE) =~ /^y/i
);
if ( $TT_LATEX_ENABLE ) {
- if (ttprompt('Are the pdflatex, latex and dvips paths ok?', 'y') !~ /^y/i) {
+ if (ttprompt('Are the pdflatex, latex, bibtex, makeindex and dvips paths ok?', 'y') !~ /^y/i) {
$TT_PDFLATEX_PATH = ttprompt('pdflatex path', $TT_PDFLATEX_PATH);
$TT_LATEX_PATH = ttprompt('latex path', $TT_LATEX_PATH);
+ $TT_BIBTEX_PATH = ttprompt('bibtex path', $TT_BIBTEX_PATH);
+ $TT_MAKEINDEX_PATH = ttprompt('makeindex path', $TT_MAKEINDEX_PATH);
$TT_DVIPS_PATH = ttprompt('dvips path', $TT_DVIPS_PATH);
}
} else {
@@ -581,6 +594,8 @@
fix_file(catfile('lib','Template','Config.pm'), '$PDFLATEX_PATH', $TT_PDFLATEX_PATH);
fix_file(catfile('lib','Template','Config.pm'), '$LATEX_PATH', $TT_LATEX_PATH);
+ fix_file(catfile('lib','Template','Config.pm'), '$BIBTEX_PATH', $TT_BIBTEX_PATH);
+ fix_file(catfile('lib','Template','Config.pm'), '$MAKEINDEX_PATH', $TT_MAKEINDEX_PATH);
fix_file(catfile('lib','Template','Config.pm'), '$DVIPS_PATH', $TT_DVIPS_PATH);
}
@@ -925,6 +940,8 @@
\$TT_SPLASH_THEME = '$TT_SPLASH_THEME';
\$TT_LATEX_ENABLE = '$ttlatex';
\$TT_LATEX_PATH = '$TT_LATEX_PATH';
+\$TT_BIBTEX_PATH = '$TT_BIBTEX_PATH';
+\$TT_MAKEINDEX_PATH = '$TT_MAKEINDEX_PATH';
\$TT_PDFLATEX_PATH = '$TT_PDFLATEX_PATH';
\$TT_DVIPS_PATH = '$TT_DVIPS_PATH';
\$TT_XS_ENABLE = '$ttxs_enable';
--- lib/Template/Config.pm.orig 2004-07-15 12:07:41.000000000 +0100
+++ lib/Template/Config.pm 2004-07-15 12:53:01.000000000 +0100
@@ -29,7 +29,7 @@
use base qw( Template::Base );
use vars qw( $VERSION $DEBUG $ERROR $INSTDIR
$PARSER $PROVIDER $PLUGINS $FILTERS $ITERATOR
- $LATEX_PATH $PDFLATEX_PATH $DVIPS_PATH
+ $LATEX_PATH $BIBTEX_PATH $MAKEINDEX_PATH $PDFLATEX_PATH $DVIPS_PATH
$STASH $SERVICE $CONTEXT $CONSTANTS @PRELOAD );
$VERSION = sprintf("%d.%02d", q$Revision: 2.68 $ =~ /(\d+)\.(\d+)/);
@@ -49,13 +49,15 @@
$PLUGINS, $PROVIDER, $SERVICE, $STASH );
# the following is set at installation time by the Makefile.PL
-$INSTDIR = '';
+$INSTDIR = '/usr/local/tt2';
# LaTeX executable paths set at installation time by the Makefile.PL
# Empty strings cause the latex(pdf|dvi|ps) filters to throw an error.
$LATEX_PATH = '/usr/bin/latex';
+$BIBTEX_PATH = '/usr/bin/bibtex';
+$MAKEINDEX_PATH = '/usr/bin/makeindex';
$PDFLATEX_PATH = '/usr/bin/pdflatex';
-$DVIPS_PATH = '';
+$DVIPS_PATH = '/usr/bin/dvips';
#========================================================================
# --- CLASS METHODS ---
@@ -287,14 +289,14 @@
#------------------------------------------------------------------------
# latexpaths()
#
-# Returns a reference to a three element array:
-# [latex_path, pdf2latex_path, dvips_path]
+# Returns a reference to a five element array:
+# [latex_path, pdflatex_path, dvips_path, bibtex, makeindex]
# These values are determined by Makefile.PL at installation time
# and are used by the latex(pdf|dvi|ps) filters.
#------------------------------------------------------------------------
sub latexpaths {
- return [$LATEX_PATH, $PDFLATEX_PATH, $DVIPS_PATH];
+ return [$LATEX_PATH, $PDFLATEX_PATH, $DVIPS_PATH, $BIBTEX_PATH, $MAKEINDEX_PATH];
}
#========================================================================
--- lib/Template/Filters.pm.orig 2004-01-30 19:32:25.000000000 +0000
+++ lib/Template/Filters.pm 2004-07-16 09:36:58.000000000 +0100
@@ -599,20 +599,79 @@
# the entire doc.(pdf|ps|dvi) output or throws an error with a summary
# of the error messages from doc.log.
#
+# Re-runs latex (or pdflatex) if there are undefined references.
+# Runs bibtex if the latex log reports undefined citations
+# Runs makeindex if the latex run leaves a ".idx" file in the temporary directory.
+# Re-runs latex if bibtex or makeindex was run
+# Re-runs latex up to three more times if labels change.
+#
# Written by Craig Barratt, Apr 28 2001.
# Win32 additions by Richard Tietjen.
+# BibTex/MakeIndex/forward reference changes by Andrew Ford.
#------------------------------------------------------------------------
use File::Path;
use File::Spec;
use Cwd;
+sub _run_texish_command
+{
+ my($context, $fName, $currDir, $tmpDir, $command, $logfile) = @_;
+ my $message = "";
+
+ my $fileName = File::Spec->catfile($tmpDir, $logfile);
+ if ( system($command) ) {
+ if ( open(FH, "<$fileName") ) {
+ my $state = 0;
+ #
+ # Try to extract just the interesting errors from
+ # the verbose log file
+ #
+ while ( <FH> ) {
+ #
+ # TeX errors seems to start with a "!" at the
+ # start of the line, and are followed several
+ # lines later by a line designator of the
+ # form "l.nnn" where nnn is the line number.
+ # We make sure we pick up every /^!/ line, and
+ # the first /^l.\d/ line after each /^!/ line.
+ #
+ if ( /^(!.*)/ ) {
+ $message .= $1 . "\n";
+ $state = 1;
+ }
+ if ( $state == 1 && /^(l\.\d.*)/ ) {
+ $message .= $1 . "\n";
+ $state = 0;
+ }
+ }
+ close(FH);
+ } else {
+ $message = "Unable to open $fileName\n";
+ }
+ my $ok = chdir($currDir);
+ rmtree($tmpDir);
+ $context->throw($fName, "can't chdir $currDir") if ( !$ok );
+ $context->throw($fName, "latex exited with errors:\n$message");
+ }
+ elsif ($logfile) {
+ if ( open(FH, "<$fileName") ) {
+ $message = join('', (<FH>));
+ close(FH);
+ }
+ else {
+ $message = "Unable to open $fileName\n";
+ }
+ }
+ return $message;
+}
+
sub latex_filter_factory
{
my($context, $output) = @_;
$output = lc($output);
my $fName = "latex";
- my($LaTeXPath, $PdfLaTeXPath, $DviPSPath)
+ my($LaTeXPath, $PdfLaTeXPath, $DviPSPath, $BibTeXPath, $MakeIndexPath)
= @{Template::Config->latexpaths()};
if ( $output eq "ps" || $output eq "dvi" ) {
$context->throw($fName,
@@ -637,6 +696,7 @@
my $cnt = 0;
my($tmpDir, $fileName, $devnull);
my $texDoc = 'doc';
+ my ($BibTeX_cmd, $MakeIndex_cmd, $messages);
do {
$tmpDir = File::Spec->catdir($tmpRootDir,
@@ -662,48 +722,70 @@
$context->throw($fName, "can't chdir $tmpDir");
}
#
- # We don't need to quote the backslashes on windows, but we
- # do on other OSs
+ # We don't need to quote the backslashes in the LaTeX command line
+ # on windows, but we do on other OSs
#
my $LaTeX_arg = "\\nonstopmode\\input{$texDoc}";
$LaTeX_arg = "'$LaTeX_arg'" if ( $^O ne 'MSWin32' );
- if ( system("$LaTeXPath $LaTeX_arg"
- . " 1>$devnull 2>$devnull 0<$devnull") ) {
- my $texErrs = "";
- $fileName = File::Spec->catfile($tmpDir, "$texDoc.log");
- if ( open(FH, "<$fileName") ) {
- my $state = 0;
- #
- # Try to extract just the interesting errors from
- # the verbose log file
- #
- while ( <FH> ) {
- #
- # TeX errors seems to start with a "!" at the
- # start of the line, and are followed several
- # lines later by a line designator of the
- # form "l.nnn" where nnn is the line number.
- # We make sure we pick up every /^!/ line, and
- # the first /^l.\d/ line after each /^!/ line.
- #
- if ( /^(!.*)/ ) {
- $texErrs .= $1 . "\n";
- $state = 1;
- }
- if ( $state == 1 && /^(l\.\d.*)/ ) {
- $texErrs .= $1 . "\n";
- $state = 0;
- }
- }
- close(FH);
- } else {
- $texErrs = "Unable to open $fileName\n";
+ my $LaTeX_cmd = "$LaTeXPath $LaTeX_arg 1>$devnull 2>$devnull 0<$devnull";
+
+
+ # Run (pdf)?latex
+ $messages = _run_texish_command($context, $fName, $currDir, $tmpDir, $LaTeX_cmd, "$texDoc.log");
+
+ # If there are undefined (forward) references or a TOC
+ # file was reported as not found then re-run latex
+
+ if ($messages =~ m{ LaTeX \s warning: \s There \s were \s undefined \s references
+ | No \s file \s $texDoc \. ( toc | lof | lot ) }xi) {
+ $messages = _run_texish_command($context, $fName, $currDir, $tmpDir, $LaTeX_cmd, "$texDoc.log");
+ }
+
+ # If there are undefined citations then run BibTeX (don't bother about the log file)
+ if ($BibTeXPath && $messages =~ /LaTeX warning: Citation \`.*?\' on page \d+ undefined/i) {
+ $BibTeX_cmd = "$BibTeXPath $texDoc 1>$devnull 2>$devnull 0<$devnull";
+ _run_texish_command($context, $fName, $currDir, $tmpDir, $BibTeX_cmd);
+ }
+
+ # If latex left behind a ".idx" file then run makeindex
+
+ $fileName = File::Spec->catfile($tmpDir, "$texDoc.idx");
+ if ($MakeIndexPath && -f $fileName) {
+ $MakeIndex_cmd = "$MakeIndexPath $texDoc 1>$devnull 2>$devnull 0<$devnull";
+ _run_texish_command($context, $fName, $currDir, $tmpDir, $MakeIndex_cmd);
+ }
+
+ # If bibtex or makeindex was run then rerun latex
+
+ if ($BibTeX_cmd || $MakeIndex_cmd) {
+ $messages = _run_texish_command($context, $fName, $currDir, $tmpDir,
+ $LaTeX_cmd, "$texDoc.log");
+
+ # If bibtex was run then latex will report undefined
+ # references on the first run (assuming the
+ # bibliography follows the citations).
+
+ if ($messages =~ /LaTeX warning: There were undefined references/i) {
+ $messages = _run_texish_command($context, $fName, $currDir, $tmpDir,
+ $LaTeX_cmd, "$texDoc.log");
+ }
+
+ # After all that the labels may change, especially if
+ # the bibliography or index sections are included in
+ # the table of contents, so latex is run a couple more
+ # times for the labels to settle down.
+
+ if ($messages =~ /LaTeX warning: Label\(s\) may have changed/i) {
+ $messages = _run_texish_command($context, $fName, $currDir, $tmpDir,
+ $LaTeX_cmd, "$texDoc.log");
+ }
+ if ($messages =~ /LaTeX warning: Label\(s\) may have changed/i) {
+ $messages = _run_texish_command($context, $fName, $currDir, $tmpDir,
+ $LaTeX_cmd, "$texDoc.log");
}
- my $ok = chdir($currDir);
- rmtree($tmpDir);
- $context->throw($fName, "can't chdir $currDir") if ( !$ok );
- $context->throw($fName, "latex exited with errors:\n$texErrs");
}
+
+
if ( $output eq "ps" ) {
$fileName = File::Spec->catfile($tmpDir, "$texDoc.dvi");
if ( system("$DviPSPath $texDoc -o"
--------------060400010501010800040703--
_______________________________________________
templates mailing list
templates@xxxxxxxxxxxxxxxx.xxx
http://lists.template-toolkit.org/mailman/listinfo/templates
 |
[Templates] PATCH: enhances latex filter to cope with forward references, indexes and bibliographies (take 2)
|