[Templates] BUG REPORT Parser broken after failed parse attempt

This is a multi-part message in MIME format.
--------------090007080905070005050501
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

This is a report of what I believe to be a bug in
Template::Parser. I have included a patch which has
worked for me.

If this is not the correct place for such reports, I would
be grateful if someone could point me at a better place.

I am using the Template Toolkit under Apache mod-perl to
generate dynamic pages. I am not using Apache::Template,
but am using my own perl code to maintain a persistent
Template object in each Apache child process, and so
cache compiled templates in store.

Sometimes, after an Apache process has parsed a broken template,
it becomes unable to parse correct templates correctly. The
templates in question use named blocks. The symptoms are
spurious error messages like

  file error - myblock: not found at ...

I have managed to narrow the problem down, and can reproduce
it in a simple, stand-alone program, which I attach as
tt2test.txt.

The program uses three calls of the process method of a
single Template object. First it processes a correct template
with the expected result. Then, it processes an incorrect
template, taking the expected error, Next, it attempts to
process _the same_ correct template again. This time
it gives the spurious error

 file error - myblock: not found at ./tt2test line 29.

The debug output (attached as tt2out.txt) gives a clue to
what is going on. The first trace line from the original,
successful, process call is

 [Template::Parser] compiled block 'myblock':

The corresponding line from the third process call is

 [Template::Parser] compiled block 'hisblock/myblock':

The parser has got the name of the block wrong. 'hisblock'
is the name of the block in the broken template. The
parser was parsing this block when it gave up on the
broken template, and this bit of parser state has not
been correctly cleared out. This Parser instance
will now get block names wrong forever.

Inspection of the code shows that the culprit is the
DEFBLOCKS property of the Template::Parser object.

Further inspection suggests that there are a small number
of other properties which could also be left in unfortunate
states after a failed parse, and which are not initialised
anywhere. They are INFOR INWHILE DEFBLOCK_STACK and STYLE.
(Initially I also suspected INPERL, but this is initialised
in _parse.)

I have attached a patch (tt2patch.txt) against the 2.14
version of Template/Parser.pm, which does seem to fix
the problem. However, I am not at all confident that
this is the right way to fix it.

Perhaps it would be better to modify Template::Provider
so that it would throw away its persistent parser and
make a new one every time a parse failed.

        -- 
        Charles Jardine - Computing Service, University of Cambridge
cj10@xxx.xx.xx    Tel: +44 1223 334506, Fax: +44 1223 334679

--------------090007080905070005050501
Content-Type: text/plain;
 name="tt2test.txt"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="tt2test.txt"

#!/bin/perl -w

use strict;

use Template;
use Template::Constants qw(:debug);

$| = 1; # so stdout and stderr interleave

my $broken = <<EOF;
[% INCLUDE myblock %]
[% BLOCK hisblock %]
The output
EOF

my $mended = <<EOF;
[% INCLUDE myblock %]
[% BLOCK myblock %]
The output
[% END %]
EOF

my $template = Template->new({DEBUG=>DEBUG_PARSER}) or die Template->error;

$template->process(\$mended) or warn $template->error;

$template->process(\$broken) or warn $template->error;

$template->process(\$mended) or warn $template->error;

--------------090007080905070005050501
Content-Type: text/plain;
 name="tt2out.txt"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="tt2out.txt"

[Template::Parser] compiled block 'myblock':
sub {
    my $context = shift || die "template sub called without context\n";
    my $stash   = $context->stash;
    my $output  = '';
    my $error;
    
    eval { BLOCK: {
$output .=  "\nThe output\n";
    } };
    if ($@) {
        $error = $context->catch($@, \$output);
        die $error unless $error->type eq 'return';
    }

    return $output;
}
[Template::Parser] compiled main template document block:
sub {
    my $context = shift || die "template sub called without context\n";
    my $stash   = $context->stash;
    my $output  = '';
    my $error;
    
    eval { BLOCK: {
#line 1 "input text"
$output .=  $context->include('myblock');
$output .=  "\n";

$output .=  "\n";
    } };
    if ($@) {
        $error = $context->catch($@, \$output);
        die $error unless $error->type eq 'return';
    }

    return $output;
}

The output


file error - parse error - input text line 2: unexpected end of input at ./tt2test line 27.
[Template::Parser] compiled block 'hisblock/myblock':
sub {
    my $context = shift || die "template sub called without context\n";
    my $stash   = $context->stash;
    my $output  = '';
    my $error;
    
    eval { BLOCK: {
$output .=  "\nThe output\n";
    } };
    if ($@) {
        $error = $context->catch($@, \$output);
        die $error unless $error->type eq 'return';
    }

    return $output;
}
[Template::Parser] compiled main template document block:
sub {
    my $context = shift || die "template sub called without context\n";
    my $stash   = $context->stash;
    my $output  = '';
    my $error;
    
    eval { BLOCK: {
#line 1 "input text"
$output .=  $context->include('myblock');
$output .=  "\n";

$output .=  "\n";
    } };
    if ($@) {
        $error = $context->catch($@, \$output);
        die $error unless $error->type eq 'return';
    }

    return $output;
}
file error - myblock: not found at ./tt2test line 29.

--------------090007080905070005050501
Content-Type: text/plain;
 name="tt2patch.txt"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline;
 filename="tt2patch.txt"

--- Template/Parser.pm~	Fri Oct  8 15:48:54 2004
+++ Template/Parser.pm	Fri Oct  8 15:58:49 2004
@@ -247,6 +247,14 @@
 
     $self->{ _ERROR } = '';
 
+    # clear possible wreckage of previous failed parse
+    # attempt
+
+    foreach (qw(INFOR INWHILE DEFBLOCKS DEFBLOCK_STACK)) {
+        $self->{ $_ } = undef;
+    }
+    $self->{ STYLE } = [ $self->{ STYLE }[0] ];
+
     # split file into TEXT/DIRECTIVE chunks
     $tokens = $self->split_text($text)
         || return undef;				    ## RETURN ##

--------------090007080905070005050501--

_______________________________________________
templates mailing list
templates@xxxxxxxxxxxxxxxx.xxx
http://lists.template-toolkit.org/mailman/listinfo/templates
