Overview

Top Close Open

This design document proposes that TT3 keywords will be lower case by default.

Description

Top Close Open

In TT2 all keywords are upper case by default.

[% INCLUDE header %]

You can use the ANYCASE option to enable the use of keywords in upper, lower or mixed case.

[% include header %]
[% Include header %]

In TT3 keywords will be lower case by default.

[% include header %]

There will be a configuration option to enable upper case keywords for those who prefer them.

my $tt = Template->new( keywords => 'upper' );    # tentative

There will also be a generic facility to map the default keywords to an alternative or range of alternative keywords.

my $tt = Template->new( keywords => {
    include => 'INCLUDE',                    # upper case INCLUDE
    elsif   => ['elsif', 'elseif', 'elif'],  # aliases for elsif
});

If you have a variable that shares the same name as a keyword then you will be able to use an explicit $ prefix to indicate that it is a variable.

[% $include %]      # the variable 'include', not the keyword

This can also be used to access variables that have spaces or other reserved characters in their names. Yes, it's Bad Idea[tm] to have reserved characters in variable names, but it happens (occasionally) in the Real World[tm] (e.g. passing a hash of product codes/descriptions to a template)

[% $'XYZ-42A/m' %]        # a variable called XYZ-42A/m

This will work woth dotops Just Fine[tm].

[% $'XYZ-42A/m'.price %]  # the .price of the variable XYZ-42A/m

The C<$> serves only as a hint to the parser.  There is no additional
interpolation of the variable.
# TT3
[% foo = 'bar' %]
[% foo  %]          # bar       # variable with implicit '$'
[% $foo %]          # bar       # same as above, explicit '$'

Note that this is different to TT2 which treats a leading $ on a variable as an indicator of an extra level of indirection. It interpolates the first variable to get the name of the second variable which is then interpolated.

# TT2
[% foo = 'bar' %]
[% bar = 20 %]
[% foo %]           # bar       # value of foo
[% $foo %]          # 20        # value of bar, named in foo

TT3 will support a generic namespace mechanism for partitioning variable names separately from keywords and/or other token classes (e.g. plugin names, template names, filter names). The more verbose way to indicate that a token should be parsed as variable is to add a var: prefix.

[% $include %]      # short form 
[% var:include %]   # long form

Thus, the C<$> symbol is little more than syntactic sugar for C<var:>.

Rationale

Top Close Open

Lower case keywords are easier to type and easier to read. The reason why they aren't in lower case in TT2 (or TT1/TT0) is due to potential conflicts with variables of the same name.

TT2 cannot distinguish between keywords and non-keywords variables in a template. There is no (easy) way to define or use a variable called INCLUDE, for example.

This is because the TT2 parser first splits a template into tokens and then feeds them into the parser which recognises them according to the grammar rules. In the initial tokenising stage, the parser can only identify a keyword or non-keyword based on what it looks like. It has no way of knowing what rule is being parsed, or at what point within a rule it is currently positioned at because it hasn't even started to think about rules and grammars yet.

This kind of up-front tokenising is "dumb tokenising" (although you probably won't see that term in the Dragon Book). It means that wherever you write INCLUDE, it will be recognised as the INCLUDE keyword, even if it's not valid to have an INCLUDE keyword in that position.

[% example.INCLUDE %]       # FAIL: unexpected token (INCLUDE)

Thus, the reason for making TT2's directives UPPER CASE - to avoid potential clash with lower case variable names. Although include isn't a very likely name for a variable, for example, things like next and last are much more common. If you enable the ANYCASE option in TT2, then you can no longer write things like this:

[% page.next %]             # FAIL: unexpected token (next)
[% list.last %]             # FAIL: unexpected token (last)

It's a large helping of FAIL when enabling the ANYCASE option breaks an inbuilt vmethod. It has to go!

The TT3 parser is much smarter and can distinguish keywords from non-keywords based on position. It's also much more flexible in allowing keywords to appear in places that they weren't allowed to appear in with TT2.

[% page.include %]           # no problem: 'include' is not in a 
                         # position where keywords are allowed

Impact

Top Close Open

In TT2, a leading $ indicates that a variable should be interpolated before use. This is applied consistently for both top-level and dotted variables.

[% # TT2
   foo = 10; 
   name = 'foo'; 
   bar = { foo = 20 };

   foo          # 10
   $name        # 10 - name interpolated twice, to foo, then 10
   bar.foo      # 20
   bar.$name    # 20
%]

TT3 will subtly change the meaning of $ to indicate simply that the word following is the name of a variable. Whether or not it is interpolated depends on the context in which it is being used. Dotops will continue to treat a variable as something that needs to be interpolated first, but top-level definitions will effectively ignore it, given that those variables are already being interpolated.

[% # TT3
   foo = 10; 
   name = 'foo'; 
   bar = { foo = 20 };

   foo          # 10
   $foo         # 10
   $name        # foo - name interpolated once, to foo
   bar.foo      # 20
   bar.$name    # 20  - bar.$name is same as bar.foo
%]

In summary, a leading $ makes TT2 interpolate a variable twice, whereas TT3 will only interpolate it once. A $ following a dotop will behave the same in both TT2 and TT3.

I don't imagine that many people are relying on TT2's current symbolic interpretation rules, and any that do will have to add a second $ to really drive the message home.

[% $$name %]

Acknowledgments

Top Close Open

Bad Idea[tm], Real World[tm] and Just Fine[tm] are trademarks of Brian Badger's Fictional Trademark Factory Inc.


http://tt3.template-toolkit.org/docs/design/TT3DD01.html last modified 13:25:03 10-Dec-2009