Typophile

Lookup calls and lookup types

George Horton0 votes

I think I'm getting to grips with OpenType coding - I just birthed my first contextual one-to-many substitution, and am as proud as you'd expect - but I'm still puzzled by one thing and unsure about another.

The puzzle is that when I define-and-run a lookup, the layout engine goes back to the start of the glyph run, as I'd expect. But when I call a lookup, it doesn't, at least in FontLab. Is that just how it is?

The uncertainty is over whether all kinds of lookup types are allowed in all features, or whether it's better to stick to the lookup types specified for features at the Microsoft OT spec site. I know FontLab is liberal, but are all apps?

Thanks very much for any help!
George

If you didn't read this article will help

not all apps, I think; InDesign accepts such wrong substitution in stylistic set (at least CS5).

.

Thanks JanekZ! I knew about that article, but it doesn't talk about the fact that the following snippet will turn all 'a's into 'c's:

sub a by b;
lookup ReRun {
sub b by c;
} ReRun;

whereas this snippet won't:

lookup DoesntReRun {
sub b by c;
} DoesntReRun;
sub a by b;
lookup DoesntReRun;

Do you think the ban on lookup types in features where they're not specified only applies to substitutions that take or return the wrong number of glyphs? Or does it mean you also can't do contextual substitutions where only non-contextual are specified, or non-contextual where contextual are specified? There are places where bending the rules would be handy even for basic stuff: for instance, contextual substitutions in locl for Catalan punt volat; one-to-many in smcp for decomposing germandbls.smcp to s.smcp.DEU s.smcp.DEU; and many-to-one in salt or ss01 for replacing s.smcp.DEU s.smcp.DEU with germandbls.smcp.alt.

Warning: this is only my opinion and it may or may not be true.

"the following snippet will turn all 'a's into 'c's"
It doesn't work that way for me, neither in FontLab nor in FontForge.
1. Your example is equivalent* to:
sub a by b; # 1st rule
sub b by c; # 2nd rule
2. It will work that way:
- "abc" -> "bbc" ["a" is substituted by "b"; no more "a"s] 1st rule
- "bbc" -> "bcc" [second "b" is substituted by "c"; no more "b"s] 2nd rule
3. If your first example works as you described, then try:
sub a by b;
sub b by a;
(or better not...)

* "A lookup is a group of rules of the same type." Your feature has rules of the same type (one to one substitution) so no need to hide them in a lookups.

Odd it doesn't work for you; it does for me in FL. But my first example isn't equivalent to

sub a by b;
sub b by c;

Those two lines just turn a string of 'a's into 'b's, not into 'c's (as you know) because without a separate lookup for the second rule the layout engine doesn't go back to the start of the glyph run. What puzzles me is that, in my second example, lookup DoesntReRun; doesn't send the layout engine back to the start in the same way as defining and running the lookup in place does.

It is my understanding that the cursor never goes backwards (though contextual rules allow looking at left contexts).

If you have many substitution rules in a lookup, it is my understanding that the first that matches is applied. If you have many lookups they are applied in a row. So if the string is ab and the first lookup says to replace a by b and b by c, then you will endup with bc because the first rule matches a and will replace a by b and then the cursor moves forward one position and then the second rules applies for b. If you have a first lookup that replaces a by b and a second lookup that remplaces b by c, then a will be replaced by b by the first lookup and then b by c by the second lookup and then the cursor will move forward one position to the second character and then the second lookup will replace b by c to give cc as a result. If your first lookup replaces b by c and the second replaces a by b, then on ab the first lookup does nothing on a and the second replaces it by b and then the cursor moves to the second glyph that is replaced by c; the resulting string is bc.

I tried with FontForge with the following code: on ab, ss01 gives bc, ss02 gives cc and ss03 gives bc as expected.

feature ss01 {
lookup trysubboth {
lookupflag 0;
sub \a by \b ;
sub \b by \c ;
} trysubboth;
script DFLT;
language dflt ;
lookup trysubboth;
script latn;
language dflt ;
lookup trysubboth;
} ss01;

feature ss02 {
lookup subab2 {
lookupflag 0;
sub \a by \b ;
} subab2;
lookup subbc2 {
lookupflag 0;
sub \b by \c ;
} subbc2;
script DFLT;
language dflt ;
lookup subab2;
lookup subbc2;
script latn;
language dflt ;
lookup subab2;
lookup subbc2;
} ss02;

feature ss03 {
lookup subbc3 {
lookupflag 0;
sub \b by \c ;
} subbc3;
lookup subab3 {
lookupflag 0;
sub \a by \b ;
} subab3;
script DFLT;
language dflt ;
lookup subbc3;
lookup subab3;
script latn;
language dflt ;
lookup subbc3;
lookup subab3;
} ss03;

Thanks Michel. The Adobe Feature File spec says the cursor does go back with each lookup: "each lookup corresponds to one 'pass' over the glyph run". You can prove it by giving a string of 'a's to the following code, which returns not 'i's, as it would if the layout engine went through all lookups before moving on to the next glyph, but 'j's:
sub a by b;
lookup proof {
sub b' a by i;
sub b' b by j;
} proof;
That's why I expected lookup DoesntReRun; at the end of my second example to send the cursor back to the start, turning all the new 'b's produced by sub a by b; into 'c's. The following code does do what I'd expected, despite using only lookup calls rather than definition-and-execution in place:
lookup sub_a {
sub a by b;
} sub_a;
lookup sub_b {
sub b by c;
} sub_b;
feature xxxx {
lookup sub_a;
lookup sub_b;
} xxxx;

But now I've noticed something else odd: the last piece of code does exactly the same thing if you swap the order of the lookup calls, but changes its behaviour if you swap the order of the lookup definitions. Specifically, given a string of 'a's, it returns 'b's instead of 'c's. This language is weird.

I had noticed that the order of the declaration affected the result. I could see it with the Fontforge grahical interface. That is why I declared twice the substitutions above. Weird.

Your proof leaves me puzzled. In Haralambous' Fonts & Encodings, section D 9.4.5 contains about 5 pages of explanations on how contextual substitutions work with accompanying xml code generated by ttx. I would first look there.

The Terrible Secret of OpenType 2

[7.b. Ordering of lookups and subtables]
"Lookups will be created in the GSUB/GPOS table's LookupList in the same order as the corresponding named lookup blocks or runs of rules in the feature file"
so if you have named lookups:
lookup sub_b
lookup sub_a

in that order, then:
feature xxxx {
lookup sub_a;
lookup sub_b;
} xxxx

is executed in order of named lookups.

Thanks both of you. My provisional understanding, after some more experimentation, is that it works as if
1) Sequences of rules outside explicit lookups within the same script and language, with the same lookupflag, and of the same lookup type (e.g. non-contextual one-to-one or reverse contextual) are converted to anonymous lookups. For this purpose, lookup calls are treated as boundaries between rule sequences.
2) Lookup definitions written outside a feature but called within one are copied in written order to the top of the feature's code.
2) All lookup calls themselves are then deleted.
Which is impressively user-hostile, but no more so than the fact that rules written after script latn;
language dflt; are actually excluded from all latin-script languages not explicitly mentioned (hope I've got that right).

2) Lookup definitions written outside a feature but called within one are copied in written order to the top of the feature's code.
2) All lookup calls themselves are then deleted.

This is incorrect. To understand what's going on you need to understand that the GSUB table within a font is a set of ordered lookups, and that feature tags simply indicate a set of lookups which are to be applied when that feature is active. To illustrate this, consider the following:

feature xxxx {
sub a by b;
sub x by y;
lookup lookupA {
sub c by d;
} 1ookupA;
} xxxx;

feature yyyy {
sub a by c;
lookupA;
} yyyy;

The above code will create a GSUB table containing three lookups as follows, which appear in the order in which they are defined (implicitly or explicitly) in your code:
lookup1 {
sub a by b;
sub x by y;
} lookup1;
lookupA {
sub c by d;
} lookupA;
lookup2 {
sub a by c;
} lookup2;

Feature tag xxxx will specify that lookup1 and lookupA are to be applied.
Feature tag yyyy will specify that lookup2 and lookupA are to be called.

The fact that the call to lookupA appears after the line sub a by c; does not override the fact that lookupA has been defined before lookup2 in the feature file.

So, if you apply feature yyyy to the string 'abc' the output will be 'cbd' rather than 'dbd' since it is the order of the lookup definitions, not the order of the lookup calls which is relevant.

André

I should also add that this is one reason I strongly object to attempts to treat opentype coding as a type of programming language, since it leads one to think of lookup calls as analogous to procedure calls which they really aren't (this issue came up recently in a different thread).

André

Thanks André. You're right that the model I gave isn't the way things actually work, though I think it's behaviourally identical to the way things actually work. And it includes the fact that the position of lookup calls in the code of a feature still has semantic relevance, in that rules written above and below will be put into different lookups.

Type design is not in Kansas anymore ;-)

:-)

The opentype engine's layout algorithm is indeed rather clear. Another "proof" that there is more than one pass is that with

feature calt {
script latn;
language dflt ;
sub \a \a' by \b;
} calt;

feature dlig {
script latn;
language dflt ;
sub \a \b by \c ;
} dlig;

'aaaaaa' is replaced by 'ccc' if calt and dlig are activated.

Captured from web.archive.org on 16 Jul 2018. view this capture