Maybe you should add it to the bug tracker at SourceForge.

http://sourceforge.net/tracker/?group_id=29196&atid=395372

@ Andreas

It’s not included in FontTools. TTXN is different from TTX. The options are the same, but it is a separate Python file. It seems to be developed by Adobe.

I don’t know anything about Python, but I assume, that the problem with the "C sub …" is located at the following lines in TTXN.py:

ChainINDENT = INDENT + " C "

May it be, that not the character "C" was meant, but a control character?

And the other problem may be located here, although there are more lines like that:

rule = "sub %s %s' %s by " % (backTxt, inputTxt, lookAheadTxt)

%s seems to be a placeholder for the variables (?) in the parentheses.

In contradiction to other lines, in which anything is substituted there is no placeholder after the word by.

I don’t want to try to correct anything in code, that I don’t understand. But I assume, it is not so difficult to fix these bugs. But maybe there are more bugs, which I have not yet discovered.

-----------

I wish, more of you would use the AFDKO instead of FontLab for generating the features. I mean, I am amateur, but there seem to be many professionals on the forum, that use FontLab for the OT features. But the AFDKO is really cool and as more people find those bugs, as easier it is, to improve the AFDKO.

You can also generate the feature file with FontForge. PC users can use fontforge-mingw, a self contained binary (cf http://www.typophile.com/node/71038). Mac users can install FontForge with Fink (binary) or Macports (requires compiling).

On the mac I dot it with a script; doing the same on the PC is more tricky and I won't have access to that machine before wednesday and I tried just once, to check. I'll thus just say how to do it with the graphical interface. After opening the font.

You click "Element > Font Info"

You then click "Lookups", right click on a feature and click on "Save Lookup..." to save the feature file for that lookup. You click on "Save Feature File..." to save the full feature file of that font.

Thanks, Michel!

A few months ago, I already wanted to install FontForge. It is not yet installed. What is true for the AFDKO, is also true for FontForge. More people should try it. I will do.

I think, I could make out, where the problem is, if I do the follwing:

When I replace this

rule = "sub %s %s' %s by " % (backTxt, inputTxt, lookAheadTxt)

by this

rule = "sub %s backTxt %s' inputTxt %s lookAheadTxt by " % (backTxt, inputTxt, lookAheadTxt)

I probably can make out, which rule in the Python code is responsible for which decompiled sub rule.

Thanks, Michel!

Welcome. And I should try Volt. I would have if it had been available for OS X (or X Windows on Unix). For me, Microsoft Windows is the last resort.

Arno,

As was noted in the Calibri kerning thread and in John Hudson's comment in the Checking Kerning Pairs in fonts thread, ttx does not output all a kern subtable when it overflows.

Here is a patch for the file _k_e_r_n.py.

34c34,35
< version, length = struct.unpack(">HH", data[:4])
---
> version, length, coverage, nPairs = struct.unpack(">HHHH", data[:8])
> if ((self.version == 0.0) & (length < 14+6*nPairs)): length = 14+6*nPairs # Undocumented hack

If I apply it, I get all kerning pairs of Calibri's kern table with ttx and ttxn.

I have no time to check if this is ok or not for all possible cases. If you want to have a look at that, that would be great.

@ Michel

34c34, 35

Does this mean, that I shall replace line 34 by the other line and add a new line above the line, that is actually line 35?

This

version, length = struct.unpack(">HH", data[:4])
length = int(length)

shall be replaced by?:

version, length, coverage, nPairs = struct.unpack(">HHHH", data[:8])
if ((self.version == 0.0) & (length < 14+6*nPairs)): length = 14+6*nPairs # Undocumented hack
length = int(length)

I have no time to check if this is ok or not for all possible cases. If you want to have a look at that, that would be great.

I don’t know, how to check all possible cases. The Calibri kerning thread looks complicated. I feel, that I miss theory about these overfloating (?) tables and sub tables. I could compare dumps of some system fonts with the original and the unpached kern.py. Most of the new MS systems fonts are not installed here (except from Consolas), but I think, I have them somewhere.

Edited

I replaced it now in the way described in this post, but I get an error. Obviously I did replace the wrong code. I would try it again, if you explain a bit more precisely, what shall be replaced by what.

And I found the Windows 7 system fonts. Calibri 5.62 is the latest version, that I have here.

By the way, since this is another TTX bug, we maybe should ask a moderator to move the discussion.

Okay, I tried it again and now it seems to work.

I replaced

version, length = struct.unpack(">HH", data[:4])
length = int(length)

by

version, length, coverage, nPairs = struct.unpack(">HHHH", data[:8])
length = int(length)
if ((self.version == 0.0) & (length < 14+6*nPairs)): length = 14+6*nPairs # Undocumented hack

and get 26706 pairs.

Did I replace it correctly this time?

calibri.ttx: 26706
cambriab.ttx: 31738
cambriai.ttx: 36452
cambriaz.ttx: 35714
constan.ttx: 36740
constanb.ttx: 36573
constani.ttx: 38688
constanz.ttx: 38417

But this was probably not the check, for which you have asked (all possible cases).

Edited

Something very odd has just happened here: I had decompiled the kern tables of the fonts above only. And then I have merged them back (likewise with TTX). The checksums are the same as the ones of the original fonts. And the digital signatures are valid! That’s very odd!

I was indeed replacing line 34 with two lines, 34 and 35, with the same indentation as old line 34, i.e. the if of line 35 is aligned with version of line 34 (which remains where it was). Here is how it looked on the screen with the proper indentation.

What worries me is the specs, that I don't want to read, and I did not check either what struct.unpack actually does.

Sorry, it is not "overfloat" but "overflow". According to http://www.microsoft.com/typography/otspec/kern.htm, the kern table starts with two fields that are USHORT, the first containing a version number, the second the number of subtables. USHORT is a 16 bit unsigned short interger, and thus may be from 0 to 2**16-1, i.e. from 0 to 65535. Then follow the subtables, each starting with the fields version, length and coverage, which are also USHORT. The definition of the field "length" is "Length of the subtable, in bytes (including this header)." That means that the value of the length field may be from 0 to 65535, and thus the subtable may take at most 65535 bytes, else it "overflows". Of course, a subtable with 26706 kerning pairs, each taking 6 bytes, gives much more than the alloted 65535 bytes. It seems someone has done a "theological reinterpretation" of the specs. I'd like to see it in some "official" written text.

Added: if in the spec they replaced "Length of the subtable, in bytes (including this header)." with "Length of the subtable modulo 65536, in bytes ...", with some explanations, I would like it better.

I was indeed replacing line 34 with two lines, 34 and 35, with the same indentation as old line 34, i.e. the if of line 35 is aligned with version of line 34 (which remains where it was). Here is how it looked on the screen with the proper indentation.

That’s odd, because it did not work here. I will check that again.

@ Michel

Quoting myself: I will check that again.

I obviously made a mistake in my first attempt yesterday. The replacement, as it was explained by you, is working.

Back to the bug in TTXN

There is a bug in the lines above line 1235. At least two of the variables backTxt, inputTxt, lookAheadTxt are not correctly defined. And maybe the word "by" is missing.

And the other bug is in in line 111 (ChainINDENT = INDENT + " C ").

I am not competent to fix that. And it is so obviously, that there is a problem with the part of TTXN, which is responsible for the features, that I assume, there are more bugs.

Hi Arno;

I am not about to fix this particular problem, because ttxn is not meant for making a compile-able font - it is meant only to help compare different versions of basically the same font, and has output features that work against what you want. (see below).

You have lots of company in the desire for a tool that will decompile GPOS and GSUB tables to something directly usable. In the long run, I do plan to add this ability to 'spot'. In the short term - the next several months - the only path to what you want with the FDK tools is use the "spot -t GPOS=7" or "spot -t GSUB=7" commands, and then expect to do a lot of manual editing.

What 'ttxn' is about is producing a text dump of a font, that makes it easy to compare two different versions of a font. The idea is that you use ttxn to do a text dump of each of the two versions, and you use a text comparison tool, like the "Search for Differences.." in BBedit.It is useful only for relatively small differences, such as you get with most bug fixes. If you have just added a few glyphs, or corrected a few paths, or tweaked some kern values, or fixed some GPOS and GSUB differences, then ttxn is great for confirming that what you think you fixed did in fact change, and - just as important! - that nothing else changed.

What makes ttxn more interesting than other tools which make text dump of fonts is that ttxn tries to remove all the things that make fonts look different inside, but have no functional consequences. For example, glyph order is normalized to alphabetic in all places where you see a list of glyphs. In the feature file, I tried to flatten all the different ways in which you can express a particular functional idea to the simplest common denominator. So, lookup order is important, but language system order or offset or whether coverage tables are shared aren't.

In the case that you report, the "by " is missing from the end of the contextual rule, and is instead replaced by a series of simple substitution rules, prefixed by "C", which comprise the set of simple substitutions which are applied to the contextual target string.The first 'C' rule is applied to the first glyph in the input string, the second to the second glyph in the input string, and so on. I did it this way just as a matter of expediency - it was easier, given the several different paths by which you end up with the secondary lookup tables - and I wasn't thinking about making a compile-able output.

All that said, feel free to re-purpose the tool as you want - just give the variant a different name. If you are in doubt about just what is going to happen as a result of a given edit, just try and see. You can always check that you are getting what you want out of your fonts.

@ Read Roberts

Thank you!

All that said, feel free to re-purpose the tool as you want - just give the variant a different name.

This would require, that I understand Python. I am probably not going to try re-purposing ttxn.

What’s the best place for reporting bugs or proposing new functions of the AFDKO?

When will the next version of the AFDKO be released?

@ Michel

I have exported the features of a font with Fontforge-mingw now, but I was searching for a tool, that rebuilds the feature file as close as possible to the original file with the syntax as it is required by the AFDKO and with classes outside the features – the same output, as you get it, if you export the features with FontLab, but without the FontLab bugs.

I have never "massaged" feature files exported by FontForge and I don't use FontLab (all I have is a demo version). As for using FontForge's fea output in AFDKO, I think that is what John Hudson wanted to do after this post: http://typophile.com/node/69658. From a subsequent message, I understand John had a bug in some .fea file and a little afterwards was posted a message concerning a bug that had been corrected in makeotf http://typophile.com/node/70019. My guess is that the match is not perfect yet though these posts may not be related...

Arno: [...] a tool, that rebuilds the feature file as close as possible to the original file with the syntax as it is required by the AFDKO [...]

Perhaps the Export... function in OTM (Light) is useful?

Michel, to my my knowledge those two posts are not related.

The feature syntax extracted from the FontForge source turned out to have a number of bugs (identified for me by Karsten Leucke), which caused makeotf to fail. When these were fixed, makeotf worked fine. The question I was left with was whether FontForge would compile the feature code with these syntax errors, but since I didn't need an answer to that question to complete my project, I didn't pursue it. The feature code was part of an open source FontForge file, so unless there were a disconnect between that source and the shipping version of that font, perhaps FontForge is more forgiving of syntax errors than makeotf.

@ Frank

OTMaster is useful. But for me it is easier to fix the bugs in the feature file exported by FontLab. FontLab rebuilds the classes, OTMaster doesn’t. I loose the overview in feature files, that were exported by OTMaster. The problem with FontLab is, that the (de)compiler is old. (Which version of the AFDKO decompiler is built in FontLab? It is the AFDKO decompiler, isn’t it?)

@ John Hudson

It is not clear what version of FontForge were you using, but there were indeed a number of bugs in feature file generation (and application) code that I reported and was fixed in the last release, not sure if those are the bugs you encountered, but if you can elaborate more, I can try reporting any remaining bugs.

Arno,

there is no "AFDKO decompiler". As Read Roberts (the principal developer of AFDKO) mentioned above, the ability to fully decompile binary OpenType Layout tables into feature definitions in AFDKO syntax isn't part of AFDKO.

Several products allow partial decompilation into syntax that is fully or mostly compatible with AFDKO:
— the Adobe FDK for OpenType itself (in the spot and ttxn tools, but the output is only partially compatible with the strict AFDKO syntax)
— FontLab Studio 5 (but this is based on the AFDKO version 1.6 syntax, not the new 2.5 syntax, since FLS5 includes the AFDKO version 1.6 compiler)
— FontForge (which uses syntax mostly compatible with AFDKO, but uses its own code to compile from the syntax into binary tables)
— DTL OTMaster (which does decompile into AFDKO 2.5 syntax, but in a way that I find somewhat difficult to read)

In addition, FontLab Studio 5 includes an AFDKO-to-VOLT syntax converter, but it’s not quite perfect. Also, the Font-TTF package written in Perl by Martin Hosken includes a ttf2volt decompiler (which also works on .otf fonts despite its name) and a volt2ttf compiler, which use the VOLT syntax, but also only have partial support.

So, there are two principal syntaxes for defining OpenType Layout features support: AFDKO and VOLT. There are two "full compilers", i.e. AFDKO and VOLT themselves. There are two "partial compilers" (FontForge for the AFDKO syntax, and volt2ttf for the VOLT syntax). And there are several partial decompilers, but none that would support the formats fully. There is even a partial AAT-to-AFDKO syntax decompiler (in FontLab Studio), but none that would do the reverse thing.

There is one partial AFDKO-to-VOLT converter (in FLS5), and there is no direct VOLT-to-AFDKO converter. Conversion can, however, be done using the binary tables as an intermediary (i.e. compile from one syntax, then decompile into another). But since all the decompilers are partial, the conversion is not hassle-free.

Of course, we at FontLab are working towards the goal of being able to complete these tasks better. I hope that Read Roberts can help us here by writing a fully-featured decompiler into the AFDKO syntax that would become part of the standard AFDKO distribution, and could be integrated into font development tools that incorporate the AFDKO libraries.

Best,
Adam Twardoch
Fontlab Ltd.

Thanks, Adam!

Do you have a list of problems with FontLab, if it tries to decompile fonts, that were built with the AFDKO 2.5? A way, how the user can identify problems, that are caused by the changes of the syntax? A comparison of input Makeotf and output FontLab?

This is from NewFeatureSyntaxIn2.5.txt:

# Examples of new feature file syntax for FDK 2.5
# Oct 21 2008

# languagesystem DFLT dflt;
# stand-alone lookup: definded outside of a feature block
# Names for stylistic set features
# Multiple substitution
# Reverse chaining substitution
# Contextual chaining substitution with explicit lookup references
# Named anchor and named value record
# Cursive adjustment
# Mark adjustments
# Contextual chaining positioning
# GDEF table definition

@twardoch

There are two "partial compilers" (FontForge for the AFDKO syntax, and volt2ttf for the VOLT syntax).

To the best of my knowledge, FontForge even supports the documented-but-not-yet-implemented parts of adobe's feature file syntax. I'd like to know about what is not implemented in FontForge to qualify it for a "partial compiler".

Khaled,

I think my point of being "partial" is that George does not claim to support the AFDKO syntax per se, but is using a syntax that is "almost identical". As far as I remember reading the docs.

I'm not saying it's a bad thing, it's just not AFDKO.

OK, it was a genuine question, since I intend to use feature files as an exchange format, but I don't have regular access to non-linux machines to actually test with AFDKO, so I was interested in knowing any "known" incompatibilities.

Here is a copy-paste from http://fontforge.sourceforge.net/featurefile.html. This is what George Williams says about it (or at least about the specs)

Major differences between FontForge's and Adobe's interpretation of feature files

Not really any, any more. Except that FontForge supports much of the syntax which adobe documents but does not support. Adobe continues to reserve the right to change the syntax of anything they do not currently support.

In November 2008 Adobe made radical changes to the feature file format. This meant that my extensions were no longer needed. It also as an incompatible change from the earlier format. FontForge should (still) be able to parse files written in the earlier format, as well as files written in the new format. It will only produce files in the new format now (actually there is a compile time flag which will revert output to the old format, but I haven't tested it).