I leave them blank because I figure if you've included all the characters in the code page(s) supported, there's no need for them.

Besides, don't layout applications insert missing characters from large system fonts, e.g. Arial?

So some apps swap it out with another font. OpenOffice writer does. Photoshop and Illustrator both show the notdef if a glyph isn't present in the font. I'm not sure about other Adobe apps.

So if I copy and paste

abcdǻabd

into Illustrator or Photoshop and the font doesn't contain aringacute, it's replaced by a notdef glyph.

The .notdef box is a safety mechanism. Is there a case for deliberately leaving them blank? Are they just uncool now? Clue me in, please.

Nick, it's not a good idea to leave the .notdef glyph blank. The point of the glyph is to signal the presence of characters not supported by the font, and that includes characters beyond the codepages that the font aims to support. If your .notdef glyph is blank or, worse, blank and zero-width, those characters will simply disappear from the text in many applications. Consider what happens if someone is using one of your fine fonts, but is sent a text for typesetting that includes e.g. one Chinese character. There should be some visible indication that this character is present and that it requires a different font to be selected.

My preferred .notdef design is a box with a question mark in it. Recently, seeking to give this maximum visual impact on the page, I've taken to reversing the question mark out of a black box. I usually align the height of the box to the cap-height, because this producing a proportion that doesn't have an atypical impact on the line-layout, i.e. its width is representative of a typical missing character.

I would put something in there, and a box is as good as a squiggle or a question mark. I’m surprised we don’t see more creative shapes – logos, alien heads etc.,

I do recall a customer complaint. One chap had been using a random Arial codepoint that resulted in a missing glyph. He used it as a check-box on all his forms. When we updated the font his check boxes were all replaced with the obscure character that belonged in that slot.

Also there are “blank” code-points in Unicode – all those different spaces for example – if you don’t populate them and have a blank missing glyph, expected spacing could be messed up.

I would put something in there, and a box is as good as a squiggle or a question mark ..

I like to make pretty sure I don't miss a single missing glyph:

I like that, it really stands out.

Yes, please make sure it really stands out. Framing or reversing sounds like a good solution.
Related: Filling in empty slots with placeholder junk

I always kind of liked Palatino Linotype Italic's little bird (of Character Detail fame on the old MyFonts site).

Thanks John, I don't think it's a huge error to omit the .notdef glyph, but I will include it from now on.
Should this be a Unicode character?

Nick,

Per definition it's not a Unicode character :-)
Giving it the name ".notdef" and putting it as first glyph in your font (index #0) should be enough.

Per definition it’s not a Unicode character :-)

Then what's it doing in a font?

Nick, the .notdef glyph dates back to Apple's original TT spec: it is to be the first glyph in a font, it is to be named .notdef, and it is to be unencoded. It provides the ultimate fallback -- after font switching or other mechanisms that an application may choose to employ -- to display something when text includes a character that is not supported in the font. The .notdef glyph is not a Unicode characters because it may represent any Unicode character that is not mapped in the font cmap table.

Si wrote: I’m surprised we don’t see more creative shapes – logos, alien heads etc.

I'm glad we don't. I recall some users being confused by the spiral in the Palatino Linotype fonts. It is better that this glyph approximate the evolved conventions: an empty rectangle, a rectangle with diagonal lines corner-to-corner (the form Adobe uses), a rectangle with a question mark in it, a reversed rectangle with a question mark in it.

In traditional Japanese typography, a placeholder sort was used for a similar purpose: to mark a place in the text that would require a character from a different font. This enabled the typesetter to keep working and to leave a visible marker in the text to indicate the place where the other-font character should be inserted. If I recall correctly, the sort had a horizontal rectangle in the upper half, so would print as a black rectangle above a white rectangular space. This form might be appropriate for the .notdef glyph in an East Asian font, although Ken Lunde should comment on this and correct me if I am wrong. It led me to experiment with this form:

It provides the ultimate fallback

That doesn't seem to be the case, as there is a variety of glyphs used to represent absent characters, as well as emptiness.
Perhaps it would be better to handle this at a deeper level than fonts?

Alternatively, Unicode could specify a preferred glyph shape for ".notdef", even if it has no code number.
That would at least apply some standard to the situation, and provide direction to font developers, internationally.

>That doesn’t seem to be the case,

True, due to app/system fall-back, font-linking etc., but this is the only fallback marker that's within the type designer's control.

I think it specifies in the Microsoft TT spec (paraphrasing from memory, don't sue me if I'm off by a bit) that the .notdef glyph should be a rectangle, acceptable variants including an X or ? enclosed in the rectangle, but variants beyond that being highly discouraged so as to avoid confusion.

I suppose to make it fit with a particular font the rectangle could be approximately the average glyph height and width of the particular font, and some consideration given to stroke weight too.

U+3013 (〓) is commonly used as a placeholder glyph in Japanese typography when the desired ideograph is not available.

Dr. Ken Lunde
Senior Computer Scientist, CJKV Type Development
Adobe Systems Incorporated
lunde@adobe.com

Interestingly, of the 64K glyphs that can be in a CIDFont resource, the only glyph that is required is that for CID+0, which serves as the .notdef. Its function is for rendering as part of a fallback mechanism, and is not encoded. In fact, I would argue that it should not be encoded, which is stronger.

Dr. Ken Lunde
Senior Computer Scientist, CJKV Type Development
Adobe Systems Incorporated
lunde@adobe.com

Nick, as I said, it is the ultimate fallback, if an application doesn't attempt any other kind of fallback or such other methods, e.g. font-switching, fail, the .notdef glyph is supposed to be displayed.

For instance, let's say a piece of text includes a character not supported by the current font cmap table. An application (or system/library/engine) may automatically use the .notdef glyph, or it may go through a series of degrading fallbacks. It may try to find a default font on the system for displaying the character; if it can't find such a font, it may try to find a default fallback font such as Apple use in order to at least identify the Unicode range and script to which the missing character belongs; if the system does not have such a font, the ultimate fallback should still be the .notdef glyph.

Alternatively, Unicode could specify a preferred glyph shape for “.notdef”, even if it has no code number.

This has nothing to do with Unicode. The .notdef glyph is not a Unicode character; its function is defined by the Apple TT spec.
___

Thanks, Ken. I remember now the three-band placeholder. I did try a .notdef glyph based on this with a superimposed question mark, but it was too messy at small sizes.

John, wouldn't it make more sense for applications to substitute a standardized ".notdef" glyph from a system font, rather than rely on the vagaries of commercial fonts?

How are we going to get all the foundries to play ball, especially when there is no Unicode guide to suggest a standard glyph shape?
Having the standard in the Microsoft TT spec is not comprehensive enough.

This has nothing to do with Unicode.

By the same token, it's not really a responsibility of font formats
As Unicode is all about character encoding, it may be better positioned to deal with the lack of an encoded character than font formats.

OpenType (and TrueType) fonts are designed to be fairly self-contained. Depending on external resources, such as the rendering of the ".notdef" glyph goes against this, though some clients may choose to implement their own handling of ".notdef" situations. This is also the reason why the mapping from character codes to glyphs are included in the fonts themselves, in the form of the 'cmap' table.

Dr. Ken Lunde
Senior Computer Scientist, CJKV Type Development
Adobe Systems Incorporated
lunde@adobe.com

How are we going to get all the foundries to play ball, especially when there is no Unicode guide to suggest a standard glyph shape?
Having the standard in the Microsoft TT spec is not comprehensive enough.

Nick, it's also in the Adobe CFF specification, so pretty much any font that doesn't have it is buggy.

But I think your point is that it's not required to be a particular mark, or any mark at all. (Maybe there is some suggestion somewhere in Adobe's documentation, I'm not sure.) It would maybe be nice if FontLab and other significant tools had a reasonable default notdef shape built-in.

Um... Isn't .notdef Unicode #FFFD?
If not, shouldn't it be?

I think all that's needed is some awareness that a blank .notdef could cause problems. A default .notdef box in Fontlab would be a nice touch. Didn't Fontographer generate a default .notdef box?

Another windmill . . . Where's my donkey.

Nick, We key about 2 manuscripts a year. Well, no, we no longer key, we send out about 2 manuscripts a year to be keyed . . . The other 398 arrive "editorially correct on disk." We set them with one of the few apps that supports OT - Adobe's InDesign. Not one of our customers would accept, say, an Ariel character if the MS called for a glyph not in the font. And not one of the graphic designers will check to see if the typeface they have selected in fact has all the characters called in the MS, (which may, BTW, have substituted characters from another font).

Yes, we do have a little script we run that shows all the Unicode characters called in the MS. But you can miss something; each proofing pass catches about 98 percent of the errors.

So, (1) The appearance of the missing glyphs is important. The starker, the better. And (2) it is important the end user has the right to add the missing glyph -- like the ygrave I encountered today.

I understand ID-CS4 won't let you export to PDF unless all the needed characters are present. But sadly, I flat guarantee you ID-CS2 will.

Don't you get a big pink em-square in InD if the character is missing?

Kevin: Isn’t .notdef Unicode #FFFD? If not, shouldn’t it be?

U+FFFD is used to replace an incoming character whose value is unknown or unrepresentable in Unicode, e.g. where you have text in a different encoding for which software can't figure a way to convert to Unicode. So U+FFFD indicates a character not recognised as Unicode, whereas the .notdef glyph indicates a character not supported by the font. U+FFFD appears in text up-stream of display; .notdef appears in text during glyph display.

It is one of the fun glyphs so I always include it--sometimes in a humorous vein.

ChrisL

Don’t you get a big pink em-square in InD if the character is missing?

Well, it is pink. I've seen fonts where the size -- width -- is quite small. I'd assumed the "pink" is the width assigned to the .notdef character; I suppose I should check. I've missed a few.

Pink doesn't work when your primary checking/proofing method is to print out the job and have it read (or skimmed) by someone other then the comp who set it. An old idea, but still a good one. BTW, with "editorially correct" files supplied, we don't read against copy.

I thought that pink highlight in ID shows font substitution, not missing character?

@Jan (paragraph): Both. In InDesign, missing fonts and missing characters are both marked with a pink highlight that can be toggled in the preferences (Compositon > Substituted Fonts). If the font has a .notdef glyph, missing characters are substituted with that glyph and get highlighted. Otherwise it’s just a highlighted blank space. I reckon the width of that blank space is one en space.
F

Otherwise it’s just a highlighted blank space..

For entire missing fonts, it seems ID checks for a reasonable match -- the same Unicode in its default font (Myriad?), or in a font matching the name but not the style. A missing glyph in an otherwise good font usually displays the notdef glyph.

Pre-CS4 InDesign, it used to be a matter of carefully examining the document (in InDesign), but hip-hip-hooray! CS4 finally marks 'missing glyphs' as an error in its live preflight.

By the way:
Is there any other app (beside InDesign) that allows to check fonts against certain Unicode texts and output missing characters?
If people need to perform such tests, one can hardly suggest a CC subscription just for that. ;-)

Ralf, https://github.com/davelab6/pyfontaine does exactly that :)