It might be customised Avant Garde, with the serifs & swash-like extensions added. There's a similar feel to Cortez & Serif Gothic.

Agree with you about Lubalin... Looks like a custom based on Avant-Garde Gothic Bold with stylistic alternates or on Lubalin Graph Bold which have been designed by Herb Lubalin.

I hadn't thought about Graph. It would be a lot easier to get those serifs by cutting across Graph's slabs than to add them to Avant Garde.

I've just taken a look at the album cover and it says ALBUM DESIGN: Lubalin, Peckolick Associates.

Looks like you were spot on.

In his 2013 book Teaching Type to Talk, Alan Peckolick says:

I designed the logo, record jacket, and posters for musician Paul Simon’s album and movie, One Trick Pony, which came out in 1980. I used the typeface Avant Garde Latin with pointy serifs, which had never been released to the public, and broke up the word “One-Trick” with a hyphen. It’s strong, in your face, and the right look.