Is there such a thing as a perfect letter spacing without kerning

Yes, if your font is monospaced or the design lends itself naturally to harmonious spacing (and even then it is a matter of perspective based on your perception vs. that of the people that use your font). It sounds like you are ready for the kerning phase.

From sharing experiences with other designers, my conclusion is that letterfit is always a matter of compromise. Letters will space too tightly in certain combinations, and too loose with others. Letterfitting is a matter of balancing out the tight combinations with the loose ones so that the overall texture is as even as possible.

As Karl says, spacing is very subjective, but it's advisable to give letters sidebearings that work in the largest number of combinations and use kerning to fix difficult pairs. Most designers would not for example kern rounds positively/oo/, some will kern /fo/ and /ro/ and most (?) would kern /vo/ and similar shapes.

I know of no way to be sure one has done enough spacing before resorting to kerning. What I would tell you is that with experience: you will -in advance- make the black bodies themselves in a way to reduce spacing and kerning issues, and you will -in advance- make the sidebearings in a way to reduce the need for kerning irrespective of specific pairs - meaning that you will come very close to what the sidebearings need to be even before you try out specific problem pairs for reasons of tweaking. So: you cannot Know, but the more and the earlier you think the better.

Also useful is to determine a "granularity" to work with: the finest tweak you will make. In an Em of 1000 many accomplished designers don't go finer than 5 units (and often as coarse as 10 units for kerning). Having this parameter not only saves time/stress but puts you more in control, able to robustly increase the quality -by reducing the granularity- across the board if needed.

hhp

The trick is to first come up with spacing for key letters that will never require kerning to each other. Typically, in Latin fonts these will be the straight sided letters H and i. Then you assign sidebearings to all other glyphs based on spacing relative to these key letters, such that none of the glyphs require kerning to the key letters:

HHAHH
HHBHH
HHCHH
...
iiaii
iibii
iicii
...
etc.

That way you know that your basic spacing is sound, and after that anything that looks wrong is a situation for kerning.