Think not as a type designer, but as 1) typesetter/graphic designer, and 2) reader ("can I imagine someone needs to present/recieve this element in a cursive manner?").
I would say that pictorial elements like arrows aren't worth of slanting or italicising in a font, but in the other hand, one can foresee a need of having italicised arrows, for certain purposes...

Unless you are producing a math font, there is no harm in italicizing/slanting just about everything.
Except ©, ® and •.
Also, I don’t usually italicize small marks: tittles/dots/periods, and the degree symbol and ring accent.

“There is no harm in doing it” is not the same as “that is the optimal thing to do.” I think the question is, what is optimal?

One notable omission: the estimated symbol.

I made a rather small .enc file for FontLab Studio, with the glyph names of the glyphs that I keep as upright in an italic face, based on what I did at Adobe with Hypatia Sans. It has a fairly large character set.

Here is a link to the file: http://www.thomasphinney.com/wp-content/uploads/2013/03/Upright.enc_.zip

Here is the info to cut and paste into a new .enc file:

%%FONTLAB ENCODING: 84131926; Upright in Italic
.notdef 0
asterisk 1
plus 2
less 3
equal 4
greater 5
asciicircum 6
bar 7
asciitilde 8
logicalnot 9
trademark 10
plusminus 11
degree 12
registered 13
copyright 14
revlogicalnot 15
uni2120 16
uni2117 17
estimated 18
uni2190 19
uni2192 20
arrowup 21
arrowdown 22
uni25C0 23
uni25B6 24
uni25B2 25
uni25BC 26
uni25C1 27
uni25B7 28
uni25B3 29
uni25BD 30
uni25C9 31
uni25C6 32
uni25A0 33
uni2752 34
uni2713 35
uni2610 36
uni2611 37
lessequal 38
greaterequal 39
approxequal 40
notequal 41
uni2213 42
infinity 43
lozenge 44
integral 45
radical 46

What is optimal?
As Marcin says, consider usage.
IMO, it looks like a mistake if text is in italics, with mathematical symbols upright.
Unless you’re making a math font, there is absolutely no reason* to put any upright glyphs in an italic font — except for ©, ®, •, ˚,° and Estimate (if you are doing WGL4 encoding).

*Except laziness.

I'm with you guys on the Estimated*, but do © and ® have to be upright to be legal? I don't think so, since even "(c)" is acceptable AFAIK. I also don't grasp why any of the others should be upright. You want the upright one? Use the Roman. You want the slanted one? There it is.

* Related: http://typophile.com/node/98229

hhp

Thomas, do you really slant your hyphen/endash/emdash?

Hrant, © and ® are problematic to slant for aesthetic reasons.
However, I have made some typefaces where they are slanted: Alphaville, Fontesque, and Preface.

Slanted © or ® look strange (being rather pictorial elements), but I can imagine a dense, technical or business material, full of quotations, where they could be an advantage.
Leaving upright periods makes utility sense, because people often don't make exact selections...
Anyway, I have problems in saying what sould and what shouldn't be italicised, because there are such texts and such conditions, where I find those upright elements as details of incredible beauty, and I would even accept the geometric zeros there.