Ha! I've known about that thing since I was seven years old and became a TOS fan - but it never looked like an "E" to me! I just called it the "swirl" or "the swirly thing." But now that you point it out - it DOES look like an "E"! Except it's also worn by Security personnel.
The Science symbol always looked like a basketball to me!