Removing them by hand is as good as anything provided you have time and there aren't too many. If you need to spray as the trees get bigger I've had good results from BT. It's safe for you and the tree and organic.

Any spray will work better on smaller insects and result in less damage.

Thank you. :) any idea what they are though? If they are butterflies that will eat a few leaves but end up pollinating many plants and looking pretty and doing no harm, then I really wouldn't mind them. But if they are the beginning of an infestation that will defoliate my trees, then I'd like to send them far away. Like the after life. :)

I did a Google search and came up with this one that looks similar.

More info but no pic of larvae:
http://www.massaudubon.org/learn/nature-wildlife/insects-arachnids/butterflies/find-a-butterfly/(id)/176

Here is a link that might be useful: Red-spotted purple caterpillar (Life cycle series) - Limenitis arthemis

Thank you, milehighgirl! I did some more googling and found the photo of the larvae. They look exactly what I've been seeing!

Okay, so it's a butterfly. I know butterflies will benefit my other plants later on and I don't think admiral butterflies normally do serious damage to their hosts (but I will research that further). So I probably should just leave them alone when I see them. :)

Angie

No way in a million years I could have figured that out without the internet. I can identify a tomato hornworm and that's about it.