Sounds like a fungus but a picture would help.

It's a stinkhorn fungus.

http://botit.botany.wisc.edu/toms_fungi/july99.html

Scroll down to see a picture of an orange stinkhorn.

carrot is putting it mildly, IMO.

I'm really surprised that the OP didn't mention the lovely aroma wafting in the vicinity of these things. Yikes!

Thanks for the pics--the one I have is the second one down on the left.

I didn't smell anything, but I didn't get too close, either. I've only found three "carrots" so far.

Oh yes, the heavenly stinkhorn. I've seen a few and smelled the fetid aroma. They actually draw flies they smell so bad. The first I ever encountered was beside a headstone in an old graveyard. All sorts of images came to mind until I realized it was a mushroom (fungus). I can't imagine eating those things although they are edible, ugh.