You can if you wish. It won't hurt anything if you do other than maybe lose a few small ones that might develop from those blooms. Personally I have never found bloom removal to have much effect on existing growth. It usually just causes more blooms. So I tend to just let nature takes its course.

You might ask over on the Gourds forum here for more info specific on gourd growing. They may have more experience.

Dave