There are lots of large chewing insects, beetles and caterpillars, that eat rose foliage. The damage shown is not, IMO, significant.

If you find a cluster of roughly chewed leaves, look for a worm hiding in the vicinity. Beetles are more mobile.

It means a bad insect is hungry.

Ditto on what MichaelG said...

Thanks for the responses. What would you recommend as an organic way of controlling this? Maybe a hot pepper spray or something like that?

Try a repellent if you want to, but there's no way of knowing whether the perp is still in the garden. Personally, I wouldn't do anything.

Remember, caterpillars can be baby butterflies. Do you really want to kill butterflies? Not all insects that eat foliage are bad insects. Bugs got to eat too!
Are perfect leaves and blooms all that important?
Do you have a perfect leaf phobia?
If you walk quickly past the bushes you won't see the damage. Your roses will survive without running for toxic insecticides. Any insecticide that kills is toxic, even organic ones.
They're toxic to the insects, even good bugs.

Sounds okay then. I will leave them be. :)

A few holes here and there aren't a big deal. If they begin to eat large portions then I'd spray. If you do decide to spray make sure you ID the insect before you chose a defense. Usually an insecticidal soap spray will do the trick depending on the culprit of course. But spraying anything before you know what you have is a waste of time, money and effort and could kill the good bugs, who eat the bad bugs, leaving you with an even worse problem.

br, I saw some little holes like that on my leaves too.

The attached link looks like my leaves, and some other articles said these bees are good guys.

Here is a link that might be useful: Leaf cutter bees?