Here is a close up of the leaves. Maybe it will help with the identification.

Leaves look like mock orange, Philadelphus. Often, these shrubs do take some time to mature before they bloom and flowering can also be affected by when or how one prunes.
Personally, I'm not sure where it is now is necessarily the best location, both for appearance or inclination to bloom - I might consider moving to a more open, sunnier spot.

When I googled your answer I found another name is English dogwood and I recall finding a plastic landscaping tag calling it a dogwood. It confused me since it wasn't like the dogwood I know but now I think it must have been "English dogwood" and that confirms your ID as a mockorange. Thank you.

I looked up some pictures and this plant might turn into something, although I've never seen a blossom on it yet.

Yes, Philadelphus/mock orange. From the look of it, it should bloom next year if you don't cut off the new growth.
Mock orange blooms on last year growth. Once it blooms, you can cut off that branch to allow new one to develop.

It looks like a regular Mock Orange, so it will grow to 8 - 10 ft. I have seen Mock Orange grown as a hedge, so I guess you can shear it to the height you want as well.