The limbs won't take a year off then come back the next year, no.
If they don't come back this year (and they should have already) remove them and be happy that it is at least coming back from the base.

You can cut a black lace down to the ground every spring...if the root ball is big, you'll still get a big plant in this growing season.
My friend has one that's leafing out all over, and one like yours...we had a good look at it and felt that it MIGHT still leaf out, so she is going to wait 2 more weeks before cutting it down.

It will be fine, just prune it in a couple of weeks to make it look nice and you'll be surprised how beautiful (and fast) it will fill out again.

Thanks so much for your responses. I've already started to cut it way back. It is just so disappointing as it had become quite large and produced beautiful flowers.

I had this awhile back and it died for whatever reason, but I have heard it likes the colder zones so you should not have a problem with it dying from that. I also cut mine almost to the ground...think mine died from too much heat and humidity. but I LOVED it. I will try it again in another place.