We have 1 honeyberry bush. We have done nothing to it other than plant it. I think it's on it's 4th or 5th season. It grows slower than blueberry bushes but is extremely healthy. The fruit tastes awful though, so it is kind of a running joke in our family. Not sure how they got their name. My kids will eat a couple off but then the rest usually falls off.

I'm surprised that you get any fruit with only 1 bush. Everything I have seen says that they need at least one other variety to pollinate with.

As to the taste, were the berries fully purple on the inside? If they are green on the inside they need more time to ripen. Of course, you could have a bad variety. Per the University of Saskatchewan:

"The good varieties taste something like blueberries & raspberries. The bad ones taste like tonic water"

I've just started with honeyberries. The first 3 are already planted, with the next 3 coming in the next few days.

Blue Velvet
Berry Blue (Czech #17)
Blue Moon (Sergei)

Midnight Blue
Borealis
Tundra

The last 2 on the list are relatively new varieties from UoS.

Agreed that there's nothing magical to growing it. My 4 varieties grow pretty slowly, but I figured that's because they get a lot of shade. I too am surprised that yours taste bad. I like mine quite a bit. A lot more juicy than a blueberry... but since my plants are small, I don't get too many. At least the fruit seems to be hidden from the birds so they don't steal 'em.
-Glenn