Sounds like overcrowding and, perhaps, not enough food.

Maybe three to an eight inch pot. Topdress with good compost and well-rotted or pelletised animal manure (steer, horse, or sheep but not chicken because it can be too 'hot' and alkaline.) And - cover with oak leaves in winter.

Morning sun up to about 10am could be useful, too.

Repot every year if they are going to stay in a small-ish pot.