You just turned the pile so that is the first & most important step for getting rid of unwanted guests in the pile.
Provided there is ample moisture, a good mix of green/brown matl. & the pile is 3x3x3 roughly, those baby mice will disappear in no time. At some point every composter experiments with bones & meat in small quantity. The result is simply compost & some added trace minerals.
Hope that helps.

Thank you!

I empathise with your baby mice killing trauma.
And yes, they make great compost.
I don't turn my piles often and have traps set permanently-
less adults means less babies and less trauma!
I've buried 12 cockerel carcasses in a good hot compost pile, and they'd basically vanished when I turned it the week after.
Just an example, but the compost mantra 'if it has lived, it can live again' definitely applies...

Maggots or other insects will take care of the baby mice flesh. As for the bones, they should decay also.