It is plants--those devices with roots anchored into the soil--that do the best job of holding soil in place. A tremendous volume of water cannot wash them loose. Mulch, even if rocks, wash if the water flow is heavy enough. Mulch is a temporary solution, not a permanent fix. A plastering over of concrete will probably be the lowest maintenance solution and prevent bugs and critters from feeling welcomed. But is there no eventual interest in having something nice looking?

You already have hostas. They are free to dig and divide as you wish and will fill that space. No mulch needed.

Solidly paving the ledges with concrete would make the water flow worse, not better. You need to slow it down, and get it to soak in. Plants are the absolutely best way to do that. They do not harbor vermin you need to worry about.

Start your flood control efforts by the trees at the rear, or by the fence behind them.

For now, use anti-erosion tubes : make rolls of straw or dry leaves or wood shavings and burlap and lay them across the slope of the terraces every couple of feet, held in with wooden stakes. You have probably seen these, but way longer, across the slopes of construction sites.

On the uphill side of the tubes, start planting perennials with deep roots: ornamental grasses, shrubs, dense ground covers, etc. Whatever grows well in your area.