sounds like the pea gravel is your problem. Setting stones on pea gravel is like setting stones on ball bearings. If you like the look of the stone steppers and that aspect is working for you, I'd recommend removing the pea gravel and replacing it with a #8 (3/8") chip gravel. You want a gravel that's very angular so it all locks together.

To go the concrete route, especially where you are it's not setting "in" concrete as much as it's setting ON concrete. You'd pour a minimum 4" slab and mortar the stones to the slab, then mortar and point up the joints. If I'm envisioning your stone correctly I imagine it varies a lot in thickness, which would make laying on a slab very tedious.

ditto what marcinde said. Every part of the country seems to have slight variation in what is available for walkway material that packs, so explore options where you live. I'd also suggest an aggregate where the largest pieces are 3/8," but that includes sand and dust sizes so it packs tight.

I think it's six of one, half dozen of another whether you use 3/8" clean stone or the 3/8" minus that Yard mentioned. If I was doing a totally clean install with an appropriately thick (4-6") base, I'd absolutely use a 3/8" or 3/4" minus for awesome compaction, and then a bedding layer of clean 3/8" stone.

Reading your description, though, I'd wager you have some spots where you have, at best, an inch of stone under your steppers. At that point you have no base, just a bedding layer. Depending on which is cheaper, go with the cheaper ANGULAR aggregate stone.

just in case I wasn't clear, what I'm saying is either solution (mine or Yard's) is wholly appropriate. Don't want anyone thinking otherwise.