Hello Nancy, In my experience with pavers/flag (pavers...two driveways, two walkways, 3 patios, flagstone...4 walkways, one patio and I learned from a guy that did them for 30 years) the base is by far number one. For garrentee one and done jobs (no rework 2-5 years later) 4" of compacted base. Number two....DO NOT USE SAND to level, sand can AND will shift given enough time and rain and will not offer enough support for thinner types of material as you found out. Is leveling base harder (more time consuming) than leveling sand? YES it is. But then you are finished with it. No chance of sand shifting or washing out over time.

I know my advice flies in the face of DIY and most beliefs out there....it's simple...if you enjoy redoing your patio/walkway every 3-7 years, by all means USE sand, it's faster and less time consuming (cheaper labor cost).

Good luck in your choices and project