A water softener exchanges useful calcium and magnesium ions for sodium ions. Your faucet filter is most likely just a carbon filter. It will remove large organic molecules, but not the sodium ions you're concerned about. Do you have a water outlet upstream from the softener that you could use? Distilled or RO water are very good choices, but are a pain to buy and transport.

If your plants are doing fine with the water you're giving them, then your softened water may be good enough to get them through the winter. Watering until water runs out the bottom of your pots will prevent sodium and other minerals from increasing any further.