Does anyone know the name of this purple/blue shrub?
Ruellia simplex (Ruellia brittoniana)
Here is a link that might be useful: Ruellia simplex
You are right, another name for it is a Mexican Petunia.
They sell them at all the garden centers here.
If you can, either keep it in a container, or a part of the
garden where you don't mind it taking over. It is very
invasive and has underground roots that go all over
the place. If you love it and want more of them, they
root very easily in a cup of water.