Move the garden? Trees shading some now.
My huge 60 + year old oak trees are starting to shade the garden around 4:30 PM. It is a blessing to work in the shade after working most of the day in the sun on my job. They are 50' away from the garden so I don't think they are drawing much water.
It gets full sun at the crack of dawn, so its getting 10 hours of full sun, more on the east side. I have room to move it about 50' east with a little sod busting.
Personally, I think it may be better as is. The shade keeps the late afternoon sun from cooking me and the plants. But on the other hand, more sun is supposed to be better from what I read here and elsewhere.