Gelding likes to prance sideways and wont straighten out?

I bought a gelding a couple of months ago. He is an amazing trail horse...he doesn't buck or spook at all and will go through anything...alone or with other horses. His only vice is that he gets excited. Normally that wouldn't be a problem for me, but when he gets excited he will trot/prance sideways. He will turn his head to the right and shift his body to the left. I have tried pushing his hindquarters back over and pulling his back to the left but he seems to get more agitated the more I try to straighten him out. Any tips? He doesn't do this at all in a round pen...just on the trails and it doesn't matter if we are going up hill or down or on a flat trail.