Units pass thru neutral (giants) on path

When I click to gather converted units from main tile all the way to enemy tile so...they sometimes pass thru neutral (giants tile). This causes unintended aggression from giants.