A manifestly Lorentz and diffeomorphism invariant form for the abelian gauge field action with local duality symmetry of Schwarz and Sen is given. Some of the underlying symmetries of the covariant action are further considered. The Noether conserved charge under continuous local duality rotations is found. The covariant couplings with gravity and the axidilaton field are discussed.