We investigate exceptional generalised diffeomorphisms based on E_8(8) in a geometric setting. The transformations include gauge transformations for the dual gravity field. The surprising key result, which allows for a development of a tensor formalism, is that it is possible to define field-dependent transformations containing connection, which are covariant. We solve for the spin connection and construct a curvature tensor. A geometry for the Ehlers symmetry SL(n+1) is sketched. Some related issues are discussed.