Traveling on horseback:
— A file of men ahorseback passed by.

synonym: ahorseback.

On the back of a horse:
— Managed to escape ahorse.
— Policeman patrolled the streets ahorseback.

synonyms: ahorseback, horseback.