Regular horses and ponies can be purchased in the Turbine store or from Eogar at Hengstacer Farms in the Bree-Land Horsefields for 500s. Dwarves and Hobbits are able to ride Pony's, Elfs and Humans are able to ride Horses. You can choose between four different horses: Chestnut, Blonde Sorrel, BloodBay and Bay. They are fast horses with 100 moral points.
As a reward for achieving reputation with different factions you can obtain Faction mount. These horses can be purchased with either gold (~5 gold) or reputation items. They are equally as fast as regular mounts but they are tougher (150 morale points compared to your normal horse which is 100 morale points).
Festival horses can only be obtained during the specific festival time. Horses are given as rewards for participating in the four different festivals. Usually you have to run and win the race, earn a token and barter for the horse. The festival vendor at Hengstacer Farms sells Receipt of Purchase for 4g, but if you already own a horse the price will be 1 gold, 600 silver.
Special Deed mounts
Special mounts are available for specific achievements like: achieving PvMP rank 9, completing the meta deeds: Tempest of Dol Goldur, the Savior of Eriador, the Savior of Khazad-Dum or the Savior of Lothlorien.
Quest Mount - Grey horse
After completing Volume I, Book 15 your reward is a beatiful Grey horse level 50, fast with 150 Moral Points.
Collection of Lotro mounts can be seen in this awesome HD video. This video includes 27 horses and goats currently available in LOTRO. All credits for this video go to [Eldarond].
Very beautiful guide to the horses in Middle Earth Horse and Pony Guide