The Central Park fountain that Marian, Tom, and Peggy visit is Bethesda Fountain. The statue atop it is "Angels of the Waters" and was indeed designed by sculptor Emma Stebbins, the first woman commissioned by New York City to create public artwork.
Richard Morris Hunt was a 19th century architect, famously designing buildings and homes in New York and Newport RI, including many Fifth Avenue mansions in NYC.
When being pushed by Oscar to meet the Russells, Aunt Agnes says that she "fells like King Canute pushing down a tide of vulgarians", to which Oscar mockingly replies that Agnes' policies make as much sense as the King's. Famously, King Canute I of Denmark once stood at the shore and ordered the sea to retreat. King Canute allegedly did this to prove to his court that his power was nothing compared to God's, and the story has passed on as an example of a useless command against an unstoppable force.
At the dressmaker's, Marian says that soon she'll have more clothes than the Princess of Wales. She is likely referring to Mary of Teck who would go on to be Queen Mary and appears as a character in the Downton Abbey movie which takes place in the same universe as The Gilded Age.