The Rise of the Astor Dynasty: Part II

William Backhouse Astor was the second son of John Jacob Astor I and one of five children who survived to adulthood. When John Jacob I died in 1848, his eldest son, John Jacob Astor II, could not feasibly inherit due to mental health issues; William Backhouse was named as his father’s heir and became the richest man in America virtually overnight.

Born in 1792 in New York, William grew up under the shadow of his aggressive and calculating father. As the heir, he was expected to take over and expand the family business, which largely consisted of exports and real estate. Educated at a boarding school in Connecticut as a boy, he later attended Columbia, and then Heidelberg and Gottingen Universities in Germany. With his diverse education, William was exposed to multiple areas of study which allowed him to fully embrace the various cultures he would experience during his four year Grand Tour of Europe. The Grand Tour was a rite of passage for all wealthy young men at that time and William traveled throughout Italy, Germany, and France, and other places with his tutor. He was called home from his travels in 1818, at the age of twenty-six, with the expectation that he would go straight to work at the Astor & Son’s Prince Street office in New York City.

Shortly after his arrival back in New York, he married Margaret Rebecca Armstrong. While the relationship was not necessarily a love match, William and Margaret were considered a very compatible couple and had known each other since childhood. Margaret was a descendant of both the Beekmans and the Livingstons: two long-standing Hudson Valley families who could trace their lineage to the ancestors who purchased vast acreage in what became New York State directly from King James II in the late 1600s. Their marriage tied Margaret’s rich family history with the Astor name, ultimately making the names of Beekman, Livingston and Astor synonymous with American aristocracy. Together, William and Margaret had seven children, six of whom would grow into adulthood. Two of their sons, William Backhouse Jr. and John Jacob III, would later inherit the majority of their estate and find themselves at the front and center of New York Society amidst the dichotomous world of the Gilded Age.

William Backhouse Sr. and Margaret lived quietly: they were steadfast in their religion and did not flaunt their wealth as some other families in their peer group. However, they could not escape the spotlight of society and newspapers: as the richest family in America, whatever the Astors did, society emulated or disdained. When they moved their family to a new townhouse on LaFayette Place, many others bought up the land around them to build there as well. When they decided to purchase property in the countryside of upstate New York, others followed and built along the Hudson River too. Suddenly, having “a country home” away from the City became de rigueur in Society.

In 1830, they purchased La Bergerie in Barrytown, NY from Margaret’s family as a second home. It was an estate of four-hundred-twenty plus acres located along the Hudson River, about a hundred miles north of the City. Margaret quickly renamed it Rokeby, after Sir Walter Scott’s poem of the same name.

From all accounts, William seemed content to handle the day-to-day activities of the family business. He was known to be boring and staid and had an insatiable love of learning. He preferred to be in his library with a book (there was a beautiful octagonal library at Rokeby) and only socialized when it was necessary for business. His mind was so sharp that it is said he was able to recite the rent rolls and every bit of minutiae about all of the properties held under the vast Astor holdings. Whatever creativity for business William may have espoused as a young man, it was unfortunately squashed under his father’s continued oversight of the Astor & Son enterprise until John Jacob I’s death in 1848 when William was fifty-six.

As the richest man in America, he had become a man ultimately set above all others, yet he remained separate from it all. He didn’t flaunt his wealth nor did he get involved in public life, but life was rapidly changing all around him and he tried to fiercely cling to the old ways. He made a point to support the Tammany Hall regime because he valued the stability he thought they represented. Social reformers, progressives and radicals were unknown entities in William Backhouse Sr.’s eyes, regardless of the fact that some were working for the greater good. From Astor’s viewpoint, they were all a threat to the family’s business practices and a possible drain on revenue. Astor continued to line the Tammany Hall politicians’ pockets in an effort to help them control the City and its laws.

When Tammany Hall was eventually found to be corrupt, the family’s connection didn’t help the overwhelming perception that the Astors were really slumlords. In the eyes of many, they were taking advantage of the poor through their blatant manipulation of zoning and building laws, which allowed them to create as many tenements as possible per lot. From the outside, the Astors seemed to care little for the living conditions of their tenants. As their fortunes increased through the growth and management of their vast properties, they ultimately contributed to the poverty that was running rampant across many areas of New York City.

Shortly after her father-in-law’s death, Margaret Armstrong Astor established Mrs. Astor’s Orphan Asylum (later known as St. Margaret’s Home) in 1853. The home was located in Red Hook, NY near their Rokeby estate. It was created as a home for orphaned and neglected girls from northern Dutchess County to learn the skills needed to either find a husband or be employed on the many country estates along the Hudson River. She paid a matron to run the home and the family paid expenses for upkeep of the property. By 1872, when Margaret died, the home was housing ten girls.

William Backhouse Astor Sr. left the world in 1875, three years after Margaret. Although he led a quiet, unassuming life collecting rents and avoiding the society he disdained, he had managed to double the estate his father had left him. He left no other legacy behind. Of an estate of over $55 million, only $1.5 million dollars (less than three percent) was divided between several charities, including the Astor Library. The remaining funds were dispersed to his heirs: $1.1 million was bequeathed to each of his daughters, or their heirs, with a caveat that this was “to relieve them of the care and exposure a larger fortune would carry.” His granddaughter, Margaret “Maddie” Ward, whom William Backhouse and Margaret had raised after the death of her mother, Emily, was also given Rokeby (and the property still remains in the Astor family’s hands to this day). His two sons, John Jacob III and William Backhouse Jr. would split the remaining funds of more than $50 million in accordance with the rules laid out in John Jacob I’s will. His third son, Henry, received only $30,000 as he had married against the family’s wishes in 1871.

Newspapers were quick to point out that between his wealth and the vast acreage he owned across the City, he could have created a foundation for benevolent reform. They surmised that he was in a prime position to use his wealth to change the lives of the masses, whose populations continued to swell in his slum tenements, and who ultimately increased his fortune through the rents they paid. Instead, there were no schools, no churches, no parks which bore his name; there were no improvements he instituted. William Backhouse Astor Sr. had left no mark on the City that had built his family’s name for two generations. At the time of his death, many critics in the newspapers would say he had been “derelict in performing his duty as The Landlord of New York and as its leading citizen”.

Afterword: The Hudson Valley is as enmeshed in the Gilded Age as New York City. I was raised in the heart of it all: my hometown is surrounded by the country estates of the Livingstons, Astors, Roosevelts, and so many other notable families. My mother grew up on Rokeby Road in Red Hook. It was named after the Astor estate, and connects Route 9 (the Old Post Rd) to the Barrytown docks down at the Hudson River.

The building which housed St. Margaret’s School is still standing at the end of Rokeby Road just north of the Route 9 intersection. It was closed in 1932 when the house and property were sold, and it served as a private residence for almost forty years until it was repurposed as a halfway home for teens. It was sold again in the early 2000’s, to the Hannaford grocery chain, which donated the house and about two acres of land back to Red Hook in 2006. St. Margaret’s was named to the National Historic Register in that same year and has since been restored. If you’re curious to learn more of the Rokeby Astors (theirs is a tragic and bittersweet tale) I highly recommend The Astor Orphans: A Pride of Lions, by Lately Thomas.

Sources: The Astors, Derek Wilson, 1993 and Dynasty: The Astors and Their Times, David Sinclair, 1984; Life Along the Hudson, The Historic Country Estates of the Livingston Family, Pieter Estersohn, 2018.