For years, it’s been getting harder for American women to access abortion services as clinics have closed and state laws have raised barriers to getting these procedures. In coming years, the Supreme Court may move closer to overturning Roe vs. Wade altogether.

The first time abortion was criminalized in the United States was in the mid-nineteenth century. Nicola Beisel and Tamara Kay write that a key factor in the anti-abortion arguments of that time was that too many native-born white women were ending their pregnancies, opening the door for the country to be overrun by fertile foreigners.

Beisel and Kay write that nineteenth-century doctors, who led the fight for anti-abortion laws, saw ending a pregnancy as anathema to women’s role in the world. Anti-abortion leader Hortio Storer wrote that woman was “what she is in health, in character, in her charms, alike of body, mind and soul because of her womb alone.”

Abortion, the doctors argued, was very likely to be fatal, or at least to drive a woman insane. Beisel and Kay note that it certainly was a scary and painful procedure at the time, but that historians have found it was still safer than childbirth. Many doctors were opposed to contraceptives as well, seeing any sexual relationship in which conception was prevented as a form of prostitution.

And yet, women—white, middle-class women in particular—were clearly avoiding having children. Doctors’ writings expressed horror at the contrast between the declining fertility of native-born Anglo-Saxon women and the large families of Irish Catholics and other immigrants. Of ten abortion-related books and tracts directed to the public that Beisel and Kay found at the Library of Congress, six included warnings about the potential demise of the Anglo-Saxon race, and two others alluded to the different fertility rates of Protest and Catholic families.

Doctors contrasted middle-class native-born women’s supposed use of abortion to serve their vanity with the Catholic Church’s firm stance on the subject. One wrote that “in the Romish church, murder and suicide in any form is regarded in all its horror and enormity, and as a natural result, the Catholic element in this country is rapidly increasing.” Another wrote of the duty of patriotic white couples “to the great American idea of free schools and a free Protestant religion and free institutions for all, that they stop murdering their children, and stop trying to defeat nature in any way, so that our American homes many again become populous with incipient citizens and voters, and so that the American family shall not become an extinct institution in this country.”

The doctors succeeded in helping to pass a series of state anti-abortion laws in the 1850s and 1860s, and continued to crusade against illegal abortion (as well as birth control) through the end of the century. And yet, women clearly took increasing control over pregnancy and childbirth. The total fertility rates for whites—including both native-born Protestants and immigrants—ended up dropping from seven children in 1800 to 3.6 in 1900.