As the Powerball jackpot recently climbed to $1.3 billion, the lottery has captured our imagination once again. Whenever the lottery is in the news, we hear complaints that it is a regressive tax on the poor. Such claims and criticisms have been around since the beginning of the lottery in America.

This attitude can be seen in an act barring lotteries passed by the Pennsylvania legislature in 1762, which fined anyone running a lottery and diverted the proceeds to relief for the poor:

Whereas many mischievous and unlawful games, called lotteries, have been set up in this province, which tend to the manifest corruption of youth, and the ruin and impoverishment of many poor families; and whereas such pernicious practices may not only give opportunities to evil disposed persons to cheat and defraud the honest inhabitants of this province, but prove introductive of vice, idleness, and immorality, injurious to trade, commerce, and industry, and against the common good…

Similar statements can be found from other colonial legislatures. Nonetheless lotteries continued. Part of the reason is that despite vocal critics, they were popular. Lotteries also helped finance many public enterprises such as universities, churches, poor relief, and the French and Indian War. Lotteries were also popular ways of disposing property; for example, if someone died and the heirs needed to sell off his estate in a community where people could afford to buy lottery tickets but not to buy their deceased neighbor’s house.

Of course, in colonial America “property” was not just land but also enslaved human beings. In the Virginia Gazette an ad appeared on Thursday, April 14, 1768, announcing “a scheme of a lottery for disposing of certain Lands, Slaves, and Stocks belonging to the Subscriber.” This lottery was managed by a number of Virginia notables including George Washington. Imagine being forcibly separated from your family by a lottery, let alone won as a prize? The lottery wasn’t just a source of pain for enslaved people, though: It could also be a beacon of hope and (very rarely) freedom if a slave won the lottery. It was difficult for slaves to accumulate the funds needed to buy their freedom on the normal marketplace. However, a lottery ticket was within their price range, though of course the odds of winning were low.

A famous person who purchased his freedom in just this way was Denmark Vesey. He went on to live for more than two decades as a free man in Charleston, South Carolina, until 1822, when he was hanged for allegedly organizing slaves to burn the city down and kill all white people. (There is some scholarly debate over whether there was a real conspiracy or if Vesey and his supposed co-conspirators were the victims of white panic). Although Vesey met a dramatic end, the lottery made it possible for him to enjoy years of freedom, working as a carpenter, co-founding a church, and being (regardless of the truth of the charges against him) an active abolitionist.

Lotteries continued into the antebellum period but got more controversial as time went on.

An 1826 British ban on the lotteries inspired new debates over the practice. Over the antebellum period, the lottery became less popular and gained more disrepute, seeming a bit immoral and indulgent in a time of such suffering. The economic strains of war, too, helped to greatly diminish the lottery’s popularity, with so few able to afford tickets, and by 1878, all states except Louisiana had banned lotteries.

Louisiana’s lottery was widely profitable, earning at its height $13 million. But complaints about its unsolicited mass mailings led to the first federal anti-lottery laws, which ended the nation’s last lottery by 1895.

The first state-sanctioned lottery since that time began in 1964 in New Hampshire. Spurred by changing attitudes and increasing need for funds, more states started holding lotteries, though the federal anti-lottery laws didn’t change until a decade later.

The criticism they incur today is the same as it was in the colonial days—that they take money from the people who need it the most, usually for little return. The hope they impart is less discussed. Their re-legalization and popularity has coincided with increases in inequality and falling class mobility, making the dream of winning all the more enticing. It’s common to hear moralists tell poor people who are putting down a few dollars dreaming of a better life that winning the lotto is “impossible” and tickets are a “waste” of money. But it’s a good thing no one barred Denmark Vesey from buying a lottery ticket for his own good.