Humanity is supposed to be, and I think generally is, on an upward arc. The understanding of what is right, as well as of what can be accomplished in the moment, changes with time. We do have to judge historical individuals based upon the time in which they lived, so what they did to push the arc of humanity upward is arguably more important than whether or not they shared some of the most common beliefs/practices of their time. It doesn't make those beliefs and practices right -- they obviously weren't, but a person of good character must work to accomplish what improvements are possible. Not all problems and injustices can be solved all at once. You are almost forced to deal with them sequentially as the time/mood becomes amenable for solving a particular injustice or problem.
My wife and I were members of the National Organization For Women back during the period of trying to pass and then ratify the ERA. The women's movement had a lot of issues (injustices) it wanted to address, but the wiser heads realized that thee is only so much you can immediately correct. One of the key points of argument, which arose periodically within NOW, was whether the pursuit of equal rights for gays and lesbians should be pushed simultaneously with equal rights for women. NOW had a lot higher proportion of lesbians that the general population (close to 50% in our chapter), but,, in that time, most of them recognized that at that time equal rights for women was possible, while equal rights for gays was not, and rights for gays was parked. Ironically, far more equal rights for gays has now happened, while the chance to pass the ERA was squandered.
Our Founding Fathers lived at a time of great inequality and lack of equal rights for minorities, women, and for poor white men. Slavery was the most blatant injustice. The Constitution was a very big step forward for human rights and democracy, even though it was drafted to permit half the country to own slaves and much of the rest of the country to profit from slavery. Without its facilitation of slavery, the Constitution could not have been approved. It left big swaths of injustice untouched. Some in the Constitutional Convention tried to correct some of those injustices, but it wasn't possible. We have to , IMHO, give kudos for the progress made in adopting the Constitution, while recognizing all the work still remaining to achieve a just society. Similarly, when former slave gained the rights of citizenship after the Civil War, women were kept out. Again a combination of justice and injustice. That is just always the way it seems to go throughout history.
Does FDR deserve a pass. I think if he does, and I am a huge FDR fan, then Jefferson and the other Framers also do. FDR gave us a social safety net and beat Fascism. High marks! He beat the Depression. He also interred Japanese-Americans, incessantly pandered to the Southern Democrats, including fighting WWII with a segregated military. What good he did on social justice is more down to Eleanor. For his time in history, FDR arguably better situated to integrate the military than Jefferson, Washington, et al. were to end slavery. They all had the threats of European powers to deal with. Arguably more an existential threat during the time of the Constitution. I think FDR could have done more to promote justice and equality.