You are claiming the 1690's as your evidence? Ok...
Who is to say there weren't witches in 1692 Salem? Court trials were conducted and those deemed guilty were executed. Was each person executed innocent? Only some? Just a few?
The point being is that even if ONE innocent was executed, it was immoral. If 20, 200, or 2,000 were executed because of some sort of majority thinking, or more likely mass hysteria, it was immoral and wrong. It doesn't matter who did it then nor does it today.
Just looking at today's time period, how many innocent lives were lost to abortion? Millions! How many more will be lost? Hundreds? Thousands? Millions?
You ask why? Because injustice anywhere is injustice everywhere. It affects everyone. The same people in the 1800's who, like you, said "Well, I don't own slaves so slavery doesn't affect me". The same people who said, "What do I care what some white southerners do, Jim Crow doesn't affect me here". The same people who, like you, said "Why do I care what happens to a bunch of Jews in Europe, it doesn't affect me here".
It doesn't affect you... until one day it does. There are those who, like you, will turn a blind eye to an injustice because, for one, you can justify it, and two, think it doesn't affect you so who cares? Fortunately, there have always been those who've seen the right and fight for the right - unfortunately, it sometimes even has to be against those who are ambivalent towards it.