I discussed this also with my U.S. history classes recently. Students had expressed confusion about why businesses in slave-free states would still profit from slavery. Northern textile companies would buy their cotton at the lowest price that they could, which was generally from plantations which used slave labor. I mentioned that it's not unlike any of us who buy nearly anything in today's day and age.
I just purchased an iPhone (the X), which uses a rare earth metals that come from regions of the world with high poverty and questionable ethics and is assembled in countries that do not have the same work standards that I would want for myself or my children. Similar stories are behind nearly all of the clothes that we wear. Yet we often ignore that knowledge so we can move on with our lives.
This reminds me of a cartoon by the erudite philosopher Bill Watterson: