Because it is something that was considered the norm, historically that is

It is something that has been seen before in the past and in the current timeline. Some POC are struggling and are still discriminated against while white people are given privilege that gives them an easy life. Those people who struggle to comprehend the concept of white privilege are those who do or may have white privilege. They struggle to understand because they do not know how to struggle for things that they already have. Or because they were taught aware of their privilege.