It's cool, man, I don't think anybody thought you actually seriously meant all Doctor Who fans are racist.
But Ted's right too that Elba's casting WOULD bring the racists out of the woodwork, like it did in Thor.
But it won't be Elba. As Bleeding Cool pointed out, the reason BBC's not likely to go for an established name isn't just that they want us to be surprised, it's that the show's budget keeps getting slashed. Paying Elba's rate for a half-dozen episodes of Luther every year or two is a pretty far cry from paying him for Doctor Who.
Which isn't to say they WON'T cast a black guy -- just not a black guy who's already famous.
We're not going to see a guy who's been in a Thor movie play the Doctor. We're just going to have to settle for a guy who played the Doctor being in a Thor movie.