Here's number 1 on my list.
Imam Rauf says the purpose of the mosque and culture center is to build bridges.
That idea is not working at this location for the time being. If he truly wants to build bridges he would come out and talk to the people who doubt his motives.He would not refuse to consider another location without explaining why it is so important to have it there.
He has right to build there, but is it the right thing to do?