Hmm...looks like all of the buildings are super tough, then.

Oh! Actually, the block is gone. I was worried about it, too, but apparently the Pokémon decided to move on. You can go anywhere you'd like now!