You can remove only paired free blocks. The block is free when at least two neighboring sides of it are free.