Mostly unwillingness to challenge the status quo. It's the same reason why despite all the incredible advances in remote communication technology (eg. Google Hangout/Skype, Slack/HipChat), most companies haven't embraced remote work.
My employer has been pushing remote work for a while, and they've recently recanted and are telling everyone to come into the office. Why? Because for all the benefits you imagine those tools to provide, they are not actually helping people be more productive than they would if they were working face-to-face.
I work semi-remotely, and yes, it has several downsides, which can be mitigated by the company actively taking action.
On the flip side, companies get access to a deeper talent pool at a cheaper price. My current position used to be based in the U.S., but they couldn't manage to get developers to stay. In my case, for the price of one U.S. junior developer, they got two senior developers.
I do believe some face to face communication would make things smoother (at least two or three times a year).
A lot of great articles have been written on how to manage a remote team. It's definitely not for every company, and tools are not yet there (and probably will never be 100%, but they'll get "good enough" for the normal use case).
Of course there are downsides as well, but they can be managed. When properly managed, the pros (ability to hire from larger talent pool, increased employee flexibility and happiness, focus on results rather than facetime) can outweigh the cons.
