Dori,
There is no way to know how far out the phone would work without knowing several things that we can't know.
1) The provider
2) Many providers turn transmission power down at night.
3) The power that particular provider broadcasts at in the region.
During the daytime, most broadcast at a range of 45 miles. Being in the air as the plane was, it would receive the full 45 miles as there aren't any obstacles to limit the range.
I don't know anything about Malaysia airlines, but it would seem to me there would be satellite phones on board for emergency use if not customer use at a premium. Satellite phones use a satellite to connect to a cellular tower; thus would work in the middle of the ocean. It would also solve the tower hop problem that would occur with regular cellphones which is another reason I'd think airlines would have them installed on planes.