Even though the void is right underneath your feet, the starting island is not the only solid ground you've got. There are multiple structures to be discovered and explored. They lay on a straight 'line', so there is no need to look in all directions. 

Of course, these structures contain a reward, big or small, for exploring them. The difficulty of finding and searching these structures is compromised with loot that suits it. Some loot may also be key to progress more easily in upcoming dungeons.