The islands are made of limestone, a kind of soft stone that water can slowly dissolve. Around 500 million years ago this part of Vietnam was the bottom of a warm shallow sea. Tiny sea creatures with hard shells lived and died there, and over a very long time their shells pressed together to make rock.
Later, the land was pushed up by movements deep inside the Earth. Once it was above sea level, rain and rivers slowly carved the rock into towers. When the sea rose again, it flooded the valleys between them - leaving only the tallest towers poking above the water. That is why the islands look so steep on every side.
Many of the islands have hidden caves carved by rainwater. The biggest, Sung Sot Cave (which means 'Surprise Cave'), has rooms tall enough to fit a five-storey building inside. The cave walls drip with stone shapes called stalactites and stalagmites that have taken thousands of years to form.
People live on the bay too. Some Vietnamese families have spent generations in floating villages - houses on bamboo rafts, with boats instead of bicycles. Children growing up there often learn to swim before they can ride a bike, and some take a wooden boat to school each morning instead of a bus.