I think that the majority of prehistoric Pokemon went extinct due to the same factors as the real world, climate change etc. Some of the species didn't exactly die out, but rather changed over time, like Archen evolving (in the real world sense) into more modern birds. As they died out some of them formed fossils, and many of those fossils are yet to be discovered. Scientists revived those fossils, but the resulting Pokemon were not well-adapted to the modern world and many died/were hard to breed, therefore they are very rarely seen, especially taking into account the rarity of fossils in the first place.

Dinosaur-like Pokemon who aren't fossil Pokemon either came into existence later or stayed alive because of their power or ability to better adapt to changing surroundings. Other 'living' prehistoric Pokemon like Yanmega and Relicanth count as this too.

As for the fossil Pokemon who are naturally found in the wild in the anime, maybe some of the prehistoric Pokemon did survive whatever catastrophes wiped most of them out. However, there are very few examples.