The grandfather paradox seems a possible barrier in our temporally linear and exclusively causal reality.
But at a quantum level, things are popping in and out of reality all the time. If I pop up at my grandfather's first birthday party, it's as if I'm instantaneously created. The lineage of my information doesn't exist yet and will never need to. This is a reasoning problem, when we come in after the fact and link up the causes and effects. But it is not a paradox for the physical universe: we don't need to worry that I am not yet born because I already exist. From an anthropic principle view, if only a very tiny subset of universes or realities are cohesive enough to be observable, then this tiny paradox doesn't seem like it would prevent our universe from being an observable one.
Time travel might seem counter-intutive when time is more a property of matter than the ether we usually imagine. But assuming time travel were possible, where are all the time travellers from the future?
And if the present splits into multiple timelines each time we look at it (which is intuitively ridiculous but mathematically possible), we might even get travellers from millions, billions, uncountable numbers of possible futures converging on our present.
One possible answer is that people travel back in time repeatedly until they create a steady state universe in which time travel isn't possible. Not by altering physical laws, but by continuously changing preconditions that enabled time travel to occur.
Additionally, time travellers who make themselves more detectable might more significantly change the preconditions that make time travel possible, so we might be more likely to end up with a present that has inconspicuous time travellers.
If time travel is possible but incredibly difficult and unlikely, then we might see a small number of time travellers appear.
But if time travel is more or less inevitable and common, then more and more travellers will arrive until one of them causes time travel to not be developed.