I have another book in my queue, The Time Traveler's Wife, and that will be a similar trope, and I can recall reading and liking Jack Finney's book, Time and Again, and let's not forget about the movies that do this, like the classic Somewhere in Time, probably one of the worst acted movies from the 80s that, if it had time traveled to present day, would never made it to the big screen, but would've been instead a Lifetime original movie. That's the one where Christopher Reeve falls in love with a photograph of an woman from another time and then figures out a way to rub a penny and get back to where she lives. She miraculously falls in love with him, but the problem is that he can't stay in that time period, and so there's this scene where he gets sucked back into his present day, in an effect that's like one you've seen in a Tidy Bowl commercial where he sort of swirls away down a hole with Jane Seymour screeching, Richard! as he's pulled down the time drain. I couldn't find that scene, but here's a different one. In the end, I believe he dies from starvation or heart disease or something and he meets her in heaven.
But my point is that I am not into time travel, even in a book that is well written like the aforementioned one. Sci fi general has not ever appealed to me, I guess because I have trouble predicting what will happen when I don't know the exact rules.
In any case, enjoy this scene from Somewhere in Time.