Can time travel be a curse?

I know in the The Time Traveler's Wife, Henry thought time traveling was a curse but in actuality, his ability to time travel was a disease (a genetic one).

I wonder if someone can be cursed with the ability to time travel.

I should just write it and stop worrying about the logistics of it. I guess I wonder if it's a stupid curse. Of all the things to be cursed with, the ability to travel through time just doesn't seem that bad (considering one could be cursed with no memory, warts, a tail (or two)). But then again, there's a twist.....