In a series of tweets, Harris, who said after the song's release in April that he couldn't imagine working with his then-girlfriend, acknowledged Swift's contributions to the song, but maintained that it was her idea to keep her involvement secret.
As a result, he added that it was "hurtful to me at this point that her and her team would go out of their way to try and make ME look bad."
The couple split several weeks ago after a year of dating, and Swift has since been linked to "Avengers" star Tom Hiddleston.
"I figure if you're happy in your new relationship you should focus on that instead of trying to tear your ex bf down for something to do," he tweeted. "I know you're off tour and you need someone new to try and bury like Katy [Perry] ETC but I'm not that guy, sorry. I won't allow it."
A request for comment from Swift's rep about Harris' tweets wasn't immediately returned to ABC News.
Perry, who has long been rumored to be feuding with the "Shake It Off" singer, also has yet to respond directly. However, she did tweet a GIF of Hillary Clinton smiling and shrugging shortly after Harris' tweets were posted.
She followed up with a re-tweet of a post from a little over a year earlier: "Time, the ultimate truth teller."