Rose carded five birdies in a typically consistent display to take a one-shot advantage after 16 holes in the final round at the Antalya event but bogeyed the 17th hole to slip back into a share of first with overnight leader Li.
Lacklustre putts on the 18th from both Li and Rose meant they both finished at 17-under-par after 72 holes.
Both players returned to the 18th for the playoff and after Rose missed a 15-footer for birdie, Li surprisingly three-putted from just nine yards to hand his opponent the title.
"A tough day for me, honestly," the 23-year-old Li said.
"I think I played well the whole week but didn't hole a few putts on the last and that was it.
"It's a pretty good round, especially against Rosey who's such a nice guy. Hopefully next time I can play better."
It was the first time Rose had successfully defended a title in his career and it moved him above American Brooks Koepka at the top of the world rankings.
"It's nice to get back there and maybe it's even sweeter to get back there," said Rose, who had spent two weeks as the top-ranked player in September.
"[Last time] I lost in the playoff to Keegan Bradley [at BMW Championship] and I was pretty aware my playoff record wasn't great. This was the one I definitely wanted to win."