"I am stunned," Lemon wrote on Twitter, saying he was told by his agent he had been let go.
CNN said it "parted ways" with Lemon, who co-hosted its morning show, saying "we wish him well".
It follows on-air remarks by Lemon in which he said Republican Nikki Haley, 51, was not "in her prime".
The dismissal also comes on the heels of another major US media departure. Just moments before Lemon's announcement, Fox News announced it was parting ways with primetime host Tucker Carlson.
A long-time fixture of the network, Lemon was most recently a co-host of CNN's This Morning programme.
Ms Haley was not "in her prime", Lemon said in February, a remark widely decried as sexist.
"When a woman is considered to be in her prime - in her 20s, 30s and maybe her 40s," he said.
"I'm just saying what the facts are - Google it," he added, in response to objections from his female co-hosts, Poppy Harlow and Kaitlan Collins.
Lemon issued a statement on the same day saying he regretted his "inartful and irrelevant" comments. He also apologised to the newsroom and agreed to partake in "mandatory training" to address the incident.
But the remarks sparked widespread criticism, including from actress Michelle Yeoh, 60, who appeared to reference the comments in her Oscar's acceptance speech last month, saying: "Ladies, don't let anybody tell you you're ever past your prime."
In a tweet on Monday, Ms Haley appeared to address the firing, calling it a "great day for women everywhere" adding "#StillInMyPrime".
Lemon's reputation was further clouded by a report from Variety earlier in April, which detailed accusations of misogynistic behaviour toward his CNN colleagues.
He called a producer fat to her face, Variety reported, mocked and mimicked one colleague and allegedly sent threatening texts to another. A representative for Lemon denied the reports.
Lemon has also faced controversy last autumn for saying that the US men's soccer team should be paid more than the women's, saying that the men were "more interesting to watch".
The 57-year-old had appeared on CNN on Monday during the morning programme as normal, before reports of his dismissal were publicised later that day.
"After 17 years at CNN I would have thought someone in management would have had the decency to tell me directly," Lemon wrote on Twitter.
"At no time was I ever given any indication that I would not be able to continue to do the work I have loved at the network."
He continued: "It is clear that there are larger issues at play."
The network did not elaborate on the reason for his dismissal. But in a second statement issued on Monday afternoon, it called Lemon's description of events "inaccurate", saying he had been given an opportunity to meet with management.