Gatland will take charge for next year's Six Nations and Rugby World Cup, and potentially the 2027 tournament.
The decision follows a Welsh Rugby Union review into a dismal autumn campaign, which included a first loss to Georgia.
Former Scarlets coach Pivac took over from fellow New Zealander Gatland after the 2019 Rugby World Cup.
But during the 60-year-old's final year in charge, Wales only won three out of 12 games.
Gatland is Wales' most successful and longest-serving coach, having won three Grand Slams during his 12-year reign. He also guided Wales to the semi-finals of the 2011 and 2019 World Cups.
In his final season, Wales reached the top of World Rugby's rankings having recorded a 14-match unbeaten run.
Gatland departs Super Rugby side the Chiefs to return to Wales before Christmas.
Pivac's three-year tenure saw Wales slip to ninth in the rankings, having won 13 games, lost 20 and drawn one.
2022 was a particularly disappointing year in which Wales suffered home defeats to Italy in the Six Nations and Georgia last month.
His highlights include a 2021 Six Nations title and a first a Test victory on South African soil in the summer.
"This is one of the toughest calls to make in sport, but the review process has reached its conclusion and we acted quickly and efficiently in the very best interests of our national team," WRU CEO Steve Phillips said.
"Ultimately we're in the results business and we have agreed with Wayne that the current trajectory for Wales is not where we want it to be and we thank him sincerely for his time, enthusiasm, diligence and effort, which is unquestioned, as head coach over the last three years."
"I'm very much looking forward to returning to coach Wales," said Gatland.
"This is an opportunity to achieve something with a talented group of players in a country so passionate about rugby. A country which made my family and I so welcome, when we first arrived 15 years ago, and all the time we were there.
"Our immediate priority is the 2023 Six Nations and next year's World Cup."
Pivac, who took over from Gatland in 2019, said he was disappointed but understood the decision.
"It was a speedy review process post-Autumn Nations Series as time is of the essence with the Six Nations fast approaching," the New Zealander said.
"Unfortunately, the results or performances this year were not all as we hoped. As a group we all take responsibility for that, but me in particular as head coach.
"We've played some really good rugby at times, but needed to do that more consistently. However, I know there's a strong foundation for the squad to progress to great things in the future."