He died on Wednesday from heart complications at a hospital in Dar es Salaam, Samia Suluhu Hassan said in an address on state television.
Magufuli had not been seen in public for more than two weeks, and rumours have been circulating about his health.
Opposition politicians said last week that he had contracted Covid-19, but this has not been confirmed.
Magufuli was one of Africa’s most prominent coronavirus sceptics, and called for prayers and herbal-infused steam therapy to counter the virus.
“It is with deep regret that I inform you that today… we lost our brave leader, the president of the Republic of Tanzania, John Pombe Magufuli,” Vice-President Hassan said in the announcement.
She said there would be 14 days of national mourning and flags would fly at half mast.
According to Tanzania’s constitution, Ms Hassan will be sworn in as the new president and should serve the remainder of Magufuli’s five-year team which he began last year.
Magufuli was last seen in public on 27 February, but Prime Minister Kassim Majaliwa insisted last week that the president was “healthy and working hard”.
He blamed the rumours of the president’s ill-health on “hateful” Tanzanians living abroad.
But opposition leader Tundu Lissu told the BBC that his sources had told him Magufuli was being treated in hospital for coronavirus in Kenya.
{{John Magufuli at a glance}}
-# Born in Chato, north-west Tanzania, in 1959
-# Studied chemistry and maths at the University of Dar es Salaam
-# Worked as a chemistry and maths teacher
-# First elected as an MP in 1995
-# Became a cabinet minister in 2000
-# First elected president in 2015
Magufuli declared Tanzania “Covid-19 free” last June. He mocked the efficacy of masks, expressed doubts about testing, and teased neighbouring countries which imposed health measures to curb the virus.
Tanzania has not published details of its coronavirus cases since May, and the government has refused to purchase vaccines.
On Monday, police said they had arrested four people on suspicion of spreading rumours on social media that the president was ill.
“To spread rumours that he’s sick smacks of hate,” Mr Majaliwa said at the time.
{{A legacy set for debate}}
{{Analysis by Dickens Olewe, BBC News}}
John Magufuli cultivated an image of a rambunctious action man, a departure from his predecessors’ demure and stately styles.
His interventionist leadership won him fans beyond Tanzania, especially in the East African region where he once inspired the #WhatWouldMagufuliDo Twitter hashtag that was shared by supporters of his no-nonsense approach to fighting corruption.
There will be vigorous debate about his legacy and whether his successor should stay the course or change direction.
But this discussion cannot be divorced from the current trend on the continent, where support for democracy remains strong but most people are increasingly disillusioned by the failure to deliver the promised dividends.
So while a significant number of Africans would prefer an action-driven leader like Magufuli, they equally want leaders who govern honestly and a government that respects them and does not hide information about their president’s health.
Magufuli’s death has been attributed to a long-standing heart condition, but many will still suspect that he succumbed to Covid-19.
It is an irony that the pandemic he so strenuously denied has outlasted him, turning his once heralded presidency into a cautionary tale for the region and the continent.
Magufuli was declared president on his 56th birthday in October 2015. He was elected for a second term following a disputed poll last year.
He was hailed for his anti-corruption stance during his time in office, but he was also accused of cracking down on dissent and curtailing certain freedom.
His critics agree that Magufuli contributed to Tanzania’s development. He invested in large infrastructure projects such as a standard-gauge railway to connect the country with its neighbours, major highways, and a bus system in the commercial hub of Dar es Salaam.
He also increased electricity production, reducing the need for power rationing.
But it is his approach to Covid-19 that many analysts say will define his legacy. There has been little testing in the country and no plans made for a vaccination programme, leaving the country as an outlier.
In Tanzania, locals have reacted with grief and disbelief to the news of Magufuli’s death.
One, Joseph Petro, told the BBC he thought Magufuli was a “caring” leader, adding “he was helping people in one way or another”.
“I am really pained. I am personally pained,” he said.
Another, Illuminata Abel, offered similar sentiments: “He was not my relative, but he was someone who listened to people’s problems, and he was down to earth.”
