It would be far cheaper for the taxpayer if Premier House was demolished and completely rebuilt, rather than spending millions on renovations, says a local real estate agent.
The historic Wellington building on Tinakori Road in Thorndon, also known by the title the Prime Minister's Residence, is about a 15-minute walk from Parliament.
Prime Minister Christopher Luxon says he is not living there because it needs significant maintenance work.
His choice came under scrutiny when it was revealed he was receiving more than $50,000 a year as an accommodation payment while living in his own Wellington apartment. Luxon later pledged to pay back the amount already paid to him.
Heritage NZ says the building is in a dishevelled state partly as a result of the multiple purposes it serves.
It became a Category 1 historic place in 1988, and has been scheduled as a heritage place in the Wellington City District Plan. The site is managed by the Department of Internal Affairs and currently has a Government Valuation of $37 million.
But Tommy's Real Estate sales director Tim Clark told Checkpoint it would be far more financially viable to knock it down and start over.
Clark said the building had an interesting history since it was purchased by the Crown in 1865, including decades where it served as a place to train dental nurses.
He said there had been some renovation in the 1990s, and not much since then.
Clark said that with renovations potentially costing tens of millions of dollars, it would be "significantly cheaper" to build a whole new building.
"If you just knocked the whole thing down and started again, you're probably going to come in significantly under a renovation budget."
And if a residence for the Prime Minister was not needed, Clark had other ideas for the 1.46 hectare property in central Wellington.
"It's a wonderful block of land, so you could do some pretty cool development there.
"Wellington has a limited amount of land."
Heritage NZ Pouhere Taonga has confirmed it is working with other agencies to find solutions for a badly needed upgrade to the building.