Justice Minister Andrew Little is on the right track calling for a philosophical shift in the way New Zealand looks at criminal justice, the Salvation Army says.
But it is calling for him to put his money where his mouth is by scrapping a planned mega-prison and putting the money towards re-integration programmes.
Mr Little wants to reduce the prison population by 30 percent over the next 15 years.
There are just under 11,000 inmates around the country - nearly double the number 20 years ago.
As of September, there were 3000 prisoners on remand, compared with 1800 in 2012.
The previous National government toughened the bail laws in 2013.
Mr Little has asked officials to provide advice on changing bail laws, which he said was a cause of the spiralling prison population.
He said "tough-on-crime"-style policies had not worked and said it was time for a shift in philosophy.
The country needed to change the way criminal justice worked, and policy that emphasised harsher punishments and more prison beds was not bringing numbers down, Mr Little said.
He said many prisoners had health and mental health issues, and a proactive prevention-based approach was the way forward.
Salvation Army social policy unit head Ian Hutson said reading Mr Little's words was music to their ears.
"Often people that offend have mental health issues. And if all you're doing it investing in what is, in a sense, the punitive end of things, it's a backwards step in many respects."
Mr Little's words follow an open letter, signed by dozens of academics, calling for a planned mega-prison in rural Waikato to be scrapped.
The prison, which would be in Waikeria, would have 3000 beds - making it nearly three times as large as the country's current largest facility.
Mr Hutson said scrapping that plan and putting the money towards re-integration programmes would be a real statement of intent.
"If that money - or at least some of that money - was invested into re-integration, mental health support, addiction support ... I'm sure that money could be spent better in targeted areas around that side of things."
A decision on whether to go ahead with the planned Waikeria prison is expected in the coming weeks.
Canterbury University criminologist Jarrod Gilbert told Sunday Morning New Zealand was at a significant crossroads.
"We've switched and said 'hang on a second, recidivism rates are very high, this is becoming incredibly costly, we're not actually solving problems' ... so we must try something different."
Politicians wanted to be seen to be tough on crime because it was popular with voters but it has not been an effective policy, he said.
Dr Gilbert said the government may have to go ahead with building the new billion-dollar prison, despite its hope to reduce prison numbers.
"We've got people in prison currently who are double-bunked into cells that were only ever designed for one person.
"We've opened up emergency cells, we've got women housed in men's prisons.
"We're at a crisis point.