The coalition's plan to repeal some regulations on gun clubs and ranges is being put out for stakeholder consultation.
In the meantime, one change will be made to annual reporting for non-pistol shooting clubs.
The government agreed to repeal and replace Part 6 of the Arms Act in National's agreement with the ACT Party.
Introduced after the March 15 terror attacks, the legislation requires police to approve shooting clubs and certify shooting ranges.
Associate Justice Minister Nicole McKee on Tuesday announced she would consult with focus groups including club members, police and the Muslim Community over the next four weeks.
"What we're doing now is looking at certain concerns that we have that are immediate," she told RNZ.
"But in the interests of public safety, I think it's best that we don't repeal all of it straightaway but rather go out for that consultation, make sure that we have it right and that the public safety interests are there and paramount."
McKee would not elaborate on what her concerns were, only referring to "discussions" with her coalition colleagues.
"I've had discussions. I wouldn't call it pushback because it's really hard when people don't understand exactly what the role of a club and range is.
"From my perspective, looking at the public safety side of it, I want to make sure that I don't just repeal everything and not have the public safety concern at front of mind."
ACT campaigned on scrapping Part 6 over concern it was too costly and onerous for clubs and ranges, some of which are run by volunteers.
"It's imperative that we have clubs and ranges operating in New Zealand because they are the public safety benefit in our communities if we know where firearms owners are," McKee said.
"At the moment, they're telling us there are so many rules and regulations they feel they can't [operate]."
McKee will consider stakeholder feedback before taking her proposal back to Cabinet later this year.
Those being consulted include members of gun clubs, the police, lawyers, doctors and the Muslim community.
In a statement, McKee said the government would make a short-term adjustment to the non-pistol club reporting requirement, after clubs contacted her about having to provide an annual report.
"This is first-hand knowledge of the pressure clubs are under. In my view, this is an example of over-regulation without a clear public safety rationale," she said.
All shooting clubs would need to comply with current requirements until the changes had come into effect, she said.