The National Party has issued a strident letter to Parliament's referee Trevor Mallard, demanding he explain his role in a story about an alleged sexist remark.
Mr Mallard, who is Speaker of the House, insists he heard a National MP call the Prime Minister "a stupid little girl" in the debating chamber two weeks ago.
The remark was first reported last week by Newshub but no source was given. Mr Mallard confirmed on Wednesday those were the words he heard.
The audio recording of the incident is unclear and National's MPs have denied saying any such thing.
National MP Gerry Brownlee said it was unacceptable for the Speaker to promote stories about unverified events.
"None of us heard it and none of us said it," he said.
"The only person who says he heard it is the Speaker and we are concerned about why some days after it had been dealt with, it's now back in the media."
Mr Brownlee said the Speaker had to be completely neutral in parliament and to support the Opposition in its efforts to hold the government to account.
"The Speaker's job is the hardest in parliament ... but there is an expectation the Speaker will behave in a neutral fashion and we're having a little trouble seeing evidence of that."
The letter sets a deadline of 2pm on Thursday for Mr Mallard to respond.
"All we want is a clarification - did he brief the media about this issue? And if he did, then he's shown himself not to be in a neutral position."
National said its confidence in the Speaker had been "severely shaken".
A spokesperson for Mr Mallard said he would not be making any comment to the media about the story.
The letter also expressed frustration at the way Mr Mallard ran Question Time in Parliament.
Since becoming Speaker, Mr Mallard has introduced a new system of discipline, awarding and deducting questions from MPs who interrupt proceedings.
National's deputy leader Paula Bennett stormed out of the debating chamber this afternoon in protest after having five questions deducted.
"When you're in there it doesn't seem very fair and doesn't seem at all predictable," she said.
Mr Mallard said, to date, National MPs had actually been awarded 22 more questions than they otherwise would have had.