Speaking at a Senate estimates hearing, Philip Lowe said people were writing to him about their distressing personal circumstances.
“I read those letters and hear those stories with a very heavy heart,” he said.
“Personally I find it disturbing.”
But he said higher interest rates were necessary to rein in inflation.
“If we don't get on top of inflation it means even higher interest rates and more unemployment,” he said.
There are mounting criticisms about the central bank's interest rate predictions and how they have been communicated.
Bernie Fraser, who led the RBA for seven years before his tenure ended in 1996, said it would be better if the central bank flagged the possibility of further rises while it watched the impact of existing interest rate hikes, rather than giving firm predictions.
“The market has sort of jumped on and interpreted this as the likelihood or near-certainty of another three or four increases to interest rates,” he told ABC Radio.
“That's unhelpful and doesn't provide the kind of confidence that the bank should be striving to enlist with the community.”
Dr Lowe again dismissed calls for his resignation based on poorly communicated forecasts during the pandemic.
“It's an important job that comes with public accountability as part of the process but I intend to serve out that term,” he said.
The governor did not mention his possible reappointment however, with the treasurer due to make a decision about a second term later in the year.
Treasury secretary Steven Kennedy, who has a seat on the RBA board, said criticism of interest rate decisions should be applied to all board members.
“The criticisms or otherwise of the interest rate decisions apply to the whole board, not just the governor because it's the board that makes the decisions,” he told the parliamentary committee in Canberra.
But the governor did admit to providing too much communication about interest rates and other matters.
Responding to criticism of a private interest rate briefing he gave a group of bankers, Dr Lowe said he was revising his timetable of public speaking arrangements based on feedback he was “maybe talking too much”.
“It is possible to talk too much. I have been conscious of that,” he said.
Dr Lowe denied the bank was under pressure from the treasurer or other ministers to stop increasing the cash rate.
“It's noisy but raising interest rates is always unpopular and it affects the whole community and the representatives of the community understandably will sometimes want to talk about that,” he said.
“But we keep doing our job and our job is to make sure inflation comes down and hopefully preserve the gains in unemployment that we have made.”
NAB economist Tapas Strickland said the governor stuck with the same hawkish messaging observed after the February cash rate decision.
"Governor Lowe's comments and tone today suggests he is more worried about inflation risks than activity risks,“ Mr Strickland said.
He said the rhetoric supported the market's expectation of three more interest rate hikes, taking the cash rate to 4.1 per cent by May.