Dlomu apologised for the “mistakes”, and said she was “greatly disappointed” in the standards of work completed for the Indian-born businessmen, Reuters reported.
“I am determined that these mistakes do not happen again, which is why we have already made a number of changes,” she said.
“I am leading other reforms.”
Dlomu was appointed chief executive last month, after an internal investigation found the auditor had missed “red flags” when dealing with Gupta-owned companies.
This triggered resignations from most of the local senior management team, including former chief executive Trevor Hoole, and Dlomu promised they would be held accountable.
The probe also uncovered issues with a report produced for the South African Revenue Service (SARS), over which the agency’s commissioner, Tom Moyane, wanted KPMG blacklisted for “unethical” and “unlawful” behaviour.
The Gupta scandal saw seven clients cut ties with the Big Four firm, the most recent being Johannesburg-based disposal company Interwaste, which appointed Deloitte as their new auditor on Thursday.
“The change in audit firm, which is effective immediately, was initiated by the company following the concerns recently raised regarding KPMG,” an Interwaste statement read.
Over the past month, the Gupta scandal has deepened.
Bell Pottinger was forced into administration following accusations of stoking racial tensions in South Africa in relation to the scandal.