The bribery allegations against Mr Kuswaha led to international donors suspending millions of dollars in funding for schools.
He denies accepting bribes from more than 1,000 teachers to secure them jobs in the schools of their choice.
He also denies receiving payments from a textbook printing firm. Impoverished Nepal relies heavily on foreign aid.
The government replaced Mr Kuswaha on Wednesday amid growing controversy over the corruption allegations.
The education minister left his job "based on the recommendation" of his Terai Madhes Democratic Party, Parliamentary Affairs Minister Minendra Rijal told AFP news agency.
Mr Kuswaha strenuously denies any wrongdoing.
"I have not committed any irregularity. There is only a suspicion, they have only expressed suspicion," he told the BBC Nepali service on Tuesday.
"I want a proper investigation on the issue to determine any wrongdoing. If the money hasn't gone where it was meant to go, that should be investigated. And after the investigation, the irregularities found should be annulled."
Correspondents say losing the funds for schools would be a major blow for Nepal.
The country is one of the world's poorest and depends on foreign governments and aid agencies for nearly 25% of an education reform programme which will cost an estimated $4bn over the next seven years.
.png)
