The court's governing body's executive bureau ruled Khan had committed serious misconduct following an 18-months long probe into accusations that Khan had non-consensual sexual interactions with a lawyer in his office, a diplomatic source briefed on the decision said.
Khan has always denied the charges.
The ICC's governing body will send its conclusion on to all 125 ICC member states who are expected to put Khan's possible removal from office to a vote at a later date.
The source added the executive bureau has recommended the prosecutor should be removed from office.
In its press release, the bureau said it had made a decision on the disciplinary proceedings against Khan and referred the matter to the ICC's Assembly of States Parties, but did not give details about what it decided.
"The decision of the bureau and the related documentation will remain confidential," the press release said.
Khan's lawyers said in a statement that he rejected the decision in the strongest terms, and repeated he denies any wrongdoing.
"The decision is unlawful, procedurally unfair and unsupported by evidence," the statement said.
The International Criminal Court has been thrust into crisis by the investigations into Khan - its most prominent official - as well as by US sanctions over the court's actions, including arrest warrants for Israeli officials for alleged war crimes.
Khan has not been at the helm of the ICC office of the prosecutor since last May when he took a voluntary leave of absence pending the outcome of the inquiry. He is the first ICC prosecutor to be formally suspended from his role by the court's oversight body.
Sources told Reuters earlier that a report by United Nations investigators found a "factual basis" for the allegations of sexual misconduct made by a female aide and witness accounts "lend support to her claims".
However, a second report by three judges that analysed the UN report found the evidence insufficient to establish the truth of the allegations "beyond a reasonable doubt", they added.
Lawyers for Khan had told Reuters the judges unanimously concluded that the "factual findings do not establish misconduct or breach of duty."
