The case, the latest point of friction in an escalating spat between Japan and China, surfaced on Tuesday after Beijing protested to Tokyo.
An individual who claimed to be a Japan Self-Defence Forces officer scaled the wall and forced his way into the embassy compound on Tuesday morning, Lin Jian, a Chinese foreign ministry spokesman, said in a news conference in Beijing.
The Tokyo police on Wednesday told The Associated Press they arrested a 23-year-old member of Japan's army in the alleged trespassing into the Chinese embassy.
The suspect is a soldier based at the Camp Ebino, in Japan's southwestern prefecture of Miyazaki, and army officials are "fully co-operating" with police.
Japanese Chief Cabinet Secretary Minoru Kihara said the arrest of a soldier was "extremely regrettable".
Kihara also expressed regret that the alleged trespassing was committed even as Japanese police provided routine security around the Chinese embassy, adding that security had since been reinforced, including by increasing staffing.
Following China's protest and demand for preventive measures, Japan conveyed to China its regret over the case and explained that Japanese ministries would respond appropriately, including by taking preventive measures under relevant international and domestic laws, Kihara said.
The suspect forced his way into the embassy hoping to tell the Chinese ambassador to stop China's hardline stance on Japan, and said he would kill himself with the knife he was carrying if his demand was rejected, Japanese media quoted police as saying.
Tensions between the Asian nations have escalated in recent months after Japanese Prime Minister Sanae Takaichi said in November a Chinese military action against Taiwan could constitute "a survival-threatening situation" for Japan requiring the use of force.
Beijing, which regards the self-governing island as part of its territory, has since stepped up diplomatic and trade reprisals against Japan.
Japan's NHK public television said the suspect was detained on the spot and handed over to Tokyo police for further investigation and that nobody was injured.
The man allegedly climbed the embassy wall and a knife was left behind, NHK reported.
