His comments came as eight Chinese government ships sailed near East China Sea islands that both nations claim.
A flotilla of 10 fishing boats carrying Japanese activists was also reported to be in the area, as well as the Japanese coastguard.
Abe was speaking in parliament hours after dozens of lawmakers visited a controversial war-linked shrine, reports the BBC.
A total of 168 lawmakers paid their respects at the Yasukuni Shrine, which commemorates Japan's war dead, including war criminals, in a move likely to anger regional neighbours who say the shrine is a reminder of Japan's military past.
The warning from the Japanese prime minister was the most explicit to China since Abe took power in December, the BBC's Rupert Wingfield-Hayes reports from Tokyo.
Asked in parliament what he would do if Chinese ships tried to land on the disputed islands, Abe said they would be expelled by force.
"Since it has become the Abe government, we have made sure that if there is an instance where there is an intrusion into our territory or it seems that there could be landing on the islands then we will deal will it strongly," he said.
The warning came as eight Chinese ships sailed around the islands – called Senkaku in Japan and Diaoyu in China.
The Japanese coast guard said it was the highest number of Chinese boats in the area since Tokyo nationalised part of the island chain in September 2012.
China said its ships had been monitoring Japanese vessels. The State Oceanic Administration issued a statement saying three of its ships had "found" several Japanese ships around the islands and "immediately ordered another five ships in the East China Sea to meet the three ships".
Ten Japanese boats carrying around 80 activists arrived in the area early on Tuesday, Reuters news agency reported, monitored by Japanese Coast Guard vessels. Public broadcaster NHK said the boats were carrying "regional lawmakers and members of the foreign media".
Japan's top government spokesman said the "intrusion into territorial waters" was "extremely regrettable". Japan also summoned the Chinese ambassador to protest, reports said.
The territorial row has been rumbling for years but was reignited last year when Japan bought three of the islands from their private Japanese owner.
China claims the island chain, which is controlled by Japan. Taiwan also claims the islands, which offer rich fishing grounds and lie in a strategically important area.
The dispute has led to serious diplomatic tension between China and Japan, most recently in January when Japan said a Chinese frigate locked weapons-controlling radar on one of its navy ships near the islands – something China disputes.