The biggest waves are forecast to hit Iwate and Aomori prefectures at the top of Japan's main Honshu island, and the northern island of Hokkaido, authorities said.
In the hour following the earthquake, which struck at 4.52pm Japanese time (0752 GMT), tsunami waves as high as 80cm had been detected, while warnings remained for waves as high as three metres.
Several port towns including Otsuchi and Kamaishi - both hard-hit by a massive earthquake and tsunami in 2011 - issued evacuation orders for thousands of residents, according to public broadcaster NHK.
Prime Minister Sanae Takaichi said the government had set up an emergency task force and urged citizens in the affected areas to evacuate to safety.
"Possible damage and casualties are now being looked into," Takaichi told reporters at her offices in Tokyo.
Big aftershocks could occur in the following days and weeks, an official from Japan's Meteorological Agency (JMA) said at a separate televised press conference.
Ships sailed out of Hachinohe port in Hokkaido in anticipation of the waves, footage aired on NHK showed, as a "Tsunami! Evacuate!" alert flashed across the screen.
A three-metre tsunami could cause damage to low-lying areas, flooding buildings, and anybody exposed would be caught in its currents, according to JMA
Bullet train services in Aomori at the northern tip of Japan's main Honshu island were halted due to the tremors, Kyodo news agency reported.
The quake measured an "upper five" on Japan's seismic intensity scale - strong enough to make it difficult for people to move around and cause unreinforced concrete-block walls to collapse.
The tremor had an epicentre in the Pacific Ocean and was 10km deep, JMA said.
Located in the "Ring of Fire" of volcanoes and oceanic trenches partly encircling the Pacific Basin, Japan is one of the world's most earthquake-prone countries, with a tremor occurring at least every five minutes.
It accounts for about 20 per cent of the world's earthquakes of magnitude 6.0 or more, such as the 2011 disaster that caused nuclear meltdowns at a Fukushima power plant.
There are no nuclear power plants currently in operation in the affected areas and Hokkaido Electric Power Co and Tohoku Electric Power Co said there were no abnormalities reported at their idled facilities there.
