Police have described the camouflage-clad attacker as a man wearing a hat, mask and sunglasses.
Officers rushed to the scene on Thursday after a pedestrian reported a commotion in Nakano city in the central Japanese prefecture of Nagano.
A witness told NHK public television a woman fell while being chased by the suspect, who then stabbed her with a knife and shot at two police officers with a shotgun as they arrived at the scene.
“A woman came running from the road saying ‘help me’ and she fell down,” the man said. “Behind her came a man wearing camouflage and carrying a large knife, who stabbed her in the back.”
A fourth person who was wounded could not be rescued because he was near where the suspect was holed up.
Police described the attacker as a man wearing a hat, mask and sunglasses, Kyodo News agency said. City officials urged those in the area to stay home. No other details, including about the suspect and his motives, were immediately known.
Crimes involving guns are extremely rare in Japan, where gun ownership is tightly regulated and anyone seeking to own a gun must go through a rigorous process before a licence is granted.