The restrictions are being imposed in areas surrounding the capital as well as in the city of Osaka.
Japan’s Prime Minister Yoshihide Suga warned infections were spreading at an unprecedented rate, urging the country to watch the Games from home.
New cases are being fuelled by the more infectious Delta variant.
“If the increase of infection does not stop, the severe symptoms cases will increase and the medical system may possibly be further under strain,” Mr Suga said.
Earlier, Health Minister Norihisa Tamura warned that the country had entered a new, “extremely frightening”, stage of the pandemic.
“I think that people cannot see ahead and, worrying how long this situation will last, they find it unbearable that they cannot return to normal daily life,” Reuters quoted him as saying.
Japan has had some success fighting Covid-19, keeping cases and deaths low for months, but is now seeing record cases.
Daily cases nationwide topped 10,000 for the first time on Thursday, more than a third of them in the capital.
Tokyo – where the Olympics are mainly taking place – has seen three straight days of record cases, even though it is already under a state of emergency. Hospitals are under increasing pressure in the city.
Olympic organisers reported 27 new infections at the Games on Friday, bringing the total since the start of July to more than 200. But with strict rules in place, including a ban on spectators, organisers deny the event is driving the rise in cases.
Despite this, some experts worry that holding the Olympics in such circumstances sends a confusing message to the public about the need to limit daily life.
Under the state of emergency, bars and restaurants must stop serving alcohol and close early.
Three regions, known in Japan as prefectures, along with Osaka, will go under the restrictions from 2-31 August, with the measures in place in Tokyo extended to the end of the month.
While other countries have imposed legal restrictions on residents, in Japan people are only advised to work from home, so questions have been raised as to how effective the new measures will be.
Tokyo’s Governor Yuriko Koike has said the key to controlling the outbreak is young people, urging them to get vaccinated, but the BBC’s Mariko Oi says this is a little unfair given the low availability of doses.
Less than 30% of the population has been fully vaccinated, with officials aiming to jab all those who want a dose by October or November.
Speaking to the BBC, some young Japanese people were sceptical about whether the new measures would work.
“I can sense that we are getting too used to the state of emergency, so it’s not stopping people from going out,” said one.
“If the government really wants to stop the spread of the virus, they have to lock us down and offer financial support,” said another.