The first stage separation appeared to go as planned, but soon afterwards, signs of trouble emerged.
"It seems that the velocity is coming down," announcers on the JAXA live feed said while the rocket was about 300 kilometres above ground.
The command centre then announced: "The second stage engine ignition has not been confirmed yet, we continue to confirm the situation."
The live feed was then briefly halted, with a message reading "We are currently checking the status. Please wait."
When it resumed, the command centre confirmed the bad news.
"Destruct command has been transmitted to H3 because there was no possibility of achieving the mission."
There was no immediate explanation for why the launch failed, though JAXA is expected to hold a press conference later.
The H3 rocket was developed for more frequent commercial launches as well as better cost efficiency and reliability and has been mooted as a possible competitor to Space X's Falcon 9.
"We are aiming to create an operational world where (the) Japanese industrial base can be underpinned by steadily launching the H3 six times or so annually for 20 years," JAXA says in its description of the project.
Developed by Mitsubishi Heavy Industries, it is the successor to the H-IIA model, which debuted in 2001.
Tuesday's launch was carrying the ALOS-3 observation satellite, touted as having improved resolution and intended to help with disaster management and other monitoring.
The incident is not the only recent blow for JAXA.
In October 2022, the agency was forced to send a self-destruct order to its solid-fuel Epsilon rocket after take-off. It was carrying satellites into orbit to demonstrate new technologies.
That was Japan's first failed rocket launch since 2003.
The solid-fuel Epsilon rocket has been in service since 2013. It is smaller than the country's previous liquid-fuelled model, and a successor to the solid-fuel "M-5" rocket that was retired in 2006 due to its high cost.