Analysis
The 9V battery came about in an age where boosting voltage was difficult - if not impractical - in inexpensive devices. Flash forward several decades and DC-DC circuits are common, thus it's trivial to implement a cheap, compact, high-efficiency power supply that can boost voltage from 1-2x ~1.5V cells. To the extent that the 9V endures it's a combination of servicing decades-old designs or legacy devices.
Concurrent with this, the market for N•1.5V powered devices has largely coalesced around the AA cell. The other common sizes - C, D, 6V lantern battery - have seen significant reduction in demand relative to decades past; AAA endures for more niche use cases demanding a more compact cell. I've seen a number of tools that used to call for 9V cells turn to 2xAA, a pair of which fit roughly within the same footprint as a 9V battery while broadly outperforming it in terms of energy density and power output.
I expect the 9V battery to recede into specialty status - like most other proper alkaline batteries packaging multiple cells in series - as the market moves on and legacy designs are retired from production.
Opinion
For more than a decade I've treated the 9V like I do coin cells - an unwanted oddball formfactor, expensive, low-performing, unreliable in practice - thus to be eliminated when possible. I have reduced the tools using them to the minimum legacy tools/appliances: gun safe, stud sensors, cheap multimeters. The PALight was retired many years ago after I realized all about I'd done with it was replace batteries as they expired.