I've never so much as touched a PSP, so I can't speak from a position of authority like Geo, but his arguments make sense. UMD is pretty much universally reviled as a shitty format, and represents Sony thinking it's still 1997 and storage capacity will defeat snappy load times. Thing is, this isn't the console market, and this time Nintendo has its shit together. And Nintendo is winning, both in the console AND the handheld market, by targeting users who put a premium on pick-up-and-play value instead of horsepower.
All that said, the PSP's still selling just fine, and even frequently beating out DS sales in both the US and Japan. The biggest risk to eliminating physical media is that this concept may still be ahead of its time. That's the way the industry is bound to go, but IMO eliminating brick-and-mortar stores and targeting a download service for distribution is still something that falls under the umbrella of "targeting hardcore gamers at the expense of casual ones".
At least, as far as a dedicated gaming handheld goes. On the iPhone, the concept of the App Store has become second nature to even the most casual of users. If Sony can put together something with a good interface and decent download speeds, it can mitigate the worry that a casual gamer might have about not being able to pick up a game in the store.
In fact, by all accounts, Apple's sorting of games in the App Store is fucking atrocious and could easily be improved on.
OTOH, the rub here is that Sony's downloads are likely to be much bigger than Apple's. And nobody likes waiting for a slow download.
Anyway. If I were to make a prediction, I'd say that the PSP Go does well but Sony's market share stays about where it is for now, that the machine's a success but doesn't put much of a dent in Nintendo's dominance or Apple's ascendance. If done right, it'll be remembered as a pioneering machine that set the tone for future generations of gaming; if done wrong, it'll be remembered as a nice effort that just didn't quite meet the needs of a good online distribution system.