Why is this such a mission?! It's probably one of my biggest bug-bears with London, along with the lack of true 24 hour life and the cost of transport.
The best site I've found so far listing spots is: http://londonist.com/2007/05/free_wifi_in_lo.php. It's a good start, but not ideal and not always accurate.
Cafe Camera near the British Museum is tiny but quaint.
The Royal Festival Hall cafe level is a great spot but last time I was there my laptop refused to get online.
Pret have announced free wifi but I've yet to find a branch that has implemented it.
The ideal for me would be Cafe Nero, my favourite chain, having free wifi. Come on Nero, I'll live in your coffee shops if you don't make me pay for net access!
How about a new logo indicating free wifi?