Whatever it is, it's just about got to do with the hosting environment and not the site software. I'm thinking the database server is separate from the web server (on different computers) and the database server is being shared with a bunch of other sites, likely "bloggers". That or the network connecting the two is saturated at times, resulting in the very high latency everyone's talking about. I have noticed some delays in pages starting to be delivered, but not in the actual delivery once initiated. That second aspect may well be due to my limited-bandwidth connection.
I think it was either Christmas eve, or its eve, when I was at a friend's house with my laptop to fetch several hundred megabytes of source code. The stuff eventually ended up coming in at 500 KB/s average, but his Roadrunner Cable connection was so saturated elsewhere that the entire process was "excruciatingly" slow. High latency sucks, but it actually sucks worse when there's high bandwidth involved!