What's the right solution to dealing with slow 3rd party servers on the web?

With more and more mainstream web sites using widgets (like AddThis), Facebook Like buttons, and obviously ads served from external ad systems, the performance impact on web sites when these 3rd party servers are slow seem fairly significant. On top of that, even if you improve the experience for your end users (using invisible div or JavaScript defer tricks), don't we also have to now worry about the effects on SEO (since Google now considers overall site performance as a ranking factor)? This applies to web sites of all sizes, but is particularly acute for your small and medium ones, since they have far less clout when it comes to trying to influence these partners/technology providers (to try to get them to add capacity or troubleshoot their molasses boxes.) What's the right answer here? Faster timeouts in web browsers? New (X)HTML tag metadata which says "ignore this resource when it comes to ranking my performance, please?" A (requesting) server-side "widget/ad" caching protocol or system?