With overworked servers receiving and storing spam for hundreds of thousands of users, your access to your own e-mail can also slow down. E-mail servers are powerful machines, capable of doing thousands of tasks at one time. Even those big machines can get bogged down, though, and when you're eager to read your e-mail, the last thing you want is to have to compete with some spammer for access to your ISP's mail servers.
Every once in a while, an ISP gets so overloaded with spam that its servers crash. This situation causes everybody who was depending on that server to be inconvenienced. It may also be more than an inconvenience if, for example, your company's e-commerce Web site was on that same server or a bunch of customers' orders were in the e-mails that got scrambled when the server gave up the ghost. Calculating the cost in terms of lost business opportunities when customers can't reach you or think that you're ignoring them is almost impossible, but those are more costs you bear, thanks to spammers.