Some conversations that have been floating around the office related to Outlook performance:
1) If you have a large number of items in any single folder, you may experience performance issues during certain operations in Outlook. The performance issue is especially noticeable when you switch into and out of that folder. Generally, when 10,000 items or more items are in a single folder, these issues can occur.
2) We recommend that you maintain a maximum range of 3,000 to 5,000 items in any of the critical path folders.
But if your outlook is fast enough and you want to give it some more overhead, try Xobni.