One of the new features in SharePoint 2010 is Resource Throttling. It allows SharePoint farm administrators to put hard limits on the number of items that a user can display in a single view. By default the limit is set to 5,000 items for regular users and 20,000 for administrators (not site collection administrators) and auditors. You can read more about resource throttling and large lists here.
Although resource throttling is great for farm administrators, it can be a real pain for the SharePoint developer if they are not careful with their design. When you exceed the 5,000 item limit, SharePoint can throw some really intimidating errors and will even disable your ListView webpart making your list unavailable until you bring the list into compliance. The following Microsoft video demonstrates exactly what happens when you exceed the list limit and provides some techniques to address large lists.
The other day I noticed that my javascript on a site page was throwing an error in a section that makes a call to the new RESTful web service ListData.svc. If you have not used this wonderful web service, I highly recommend it–you can learn more about it here. What makes listdata.svc so powerful are the field-value pairs you can pass along in a query string. These field-value pairs allow you to define specific queries that return only the data you are interested in–instead of returning everything and then parsing it yourself. The query string that I was building was using the $filter command to search for items in my list that met my criteria. The query looked something like this:
var url = "http://localhost/_vti_bin/ListData.svc/MyList?$filter=ListColInternalName%20eq%20'My Value'";
You can build way more complex and sophisticated queries using other commands such as $select and $orderby and there are plenty of blogs that describe how to do so. Anyway, I started digging around in the code and tried entering the URL directly into a browser to see what was going on. I noticed that I was getting HTTP status code 500. At first I thought it was due to server patches that our network administrators deployed over the weekend–wishful thinking!
The reason the code was failing was that I was trying to filter a list that had passed the 5,000 limit set by resource throttling . I knew I had more than 5,000 items in the list since I was trying different techniques to create a large list without triggering the warnings. This particular list had 4,999 items and 1 folder with 50 items for a total of 5,050 items and the XSLT ListView webpart displayed just fine without any warnings and I could filter it appropriately. Folders is one of the options you have for building lists that exceed the resource throttling limits. However, ListData.svc does not take into account your folder structure and instead goes by the number of items in your list. If your list exceeds the resource throttling limits and you try to filter using $filter, the web service will return HTTP status code 500. Other commands such as $select work fine against the list, returning 1,000 items (the limit set by the web service) without issue.
Filtering is one of the problems that arises when dealing with large lists. For the XSLT ListView webpart, SharePoint displays a message “Displaying only the newest results below.” You may not get back all your data, but at least you get back something. With ListData.svc, you don’t even get back a meaningful error message. I guess Microsoft’s SharePoint developers were just too busy to write one!
sharingpointers 
One Response to “ListData.svc and Resource Throttling”