ListData.svc and Resource Throttling

18 Jul

One of the new features in SharePoint 2010 is Resource Throttling.  It allows SharePoint farm administrators to put hard limits on the number of items that a user can display in a single view.  By default the limit is set to 5,000 items for regular users and 20,000 for administrators (not site collection administrators) and auditors.  You can read more about resource throttling and large lists here.

Although resource throttling is great for farm administrators, it can be a real pain for the SharePoint developer if they are not careful with their design.  When you exceed the 5,000 item limit, SharePoint can throw some really intimidating errors and will even disable your ListView webpart making your list unavailable until you bring the list into compliance.    The following Microsoft video demonstrates exactly what happens when you exceed the list limit and provides some techniques to address large lists.

The other day I noticed that my javascript on a site page was throwing an error in a section that makes a call to the new RESTful web service  ListData.svc.  If you have not used this wonderful web service, I highly recommend it–you can learn more about it here.  What makes listdata.svc so powerful are the field-value pairs you can pass along in a query string.  These field-value pairs allow you to define specific queries that return only the data you are interested in–instead of returning everything and then parsing it yourself.  The query string that I was building was using the $filter command to search for items in my list that met my criteria.  The query looked something like this:

var url = "http://localhost/_vti_bin/ListData.svc/MyList?$filter=ListColInternalName%20eq%20'My Value'";

You can build way more complex and sophisticated queries using other commands such as $select and $orderby and there are plenty of blogs that describe how to do so.  Anyway, I started digging around in the code and tried entering the URL directly into a browser to see what was going on.  I noticed that I was getting HTTP status code 500.  At first I thought it was due to server patches that our network administrators deployed over the weekend–wishful thinking!

The reason the code was failing was that I was trying to filter a  list that had passed the 5,000 limit  set by resource throttling .  I knew I had more than 5,000 items in the list since I was trying different techniques to create a large list without triggering the warnings.  This particular list had 4,999 items and 1 folder with 50 items for a total of 5,050 items and  the XSLT ListView webpart displayed just fine without any warnings and I could filter it appropriately.  Folders is one of the options you have for building lists that exceed the resource throttling limits.   However, ListData.svc does not take into account your folder structure and instead goes by the number of items in your list.  If your list exceeds the resource throttling limits and you try to filter using $filter, the web service will return HTTP status code 500.  Other commands such as $select work fine against the list, returning 1,000 items (the limit set by the web service)  without issue.

Filtering is one of the problems that arises when dealing with large lists.  For the XSLT ListView webpart, SharePoint displays a message “Displaying only the newest results below.”  You may not get back all your data, but at least you get back something.  With ListData.svc, you don’t even get back a meaningful error message.  I guess Microsoft’s SharePoint developers were just too busy to write one!


How to Display Nicely Formatted Code in Your WordPress Blog

8 Jun

Since I will be posting chunks of code in future blogs, I was looking for a way to format it easily in WordPress.  I always liked webpages with the neat little widgets on them that display code nicely formatted.    I found some WordPress code highlighting plug-ins and set out to install the one I liked the most.  Imagine my surprise when I tried to follow the instructions provided by author and found out that blogs from, like the one I am using, are not allowed to use plug-ins1.  This WordPress support page explains the differences between WordPress the hosted blog versus the downloadable, self-hosting solution.

I was a little disappointed because the <code> tag is horrible and a pain to use and I couldn’t imagine posting large blocks of code with that tag.  But as luck would have it, does offer a code highlighter for users of blogs.  I won’t go into the details as you can read more about it here.   I went back to one of my posts and used the tags in the page and viola! nicely formatted code!

Here is a trivial example:

      <p>Nicely formatted code!</p>

There is another article you may come across on that talks about using the code tag, but trust me, you don’t want to use it! The codex article was the first search result that came up for me using Google and the following search terms: code wordpress blog. The article I link above was further down the results page, which is a shame since I think the lower rank article I linked above is what bloggers are looking for rather than the article.

1The VIP Program by allows for plug-ins…for $3,750/month!

Author Field in a DataForm WebPart

7 Jun

The other day I was trying to access the Author field in a DataForm WebPart and noticed that it wasn’t listed in the Data Source Details taskpane.  If you are new to SharePoint, you recognize the Author field as the Created By column in a SharePoint list.  Created By is the display name while Author is the name used internal by SharePoint.  There are several great sites that list the display/common names for built-in SharePoint fields:

Link 1Link 2

If you look towards the bottom of your form, you will notice that SharePoint automatically takes care of displaying the Created/Created By and the Modified/Modified By information for you.

SharePoint automatically adds this information to the bottom of any form created.

SharePoint bundles this information into the following server control:

<SharePoint:CreatedModifiedInfo ControlMode="Display" runat="server"/>

Although convenient for SharePoint, it is not very convenient for the developer.  How else do you display the Created By field or Modified By?  Well, after some searches online, I discovered that this tag can be customized using templates which shouldn’t come as a surprise since it is an offspring of ASP.NET server controls.  It is through this customization that you can access the fields shown above in your DataForm WebPart.  After adding the CreatedModifiedInfo server control in the desired location, you need to add CustomTemplate tags, and add a FormField server control between those tags while setting the FieldName to the field you want using the internal name.  Below is some sample code:

<SharePoint:CreatedModifiedInfo ControlMode="Display" runat="server">
      <SharePoint:FormField FieldName="Author" runat="server" ControlMode="Display" DisableInputFieldLabel="true"/>


If you want to display the Modified field, you would replace Author with Editor since Editor is the internal name for Modified-remember to check out the links above to learn more about internal names.  Or if you want to display both fields, you can just add another FormField control on the page and style it any way you want by adding HTML code between the CustomTemplate tags.  One thing to keep in mind is that ControlMode needs to be set according to the type of form you are creating.  In my example, I am modifying a display form so ControlMode is set to Display.

One final note, you could modify the ASCX for CreatedModifiedInfo server code by modifying the templates stored on the SharePoint server.  Of course this is NOT RECOMMENDED, but if you are feeling courageous or curious, they are found under the following directory:


If anything, you will learn just how the control bundles the information you see by default.

DataForm Webpart, XSLT, and NaN

3 Jun

The other night I was playing around with the dataform webpart, trying out different XSLT code to get a better grasp on it.  Where I work, we are only allowed to use SharePoint Designer so I have had to spend a lot of time learning about the DataView webpart, and the more I learn, the more impressed I am.  Now I am less inclined to jump straight into Visual Studio when developing SharePoint solutions.  Anyway, I was playing around with a inventory cost list based on a spreadsheet I found online and I was trying using the format-number function in XSLT.

The function does exactly what it says, format numbers.  You can read more about the function at the following link.  I used the following code to format currency:

<xsl:value-of select="format-number(@Total,'$#.##')"/>

Everything looked fine, until I noticed that cells containing values over a 1,000 were empty.  I tried entering the function using the Formula button in the SharePoint Designer ribbon and noticed that values over 1,000 were turning up as NaN (or Not a Number).

Values greater than 1,000 are not displayed when using format-number function in XSLT.

Since I imported the spreadsheet into SharePoint (for which I encountered a whole other set of issues I will blog about later) I thought that maybe the type of the column was set to Single line of text and maybe that was causing the issue.  Looking at the List Information  screen, the column was set to Number.  Since the values under a thousand were displaying ok, I wasn’t 100% convinced that was the issue.  I did notice though that the column automatically digit-grouped the numbers, adding a comma to values greater than 1,000.  It then dawned on me that the comma must be causing the function to spit out NaN.  I adjusted my formula to the following:

<xsl:value-of select=”format-number(translate(@Total,’,’,”),’$#,###.00′)” />

Sure enough, values over 1,00o displayed correctly.  Basically the translate function takes the first parameter, searches it for instances of the second parameter, and replaces it with the third parameter.  In my case, it took the Total value and replaced the comma with nothing before processing.  The format-pattern in the format-number function adds back the comma in the proper place.   The other thing you will notice is that I replaced the place-holders after the period from # to 0. When it was set to #, I notice that values that ended in zero were truncated.  For example, 60.00 would be turned to $60, or 49.90 would become $49.9, but 59.09 would display as $59.09.   The # place-holder displays 0 as absent when it starts or ends the number and that is why some numbers were correct while others were truncated.

If you have encountered this issue, share how you solved it below.


2 Jun

Just starting out and trying to figure out how to use WordPress.  I goal with this site is to provide tips and references, etc. for SharePoint as I come across them during the course of my work and to share my ideas and solutions on projects that I am currently working on.  I hope to update this daily, but we’ll see how that goes!

Thanks for stopping by!