How to enumerate large document library

In this post by the size of the library i mean a total size of the documents in the library, not an item count.

It is relevant for cases when you need to enumerate over all documents in the library to process they, but the size of the library greater then an amount of the RAM on the SharePoint machine.

If you will do it using SPListItemCollection or ContentIterator and try to process all items as a single batch then you will get out of memory exception. It is happens because SharePoint OM download all binaries to the worker process (before or during enumeration).

This problem could be solved using content paging. You can split the library content into small pages and process it page by page. Before page processing we should release all resources allocated for previous page. Also, exist approach that rely on the  humanity of the content structure. We can assume that the size of the documents from one folder is not large and can be processed as a single batch. Such processing order also has advantages over simple paging.

Below you can find an C# example of processing:

using Microsoft.Office.Server.Utilities;
using Microsoft.SharePoint;

public static void EnumerateFolder(SPFolder root, Action<SPListItem> processAction, Action<SPListItem, Exception> exceptionAction)
{
  foreach (SPFolder folder in root.SubFolders)
  EnumerateFolder(folder, processAction, exceptionAction);

  var contentIterator = new ContentIterator();
  contentIterator.ProcessFilesInFolder(root, false,
      (file) => { processAction(file.Item);},
      (file, exception) =>
      {
         exceptionAction(file.Item, exception);
         return false;
      });
}

EnumerateFolder method enumerate over all files into provided SPFolder and all subfolders and execute processAction on each one. The last parameter into ProcessFilesInFolder is an error handler that will be executed after each exception from item processing. Line 13 mean that we do not stop document processing after each exception. More details about ProcessFilesInFolder method you can find here.

Below you can find the same F# example.


open Microsoft.SharePoint
open Microsoft.Office.Server.Utilities

let rec enumerate (root:SPFolder) processAction exceptionAction =
  for folder in root.SubFolders do
    enumerate folder processAction exceptionAction
  ContentIterator().ProcessFilesInFolder(root, false,
    (fun file -> processAction(file.Item)),
    (fun file ex -> exceptionAction(file.Item, ex); false));

P.S. To use ContentIterator you should add Microsoft.Office.Server to the project references.

How to change SPListItem Created or Modified date

Another often asked question is a ‘How to change SPListItem Created/Modified date’. Such task occur when you importing documents to the SharePoint  from another source when you want to save authorship and time information.

To update Created and Modified date you can use SPListItem indexer to modify infomation and call Update() to save changes.

To modify CreatedBy and ModifiedBy properties, you can use SPListItem indexer to modify fields with internal names Author and Editor. But you should convert SPUser object into string with following format “{userId};#{userName}”

Example you can find below:

public void ApplyMetadata(SPListItem item, DateTime created, DateTime modified, SPUser createdBy, SPUser modifiedBy)
{
  item["Created"] = created;
  item["Modified"] = modified;

  item["Author"] = GetStringByUser(createdBy);
  item["Editor"] = GetStringByUser(modifiedBy);

  item.Update();
}

private static string GetStringByUser(SPUser user)
{
  return user.ID + ";#" + user.Name;
}

Visual Studio minimap

When I saw a Sublime Text 2 with their features at first time – I loved that. One of my favorite feature is a minimap.  It is a simple but very powerful idea to replace scrollbar with minimized code map. Using this feature very easy navigate directly to where you want.

I wanna this feature for Visual Studio for sure. Fortunately, it is already there. It is a part of the Productivity Power Tools.

First of all we need to setup Productivity Power Tool using Visual Studio Extension Manager.

We need to enable this feature, because it is disabled by default.  Open Tool -> Options -> Productivity Power Tools, enable Enhanced Scroll Bar and restart your Visual Studio.

Choose ‘Full map mode‘ in the Productivity Power Tools->Enchanced Scroll Bar section.

You should see minimaps in your Visual Studio now. Enjoy it!!!

How to integrate F# and Notepad++

People a faced with challenges trying to install F# interactive plugin for Notepad++ (example). I am not an exception, going through all of this I want to share my experience.

  1. First of all download latest version of the Notepad++ and install it (default installation settings). For today it is Notepad++ 6.1.5 (Jul 16 2012)
  2. Download NPPFSIPlugin Version 0.1.1.
  3. Extract dll from achieve and copy it to Notepad++\plugins\ folder. On the my Win7 64bit machine it is c:\Program Files (x86)\Notepad++\plugins\.
  4. Open or re-open you Notepad++ application.
  5. Go to Plugins\F# Interactive\Options menu item.
  6. Specify Binary Path to fsi.exe file.
    For example:
    F# 2.0C:\Program Files (x86)\Microsoft F#\v4.0\fsi.exe
    F# 3.0 – C:\Program Files (x86)\Microsoft SDKs\F#\3.0\Framework\v4.0\fsi.exe
    Before use this paths check that you you have installed appropriate version of F# in that 
  7. Save you changes. Now it should work.
    Alt+T to open F# interactive
    Alt+Enter to to send selected text to F# Interactive

It is also very nice to have a F# syntax highlighting.

  1. Download xml file with user definition language for Notepad++ from here or here.
  2. Rename file to userDefineLang.xml.
  3. Replace  ext=”fs”  to  ext=”fs fsi fsx”  in the file.
  4. Copy this file to %APPDATA%\Notepad++\ folder. (for more details go here )
    Path should be like this : C:\Users\User_Name\AppData\Roaming\Notepad++
  5. Restart Notepad++.

Note: If you already have such file open both. (instructions is copied from here)

  1. Select all of the new file, copy, and paste at the end of the current file
  2. Delete  </NotepadPlus><NotepadPlus> pair in the middle (remove 2 lines)

P.S. Post moved from http://sergey-tihon.blogspot.com/