This is a great demo of R Type Provider presented by Howard Mansell in September 2012.
Look at this, if you missed it as I did. It is really exciting!
Going its own way with F#
This is a great demo of R Type Provider presented by Howard Mansell in September 2012.
Look at this, if you missed it as I did. It is really exciting!
If you are a SharePoint developer, an Enterprise Search developer or an employee of a large corporation with Global Search through private internal infrastructure then you may be interested in search automation. Deployment of FAST Search Server 2010 for SharePoint (F4SP) is out of the current post’s scope (you can follow TechNet F4SP Deployment Guide if you need).
F# 3.0 comes with feature called “type providers” that helps you to simplify your life in daily routine. For the case of WCF, the Wsdl type provider allows us to automate the proxy generation. Here we need to note that, F# 3.0 works only on the .NET 4.0 and later, but SharePoint 2010 server side runs exclusively on the .NET 3.0 64bit. Let’s see how this works together.
Firstly, we create an empty F# Script file.
#r "System.ServiceModel.dll" #r "FSharp.Data.TypeProviders.dll" #r "System.Runtime.Serialization.dll" open System open System.Net open System.Security open System.ServiceModel open Microsoft.FSharp.Data.TypeProviders [<Literal>] let SearchServiceWsdl = "https://SharePoint2010WebAppUrl/_vti_bin/search.asmx?WSDL" type SharePointSearch = Microsoft.FSharp.Data.TypeProviders.WsdlService<SearchServiceWsdl>
At this point, the type provider creates proxy classes in the background. The only thing we need to do is to configure the access security. The following code tested on the two SharePoint 2010 farms with NTLM authentication and HTTP/HTTPS access protocols.
let getSharePointSearchService() = let binding = new BasicHttpBinding() binding.MaxReceivedMessageSize <- 10000000L binding.Security.Transport.ClientCredentialType <- HttpClientCredentialType.Ntlm binding.Security.Mode <- if (SearchServiceWsdl.StartsWith("https")) then BasicHttpSecurityMode.Transport else BasicHttpSecurityMode.TransportCredentialOnly let serviceUrl = SearchServiceWsdl.Remove(SearchServiceWsdl.LastIndexOf('?')) let service = new SharePointSearch.ServiceTypes. QueryServiceSoapClient(binding, EndpointAddress(serviceUrl)) //If server located in another domain then we may authenticate manually //service.ClientCredentials.Windows.ClientCredential // <- (Net.NetworkCredential("User_Name", "Password")) service.ClientCredentials.Windows.AllowedImpersonationLevel <- System.Security.Principal.TokenImpersonationLevel.Delegation; service let searchByQueryXml queryXml = use searchService = getSharePointSearchService() let results = searchService.QueryEx(queryXml) let rows = results.Tables.["RelevantResults"].Rows [for i in 0..rows.Count-1 do yield (rows.[i].ItemArray) |> Array.map (sprintf "%O")]
To query F4SP we use the same web service as for build-in SharePoint 2010 search, but with a bit different query XML. The last thing that we need to do is to build query.You can find query XML syntax on Microsoft.Search.Query Schema, but it is hard enough to work with it using official documentation. There is a very useful CodePlex project called FAST Search for Sharepoint MOSS 2010 Query Tool which provides a user-friendly query builder interface.
FAST has its own query syntax(FQL Syntax) that can be directly used through SharePoint Search Web Service.
let getFQLQueryXml (fqlString:string) = """<QueryPacket Revision="1000"> <Query> <Context> <QueryText language="en-US" type="FQL">{0}</QueryText> </Context> <SupportedFormats Format="urn:Microsoft.Search.Response.Document.Document" /> <ResultProvider>FASTSearch</ResultProvider> <Range> <StartAt>1</StartAt> <Count>5</Count> </Range> <EnableStemming>false</EnableStemming> <EnableSpellCheck>Off</EnableSpellCheck> <IncludeSpecialTermsResults>false</IncludeSpecialTermsResults> <IncludeRelevantResults>true</IncludeRelevantResults> <ImplicitAndBehavior>false</ImplicitAndBehavior> <TrimDuplicates>true</TrimDuplicates> <Properties> <Property name="Url" /> <Property name="Write" /> <Property name="Size" /> </Properties> </Query> </QueryPacket>""" |> (fun queryTemplate -> String.Format(queryTemplate,fqlString)) let fqlQueryResults = """and(string("Functional Programming", annotation_class="user", mode="phrase"), or("fileextension":string("ppt", mode="phrase"), "fileextension":string("pptx", mode="phrase"))) AND filter(and(isdocument:1))""" |> getFQLQueryXml |> searchByQueryXml
FAST also supports native SharePoint Keyword Query Syntax.
let getKeywordQueryXml (keywordString:string) = """<QueryPacket Revision="1000"> <Query> <Context> <QueryText language="en-US" type="STRING">{0}</QueryText> </Context> <SupportedFormats Format="urn:Microsoft.Search.Response.Document.Document" /> <ResultProvider>FASTSearch</ResultProvider> <Range> <StartAt>1</StartAt> <Count>5</Count> </Range> <EnableStemming>false</EnableStemming> <EnableSpellCheck>Off</EnableSpellCheck> <IncludeSpecialTermsResults>false</IncludeSpecialTermsResults> <IncludeRelevantResults>true</IncludeRelevantResults> <ImplicitAndBehavior>false</ImplicitAndBehavior> <TrimDuplicates>true</TrimDuplicates> <Properties> <Property name="Url" /> <Property name="Write" /> <Property name="Size" /> </Properties> </Query> </QueryPacket>""" |> (fun queryTemplate -> String.Format(queryTemplate,keywordString)) let simpleKeywordQueryResults = """"Functional Programming" scope:"Documents" (fileextension:"PPT" OR fileextension:"PPTX")""" |> getKeywordQueryXml |> searchByQueryXml
One of the principal differences between two syntaxes is that Keyword Query needs to be converted into FQL on the SharePoint side. Keyword syntax also supports scope conditions, which will be converted into FQL filters. For example “scope:”Documents”” will be translated into ” filter(and(isdocument:1))” (In the case when Documents scope exists in the SharePoint Query Service Application). Unfortunately, we can not specify SharePoint scope in FQL query.
I have thought about possible directions of FSI development. I have some ideas that I would like to share and discuss.
In general, it would be great to make FSI more than just an execution shell. It can become a really powerful development tool. What exactly we can do:
It will be nice to have an ability to save current FSI state into the file and continue work in the next time.
It could sound crazy, but it would be cool to save FSI session to assembly and provide an easy C#-F# interaction with such assemblies. For example, it might be useful for Machine Learning tasks. You will be able to create, train and tune your model directly from FSI then save it to assembly and use it from your C# application.
Provide an ability to see the difference between current FSI state and default state. For example:
There is a tool that provides a part of this functionality. This is FsEye – a visual object tree inspector for the F# Interactive.
One of the main cons in type providers is impossibility to explore the schema of the provided world. For example, I want to use Freebase type provider, but I do not know the nature of the provided types. I cannot use provided types fully, because I do not know what data they have.
I am pretty much sure that we need to supply a way to explore provided types. This feature for sure should be tightly integrated with F# IDEs (into Object browser for example).
All we use MSDN to understand how things work. We copy type name + function name and paste them to Google, find first MSDN link and go there. Why cannot we teach FSI to do it? We just need to build an easy syntax for that.
P.S. I will be happy to hear your feedback or new ideas. Feel free to post comments here or to contact me in twitter (@sergey_tihon).
‘Type Providers’ is an extremely cool F# feature that was introduced with F 3.0 and shipped with VS 2012 and .NET 4.5. You can find Type providers’ explanation here and details about OData type provider here.
It maybe not a news, but OData Type Provider works pretty well with SharePoint 2010.
SharePoint 2010 has a OData Service. You can find mode details about this service at the MSDN article Query SharePoint Foundation with ADO.NET Data Services
One special thing that distinguishes SharePoint service from the others is that you should be authenticated. Just set your credentials into created data content before using it. You can find full code sample below.
#r "FSharp.Data.TypeProviders.dll" #r "System.Data.Services.Client.dll" open Microsoft.FSharp.Data.TypeProviders open System.Net type sharepoint = ODataService<"http://server_name/_vti_bin/listdata.svc"> let web = sharepoint.GetDataContext() in web.Credentials <- (NetworkCredential("user_name", "password", "domain") :> ICredentials) (web.Documents) |> Seq.iter (fun item -> printfn "%A : %s" item.Modified item.Name)