Google Cloud Vision API from .NET\F# (OAuth2 with ServiceAccount.json)

Google Cloud Platform provides a wide range of APIs, one of which is Cloud Vision API that allows you to detect faces in images, extract sentiments, detect landmark, OCR and etc.

One of available annotators is “Logo Detection” that allows you to find company logo in your image and recognize it.

.NET is not the part of mainstream Google Cloud SDK. Google maintains google-api-dotnet-client that should allow you to authenticate to and call all available services. API design looks slightly not intuitive for .NET world (at least from my point of view).

I spent some time on Google/SO/Github trying to understand how to use OAuth2 in server-to-server authentication scenario with ServiceAccount.json file generated by Google API Manager.

s2s

You cannot use this API without billing account, so you have to put your credit card info, if you want to play with this API.

Also, note that you need to have two NuGet packages Google.Apis.Vision.v1Google.Apis.Oauth2.v2 (and a lot of their dependencies)

So, here is the full sample:

#I __SOURCE_DIRECTORY__
#load "Scripts/load-references-debug.fsx"

open System.IO
open Google.Apis.Auth.OAuth2
open Google.Apis.Services
open Google.Apis.Vision.v1
open Google.Apis.Vision.v1.Data

// OAuth2 authentication using service account JSON file
let credentials =
    let jsonServiceAccount = @"d:\ServiceAccount.json"
    use stream = new FileStream(jsonServiceAccount, 
                         FileMode.Open, FileAccess.Read)
    GoogleCredential.FromStream(stream)
        .CreateScoped(VisionService.Scope.CloudPlatform)

let visionService = // Google Cloud Vision Service
    BaseClientService.Initializer(
        ApplicationName = "my-cool-app",
        HttpClientInitializer = credentials)
    |> VisionService

// Logo detection request for one image
let createRequest content = 
  let wrap (xs:'a list) = System.Collections.Generic.List(xs)
  BatchAnnotateImagesRequest(
    Requests = wrap
      [AnnotateImageRequest(
        Features = wrap [Feature(Type = "LOGO_DETECTION")],
        Image = Image(Content = content))
      ])
  |> visionService.Images.Annotate


let call fileName = // Call and interpret results
    let request =
        File.ReadAllBytes fileName
        |> System.Convert.ToBase64String
        |> createRequest
    let response = request.Execute()

    [ for x in response.Responses do
        for y in x.LogoAnnotations do
          yield y.Description
    ] |> List.toArray


let x = call "D:\\fsharp256.png"
// val x : string [] = [|"F#"|]

Twitter Followers Map with RProvider

Today @oppenheimmd re-tweeted a nice tweet about building Twitter Followers Map with R. Certainly, I decided to build my own map and here it is:

twitterMap

Total number of followers on the screen is smaller than in Twitter. I think it happens because not all people specified the location in the account settings. Take a note that to be able execute this script you need to specify the location in your Twitter account.

As you probably understand from the title, I did this picture using RProvider instead of executing existing R code. Actually, use of twitterMap.R is pretty simple, if not to pay attention to difficulties with Twitter authorization and SSL certificate validation (this part is ugly a bit).

For this demo we need two R packages twitteR and RCurl (with all their dependencies). Please install them:

#I @"..\packages\RProvider.1.0.5\"
#load "RProvider.fsx"

//open RProvider.utils
//R.install_packages("twitteR")
//R.install_packages("RCurl")

open RDotNet
open RProvider
open RProvider.utils
open RProvider.``base``
open RProvider.twitteR
open RProvider.RCurl
open RProvider.ROAuth

I am lazy a bit to fight with RProvider syntax in some places. Actually, I do not even know if it is possible to rewrite such R code using RProvider or not… I have decided to cheat a bit and define a function that gets R expression as a string and evaluates it.

let eval (text:string) =
    namedParams ["text", box text] |> R.parse |> R.eval

You need to have consumerKey and consumerSecret from your registered Twitter application. If you do not have such ones yet, please follow the steps from this article that helps you to register a new Twitter application. The following code is tended to authenticate you in Twitter:

let twitCred =
    namedParams [
        // TODO: insert your consumerKey and consumerSecret
        "consumerKey", box "xxxxxxxxxxxxxxxxxxxxxx"
        "consumerSecret", box "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
        "requestURL", box "https://api.twitter.com/oauth/request_token"
        "accessURL", box "http://api.twitter.com/oauth/access_token"
        "authURL", box "http://api.twitter.com/oauth/authorize" ]
    |> R.OAuthFactory

R.download_file(url = "http://curl.haxx.se/ca/cacert.pem", destfile="cacert.pem")
R.assign("twitCred", twitCred)
eval """twitCred$handshake(cainfo="cacert.pem")"""

Here you need to do some manual work. These are the last authentication steps:

  1. Copy URL from FSI window and paste it in your browser
  2. Allow your twitter app to access your account data
  3. Copy authorization number-code from browser
  4. Paste code in FSI and press Enter

After that, you need to save your authorization data and set SSL certificate to be used globally, which allow twitterMap.R to communicate with Twitter under your account.

R.registerTwitterOAuth(twitCred)
eval """options(RCurlOptions = list(cainfo = system.file("CurlSSL", "cacert.pem", package = "RCurl")))"""

The last step is to run the script and plot your own map:

R.source("http://biostat.jhsph.edu/~jleek/code/twitterMap.R")
// TODO: do not forget to specify your twitter login and increase nMax if you have more than 5000 followers
eval """twitterMap("your_login", fileName="d:\\TwitterMap.pdf", nMax=5000)"""

P.S. If you rewrite this script without eval please post a link in comments 😉

New Twitter API or “F# Weekly” v1.1

Good news for Twitter and no so good for developers:twitter_app

Today(2013-06-11), we(Twitter) are retiring API v1 and fully transitioning to API v1.1.

What does it all mean? This means that all old services are no longer available. Twitter switched to new ones with mandatory OAuth authentication. From now, to work with twitter services we must register new apps and use OAuth.

Also, it means that:

As I know, there are two alternatives available instead of Twitterizer:

  • Tweetsharp (TweetSharp is a fast, clean wrapper around the Twitter API.)
  • LINQ to Twitter (An open source 3rd party LINQ Provider for the Twitter micro-blogging service.)

I have chosen Tweetsharp because its API similar to Twitterizer. This is a new F# Weekly under the hood script:

#r "Newtonsoft.Json.dll"
#r "Hammock.ClientProfile.dll"
#r "TweetSharp.dll"

open TweetSharp
open System
open System.Net
open System.Text.RegularExpressions

let service = new TwitterService(_consumerKey, _consumerSecret)
service.AuthenticateWith(_accessToken, _accessTokenSecret)

let getTweets query =
    let rec collect maxId =
        let options = SearchOptions(Q = query, Count =Nullable(100), MaxId = Nullable(maxId),
                                    Resulttype = Nullable(TwitterSearchResultType.Recent))
        printfn "Loading %s under id %d" query maxId
        let results = service.Search(options).Statuses |> Seq.toList
        printfn "\t Loaded %d tweets" results.Length
        if (results.Length = 0)
            then List.empty
            else
                let lastTweet = results |> List.rev |> List.head
                if (lastTweet.Id < maxId)                     then results |> List.append (collect (lastTweet.Id))
                    else results
    collect (Int64.MaxValue) |> List.rev

let urlRegexp = Regex("http://([\\w+?\\.\\w+])+([a-zA-Z0-9\\~\\!\\@\\#\\$\\%\\^\\&\\*\\(\\)_\\-\\=\\+\\\\\\/\\?\\.\\:\\;\\'\\,]*)?", RegexOptions.IgnoreCase);

let filterUniqLinks (tweets: TwitterStatus list) =
    let hash = new System.Collections.Generic.HashSet();
    tweets |> List.fold
        (fun acc t ->
             let mathces = urlRegexp.Matches(t.Text)
             if (mathces.Count = 0) then acc
             else let urls =
                     [0 .. (mathces.Count-1)]
                     |> List.map (fun i -> mathces.[i].Value)
                     |> List.filter (fun url -> not(hash.Contains(url)))
                  if (List.isEmpty urls) then acc
                  else urls |> List.iter(fun url -> hash.Add(url) |> ignore)
                       t :: acc)
        [] |> List.rev

let tweets =
    ["#fsharp";"#fsharpx";"@dsyme";"#websharper";"@c4fsharp"]
    |> List.map getTweets
    |> List.concat
    |> List.sortBy (fun t -> t.CreatedDate)
    |> filterUniqLinks

let printTweetsInHtml filename (tweets: TwitterStatus list) =
    let formatTweet (text:string) =
        let matches = urlRegexp.Matches(text)
        seq {0 .. (matches.Count-1)}
            |> Seq.fold (
                fun (t:string) i ->
                    let url = matches.[i].Value
                    t.Replace(url, (sprintf "<a href="\&quot;%s\&quot;" target="\&quot;_blank\&quot;">%s</a>" url url)))
                text
    let rows =
      tweets
        |> List.mapi (fun i t ->
            let id = (tweets.Length - i)
            let text = formatTweet(t.Text)
            sprintf "</pre>
<table id="\&quot;%d\&quot;">
<tbody>
<tr>
<td rowspan="\&quot;2\&quot;" width="\&quot;30\&quot;">%d</td>
<td rowspan="\&quot;2\&quot;" width="\&quot;80\&quot;"><a href="\&quot;javascript:remove('%d')\&quot;">Remove</a></td>
<td rowspan="\&quot;2\&quot;"><a href="\&quot;https://twitter.com/%s\&quot;" target="\&quot;_blank\&quot;"><img alt="" src="\&quot;%s\&quot;/" /></a></td>
<td><b>%s</b></td>
</tr>
<tr>
<td>Created : %s</td>
</tr>
</tbody>
</table>
<pre>
"
id id id t.Author.ScreenName t.Author.ProfileImageUrl text (t.CreatedDate.ToString()))
        |> List.fold (fun s r -> s+" "+r) ""
    let html = sprintf "<script type="text/javascript">// <![CDATA[
function remove(id){return (elem=document.getElementById(id)).parentNode.removeChild(elem);}
// ]]></script>%s" rows
 System.IO.File.WriteAllText(filename, html)

printTweetsInHtml "d:\\tweets.html" tweets

LinkedIn OAuth in F#

First of all great thanks to Luke Hoban for his ‘Twitter OAuth in F#‘. His OAuth implementation works fine for LinkedIn as well.

All you need to do is to switch the requestTokenURI/accessTokenURI/authorizeURI addresses from Twitter on LinkedIn.

Additional materials about LinkedIn public API you found at the LinkedIn developers site: https://developer.linkedin.com/

LinkedIn OAuth implementation:

open System
open System.IO
open System.Net
open System.Security.Cryptography
open System.Text

// LinkedIn OAuth Constants
let consumerKey : string = failwith "Must provide the consumerKey for an app registered at https://www.linkedin.com/secure/developer?newapp="
let consumerSecret : string = failwith "Must provide the consumerSecret for an app registered at https://www.linkedin.com/secure/developer?newapp="
let requestTokenURI = "https://api.linkedin.com/uas/oauth/requestToken"
let accessTokenURI = "https://api.linkedin.com/uas/oauth/accessToken"
let authorizeURI = "https://api.linkedin.com/uas/oauth/authorize"

// Utilities
let unreservedChars = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-_.~";
let urlEncode str =
    String.init (String.length str) (fun i ->
        let symbol = str.[i]
        if unreservedChars.IndexOf(symbol) = -1 then
            "%" + String.Format("{0:X2}", int symbol)
        else
            string symbol)

// Core Algorithms
let hmacsha1 signingKey str =
    let converter = new HMACSHA1(Encoding.ASCII.GetBytes(signingKey : string))
    let inBytes = Encoding.ASCII.GetBytes(str : string)
    let outBytes = converter.ComputeHash(inBytes)
    Convert.ToBase64String(outBytes)

let compositeSigningKey consumerSecret tokenSecret =
    urlEncode(consumerSecret) + "&" + urlEncode(tokenSecret)

let baseString httpMethod baseUri queryParameters =
    httpMethod + "&" +
    urlEncode(baseUri) + "&" +
      (queryParameters
       |> Seq.sortBy (fun (k,v) -> k)
       |> Seq.map (fun (k,v) -> urlEncode(k)+"%3D"+urlEncode(v))
       |> String.concat "%26")

let createAuthorizeHeader queryParameters =
    let headerValue =
        "OAuth " +
        (queryParameters
         |> Seq.map (fun (k,v) -> urlEncode(k)+"\x3D\""+urlEncode(v)+"\"")
         |> String.concat ",")
    headerValue

let currentUnixTime() = floor (DateTime.UtcNow - DateTime(1970, 1, 1, 0, 0, 0, 0)).TotalSeconds

/// Request a token from LinkedIn and return:
/// oauth_token, oauth_token_secret, oauth_callback_confirmed
let requestToken() =
    let signingKey = compositeSigningKey consumerSecret ""

    let queryParameters =
        ["oauth_callback", "oob";
         "oauth_consumer_key", consumerKey;
         "oauth_nonce", System.Guid.NewGuid().ToString().Substring(24);
         "oauth_signature_method", "HMAC-SHA1";
         "oauth_timestamp", currentUnixTime().ToString();
         "oauth_version", "1.0"]

    let signingString = baseString "POST" requestTokenURI queryParameters
    let oauth_signature = hmacsha1 signingKey signingString

    let realQueryParameters = ("oauth_signature", oauth_signature)::queryParameters

    let req = WebRequest.Create(requestTokenURI, Method="POST")
    let headerValue = createAuthorizeHeader realQueryParameters
    req.Headers.Add(HttpRequestHeader.Authorization, headerValue)

    let resp = req.GetResponse()
    let stream = resp.GetResponseStream()
    let txt = (new StreamReader(stream)).ReadToEnd()

    let parts = txt.Split('&')
    (parts.[0].Split('=').[1],
     parts.[1].Split('=').[1],
     parts.[2].Split('=').[1] = "true")

/// Get an access token from LinkedIn and returns:
/// oauth_token, oauth_token_secret
let accessToken token tokenSecret verifier =
    let signingKey = compositeSigningKey consumerSecret tokenSecret

    let queryParameters =
        ["oauth_consumer_key", consumerKey;
         "oauth_nonce", System.Guid.NewGuid().ToString().Substring(24);
         "oauth_signature_method", "HMAC-SHA1";
         "oauth_token", token;
         "oauth_timestamp", currentUnixTime().ToString();
         "oauth_verifier", verifier;
         "oauth_version", "1.0"]

    let signingString = baseString "POST" accessTokenURI queryParameters
    let oauth_signature = hmacsha1 signingKey signingString

    let realQueryParameters = ("oauth_signature", oauth_signature)::queryParameters

    let req = WebRequest.Create(accessTokenURI, Method="POST")
    let headerValue = createAuthorizeHeader realQueryParameters
    req.Headers.Add(HttpRequestHeader.Authorization, headerValue)

    let resp = req.GetResponse()
    let stream = resp.GetResponseStream()
    let txt = (new StreamReader(stream)).ReadToEnd()

    let parts = txt.Split('&')
    (parts.[0].Split('=').[1],
     parts.[1].Split('=').[1])

/// Compute the 'Authorization' header for the given request data
let authHeaderAfterAuthenticated url httpMethod token tokenSecret queryParams =
    let signingKey = compositeSigningKey consumerSecret tokenSecret

    let queryParameters =
        ["oauth_consumer_key", consumerKey;
         "oauth_nonce", System.Guid.NewGuid().ToString().Substring(24);
         "oauth_signature_method", "HMAC-SHA1";
         "oauth_token", token;
         "oauth_timestamp", currentUnixTime().ToString();
         "oauth_version", "1.0"]

    let signingQueryParameters =
        List.append queryParameters queryParams

    let signingString = baseString httpMethod url signingQueryParameters
    let oauth_signature = hmacsha1 signingKey signingString
    let realQueryParameters = ("oauth_signature", oauth_signature)::queryParameters
    let headerValue = createAuthorizeHeader realQueryParameters
    headerValue

/// Add an Authorization header to an existing WebRequest
let addAuthHeaderForUser (webRequest : WebRequest) token tokenSecret queryParams =
    let url = webRequest.RequestUri.ToString()
    let httpMethod = webRequest.Method
    let header = authHeaderAfterAuthenticated url httpMethod token tokenSecret queryParams
    webRequest.Headers.Add(HttpRequestHeader.Authorization, header)

type System.Net.WebRequest with
    /// Add an Authorization header to the WebRequest for the provided user authorization tokens and query parameters
    member this.AddOAuthHeader(userToken, userTokenSecret, queryParams) =
        addAuthHeaderForUser this userToken userTokenSecret queryParams

let testing() =
    // Compute URL to send user to to allow our app to connect with their credentials,
    // then open the browser to have them accept
    let oauth_token'', oauth_token_secret'', oauth_callback_confirmed = requestToken()
    let url = authorizeURI + "?oauth_token=" + oauth_token''
    System.Diagnostics.Process.Start("iexplore.exe", url)

    // *******NOTE********:
    // Get the 7 digit number from the web page, pass it to the function below to get oauth_token
    // Sample result if things go okay:
    // val oauth_token_secret' : string = "9e571e13-d054-44e6-956a-415ab3ee6d23"
    // val oauth_token' : string = "044da520-0edc-4083-a061-74e115712b61"
    let oauth_token, oauth_token_secret = accessToken oauth_token'' oauth_token_secret'' ("78846")

    // Test 1: Get your profile details
    let streamSampleUrl2 = "http://api.linkedin.com/v1/people/~"
    let req = WebRequest.Create(streamSampleUrl2)
    req.AddOAuthHeader(oauth_token, oauth_token_secret, [])
    let resp = req.GetResponse()
    let strm = resp.GetResponseStream()
    let text = (new StreamReader(strm)).ReadToEnd()
    text

    // Test 2: Get a connections list
    let streamSampleUrl3 = "http://api.linkedin.com/v1/people/~/connections"
    let req = WebRequest.Create(streamSampleUrl3)
    req.AddOAuthHeader(oauth_token, oauth_token_secret, [])
    let resp = req.GetResponse()
    let strm = resp.GetResponseStream()
    let text = (new StreamReader(strm)).ReadToEnd()
    text