FAKE Final Targets

Today I discovered a new (for me) FAKE feature called FinalTarget. I worked on integration with our new product ReportPortal (in beta right now) that collects and visualizes test results and faced a problem …

The integration process is the following:

  1. Start a new launch (test session) on the server
  2. Execute all tests: unit & integration tests (in my scenario).
  3. Close a launch on the server

As you see, I need to close a launch even in the case when tests failed. So I cannot stop an execution of build script directly on the failure. Unexpectedly, but FAKE provides an elegant solution for this scenario – FinalTarget (target that will be executed in any case if you activate it).

FAKE dependencies look like this in my script:

"Clean"
 ==> "RestorePackages"
 ==> "Build"
 =?> ("RP_StartNewLaunch", not <| hasBuildParam "skipTests")
 =?> ("RunUnitTests", not <| hasBuildParam "skipTests")
 =?> ("RunIntegrationTests", hasBuildParam "allTests")
 =?> ("RP_FinishLaunch", not <| hasBuildParam "skipTests")
 ==> "All"

Targets look like this:

Target "RP_StartNewLaunch" (fun _ ->
    ...
    ActivateFinalTarget "RP_FinishLaunch"
)

FinalTarget "RP_FinishLaunch" (fun _ ->
    ...
)

Target "RunUnitTests" (fun _ ->
    ...
)

Target "RunIntegrationTests" (fun _ ->
    ...
)

As you see, I defined FinalTarget instead of usual Target for “RP_FinishLaunch” and activated it on start of a new launch.

FAKE is awesome 😉

Twitter Pulse #fsharp 2014

Past two years, at the beginning of the year, I did a post where I have tried to sum up year results and find some interesting facts/stats from tweets:

This year was really awesome, a lot of things happened:

There are so many things occurred, so I need your help to find everything that happened or changed this year.

Especially for this, I prepared a data set with tweets starting from Jan 1, 2013 that is available here, and ask you to help me analyze it.

How to start:

  1. Download  the data set & unzip an archive.
  2. Download latest version of Fsharp.Data.
  3. Copy-paste following code snippet:
#r @"..\packages\FSharp.Data.2.1.1\lib\net40\FSharp.Data.dll"
open FSharp.Data

type Tweets = CsvProvider<"fsharp_2013-2014.csv">
let tweets = Tweets.GetSample()

//TODO: Your awesome analytics here

My sample analysis:

As an example of what you can do with data, I prepared a calculation of people activity that shows who had more tweets this year.

tweets.Rows
|> Seq.filter (fun x -> x.CreatedDate.Year = 2014)
|> Seq.groupBy (fun x -> x.FromUserScreenName)
|> Seq.map (fun (group, items) ->
    (group, Seq.length items))
|> Seq.sortBy (fun (_, cnt) -> -cnt)
|> Seq.take 100
|> Seq.iter (fun (group, cnt) ->
    printfn "%s:%d" group cnt)

When you execute these rows you will get a statistical output in FSI with user names and number of tweets. You can copy this output to wordle.net and play with settings to visualize it in the nice way:
2015-01-02_1007

Please help me to observe data set and share your results with me on Twitter (@sergey-tihon). I will include your plots/charts in the end of this post. Thank you!

Deploying .NET code instantly using Vagrant

The absolutely awesome post !!!

Eirik Tsarpalis's avatarEirik Tsarpalis' blog

This is post #30 of the English 2014 F# advent calendar. Thanks to Sergey Tihon for inviting me and suggesting the topic. Be sure to check out all the other awesome posts!

In this post I will be describing Vagrant, a dependency management library of mine. I will try to walk through some of the subtleties in the .NET framework that drove the library implementation, in an attempt to make it more accessible. All code and examples presented can be found in this repository.

Prelude

It is often claimed in functional programming that functions are treated as values. This is a valid assumption in simple applications, permitting powerful patterns when writing code. But is this a general truth, or simply a linguistic abstraction? If functions really are values, one would expect that they exhibit all properties normally expected of values. For instance, it should be possible to serialise functions…

View original post 1,027 more words

Review: F# Deep Dives

F# Deep Dive

Discount code: fshserb (39% off the eBook)

Some days ago, Manning published the final version of a new F# book: “F# Deep Dive“.  I have read MEAP version of the book, and even had a chance to review the final one.

Book is written by the team of famous F# experts such as: Chris Ballard, Keith Battocchi, Colin Bull, Chao-Jen Chen, Yan Cui, Johann Deneux, Kit Eason, Evelina Gabasova, Dmitry Morozov, Don Syme and composed by Tomas Petricek and Phillip Trelford.

First of all, I want to mention that it is not a usual book about F# or another programming language. If you are novice to F# and looking for a guide of language constructs – this book is not the best choice for you. BUT,  if you are already familiar with F# basic syntax constructs, can read simple F# code samples and want to see the language in action – this book is for you.

Book is organized as a collection of independent packages written by different authors. Each author tells his own story about F# application to a practical domain, shows how it was used, how it affected an application design and describes why it was done in such a way. Experts show an application of F# to financial and statistical models, internals of Fsharp.Formatting, numerical computing, social networks analysis, real-world usage of type providers, developing MVC application in idiomatic F#, power of F# Async in action, game development, social web apps in F# and many other interesting topics.

One problem with this book, at least for me, that it is difficult to read it cover to cover. Chapters are written by different authors in a different language, sometimes it was hard for me to catch a sense of computation in a completely new domain. So, for all future readers, I recommend to reorganize chapters according to interests and read parts that most valuable for you first.

Finally, I want to say that the book is really good. Everyone will find interesting topics and a new experience. You will get a broader vision of F# application in a real life. You will see how a functional approach can change and simplify the architecture of real life apps. If you are looking for what to read next – take a look at “F# Deep Dives“, this book is really deserves it.

If you want more F# books – visit fsharp.org.

Distributing the F# Mailbox Processor

Everyone should read it

Isaac Abraham's avatarThe Cockney Coder

Note: This blog post is part of the 2014 F# Advent Calendar. Be sure to check out yesterday’s Intro to Data Science post by Jon Wood!

Mailbox Processors 101

If you’ve been using F# for any reasonable length of time, you’ll have come across the MailboxProcessor, AKA the F# Agent (or Actor). Mailbox Processors are cool. They give us the ability to offload load to background processors without worrying about managing the thread that they live on (as agents silently “go to sleep” when they aren’t processing anything), and they take away the pain of locking as they ensure that only one message will be processed at a time whilst automatically queuing up backed up messages. They also allow us to visualise problems differently to how we might do so when just using a raw Task, in terms of message passing. We can partition data based by pushing them…

View original post 2,583 more words

NuGet dependency visualizer with F# and Graphviz

Script for this article is available as public Gist.

For a long time I was interested in what is going on on NuGet. I think that NuGet UI does not provide one important piece of information – which packages depend on current packages. This information is very useful for package authors and also can help a user to find packages that provide more sophisticated implementation.

Other interesting thing is to see the big picture and answer some global questions:

  • What is inside the technology I use? What is the dependency of packages that I use? What are dependencies of dependencies and so on.
  • What is built on top of packages that I maintain? What beautiful applications of my ideas other can find? What is the cost of releasing a broken package? Who can be harmed?

I believe that there are a lot of other answers we can find from the high view.

Some time ago I have found that NuGet team provides NuGet.Core package that has all  API required to communicate with NuGet. The API is not really fast if you are going to download information about all versions of all NuGet packages 😉 But NuGet team is working on a new v3 API that is going to be much faster than current v2. For current research I have downloaded info about all packages and all their versions to my FSI session to be able to run different types of analysis and create visualizations without further communication with NuGet. This operation is slow enough: it took about 1 hour last time when I run it, but it really depends on NuGet workload and your internet connection.

The second thing is visualization of the result, here I want to say thank you to Scott Wlaschin for his great script type-dependency-graph.fsx (that was built for ‘Cycles and modularity in the wild‘ analysis). I took his GraphViz module and slightly modified it to allow colorful graphs printing. To use it you need to download GraphViz from the official site.

That is all tooling that we need. So we are ready to describe a structure of an analysis – script extracts 3 subsets of NuGet packages and visualizes them in different colors with all dependencies.

  1. Packages that are in scope of analysis (green on the graphs)
  2. Packages that we depend on (grey on the graph) – Package is in the set 2 if exists a dependency path from any package from Set 1 to this package.
  3. Packages that depend on us (blue on the graph) – Package is in the set 3 if exists a dependency path from this package to any package from Set 1.

NOTE: In this analysis I ignored package version and considered only latest package version and dependencies of latest version. If you need more accurate analysis you should adjust script a bit.

Running this script I did some interesting observations:

FsPickler is already highly used by other packages!
FsPickler is already highly used by other packages!
FSharp.Compiler.Service is already deeply incorporated in tooling!
FSharp.Compiler.Service is already deeply incorporated in tooling!
Too many different Fsharp.Core packages of NuGet adopted by different tools.
Too many different Fsharp.Core packages on NuGet adopted by different tools.
FSharpx is still alive! ;)
FSharpx is still alive! 😉
Roslyn.dot
Roslyn ohhhh Roslyn

 

F# Ecosystem is HUGE! (see full svg version here, surry bug without FunScipt ;))
F# Ecosystem is HUGE! (see full svg version here, sorry but without FunScipt ;))

I hope that you’ll also find this script useful and discover something interesting on NuGet.

 

Lightweight websites with F#

Isaac Abraham's avatarThe Cockney Coder

There are several common approaches I’ve seen people take on the .NET platform when writing web-based applications that I want to review in terms of language and framework choice: –

  • Adopt a conventional MVC application approach. Write static HTML that is emitted from the server using e.g. Razor markup + C# / VB .NET, write your controllers and any back-end logic in C#.
  • As above, but replace your back-end logic with F#. This is a reasonable first step to take, because essentially all your data access “back-end” processing are performed in a language that it’s best suited for, whilst your C# is relegated to essentially thin controllers and some simple markup logic.
  • Adopt a “SPA”-style approach. But this I mean split your web application into two distinct applications – a client-side application that is self-managing, typically using Javascript and some framework like Knockout or AngularJS; meanwhile your back-end is a…

View original post 1,115 more words

F# Kung Fu #4: Avoid using relative paths in #r directives

Today, Vladimir Makarov faced with quite interesting ‘bug'(very unexpected behavior) of FSI. The initial goal was quite simple – count number of NuGet packages, which have “ASP.NET” in title. As a result, there was created a script that perfectly works in compiled form and crashes in FSI, here it is:

#r "../packages/nuget.core.2.8.2/lib/net40-Client/Nuget.Core.dll"
#r "System.Xml.Linq.dll"

let repository =
    NuGet.PackageRepositoryFactory.Default.CreateRepository
        "https://nuget.org/api/v2"

let aspnet = query {
    for p in repository.GetPackages() do
    where (p.Title.Contains "ASP.NET")
    count
}

When we run this in FSI we got an exception

System.ArgumentException: Incorrect instance type
Parameter name: obj
at Microsoft.FSharp.Quotations.PatternsModule.mkInstanceMethodCall(FSharpExpr obj, MethodInfo minfo, FSharpList`1 args)
at Microsoft.FSharp.Quotations.ExprShapeModule.RebuildShapeCombination(Object shape, FSharpList`1 arguments)
at Microsoft.FSharp.Primitives.Basics.List.map[T,TResult](FSharpFunc`2 mapping, FSharpList`1 x)
at Microsoft.FSharp.Linq.QueryModule.walk@933-1[a](FSharpFunc`2 f, FSharpExpr p)
at Microsoft.FSharp.Linq.QueryModule.EvalNonNestedInner(CanEliminate canElim, FSharpExpr queryProducingSequence)
at Microsoft.FSharp.Linq.QueryModule.EvalNonNestedOuter(CanEliminate canElim, FSharpExpr tm)
at Microsoft.FSharp.Linq.QueryModule.clo@1741-1.Microsoft-FSharp-Linq-ForwardDeclarations-IQueryMethods-Execute[a,b](FSharpExpr`1 )
at <StartupCode$FSI_0002>.$FSI_0002.main@() in D:\Downloads\test.fsx:line 4

When we looked more carefully to FSI output, we saw that:

nuget.core

WAT? FSI lies to us?! First message says that correct version of DLL was referenced, but then FSI loads completely wrong old version installed with ASP.NET. Why? Let’s check what #r actually does…

#r means to reference by dll-path; focusing on name. This means that FSI will use the file name first, looking in the system-wide search path and only then try to use the string after #r as a directory-relative hint

So that means, #r is not reliable way to reference assemblies. You can get into the situation when your script depends on the environment: assemblies in GAC, installed software (like version of ASP.NET) and so on. To avoid this it is better to explicitly specify an assembly search path (#I) and then reference the assembly:

#I "../packages/nuget.core.2.8.2/lib/net40-Client"
#r "Nuget.Core.dll"
#r "System.Xml.Linq.dll"

let repository =
    NuGet.PackageRepositoryFactory.Default.CreateRepository
        "https://nuget.org/api/v2"

let aspnet = query {
    for p in repository.GetPackages() do
    where (p.Title.Contains "ASP.NET")
    count
}

Thanks to Vladimir Makarov for interesting challenge and be careful in your scripts.