HashiCorp Vault and TLS Certificate Authentication for .NET Applications (Comprehensive guide)

HashiCorp Vault is a tool for secrets management, encryption as a service, and privileged access management. It is quite popular nowadays, especially if you own your own infrastructure, private cloud or just cannot store your secrets using Key Vault services provided by Azure/AWS/GCP.

I assume that you already have one up and running instance of HashiCorp Vault, otherwise you may install one using official Installing Vault guide.

Why TLS certificate authentication?

Vault supports many Auth Methods. But what if you are still deploying your app on plain old Windows Server VMs or develop SharePoint application (like I am ūüėĚ).

The challenge in this case, that you have to authenticate in Vault in order to get a secret. This means that we need to choose auth method that protects our auth secrets from an  accident IT guys who may login on the VM (or malicious code that may find it on file system)

TLS Certificate Auth is a good solution candidate, because we can install certificate into windows certificate store, protect private key (mark it as not-exportable) and even specify list of service accounts, allowed to use this certificate for authentication.

TLS certificate generation

I will be using ssh command on my macOS for certificate generation and Vault configuration, but you can repeat the same step from Window for sure.

For our needs we will use self-signed certificate. You can generate one using OpenSSL. If you do not have OpenSSL installed, you can install from Homebrew.

brew install openssl

First of all we generate private key (it is highly secured, do not share it)

openssl genrsa 2048 > vault_private.pem

Then we generate public part of the key in .pem format (.pem file will be uploaded to Vault for client validation during authentication)

openssl req -x509 -new -key vault_private.pem -out vault_public.pem -days 365

Answer all questions properly, it will help you identify this certificate in future (I’ve created certificate that is valid for 365 days, but you should follow security standards defined in you company).

vault-cfg_sh_‚ÄĒ_private-key.png

Note: Common Name cannot be empty, otherwise you will not be able to use this certificate to retrieve the secret (Vault returns ‘missing name in alias’ error). Thank you ¬†Vadzim Makarchyk for this note.

The final step is to archive both parts in .pfx format (.pfx file will be deployed into Windows Server certificate store on all machines from where our code should have access to Vault)

openssl pkcs12 -export -in vault_public.pem -inkey vault_private.pem -out vault.pfx

vault-cfg_sh_‚ÄĒ_vault-cfg.png

Remember the password entered during *.pfx creation, you’re gonna need it every time you decide to install it on Windows machine.

Vault configuration

In order to configure HashiCorp Vault we will use Vault CLI interface, that can be installed from Homebrew on macOS.

brew install vault

Vault CLI uses environment variables for configuration. My Vault server is hosted on different machine so I need to provide server Url.

VAULT_ADDR=https://my.server.com:8200

export VAULT_ADDR
I uses Enterprise version of Vault that is used by several teams, that it why I also specify namespace (aka folder for my secrets)
VAULT_NAMESPACE=dev/my-team

export VAULT_NAMESPACE

I am lazy to properly setup certificates for Vault CLI, that is why I skip certificate validation (never repeat it in production ūüėČ)

VAULT_SKIP_VERIFY=true

export VAULT_SKIP_VERIFY

We are almost ready to login. The easiest option is to login using Web UI and then reuse issued token in the terminal. Login using your favorite browser, pass authentication and copy token in buffer.

EPAM_Laptop

vault login s.fJTY5S51oIfXKnBAG3Qq5eWp.9GKyY

That is it! Token is saved into ~/.vault-token and CLI is ready to use!

Key/Value secret engine creation

Vault supports multiple Secret Engines, but for our demo we create simple Key/Value storage for secrets (for example to store logins and passwords)

vault secrets enable -path=kv kv
This command enable key/value engine (V1) and name kv (-path param)
NOTE: The kv secrets engine has two versions: kv and kv-v2. To enable versioned kv secrets engine, pass kv-v2 instead.

Engine is ready, but it is empty – let’s fix it.

vault write kv/my-secret value="s3c(eT"

This command effectively creates¬†my-secret secret inside kv secret engine and store one key/value pair inside value=”s3c(eT”

ACL Policy creation

Secret engine is secured, nobody (except you, admin) has access to secrets. We need to create rules/policy that define what access we want to provide. Create new files policy-file.hcl and put following content inside.

path "kv/*" {
  capabilities = ["read", "list"]
}

This policy allows to read and list all secrets inside kv secret engine. All users with this policy will be able to read secrets from our engine. Read more about policies.

Write this policy to the server (and name it policy-name)

vault policy write policy-name policy-file.hcl

TLS Certificates – Auth Method

The last step is to assign this policy. But we want to assign it to all clients authenticated in Vault using TLS certificate created by us earlier.

Fist of all we need to enable certificate authentication in our namespace

vault auth enable cert

and create certificate auth in Vault (name it app), assign policy-name to it and upload the public part of generated key (vault_public.pem)

vault write auth/cert/certs/app policies=policy-name certificate=@vault_public.pem

That is it! Vault is configured and waiting for first connection.

TLS certificate deployment

TLS certificate allows us to deploy it to certain set of machines that should have access to the Vault and then specify which accounts (on these machines) may use it for authentication.

If you are lucky enough and your deployment is automated you can add one more build step in your deployment process that ensures that certificate is provisioned on all target machines. Octopus Deploy is one of such tools that provides built-in template for certificate provisioning. (BTW, it is free for small teams starting from Sept 2, 2019)

importcert.png

On the screenshot you see the step that imports certificate on all target machines with tag SharePoint (in my case) to LocalMachine certificate store to My/Personal store, mark private-key as not exportable and provide access to private key to 2 service accounts.

If your deployment is not automated, you may script the same steps using PowerShell and run it on all machines.

If you are brave, you can click it even manually! ūüôą

  1. Double click on vault.pfx file and choose LocalMachine store location
    store.png
  2. Click Next, Next and type password used during *.pfx creation and Next again.
  3. Choose Personal certificate store.
    EPAM_Laptop
  4. Click Next, Finish, OK – your certificated in the store!
  5. Execute mmc (Microsoft Managed Console) from start menu.
  6. File -> Add/Remove Snap-in …
    addmmc.png
  7. Certificate, Add, Computer account and click Next & Ok
    certaddacc.png
  8. Find our certificate and click Manage Private Keys…
    prk.png
  9. On this screen you can manage the list of accounts that will be able to  use this certificate for authentication on the current machine.
    acl.png

.NET client application

Vault is ready, machine is ready (service account / current user is allowed to use certificate from the LocalMachine/Personal store). ¬†Few lines of code are separating us from success ūüėä.

I will use VaultSharp NuGet Package. It is more or less up to date, it supports namespaces feature and starting from next release usage of namespaces will become even more intuitive.

VaultSecretProvider find X509 certificate in StoreName.My / StoreLocation.LocalMachine, then create CertAuthMethodInfo using certificate and VaultClient that X-Vault-Namespace header to each request with vaultNamespace name.

Using configured instance of VaultClient we can request our secret from Vault _vaultClient.V1.Secrets.KeyValue.V1.ReadSecretAsync(path, mountPoint) specifying path to the secret and mountPoint (name of secret engine).

We are ready to call and receive secrets

Conclusion

Wow, this became a long read, but I hope it was a good one.

TLS certificate authenctication in Vault is a good option for apps that uses Full .NET Framework and runs inside Windows Server VMs.

Just do not forget renew/replace certificates regularly.

Be better WPF / MvvmLight developer in 2018

It is 2018, the time of .NET Core, x-plat, clouds, microservices, blockchain and shine of JavaScript. But, there are guys, like me, who still maintain and sometimes develop classic  .NET desktop applications for Windows.

I am not a WPF expert, but I spent a couple of days reviewing, testing and fixing one of our desktop apps and I definitely learned a couple of new tips & tricks that worth to share with other non-experts.

Part #1: General Tips

Tip #1.1: Choose right library/framework

It happened that we use MvvmLight. The library is lightweight and already exists for almost 10 years, MVVM pattern is well-known and lets us keep solution code reasonably well-structured.

But this is definitely not the only choice, there are many other different-purpose libraries and frameworks that may suit you better, especially if you do green-field development. So choose carefully:

Tip #1.2: Distribute using Squirrel.Windows

Installers always were¬†hard, the seamless auto-update¬†process is even harder. But, today, we have the solution that works for simple user-oriented applications that don’t do crazy things during installation. This solution is¬†Squirrel.Windows¬†– an installation and update framework for Windows desktop apps, designed for C# apps.

It’s definitely worth to learn it once and use it for all apps that you develop.

Tip #1.3: Think about monitoring

Aggregated analytics from user’s machine is priceless for successful¬†apps. There are plenty amount of data that can help you deliver better apps:

  1. Crash reports
  2. Application version distribution
  3. User’s count / Active user
  4. Performance / Integrations tracking
  5. Custom events / Logs

It is not always possible to collect all kinds of data from user’s machine, but do it if you can. There are a couple of services that may help you, like Application Insights, HockeyApp and others.

Tip #1.4: Use the full power of IDE

Learn tools that MS baked for you and use them

Tip #1.5: Debug Data Binding Issues

When data bindings do not play nice you have a possibility to debug. It is not super intuitive, but there are ways to step into the binding process to better figure out what is actually going on. Check this nice article from Mike Woelmer РHow To Debug Data Binding Issues in WPF 

Part #2: MVVM Light – Code Tips

C# quickly evolves over time, more and more features become available to us. It is not always obvious how to use new async code with an old API.

Tip #2.1: “New” INotifyPropertyChanged syntax

I think almost any WPF developer knows how to implement INotifyPropertyChanged interface

public class MyViewModel : INotifyPropertyChanged
{
    private string _isBusy;
    public event PropertyChangedEventHandler PropertyChanged;

    public MyViewModel() {}

    public string IsBusy
    {
        get { return _isBusy; }
        set
        {
            _isBusy = value;
            OnPropertyChanged("IsBusy");
        }
    }

    protected void OnPropertyChanged(string name)
    {
        if (PropertyChanged == null)
            return;
        PropertyChanged(this,
            new PropertyChangedEventArgs(name))
    }
}

Using MVVM Light you can do way shorter (such syntax probably exists for a while, but I discovered it only recently)

public class MyViewModel : ViewModelBase
{
    public MyViewModel() {}

    private string _isBusy;
    public string IsBusy
    {
        get => _isBusy;
        set { Set(() => IsBusy, ref _isBusy, value) }
    }
}

All property change events will happen under the hood of Set method. Also, Set method returns true when the value changed so you can use it to do additional actions on property change.

private string _isBusy;
public string IsBusy
{
    get => _isBusy;
    set {
        if (Set(() => IsBusy, ref _isBusy, value)) {
        // Do whatever you need on update
        }
    }
}

Update from Chris Jobson:

There are overloads that allow us to omit first argument – propertyExpression. In this case [CallerMemberName] will be used as the property name, so the code will be even shorter. Not bad for 2018 =)

private string _isBusy;
public string IsBusy
{
    get => _isBusy;
    set => Set(ref _isBusy, value);
}

Tip #2.2: Async to Action glue

C# async was designed to be better compatible with old APIs and consume Action or delegate. Also, it is one of the reasons why async void exists in the language, but we should always use async Task in our own code.

Two following casts are valid

Action task = async () => await Task.Yield();
Func task2 = async () => await Task.Yield();

Read “Do async lambdas return Tasks?” to better understand what’s actually going on here. It means that you can pass your async method as Action to RelayCommand.

new RelayCommand(async() => await Download());

TBH, you should use it like this (explanation in the next tip)

new RelayCommand(async() => await Download(), keepTargetAlive:true);

Tip #2.3: Do not use lambdas with RelayCommand

Lambdas as a parameter for RelayCommand is a bad idea unless you know what can go wrong and use the latest version of MvvmLight.

Actually, I have spent almost 2 days of my life to figure out why at some point of time several buttons in our application stopped working, even though all commands defined in the ViewModel are read-only and assigned once in the constructor.

We had simple commands that do some trivial actions on click, so the developer decided to use lambda in command declaration to save space and simplify the code.

new RelayCommand(() => IsBusy = true);

The code looks simple and correct, but RelayCommand under the hood stores only weak reference to the delegate and any GC cycle can recycle local lambda function. So at some point in time (after¬†next cycle of GC)¬†RelayCommand may not find delegate to call and nothing will happen after the click. For a deeper analysis of this behavior, you can read “RelayCommands and WeakFuncs“.

At the time of writing this post, the issues in MvvmLight library were fixed (Using RelayCommand and Messenger (and WeakAction) with closures) and released in version 5.4.1. But fix does not apply by default.

If you really want to use lambdas with RelayCommand & Messenger you should manually set keepTargetAlive:true (false by default), but probably better do not use them at all.

new RelayCommand(() => IsBusy = true, keepTargetAlive:true);

P.S. Worth to mention that Laurent Bugnion has the course on Pluralsight “MVVM Light Toolkit Fundamentals” that provides detailed MVVM Light overview.

ASP.NET MVC with Simple Windows Authorization

A lot of enterprises use Active Directory (AD) to manage user accounts and Security Groups to manage access to resources.

So (I think) that there is a common task when you want to create some internal resource that will provide certain functionality for your team, but you do not want to expose your data outside. We can easily enable Windows authentication, however usually we also need to add an authorization(limit access to certain groups)

The task is simple, but I do not know why it is so hard to find manual for this. Steps are as follows:

  • Enable Windows authentication in web.config
  • Add WindowsTokenRoleProvider¬†that transforms all Security Groups to ASP.NET Roles
  • Configure Authorization rules based on roles
  • Disable anonymous authentication for IIS Express.

Changes in Web.config:

<?xml version="1.0" encoding="utf-8"?>
<configuration>
  ...
  <system.web>
    ...
    <authentication mode="Windows" />
    <authorization>
      <allow roles="DOMAIN\MyTeam" />
      <deny users="*"/>
    </authorization>
    <roleManager cacheRolesInCookie="false" defaultProvider="WindowsProvider" enabled="true">
      <providers>
        <clear />
        <add name="WindowsProvider" type="System.Web.Security.WindowsTokenRoleProvider" applicationName="/" />
      </providers>
    </roleManager>
  </system.web>
  ...
</configuration>

Changes in project file:

<Project ToolsVersion="12.0" DefaultTargets="Build" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
    <Platform Condition=" '$(Platform)' == '' ">AnyCPU</Platform>
    ...
    <TargetFrameworkVersion>v4.6.1</TargetFrameworkVersion>
    <UseIISExpress>true</UseIISExpress>
    <IISExpressSSLPort />
    <IISExpressAnonymousAuthentication>disabled</IISExpressAnonymousAuthentication>
    <IISExpressWindowsAuthentication>enabled</IISExpressWindowsAuthentication>
    <IISExpressUseClassicPipelineMode />
    <UseGlobalApplicationHostFile />
    ...
  </PropertyGroup>
  ...

P.S. You can use security groups to restrict access to Controllers/Views based on the roles (AuthorizeAttribute)

About JavaScript memory leaks. Again.

Worth to read, at least, if you are not JS dev

Anton Gorbikov's Blog

Web applications are getting more and more complex from year to year. Couple of years ago nobody cared about the memory leaks on the web pages (yeah, they were really a set of web pages, but not web applications). Even if you forget to clean up some memory ‚Äď it‚Äôs not expected, that user will spend a lot of time on the same page. There were a lot of navigation links, which removed information about the entire page from memory and loaded new page.

But now you cannot relate on this behavior, because now web sites turned into web applications. User loads one small HTML file, one script file and some other stuff (like CSS and images) and that‚Äôs it. Making requests from browser to servers user can stay on on the ‚Äúpage‚ÄĚ for ages. In the worst scenario you will receive a report from production user: ‚ÄúOh, your application‚Ķ

View original post 2,020 more words

How to restore Visual Studio 2015 after the Update 1 (dependency dance)

Update 12/11/2015: Two issues were opened as a result of investigation “Installing the fsharp power tools before installing adding VS2015 update 1 breaks Roslyn” in Visual F# Power Tools repository and “We read some settings from the devenv.exe.config file — devenv.exe.config is not user mutable” in Visual F# repository.

This post is prepared especially to save F# Advent Calendar in English 2015 schedule.

Short version of the fix posted on Stack Overflow.

Note: Described fix is not permanent, VS may reset your changes in devenv.exe.config file after the new extensions install/update.

Yesterday Microsoft released the first update to Visual Studio 2015¬†that contains some pretty cool features and improvements, but this update has broken “some”¬†machines.

During the first run after the update, you may see errors like this one:

VSexception

or even NullReferenceException when you try to open the list of installed extensions

In order to fix all this stuff you need to check ActivityLog.xml file (c:\Users\{user_name}\AppData\Roaming\Microsoft\VisualStudio\14.0\) and find the exact error message. Most probably, you will see something like this

SetSite failed for package [CSharpPackage][Could not load file or assembly ‘System.Collections.Immutable, Version=1.1.36.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a’ or one of its dependencies. The located assembly’s manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)]:{ at Microsoft.VisualStudio.LanguageServices.Implementation.LanguageService.AbstractPackage`2.Initialize() at Microsoft.VisualStudio.LanguageServices.CSharp.LanguageService.CSharpPackage.Initialize() at Microsoft.VisualStudio.Shell.Package.Microsoft.VisualStudio.Shell.Interop.IVsPackage.SetSite(IServiceProvider sp)}

This error says that bindingRedirect set up incorrectly for VS process. You need to find file devenv.exe.config in c:\Users\{user_name}\AppData\Local\Microsoft\VisualStudio\14.0\ and update it. (Or c:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\, depending of location of your devenv.exe file).

For this particular case, you should find rows that setup redirects for System.Collections.Immutable and change newVersion from 1.1.36.0 to 1.1.37.0. Final config should look like this

<dependentAssembly>
  <assemblyIdentity name="System.Collections.Immutable" publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
  <bindingRedirect oldVersion="1.0.27.0-1.1.65535.65535" newVersion="1.1.37.0"/>
</dependentAssembly>

After this fix, I was able to load my IDE and setup latest version of Azure SDK 2.8.1 that reset my changes in .config file and I needed to fix it once again.

Everything was OK, until I have tried to open my web project. This time it crashed with the following error in ActiveLog.xml:

SetSite failed for package [JavaScriptWebExtensionsPackage][Could not load file or assembly &apos;Microsoft.VisualStudio.ProjectSystem.V14Only, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a&apos; or one of its dependencies. The located assembly&apos;s manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)]:{   at Microsoft.VisualStudio.Html.Package.Utilities.ProjectUtilities.IsImmersiveProject(IVsHierarchy hierarchy)   at Microsoft.VisualStudio.Html.Package.Project.WebProjectServices.IsWebProject(IVsHierarchy hierarchy)   at Microsoft.VisualStudio.Html.Package.Project.WebProjectServices.AdviseProject(IVsHierarchy hierarchy)   at Microsoft.VisualStudio.Html.Package.Project.WebProjectServices.AdviseOpenedProjects(IVsSolution solution)   at Microsoft.VisualStudio.Html.Package.Project.WebProjectServices.HookGlobalEvents()   at Microsoft.VisualStudio.Html.Package.Project.WebProjectServices.Microsoft.VisualStudio.Html.Package.Project.IWebProjectServices.get_OpenedProjects()   at Microsoft.VisualStudio.JavaScript.Web.Extensions.ReferenceAutoSync.ProjectServices.Initialize()   at Microsoft.VisualStudio.JavaScript.Web.Extensions.ReferenceAutoSync.ProjectServices..ctor()   at Microsoft.VisualStudio.JavaScript.Web.Extensions.JavaScriptWebExtensionsPackage.Initialize()   at Microsoft.VisualStudio.Shell.Package.Microsoft.VisualStudio.Shell.Interop.IVsPackage.SetSite(IServiceProvider sp)}

After the search on C:\ drive for Microsoft.VisualStudio.ProjectSystem.V14Only.dll I realized that my machine has version 14.1.0.0 instead of 14.0.0.0. So it looks like we need to add one more redirect in devenv.exe.config.

<dependentAssembly>
  <assemblyIdentity name="Microsoft.VisualStudio.ProjectSystem.V14Only" publicKeyToken="b03f5f7f11d50a3a" culture="neutral"/>
  <bindingRedirect oldVersion="14.0.0.0" newVersion="14.1.0.0"/>
</dependentAssembly>

After these fixes, I have not seen any other errors on my machine (at least for now). I hope that you understand the core idea on how to troubleshoot and fix such errors. So good luck to you and VS team with dependency fighting.

P.S. One more useful advice that can save you time – try to clear Component Model Cache if things start going wrong.

F# Kung Fu #4: Avoid using relative paths in #r directives

Today,¬†Vladimir Makarov¬†faced with quite interesting ‘bug'(very unexpected behavior) of FSI. The initial goal was quite simple – count number of NuGet packages, which have “ASP.NET” in title. As a result, there was created a script¬†that perfectly works in compiled form and crashes in FSI, here it is:

#r "../packages/nuget.core.2.8.2/lib/net40-Client/Nuget.Core.dll"
#r "System.Xml.Linq.dll"

let repository =
    NuGet.PackageRepositoryFactory.Default.CreateRepository
        "https://nuget.org/api/v2"

let aspnet = query {
    for p in repository.GetPackages() do
    where (p.Title.Contains "ASP.NET")
    count
}

When we run this in FSI we got an exception

System.ArgumentException: Incorrect instance type
Parameter name: obj
at Microsoft.FSharp.Quotations.PatternsModule.mkInstanceMethodCall(FSharpExpr obj, MethodInfo minfo, FSharpList`1 args)
at Microsoft.FSharp.Quotations.ExprShapeModule.RebuildShapeCombination(Object shape, FSharpList`1 arguments)
at Microsoft.FSharp.Primitives.Basics.List.map[T,TResult](FSharpFunc`2 mapping, FSharpList`1 x)
at Microsoft.FSharp.Linq.QueryModule.walk@933-1[a](FSharpFunc`2 f, FSharpExpr p)
at Microsoft.FSharp.Linq.QueryModule.EvalNonNestedInner(CanEliminate canElim, FSharpExpr queryProducingSequence)
at Microsoft.FSharp.Linq.QueryModule.EvalNonNestedOuter(CanEliminate canElim, FSharpExpr tm)
at Microsoft.FSharp.Linq.QueryModule.clo@1741-1.Microsoft-FSharp-Linq-ForwardDeclarations-IQueryMethods-Execute[a,b](FSharpExpr`1 )
at <StartupCode$FSI_0002>.$FSI_0002.main@() in D:\Downloads\test.fsx:line 4

When we looked more carefully to FSI output, we saw that:

nuget.core

WAT? FSI lies to us?! First message says that correct version of DLL was referenced, but then¬†FSI loads completely wrong old version installed with ASP.NET. Why? Let’s check what #r actually does…

#r means to reference by dll-path; focusing on name. This means that FSI will use the file name first, looking in the system-wide search path and only then try to use the string after #r as a directory-relative hint

So that means, #r is not reliable way to reference assemblies. You can get into the situation when your script depends on the environment: assemblies in GAC, installed software (like version of ASP.NET) and so on. To avoid this it is better to explicitly specify an assembly search path (#I) and then reference the assembly:

#I "../packages/nuget.core.2.8.2/lib/net40-Client"
#r "Nuget.Core.dll"
#r "System.Xml.Linq.dll"

let repository =
    NuGet.PackageRepositoryFactory.Default.CreateRepository
        "https://nuget.org/api/v2"

let aspnet = query {
    for p in repository.GetPackages() do
    where (p.Title.Contains "ASP.NET")
    count
}

Thanks to Vladimir Makarov for interesting challenge and be careful in your scripts.

F# Kung Fu #3: Exceptions recap.

Define

Usually, you do not need to define custom exceptions during programming in F#. If you do scripting in F#, in the most cases you will be happy with standard .NET exception types and F# built-in helper functions. But when you create a custom library or design complex enterprise software, you will need to use power of .NET exceptions. How you can do it from F#:

// C# style exception (possible, but not recommended)
type MyCsharpException(msg, id:int) =
  inherit System.Exception(msg)
  member this.Id = id

// F# style exceptions
exception MyFsharpSimpleException
exception MyFsharpException of string * int

Note that F# has a special keyword exception for defining “handy” exceptions.

Raise(Throw)

F# provides set of functions (failwith, failwithf, invalidArg, invalidOp, nullArg) that help to raise most common exceptions. They are very convenient especially for F# scripts.

let rnd = System.Random()

let rnd = System.Random()

let raiseException() =
    match rnd.Next(8) with
    | 0 -> failwith "throws a generic System.Exception"
    | 1 -> failwithf "throws a generic Exception with formatted message (%d)" 1
    | 2 -> invalidArg "_" "throws an ArgumentException"
    | 3 -> invalidOp "throws an InvalidOperationException"
    | 4 -> nullArg "throws a NullArgumentException"
    | 5 -> raise <| MyFsharpException("throws a MyFsharpException", 5)
    | 6 -> raise <| MyCsharpException("throws a MyCsharpException", 6)
    | 7 -> assert (2>1)
    | _ -> raise <| System.Exception("Impossible case")

The assert expression is syntactic sugar for System.Diagnostics.Debug.Assert. The assertion is triggered only if the DEBUG compilation symbol is defined.

Try-With/Finally (Catch)

The last step is to catch exceptions and handle them in a proper way.

let catchException() =
    try
        raiseException()
    with
    | Failure(msg) // 'Failure' active pattern catches only Exception objects
     -> printfn "%s" msg
    | MyFsharpException(msg, id)
     -> printfn "Catch F# exceptions using pattern matching"
    | : ? System.ArgumentException as ex
     -> printfn "Invalid argument '%s'" ex.ParamName
    | : ? MyCsharpException | : ? System.InvalidOperationException
     -> printfn "You can handle multiple exceptions at a time"
    | _ as ex
     -> printfn "Log: exception '%s'" (ex.GetType().Name)
        reraise() // re-raise exception

let finaly() =
    try
        catchException()
    finally
        printfn "Now, I am ready for exceptions!"

: ?‘ is a two-symbol operator without space inside.

Note:

  • Failure active pattern catches only System.Exception objects. It is useful to handle exceptions raised by¬†failwith & failwithf functions.
  • Exceptions defined using exception keyword could be handled automatically¬†using pattern matching.
  • F# provides¬†reraise function that helps to raise a current exception one more time. This function can be used only from pattern matching rules of try-with expressions.

If you want to learn more about exceptions, read an amazing¬†The “Expressions and syntax” series from¬†Scott Wlaschin.

F# Kung Fu #2: Custom Numeric Literals.

All of you probably know that F# has a set of predefined numerical literals that allow you to clarify meaning of numbers. For example, number 32 will be interpreted as int by default. If you add ‘L’ suffix – 32L, you will get int64 number. If you add ‘I’ suffix- 32I, you will get¬†System.Numerics.BigInteger number.

But F# has an extensible point here: you can define custom interpretation for suffixes Q, R, Z, I, N, G. The trick is that you need to declare an F# module with a special name and define converters functions. (For example NumericLiteralZ for Z literal).

module NumericLiteralZ =
    let FromZero() = 0
    let FromOne() = 1
    let FromInt32 n =
        let rec sumDigits n acc =
            if (n=0) then acc
            else sumDigits (n/10) (acc+n%10)
        sumDigits n 0
    //let FromInt64 n = ...
    //let FromString s = ...

let x = 11111Z
//val x : int = 5
let y = 123Z
//val y : int = 6

Have fun, but be careful.

Update: Note that you cannot use constants integer literals with suffixes Q, R, Z, I, N, G in pattern matching.

NDepend for F# code or FSharp.Compiler.Service ‘code review’.

ndepend_logoI had the opportunity to play with NDepend and could not keep from trying it on F# code. It is interesting to see how analysis rules that designed for languages like C# apply to functional-first F#.

When I downloaded NDepend from the official site, I expected to get regular msi installer with wizard that passes me through all steps and does everything it needs. But it was not true, I got a zip archive with binaries that I needed to unzip manually and execute some binaries that integrate NDepend with my Visual Studio 2013 and Reflector. Installation guide is simple enough and detailed, I have no problems to follow it step by step and configure everything. Nevertheless, installation process seems rather unusual for today.

When I executed a standalone version of NDepend(VisualNDepend) and opened first project that I found РI got stuck! So much information on my screen, I could not understand where to look. My screen looked like a spaceship control panel and I was afraid to touch it ;). I decided to go back to Getting Started page and watch the introduction video. Luckily, NDepend looks pretty well documented.

Now I needed to choose some F# project to do “a code review”. I wanted to find something cool, large and complicated for clarity. And here was my first problem… The first condition is ‘cool’, but most of open source F# projects are cool. This constrain did not help me to reduce number of choices. The next condition is ‘large’, but we do not have really huge F# projects. Pithiness¬†is one of the main F# advantages (F# helps to dramatically decrease code size and code complexity by design). Even F# Compiler is not so big; it is much smaller than my usual C# project. The last condition is ‘complexity’; here situation is similar to ‘size’. F# really helps to keep complexity¬†at manageable level. Anyway,¬†I needed to choose one…

Finally, I have chosen FSharp.Compiler.Service project for analysis (It is a relatively new project). I have no idea what is inside, so it will be more interesting to explore source code in such unusual(for me) way. It is an extremely cool project, which is probably one step further for F# and a fundamental improvement that will open a lot of new doors. (You can read more about what it for and what it can do on the official F# Compiler Services site). This project must be large enough and complicated because it is a brand new extended version of the powerful compiler.

ndepend_homeX

Joking aside! Let’s go deeper to the code.

ndepend-dependencies

In the dependency graph picture, we see the primary assembly FSharp.Compiler.Service (orange in the picture). This assembly depends on minimal set of assemblies from .NET framework (they are marked blue). Also we see that this GitHub project contains sample projects that are built on top of the compiler service (marked green). So project structure is simple enough and quite clear. The same dependencies we can visualize as a dependency matrix:

ndepend-dependencies-matrix

In this matrix, we see more quantitative data about dependencies. Numbers in the cells show the number of assembly members used by one assembly from another. Using these numbers, we can make some conclusions like “UntypedTree sample uses more functionality from FSharp.Compiler.Service than others” or “FsiExe is only one sample that has Windows Forms user interface”.

NDepend also provides one crazy interesting report –¬†Treemap Metric View. This report is able to build a tree of namespaces where size of each node will depend on number of LOC in this namespace. Such plots can show where all complexity is concentrated. To extract more useful information from this plot, you need to have an understanding of the code.

VisualNDependView

Finally, the most intriguing part, the analysis dashboard:

NDependDashboard

Based on these stats, we see that¬†FSharp.Compiler.Service contains more than 84.000 lines of code, which is really a lot for¬†F# project; the average method complexity is 3 that is pretty nice. Also NDepend found violations of 12 critical rules, let’s see deeper what they are.

NDepend_violatedRules

Unfortunately, NDepend does not support navigation to method declaration for non-C# compiled source code and this fact complicates observion of F# code.

NDependSourceDeclarationTo avoid this error (for methods) you need to open instance of VS with your solution and NDepend navigate you directly where you wish.

Let’s dwell on critical violated rules:

Code Quality:

Methods with too many parameters – critical

This rule is violated when methods contain more than 8 parameters. I am going to agree here with NDepend – F# compiler source code has such sin. I do not know exact reason of it, but it should be reasonably for compiler/parser source code.

Methods too complex – critical

This rule is violated when methods have ILCyclomaticComplexity > 40 and ILNestingDepth > 4. As I see this happens mostly because NDepend does not understand definition of functions inside other functions (That does not supported by C#). Most of the code that violate this rule is pretty readable. Yes, functionality is wrapped into one large method, but inside it is split into small handy readable functions.

Types too big – critical

This rule is violated when types contain more than 500 lines of code. This story mostly not about F# too. F# compiles modules to the .NET classes. You are allowed to have as large modules as you need. Modules are more like namespaces than classes and constrain with 500 LOC is not applicable for them.

Object Oriented Design

Do not hide base class methods

The rule is self-explanatory. But in current case we should not pay attention because these 3 violations happened in source code of ProvidedType where this is a part of magic of type providers.

Architecture and Layering

Avoid namespaces mutually dependent

I am not sure here, but it also looks like issue related to the F# modules. NDepend reasoning about namespace dependencies without regards to that namespaces are divided into modules.

Dead Code

Potentially dead Types, Methods and Fields

Hmm… It really looks like that there are¬†some methods inside the compiler that were implemented but not used inside and not exposed to external world. It is probably a secret weapon of F#, sketches of new coming features.

Visibility

Constructors of abstract classes should be declared as protected or private

This issue is related to uses of F# discriminated unions. F# compiles discriminated unions into class hierarchy, where root abstract class has a default parameter-less constructor with default visibility (that is internal for F#)

Naming Conventions

Note that C# and F# have a different development guides with a bit different naming conventions.

Avoid having different types with same name

Mostly this rule is also violated by F# modules. It is side effect of F# modules compilation.

Exception class name should be suffixed with ‘Exception’

Exception suffix is rarely used in F# because language has a special exception keyword to define F# exceptions.

Interface name should begin with a ‘I’

F# compiles types to interfaces when all members are abstract. Actually, sometimes we forget to mention I.

Conclusion

Finally, NDepend is a really nice tool. It has some barrier of entry that forces you to refer to documentation, but it looks like a very powerful tool in skillful hand. It is absolutely invaluable in C# world when you want to understand what the hell is going on in the code, but also applicable to F# to see the big picture.

NDepend is highly customizable. Default set of code verification rules is targeted to C# source code, but you can modify existing rules and/or create new ones that will be designed to F#.  Hopefully, one day such rules will become available in default distribution and F# will be officially supported by NDepend team.

P.S. I have tried only a basic feature set; you can read more about advanced features in the official documentation.