FsShelter: a Storm shell for F#

About a year ago Prolucid adopted Apache Storm as our platform of choice for event stream processing and F# as our language of choice for all of our “cloud” development.

FsStorm was an essential part that let us iterate, scale and deliver quickly, but even from the earliest days it was obvious that the developer experience could be improved. Unfortunately, it meant a complete rewrite of FsStorm:

• FsStorm DSL is a really thin layer on top of Nimbus API model:
  • has explicit IDs when describing components in a topology
  • uses strings in all the names
  • matching of inputs/outputs is not guaranteed
• FsStorm uses Json AST as it’s public API:
  • messages, tuples, configuration
  • serialization mechanism is hard-baked into the API

We’ve worked around some of the problems, usually by writing more code.

It actually makes sense that Storm itself doesn’t care about the type of the tuples/fields. It runs on JVM, which is very much typed…