Wednesday August 3rd 2016

Edition 007

Building an OData service in F# using Entity Framework and Suave

TamizhV

by Tamizh Vendan, Lead Consultant at Ajira Tech

OData (Open Data Protocol) is an OASIS standard that defines the best practice for building and consuming RESTful APIs. In this article, you are going to learn how to implement an OData service in F# using Entity Framework and Suave. We will be using PostgreSQL as the backend datastore.

Project Setup

We are going to have three projects in this implementation
Suave.OData.CoreClass Library for having Types and OData Combinators for Suave.
Suave.OData.EFClass Library for defining DbContext and Entities.
Suave.OData.WebConsole Application for implementing functions that exposes the OData API using Suave.


src
|--Suave.OData.Core
|--Json.fs //JSON serializers and deserializers
|--OData.fs //OData Combinators for Suave
|--paket.references
|--Suave.OData.EF
|--People.fs // Entity Definitions
|--Db.fs // DbContext
|--paket.references
|--Suave.OData.Web
|--EfCrud.fs // Adapter for EF to use with Suave OData Combinators
|--Program.fs // API Server Bootstrapper
|--paket.references

In addition to these projects, we will be having the following files in the root directory.

A FAKE build script build.fsx that orchestrates the database migration and the build process

// build.fsx
#r "packages/FAKE/tools/FakeLib.dll"
let buildDir = "./build"

Target "Clean" (fun _ -> CleanDirs [buildDir;])

Target "BuildApp" (fun _ ->
  !! "src/**/*.fsproj"
    -- "src/**/*.Tests.fsproj"
    |> MSBuildRelease buildDir "Build"
    |> Log "AppBuild-Output: "
)

"Clean"
  ==> "BuildApp"

RunTargetOrDefault "BuildApp"

A paket.dependencies file specifying the NuGet packages the are required
source https://www.nuget.org/api/v2
nuget FAKE

A build.sh file to take care of downloading Paket, restoring the NuGet packages and invoking the FAKE build script.

 	#!/bin/bash
if [ -f ".paket/paket.exe" ]
then
   echo "skipping paket.bootstrapper"
else
  mono .paket/paket.bootstrapper.exe
fi
exit_code=$?
if [ $exit_code -ne 0 ]; then
  exit $exit_code
fi
mono .paket/paket.exe restore
exit_code=$?
if [ $exit_code -ne 0 ]; then
exit $exit_code
fi
mono packages/FAKE/tools/FAKE.exe $@ --fsiargs -d:MONO build.fsx

Setting Up PostgreSQL DB Migration

Let’s start with creating a new database with the name mydb in your PostgreSQL instance
For doing database migration, instead of picking a .NET tool/library, let’s use a simple & novel approach using Node.js. We will be using the node-db-migrate to do the database migration.
To use this npm package, create and update the following two files in the root directory.

packages.json specifying the npm packages that we needed and a migrate npm command to run the db-migrate up

 	{
  "name": "suave-odata",
  "scripts": {
"migrate": "db-migrate up"
  },
  "dependencies": {
"db-migrate": "^0.10.0-beta.14",
"db-migrate-pg": "^0.1.10"  
   }
}

A database.json to pass the database configuration for the node-db-migrate package.

{
  "dev" : "postgres://tamizhvendan:test@localhost/mydb"
}

Do replace the connection string with yours!
Then create the first migration file to define the schema by running the following command
node node_modules/db-migrate/bin/db-migrate create init
This would create a migrations directory and a file inside it with the name 20160713092223-init.js (the number may be different for you).
Let’s define the schema for the people table in this migration file

// ...
exports.up = function(db, callback) {
  db.createTable('people', {
    id: { type: 'serial', primaryKey: true },
    firstName: 'string',
    lastName: 'string',
    age: 'int',
    email: 'string'
  }, callback);
};

exports.down = function(db, callback) {
  db.dropTable('people', callback);
};

The next step is running the npm run migrate from the FAKE script.

For Windows, FAKE’s NPM Helper uses the Node.Js Nuget packages to execute the npm commands. So, let’s install them

paket add nuget Node.js
paket add nuget Npm.js

For Other Platforms, it assumes a local installation of node and npm. So, You need to download and install them if you don’t have one. To pass the npm installation path, let’s add the following statement in the build.sh file

# build.sh
# ...
export NPM_FILE_PATH=$(which npm)
# ...

Now it’s time to update the build.fsx file to invoke this npm run migrate command during the application build

// build.fsx
// ...
open Fake
open Fake.NpmHelper
// ...
Target "DbMigrate" (fun _ ->
    let npmFilePath =
        environVarOrDefault "NPM_FILE_PATH" defaultNpmParams.NpmFilePath
    Npm (fun p ->
              { p with
                  Command = Install Standard
                  NpmFilePath = npmFilePath
              })
    Npm (fun p ->
              { p with
                  Command = (Run "migrate")
                  NpmFilePath = npmFilePath
              })
)

"Clean"
  ==> "DbMigrate"
  ==> "BuildApp"
// ...

That’s all. If run the file ./build.sh, your database will have the shiny new people table in the database mydb
This Node.js based DB migration approach is my personal preference. I’ve used it here to showcase the NPM integration capability of the FAKE library.

Adding Entity Definition and DbContext

With the database table in place, the next step is to create a mapping Class People and expose it via DbContext. As a first step install the following NuGet packages and references it in the Suave.OData.EF project.
• EntityFramework
• Npgsql.EntityFramework
• Npgsql
• System.ComponentModel.Annotations
and follow it up with defining entities

// People.fs
namespace Suave.OData.EF

open System.ComponentModel.DataAnnotations
open System.ComponentModel.DataAnnotations.Schema

[<AllowNullLiteral>]
type Entity () =
  [<Key>]
  [<Column("id")>]
  member val ID = 0 with get, set

[<AllowNullLiteral>]
[<Table("people",Schema="public")>]
type People () =
  inherit Entity()
  [<Column("firstName")>]
  [<Required>]
  member val FirstName = "" with get, set
  [<Column("lastName")>]
  member val LastName = "" with get, set
  [<Column("email")>]
  [<Required>]
  member val Email = "" with get, set
  [<Column("age")>]
  member val Age = 0 with get, set

Then define the DbContext for the mydb

namespace Suave.OData.EF

open System.Data.Entity

type Db () =
  inherit DbContext("MyDb")

  [<DefaultValue>]
  val mutable people : DbSet<People>
  member public this.People
    with get() = this.people
    and set v = this.people <- v

For more details on defining Entity Framework entities in F# refer this MSDN tutorial
The string MyDb in the DbContext(“MyDb”) is the name of the connection string that we need to add. We also need to add some configurations to use the Npgsql Data Provider for Entity Framework to access the PostgreSQL database.
Let’s add both in the Suave.OData.Web project’s config file

<!-- App.config -->
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <configSections>
  <!-- For more information on Entity Framework configuration,
    visit http://go.microsoft.com/fwlink/?LinkID=237468 -->
  <section name="entityFramework"
    type="System.Data.Entity.Internal.ConfigFile.EntityFrameworkSection,
        EntityFramework, Version=6.0.0.0, Culture=neutral,
        PublicKeyToken=b77a5c561934e089"
    requirePermission="false" />
</configSections>
    <startup>
        <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
    </startup>
    <connectionStrings>
        <add name ="MyDb"
            connectionString="server=localhost;user id=tamizhvendan;password=test;database=mydb"
            providerName="Npgsql"/>
    </connectionStrings>
  <entityFramework>
    <defaultConnectionFactory
      type="System.Data.Entity.Infrastructure.SqlConnectionFactory, EntityFramework" />
    <providers>
      <provider invariantName="Npgsql"
        type="Npgsql.NpgsqlServices, Npgsql.EntityFramework" />
    </providers>
  </entityFramework>
  <system.data>
    <DbProviderFactories>
      <remove invariant="Npgsql" />
      <add name="Npgsql Data Provider" invariant="Npgsql" support="FF"
        description=".Net Framework Data Provider for Postgresql"
        type="Npgsql.NpgsqlFactory, Npgsql" />
    </DbProviderFactories>
  </system.data>
</configuration>

JSON Handling

As we will be using JSON format for the request and response in the OData API, let’s add some combinators to handle it.
Add Suave and Newtonsoft.Json NuGet packages in the Suave.OData.Core and define the combinators

// Json.fs
namespace Suave.OData.Core

open Newtonsoft.Json
open Newtonsoft.Json.Serialization
open System.Text
open Suave.Operators
open Suave

module internal Json =

  let toJsonStr v =
    let jsonSerializerSettings = new JsonSerializerSettings()
    jsonSerializerSettings.ContractResolver
      <- new CamelCasePropertyNamesContractResolver()
    JsonConvert.SerializeObject(v, jsonSerializerSettings)

  let JSON webpartCombinator v =
    toJsonStr v
    |> webpartCombinator
    >=> Writers.setMimeType "application/json; charset=utf-8"

  let fromJson<'a> json =
      JsonConvert.DeserializeObject(json, typeof<'a>) :?> 'a

  let getResourceFromReq<'a> (req : HttpRequest) =
      req.rawForm |> Encoding.UTF8.GetString |> fromJson<'a>

Query By Id – “/people(id)”

Now it’s time to implement the OData API. In this section, we are going to implement the Suave OData Combinator for querying the resource by its id. As a good design practice, we need to have an abstraction between the OData Combinator and the underlying data access mechanism (EF in our case)
Let’s start by defining it in the Suave.OData.Core project.

// OData.fs
namespace Suave.OData.Core

[<AutoOpen>]
module Types =
  type Resource<'a> = {
    Name : string
    FindById : int -> Async<'a option>
  }

Then define the FindById Combinator

// OData.fs
// ...
open Suave
open Suave.Http
open Suave.Successful
open Suave.ServerErrors
open Suave.RequestErrors
open Json
// ...
[<RequireQualifiedAccess>]
module OData =
  // type Webpart = HttpContext -> Async<HttpContext option>
  // ('a -> Async<'b>) -> 'a -> WebPart
  let FindById f id (ctx : HttpContext) = async {
    let! findResult = f id
    match findResult with
    | Some entity ->
      return! JSON OK entity ctx
    | _ -> return! NOT_FOUND "" ctx
  }

And then define the OData.CRUD combinator which uses this

// OData.fs
// ...
open Suave.Filters
open Suave.Operators
// ...
module OData =
  // ...
  // Resource<'a> -> WebPart
  let CRUD resource (ctx : HttpContext) = async {
    let odata =
      let resourceIdPath =
        new PrintfFormat<(int -> string),unit,string,string,int>
          (resourcePath + "(%d)")
      choose [
        GET >=> pathScan resourceIdPath (FindById resource.FindById)
      ]
    return! odata ctx
  }

That’s all it required to create a OData combinator in Suave. Let’s leverage it to expose the Query by id API through the Suave.OData.Web project

// EfCrud.fs
namespace Suave.OData.Web

open System.Data.Entity
open Suave.OData.Core
open Suave.OData.EF

let findEntityById find (id : int) = async {
  try
    let! entity = find id |> Async.AwaitTask
    if isNull entity then
      return None
    else
      return Some(entity)
  with
  | ex ->
    printfn "%A" ex
    return None
}
let resource<'a when 'a : not struct and
              'a : equality and
              'a : null>
              name (dbSet : DbSet<'a>) =
    {
      Name = name
      FindById = findEntityById dbSet.FindAsync
    }

The resource function is kind of a bridge between the Entity Framework and the Resource type that we defined in the Core project. The things inside the <> are generic constraints that are required by the DbSet
Handling the error by printing the exception details and returning the Option type is being used for simplicity. If you want to have a robust error handling, it can be enhanced using Chessie.

The final piece is putting everything together and expose the OData API

// Program.fs
namespace Suave.OData.Web

open Suave.OData.EF
open Suave
open Suave.Web
open System.Data.Entity
open Suave.OData.Core

module Main =
  [<EntryPoint>]
  let main argv =
    let db = new Db()
    let app = resource "people" (db.People) |> OData.CRUD
    startWebServer defaultConfig app
    0

Now, Build and run the Suave.OData.Web project.

curl -X GET "http://localhost:8083/people(1)"

As there is no person added to the people table, you will get a 404 response. Just add a new row in the people table directly to see a 200 response!
To keep things simple, I am ignoring the Service Document and Metadata part of the OData.

Adding a new Resource “POST /people”

The first step in adding a new resource is to validate whether it is correct. In EF, we do the validation typically by using Data Annotations

// OData.fs
// ...
open System.Collections.Generic
// 'a -> (bool * List<ValidationResult>)
let private validate entity =
  let vctx = new ValidationContext(entity)
  let results = new List<ValidationResult>()
  let isValid = Validator.TryValidateObject(entity, vctx, results)
  (isValid, results)
// ...

The next step is extending the Resource type to have the Add function

type Resource<'a> = {
  // ...
  Add : 'a -> Async<'a option>
}

and then adding the Suave combinators

// OData.fs
// ...
module OData =
  let Create add (ctx : HttpContext) = async {
    let entity = getResourceFromReq ctx.request
    let isValid, results = validate entity
    if isValid then
      let! addResult = add entity
      match addResult with
      | Some entity -> return! JSON CREATED entity ctx
      | None -> return! INTERNAL_ERROR "" ctx
    else
      return! JSON BAD_REQUEST results ctx
  }
  // ...
  let CRUD resource (ctx : HttpContext) = async {
    let odata =
      let resourcePath = "/" + resource.Name
      // ...
      choose [
        path resourcePath >=> choose [
          POST >=> Create resource.Add
        ]
        // ...
      ]
    return! odata ctx
  }

To expose it as OData API, we need to update the Add functionality in the Entity Framework Side

// EfCrud.fs
// ...
let addEntity (db : DbContext) add entity = async {
    try
      add entity |> ignore
      let! _ = db.SaveChangesAsync() |> Async.AwaitTask
      return Some(entity)
    with
    | ex ->
      printfn "%A" ex
      return None
  }
// ...
let resource<'a when 'a : not struct and
            'a : equality and
            'a : null> db name (dbSet : DbSet<'a>) =
  {
    // ...
    Add = addEntity db dbSet.Add
  }

We added a new parameter db to the resource function that carries the DbContext
The final step is passing the db argument

// Program.fs
// ...
let app = resource db "people" (db.People) |> OData.CRUD
// ...

Adding a resource is now up and running!

curl -X POST -H "Content-Type: application/json" -d '{
    "firstName": "Tamizhvendan",
    "lastName": "S",
    "email": "tamizhvendan.s@hotmail.com",
    "age": 28
  }' "http://localhost:8083/people"

Deleting a resource “DELETE /poeple(id)”

As we did in the previous section, let’s start with extending the Resource type

// OData.fs
// ...
type Resource<'a> = {
  // ...
  DeleteById : int -> Async<'a option>
}

The FindById combinator is already had what requires for implementing the DeleteById combinator. It takes a higher order function representing an action to be performed. So, for deleting a resource by id we just need to pass a different function!

// OData.fs
// ...
module OData =
  // ...
  // ('a -> Async<'b>) -> 'a -> WebPart
  let DeletById = FindById

  // ...
  let CRUD resource (ctx : HttpContext) = async {
    let odata =
      // ...
      choose [
        // ...
        DELETE >=> pathScan resourceIdPath (DeleteById resource.DeleteById)
      ]
    return! odata ctx
  }

The next step is adding the Delete operation on the Entity Framework side

// EfCrud.fs
// ...
let deleteEntityById (db : DbContext) find remove (id : int) = async {
  try
    let! entity = find id |> Async.AwaitTask
    if isNull entity then
      return None
    else
      remove entity |> ignore
      let! _ = db.SaveChangesAsync() |> Async.AwaitTask
      return Some(entity)
  with
  | ex ->
    printfn "%A" ex
    return None
}

let resource<'a when 'a : not struct and
            'a : equality and
            'a : null> db name (dbSet : DbSet<'a>) =
  {
    // ...
    DeleteById = deleteEntityById db dbSet.FindAsync dbSet.Remove
  }

Now, you can delete a resource by its id

curl -X DELETE "http://localhost:8083/people(5)"

Updating a resource “PUT /people(id)”

As we did for the other requests, we will be starting with updating the Resource type

type Resource<'a> = {
  // ...
  UpdateById : int -> 'a -> Async<'a option>
}

Then we need to define the combinator

// OData.fs
// ...
module OData =
  let UpdateById find update id (ctx : HttpContext) = async {
    let entity = getResourceFromReq ctx.request
    let isValid, results = validate entity
    if isValid then
      let! findResult = find id
      match findResult with
      | Some _ ->
        let! updateResult = update id entity
        match updateResult with
        | Some entity -> return! JSON OK entity ctx
        | _ -> return! INTERNAL_ERROR "" ctx
      | _ -> return! NOT_FOUND "" ctx
    else
      return! JSON BAD_REQUEST results ctx
  }
  // ...
  let CRUD resource (ctx : HttpContext) = async {
    let odata =
      // ...
      choose [
        // ...
        PUT >=>
          pathScan resourceIdPath
            (UpdateById resource.FindById resource.UpdateById)
      ]
    return! odata ctx
  }

The last step is updating the EfCrud.fs to handle the update request

// ...
open System.Data.Entity.Migrations

let updateEntity(db : DbContext) update id entity = async {
    try
      update id entity
      let! _ = db.SaveChangesAsync() |> Async.AwaitTask
      return Some entity
    with
    | ex ->
      printf "%A" ex
      return None
  }

let resource<'a when 'a : not struct and
        'a : equality and
        'a : null and 'a :> Entity> db name (dbSet : DbSet<'a>) =

    let update id (entity : 'a) =
      entity.ID <- id
      dbSet.AddOrUpdate entity

    {
      // ...
      UpdateById = updateEntity db update
    }

The ‘a :> Entity constraint has been added to update the ID property of the entity being updated.

curl -X PUT -H "Content-Type: application/json" -d '{
    "firstName" : "Tamizhvendan",
    "lastName" : "S",
    "email" : "tamizh88@gmail.com",
    "age" : 27
}' "http://localhost:8083/people(10)"

Querying OData Endpoint

The interesting aspect of OData is its flexibility to query the data.
To implement it, we need to parse the query strings in the URL into a query expression(LINQ) and then query the model set using the expression.
In this example implementation, we will be using the Linq2Rest library which exactly does what we required.

// OData.fs
// ...
open Linq2Rest.Parser
open System.Collections.Specialized

type Resource<'a> = {
  // ...
  Entities : IEnumerable<'a>
}
// ...
module OData =
  // ...
  let Filter dbSet (ctx : HttpContext) = async {
    let nv = new NameValueCollection()
    ctx.request.query
    |> List.filter (fun (k,v) -> k <> "" && Option.isSome v)
    |> List.map(fun (k,v) -> (k,v.Value))
    |> List.iter (fun (k,v) -> nv.Add(k,v))
    let parser = new ParameterParser<'a>()
    let filteredEntities = parser.Parse(nv).Filter(dbSet)
    return! JSON OK filteredEntities ctx
  }
  // ...
  let CRUD resource (ctx : HttpContext) = async {
    let odata =
      // ...
      choose [
        path resourcePath >=> choose [
          GET >=> Filter resource.Entities
          // ...
        ]
        // ...
      ]
    return! odata ctx
  }

Then update the EfCrud adapter

let resource<'a when 'a : not struct and
        'a : equality and
        'a : null and 'a :> Entity> db name (dbSet : DbSet<'a>) =

    // ...
    {
      // ...
      Entities = dbSet
    }
That's it!
curl -X GET 'http://localhost:8083/people?$select=Email'
curl -X GET 'http://localhost:8083/people?$select=Age'
curl -X GET 'http://localhost:8083/people?$filter=Age%20gt%2030'

Filter request is a blocking call as Linq2Rest doesn’t support async at this point of writing.

Summary

As the objective of this article is just to showcase a basic implementation of OData service in Suave, we haven’t covered much ground here. Some of the improvements include:
• Adding OData Metadata Support
• Replacing Linq2Rest with FParsec and making OData queries async
• Replacing Entity Framework with SQLProvider
In a nutshell, if you would like to extend Suave just add a new combinator! The complete source code is available in my GitHub repository

Book Promo

Liked what you read here and interested in learning more about Suave? Do check out Tamizhvendan’s new book F# Applied, Foreword by Don Syme, Henrik Feldt and Ademar Gonzalez.

“F# Applied” is an excellent introduction to applied, modern programming for the web. Starting with Suave, the F# server-side web framework, this book will teach you how to create complete applications using Functional-First Programming with F# In this book you will read:
• How to create complete application using Functional Programming Principles using F#
• An in-depth understanding of Web development in F# using Suave
• How to develop applications using EventSourcing, CQRS, and DDD in F#
• How to set up continuous integration and continuous deployment using FAKE and Docker
• How to leverage libraries like Rx, FSharp.Data and Paket

Author Bio

Tamizh is a Pragmatic, Passionate and Polyglot Programmer. He started his programming journey at the age of 12, and the passion has not left him since then.
He is a Full-Stack solution provider and has a wealth of experience in solving complex problems using different programming languages and technologies. F#, Node.js, Golang, Haskell are some of his favorites.
Tamizh is also a functional programming evangelist and blogs at P3 Programmer

Share This Article:

Would you like to learn more about F# and Suave? Check out our F# web apps course here!

The F# Gazette

Relax and enjoy a curated selection of the best F# content straight to your inbox.

Including:
  • F# news round-up
  • Interviews with industry professionals
  • Guest articles from F# experts
  • Latest F# jobs
  • F# tools, tips, tutorials and more.