8987c3bf9d6f6e80a92ded453f3fd07163881241 — Victor Freire 3 months ago cc6cc99
article: ironing-out-bugs-with-fscheck
2 files changed, 187 insertions(+), 0 deletions(-)

M content-org/content.org
A static/blog/ironing-out-bugs-with-fscheck/main.fsx
M content-org/content.org => content-org/content.org +130 -0
@@ 4561,8 4561,138 @@ services.postgres = {

** Ironing out bugs with FsCheck                             :fsharp:dotnet:
:EXPORT_DATE: 2023-05-29
:EXPORT_FILE_NAME: ironing-out-bugs-with-fscheck
:EXPORT_HUGO_SLUG: ironing-out-bugs-with-fscheck

#+attr_shortcode: :class info
You can read the [[file:/blog/ironing-out-bugs-with-fscheck/main.fsx][source code]] for this article.

Leveraging property-based testing and FsCheck to find bugs.

During this week, I was tasked to write some [[https://fsharpforfunandprofit.com/pbt/][property-based tests]] to a
slightly critical portion of our system that handles financial
transactions. You won't see the exact same tests I wrote at work due
to legal reasons[fn:41]. Instead, a buggy fictional scenario with
similar code structure will be used.

Our codebase is written in [[https://fsharp.org/][F#]], and luckily, there's an amazing library
called [[https://fscheck.github.io/FsCheck/][FsCheck]] that can be used for this purpose. But... What is
property-based testing? According to FsCheck's documentation:

The programmer provides a specification of the program, in the form of
properties which functions, methods or objects should satisfy, and
FsCheck then tests that the properties hold in a large number of
randomly generated cases. While writing the properties, you are
actually writing a testable specification of your program.

In short, property-based testing is about writing an specification of
your code through expected output /characteristics/. [[https://fsharpforfunandprofit.com/][Scott Wlaschin]]
wrote an awesome article about [[https://fsharpforfunandprofit.com/posts/property-based-testing-2/][common properties found on code]].

While breaking the code apart, I noticed that the first testable
property was applicable to the section "Different paths, same
destination". Before starting, load the needed dependencies for our

#+include: "../static/blog/ironing-out-bugs-with-fscheck/main.fsx" src fsharp :lines "0-5"

Picture the following scenario, you have two lands and you have to sum
the area of both. The sum cannot output a negative number. This is
quite easy, isn't it? How hard can it be to introduce a bug?

#+include: "../static/blog/ironing-out-bugs-with-fscheck/main.fsx" src fsharp :lines "6-12"

Executing the code shows us that our implementation is correct
according to the business rule:

#+include: "../static/blog/ironing-out-bugs-with-fscheck/main.fsx" src fsharp :lines "26-31"

But... is it, really? Let's bring FsCheck to the game and write a
simple test for the [[https://en.wikipedia.org/wiki/Associative_property][associative property]]:

#+include: "../static/blog/ironing-out-bugs-with-fscheck/main.fsx" src fsharp :lines "32-40"

A bit weird, isn't it? In F#, you can use backticks around a string to
make it a valid identifier. In this case, ~associativity of land sums~
is just as valid as ~AssociativityOfLandSums~, just easier to read.

This function takes a tuple with two values: ~landOne~ and ~landTwo~.
Thanks to F#'s type inference, there's no need to explicitly declare
their types, as they are being inferred by their usage in the
~sumAreas~ function that only takes arguments of the type ~Land~.

The test consists in getting the output of ~sumAreas~ with reversed
parameters and comparing to see if the argument order changes the
output. Which shouldn't. Let's feed FsCheck this function and see what

#+include: "../static/blog/ironing-out-bugs-with-fscheck/main.fsx" src fsharp :lines "40-41"

Falsifiable, after 1 test (0 shrinks) (StdGen (511870660, 297192323)):
{ Width = -0.0
  Height = -0.0 }
{ Width = nan
  Height = 0.0 }
val it: unit = ()

Bingo! We already found a runtime bug! If one of the values is [[https://learn.microsoft.com/en-us/dotnet/api/system.double.nan?view=net-7.0#remarks][~Nan~]],
our software does not conform to the /specification/[fn:42]. There are
a multitude of ways to solve this. Some examples are: treating this
individually at the ~area~ function or treating both values at the
same time on the ~sumAreas~ function.

Again, thankfully, Scott Wlaschin has also covered this up on
"[[https://fsharpforfunandprofit.com/posts/designing-with-types-single-case-dus/#creating-modules-for-wrapper-types][Creating modules for wrapper types]]". You can imagine it as a
constructor based on the module system, it's really neat! However,
it's impossible to apply this here as signature files are not
available on F# scripts. Here's a quick glance on how the ~create~
function could look like:

#+include: "../static/blog/ironing-out-bugs-with-fscheck/main.fsx" src fsharp :lines "12-26"

Alright, we know how to fix our code but we are unable to implement it
on our script file... Writing a custom [[https://fscheck.github.io/FsCheck/TestData.html#Generators][Generator]] that only spits valid
data is an alternative. Again, there are numerous ways to implement a
generator, and I have opted for the ~gen~ computation expression

#+include: "../static/blog/ironing-out-bugs-with-fscheck/main.fsx" src fsharp :lines "43-51"

What's [[https://fscheck.github.io/FsCheck/reference/fscheck-normalfloat.html][~NormalFloat~]]? The documentation states: "/Represents a float
that is not NaN or Infinity./". This is perfect! Then we pipe the
Generator to our function ~biggerOrEqualToZero~ to make sure we don't
get negative numbers.

Here comes a really good tip. At work we struggled with functions
taking two custom generators, we just couldn't figure it out how to
pass them. Well, we struggled so you don't need to:

#+include: "../static/blog/ironing-out-bugs-with-fscheck/main.fsx" src fsharp :lines "51-55"

Now testing our two properties is trivial:

#+include: "../static/blog/ironing-out-bugs-with-fscheck/main.fsx" src fsharp :lines "56-58"

* Footnotes

[fn:42] Which just expects the sum of the areas. 😁

[fn:41] I don't have a lawyer.

[fn:40] I need to try OpenBSD sometime in the future, by the way.

[fn:39] https://www.bounga.org/tips/2020/05/03/multiple-smtp-accounts-in-gnus-without-external-tools/

A static/blog/ironing-out-bugs-with-fscheck/main.fsx => static/blog/ironing-out-bugs-with-fscheck/main.fsx +57 -0
@@ 0,0 1,57 @@
#r "nuget: FsCheck, 2.16.5"

open FsCheck
open System

type Land = 
    { Width: float; Height: float }

module Land =
    let area value = value.Width * value.Height
    let sumAreas a b = area a + area b

    let create width height =
        let isValid x = 
            if Double.IsNaN(x) || Double.IsInfinity(x) || Double.IsNegative(x)
            then None
            else Some x
        let width = isValid width
        let height = isValid height

        match (width, height) with
        | (Some width, Some height) -> 
            Ok { Width = width; Height = height }
        | _ -> Error "Invalid measurements"

let store = { Width = 10; Height = 45 }
let factory = { Width = 5; Height = 10 }

Land.sumAreas store factory // 500.0

let ``associativity of land sums`` (landOne, landTwo) =
    let a = Land.sumAreas landOne landTwo
    let b = Land.sumAreas landTwo landOne
    b = a

let ``sum of land areas is always positive`` (landOne, landTwo) =
    (Land.sumAreas landOne landTwo) > 0

Check.Quick ``associativity of land sums``
Check.Quick ``sum of land areas is always positive``

let validLandGenerator = 
    gen {
        let biggerOrEqualToZero = Gen.filter (fun (x: NormalFloat) -> x.Get >= 0.0)
        let! width = Arb.generate<NormalFloat> |> biggerOrEqualToZero
        let! height = Arb.generate<NormalFloat> |> biggerOrEqualToZero

        return { Width = width.Get; Height = height.Get }

let twoValidLandsGenerator = 
    Gen.map2 (fun landOne landTwo -> (landOne, landTwo)) validLandGenerator validLandGenerator
    |> Arb.fromGen

Check.Quick (Prop.forAll twoValidLandsGenerator ``associativity of land sums``)
Check.Quick (Prop.forAll twoValidLandsGenerator ``sum of land areas is always positive``)
\ No newline at end of file