Playing with thread pools

In the project I've been working on I recently came across a situation where the service was hammered by updates and it started struggling under the load of simultaneous client (predominantly read) requests. The solution was maintain separate thread pools for updates and client requests. The service needs to be available, but we can accept delays in updates. This inspired me to play around with thread pools - which for some reason I've never done before - in a small demonstration.

If you run CommonThreadPoolExperiment you'll see that the number of completed read processes converges to that of the write processes. On my machine, around 10 seconds.

With SeparateThreadPoolsExperiment the numbers are starkly different, the reads are unperturbed by the much slower writes.

Imitating Scala Futures with Go channels (in Go and Clojure)

Scala's Future abstraction is one of my favourite feature of the language. It's simple, elegant and powerful. I make the beginner's attempt to mimic them in Go in a simplistic domain,
Let's assume we have two functions which will take some time (or http calls that naturally return Futures) and we need to aggregate their results.

The Scala way


The Go way


It's rather clunky compared to Scala's solution. But the main problem is that it's not polymorphic on the types. The same lump of code would need to be redefined every time the function signatures change.

Here is a more generic version below that uses explicit type casting.



Not the nicest code I've ever written. If only Go had generics... Any Golang expert out there who could suggest improvements?

The Clojure Way


Clojure borrowed its go-routine and channels idea from Golang. Yet the difference between the Clojure and Go versions in terms of brevity is striking. It's the result of 3 independent factors. One, Clojure is dynamically typed, hence there is no need to hassle with neither function signatures nor generics, or rather the lack of them. Two, it's a functional language. Maps and list comprehensions are way more terse than Go's for-loops. The third factor is async/go returns a channel containing the result of the function executed. Go needs to create a slice of channels first, loop through the slice of functions, create anonymous functions in go blocks where the function is executed and its result is put on the channel, ....lot's of hassle.

With generics many of Go's problems would go away. Rob Pike explicitly said, no generics, but hopefully someone will just force him or do it himself instead.

Solving problems in Scala, Clojure and Go - Template Method Design Pattern

I've planning for ages to write a series of posts about comparing OO and FP solutions for common problems. In a similar vein comparing Scala and Clojure. Usually my enthusiasm doesn't survive until I have time to sit down in front of the computer out of working hours. But recently I've started to read about Go and it rekindled my interest in such mental exercises. It also gives me a nice opportunity to learn Go. Also these 3 languages are quite different, which makes these little problems even more educative.

Without further ado, here is the first one.

In OO polymorphism is achieved by inheritance. Inheritance also enables sharing data and logic sparing the developer some code duplication. It is a powerful tool, but nowadays generally regarded overused. The Template Method Design Pattern is a nice example and I've implemented it dozens (if not hundreds) of times at work. Let's see how to solve the problems it solves in languages that don't have inheritance.

The idea is to devise a toy problem, solve it in Scala (it would be the same in Java, I just want keep the boilerplate down) to demonstrate the OO-way, then come up with Clojure and Go solutions.

The Problem

Our domain has employees who can fall into 2 broad categories, Office Workers and Field Workers. Their base salary and the calculation of how their years at the company contribute to their salary are different. I let the code speak for itself.

Scala/OO solution

Plain and simple for a OO developer.

Clojure solution


No inheritance, obviously and no types either. Therefore the toString and salary functions are not defined on types, but the entity maps are passed as arguments. Another solution could have been using protocols and defrecords, that would have yielded a more OO-like solution. However for a single function it seemed to be an overkill.

Go solution


It took me some time to come up with a solution. I Go there are no abstract classes or virtual methods. This required me to define to a separate employee interface and a baseEmployee struct. And I couldn't attach the salary method to the interface either, even though it has both the yearContribution() and the baseSalary() methods on it. It is not necessarily a problem, this could be idiomatic in Go.

Solution for "Docker is eating up my disk!"

I've started to play with Docker recently and quickly bumped into a problem. Although I've never experienced it on my Mac, on AWS EC2 the space was eaten up on /dev/xvda1 and killed my Docker process eventually. The solution proved to be running the following commands

    docker stop $(docker ps -a -q) || true
    docker rm $(docker ps -a -q) || true
    docker rmi $(docker images -f "dangling=true" -q) || true
    docker volume ls -qf dangling=true | xargs -r docker volume rm

New version of Principle

After a year long hiatus, I've touched Principle again. A new version is out, 0.34, that moves all configuration from the clunky pom.xml to a neat yaml file, like the one below



The next big thing will be to make it usable a SBT plugin, while also retaining the Maven plugin nature. By looking at the documentation of "Simple" Build Tool, this looks challenging.

The Reader Monad

Yet another exploration in Monadland. Like the State Monad, its sibling, Read Monad had managed to elude me until I came across an enlightening example in Debashish Gosh's excellent book, Functional and Reactive Domain Modelling. In the following example I'll describe a simple scenario where I'd usually use dependency injection and refactor it to a Reader monad using variant.

Version 1. Dependency Injection

When one tries to follow FP principles, she strives to build her application up like an onion. The core should contain pure functions, and all interaction with external services - DB, web service calls, user input, ... - , i.e. side-effects, should be confined to the outer layer. In the code above the domain logic and side-effects are undisentanglable. The next version shows an alternative.

Version 2. Higher order function

This is better. `notifyUser` is now a referentially transparent function. The actual execution of the effects is deferred to a later point, when the result function is called with the context. The Reader monad is nothing else just a convenient wrapper around such a function.

Version 3. Reader Monad

The benefit Reader monad offers over the simple HOF-solution is the monadic composability, like in the example below.


Note that inside the for comprehension the context doesn't even appear.

Struggling with the State Monad

After spending hours trying to find articles explaining the damn thing without resorting to "let's take this contrived example..." or "let's take a real life example, generating pseudo-random numbers" (seriously?!), I'm still left with frustration. Eventually I'd reached out to Kaloz, who pointed me to one of his earlier explorations of the topic. His example is here, but I'm still in the dark why is it any better than a simple foldLeft. In the code below I refactored slightly Kaloz's code to keep the generic part apart from the specific solutions. A second foldLeft-using function is provided to match the signature of the function using the State monad, although I don't see any additional value in that.

Kaloz? What am I missing?