Mars Rovers in Clojure - iteration 2

In the last couple of weeks I spent a fair amount of time partially rewriting the Mars Rovers (source is here). I added a Swing UI so the rovers can be visually tracked.

Run it from command line

You can run it from the command line with

$ lein run

You can also overwrite the default number of rovers and the number of actions each can take before "freezing".

$ lein run ":rover-number 1000 :action-number 9999999"

It's a bit inelegant to use quotes, but I haven't had time yet to implement a proper command line interface.

UI

As you can see, the rovers seem to move in discrete steps instead of a continuous flow. This is partly due to the fact that the Swing panel is to refresh every time a Rover sends a position changed message to the Plateau, which triggers a message sent to the displaying component. We are talking about ~10,000 messages per second, so I had to introduce some sampling. You can read about it in one of the previous posts.

Performance

The number of moves taken by the Rovers is printed out to the console at every 10,000th move. With the default 100 Rovers on my MacBook Pro (16 GB 1600 MHz DDR3, 2.3 GHz Intel Core i7) this number is 740,000 in 20 seconds, so 37,000 moves per second. This many messages are running through the core.async channels. I don't know how the Scala actors perform, but seems to be pretty impressive. Once I tried with 10,000 Rovers, but all the cores jumped up to 100% and the cooling system started up so vigorously that I saw it better to kill the process. The max it seems to be able to take is 1,000 Rovers, but the number of moves per second is much lower in that case then the one with a 100.

Tests

Painfully lacking...

Next steps...

will be some better documentation, marking dead Rovers with different colours, tests, etc...

Closures are the poor man's objects

I've heard this a lot without giving too much thought about it. But the Circuit Breaker made me contemplate the idea a bit. So....creating a stateful object in Clojure is actually pretty easy. The simplest I can think of is a unique id generator. This would look something like this in java

	interface IdGenerator {
	Integer nextId();
	}
	class SimpleIdGenerator implements IdGenerator {
	private final AtomicInteger generator; = new AtomicInteger(0);
	public Integer nextId() {
	return generator.incrementAndGet();
	}
	}

view raw IdGenerator.java hosted with ❤ by GitHub

In Clojure

	(defn create-id-generator
	"Returns an impure function to generate ids"
	[]
	(let [next-id (atom 0)]
	(fn []
	(swap! next-id inc)
	@next-id)))
	;; here we create our "object"
	(def generate-id! (create-id-generator))
	(println "Id: " (generate-id!))
	;; Id: 1
	(println "Id: " (generate-id!))
	;; Id: 2
	(println "Id: " (generate-id!))
	;; Id: 3
	;; here we create another "object"
	(def generate-id-2! (create-id-generator))
	(println "Id2: " (generate-id-2!))
	;; Id2: 1

view raw id-generator.clj hosted with ❤ by GitHub

Of course it's a boring example so I've come up with another, slightly more interesting one, a simplified vending machine.

	(defn create-vending-machine
	"Returns an impure function to act as an object representing a vending machine"
	[]
	(let [items (atom {:Coke {:quantity 3 :price 5}
	:Mars {:quantity 2 :price 3}
	:Sandwich {:quantity 5 :price 10}})
	wallet (atom 0)]
	(fn [item-id money]
	(cond
	(nil? (item-id @items)) :invalid-item
	(zero? (get-in @items [item-id :quantity])) :out-of-stock
	(not= money (get-in @items [item-id :price])) :price-not-match
	:else (do
	(swap! items update-in [item-id :quantity] dec)
	(swap! wallet + money)
	:success)))))
	;; the closure, or "object"
	(def buy! (create-vending-machine))
	(defn assert= [a b]
	(assert (= a b)))
	(assert= :invalid-item (buy! :Pepsi 1))
	(assert= :price-not-match (buy! :Coke 20))
	(assert= :success (buy! :Mars 3))
	(assert= :success (buy! :Mars 3))
	(assert= :out-of-stock (buy! :Mars 3))

view raw vending-machine-clojure.clj hosted with ❤ by GitHub

An epitome of applied Single Responsibility Principle. Does only one thing and hides its internals completely. Of course I could leverage dynamic typing to overload the function and perform any kind of other logic - getting information of the content, changing the state, whatever. I could make a mess, but I would have to awkwardly work against the language to do it.

Immutable datastructures vs information hiding

One generally accepted OO-concept Rich Hickey challenges with Clojure is the wisdom of information hiding. In rich object models (one proposals of DDD for example) aggregates are encouraged to eschew exposing their internal structure.
As a simple example we can imagine we are to write a card game with multiple players.  Any card game actually, the point is that the players have hole cards which they don't share with others. Not even with the rest of the code. This project implements a simplified Blackjack game what perfectly serves the purpose of this post. This is the class for the participating players.

	class Player extends Entity<PlayerID> {
	private final List<Card> cards;
	private boolean stopped;
	public Player(PlayerID playerID, List<Card> cards) {
	super(playerID);
	Validate.notNull(playerID);
	Validate.notNull(cards);
	this.cards = cards;
	}
	public boolean notStopped() { return !stopped();}
	public boolean stopped() { return stopped;}
	public void stand() { this.stopped = true;}
	/**
	* Important that the logic should be where the data is. This yields rich
	* object model.
	*
	* @return
	*/
	public int score() {
	int score = 0;
	int aceCounter = 0;
	for (Card card : cards) {
	score += card.value();
	if (card.rank == Rank.ACE) {
	aceCounter++;
	}
	}
	// if player has two ACES, then one has a value of 1
	if (aceCounter > 1) {
	score -= 10;
	}
	return score;
	}
	}

view raw blackjack-player.java hosted with ❤ by GitHub

The player doesn't expose her cards, which is fine for us. Now imagine we want to spice up our game with some fantasy elements. Some player can cast Spying Spells to peek into their opponents' hands. This is orthogonal to the existing functionality, so the code change preferably shouldn't touch the existing code. But it does, because we have to modify Player some way.

	//version one - simple getter
	class Player {
	...
	List<Cards> getCards() { return Lists.newArrayList(this.cards);}
	...
	}
	//version two - a new immutable object for that aspect of the Player
	// so the Player internals are not exposed directly
	class PlayerView {
	private final PlayerID id;
	private final List<Cards> cards;
	//constructor and getters
	}
	class Player {
	...
	PlayerView getView() {...}
	...
	}

view raw blackjack-player-2.java hosted with ❤ by GitHub

Thus the Player class now serves two independent features of the game.

In Clojure classes are not used, instead functions operate on associative information models (@Copyright Rich Hickey), namely maps, lists, vectors, sets. There is no information hiding, the player is represented something like this

	;; a player representation
	{:id 123
	:cards [[2 :spades] [:king :clubs] [10 :diamonds]]
	:stopped false}
	;; a function calculating the score
	(defn score-hand [cards]
	"Calculates the score for the hand"
	(let [sum (reduce + (map card-value cards))
	ace? #(= :ace (last %))
	ace-count (count (filter ace? cards))]
	(if (> ace-count 1)
	(- sum 10)
	sum)))

view raw brlackjack-player.clj hosted with ❤ by GitHub

The logic is decoupled from the data, thus we can avoid awkward things like creating a new View object, but because Clojure uses immutable data structure there is no harm exposing their details either. We can easily add new functionality too without touching existing code.

Dynamic typing - Sampler filter

This post is superficially related to the basic concept of the previous, but shows a very similar pattern to Circuit Breaker. I call it Sampler filter, but I'm pretty sure there is an established name in the field of electrical engineering.
By implementing a Swing UI of my Mars Rover project (source code is here) I recently faced the problem that if I run hundreds of rovers, then refreshing the panel after every rover position change is impossible. Each position change puts an event - containing the positions for all rovers - on the channel the displayer logic listens to, and the messages just kept piling up. It's a typical IO bottleneck. So I decided it's enough to display every 100th or 1000th of them. They would change faster than my eyes can process them anyway. So I took the original function performing the display - repaint! - and wrapped it in a sampler. Here is the code

	(defn repaint!
	"Refreshes the Swing JPanel"
	[positions-msg]
	...)
	(defn sampler-filter
	"Wraps around a function returning a new function with the same signature, which delegates to the original function, but only
	with the given frequency
	f - the original function
	freq - frequency
	E.g. 'f' is 'println' and 'freq' is 3, then calling the result function only will print at every 3rd time"
	[f freq]
	(let [counter (atom 0)]
	(fn [& args]
	(swap! counter inc)
	(when (= freq @counter)
	(reset! counter 0)
	(apply f args)))))
	;; this function will only perform repaint at avery 1000th invocation
	(def sampling-repaint! (sampler-filter repaint! 1000))

view raw sampler-filter.clj hosted with ❤ by GitHub

Dynamic typing - Circuit breaker

Despite my fascination for Clojure I'm yet to be sold on dynamic typing. Frankly the most frustrating thing about Clojure for me is the constant struggle with the lack of static type check. What went wrong and where? There are no types to guide me nor auto-completion, and I feel I have to hold too much in my head instead of letting the IDE help me out so I can concentrate on the problem. I read quite often arguments like "dynamic typing gives you a much greater flexibility" and "the static type system is often in your way", but honestly I don't understand them. Probably because I never have done anything serious in a dynamically typed language. So I've embarked on a journey of deliberately looking for situations when dynamic typing clearly gives an advantage. Here I'd like to show the first (and so far the only) one I found. Before looking at the more complex task of writing a circuit breaker, I'd like to start with a simple one.

Logger wrapper

Imagine you'd like to log the call of some functions. In Scala or Clojure it's an easy thing to do with higher-order functions.

	def logger(f: (Int, Int) => Int) =
	(a: Int, b: Int) => {
	println("Hello")
	f.apply(a, b)
	}
	def add(a: Int, b: Int) = a + b
	println(logger(add)(3, 4))
	;Hello
	;7

view raw LoggerHOF.clj hosted with ❤ by GitHub

Now this logger is able to transform any function with the (Int, Int) => Int signature to a logging one. Which is cool, but not cool enough. For every signature it has to be reimplemented. In a dynamically typed language you can use the same logger function for all. But this logger is a quite boring example, so I'd like to show something that is really useful and practical, the...

...Circuit Breaker

Imagine you application has to call a lot of web services, but the systems on the other end are quite unreliable. Sometimes they respond, but they have the habit of being unavailable for short time windows. You wouldn't like to waste time and thread calling them the n-th time if they haven't been responsive for a while. Better to wait a bit, then try again later. This is where the Circuit Breaker can help.

The Clojure version is

	(defn circuit-breaker
	"This higher order function will return a function with the same signature and behaviour
	as the original function, but with an added functionality.
	The arguments are:
	f - the original function
	max-try-limit - the number of times f can be called unsuccessfully before the circuit breaks
	wait-seconds - how much time after the break is the circuit functioning again
	fail-value - the value the function should return if the circtuit is broken"
	[f max-try-limit wait-seconds fail-value]
	(let [fail-count (atom 0) ; the number of failed attempts since the last reset
	break-ts (atom nil) ; timestamp of last break
	reached-try-limit? #(= max-try-limit @fail-count)
	waited-enough? #(<
	(* 1000 wait-seconds)
	(- (System/currentTimeMillis) @break-ts))
	reset-circuit! #(do
	(reset! fail-count 0)
	(reset! break-ts nil))]
	(fn [& args]
	(println (str "Fails: " @fail-count))
	(cond
	(reached-try-limit?) (cond
	(waited-enough?) (do (reset-circuit!) (recur args))
	:else fail-value)
	:else (try ; if we are below try attempts limit
	(let [result (apply f args)]
	(do
	(reset-circuit!) ;if the call is successful, reset the circuit counter
	result))
	(catch RuntimeException ex
	(do
	(swap! fail-count inc)
	(when (reached-try-limit?)
	(reset! break-ts (System/currentTimeMillis)))
	fail-value)))))))

view raw CircuitBreaker.clj hosted with ❤ by GitHub

An easy way to try it out is to find a function that is simple to call with arguments that will break it, and wrap it in the circuit-breaker. The arithmetic divison is the perfect candidate.

	user=> (defn div [a b]
	(println "Doing it")
	(/ a b))
	#'user/div
	user=> (def safe-div (circuit-breaker / 3 10 "Oops"))
	#'user/safe-div
	user=> (safe-div 6 3)
	Fails: 0
	2
	user=> (safe-div 12 3)
	Fails: 0
	4
	user=> (safe-div 12 0)
	Fails: 0
	"Oops"
	user=> (safe-div 12 0)
	Fails: 1
	"Oops"
	user=> (safe-div 12 2)
	Fails: 2
	6
	user=> (safe-div 12 2)
	Fails: 0
	6

view raw CircuitBreaker-test.clj hosted with ❤ by GitHub

Circuit Breaker is a fascinating pattern for many reasons. First it shows a clean analog to physical engineering solutions, making software engineering look like real engineering, and also making it easy to understand. Second, from an ezoteric FP point of view, Circuit Breaker is like magical entity that swallows a "stupid" function and spits out its doppelganger, same in signature and behaviour, but with an extra "intelligence".

The reason why dynamic typing fits so well to circuit breaker, I think, is that the extra functionality it adds has nothing to do with what the function does, nor with its signature.