Clojure Geek

Writing about learning Clojure

Clojure Is My Moon

I have always been a bit of a polyglot programmer. Always trying every new language at least a little. When I got my first blog RubyGeek.com I looked for a more generic name besides “RubyGeek” but I couldn’t find one that I liked that was available. I was doing Ruby at the time and totally loved it and I still love Ruby, but I needed a switch.

Three years ago I felt I was at a crossroads in my career, I had been doing web apps with php, perl, javascript or ruby for 18 years. I decided to form a LLC and do consulting. I have been able to use ruby, clojure and js/node as a remote contractor.

Earlier that year (2013) I picked Clojure as the language to learn. Lisp was an inspiration for many of the languages that I had used over the years and I always heard it was good to learn a Lisp sometime in your career. I also wanted to learn a Functional Language, hearing that it was good for your brain to learn it.

I looked at Scala and tried the Coursera class on Scala. Wow that was so hard, I am not sure I even completed the first lesson all the way. I had no idea about functional programming, so different! Then I looked at Clojure.

In Austin at the time there was an active (10-15 people) local user group for Clojure. I went to my first meeting after only learning a little Clojure. I only talked to one person, and soon as it was over, I left. I felt a little bad about leaving so quickly, I mean after all, I am not usually so shy at meetups. I run AustinRuby and Women Who Code – South Austin now helping to organize AustinClojure.

Why did I run out when it was over?I had no idea what they were talking about. I was like a beginner again. I distinctly remember seeing datomic code at that first meeting and thinking “wow what a strange database” and what is an immutable database?!?! how is that possible…

I took notes furiously so I could look up those things at home.

At the next meetup, I apologized to the organizers for making a mad dash the previous month and went out after for post-meeting food and conversation.

Every 3-4 months I have stopped and tried to decide if I want to keep up with Clojure. It is hard sometimes. I feel stupid and think, OH GEES, maybe this is not for me. Whenever I think that, I remember Russ Olsen’s talk “To The Moon!“. Not that my learning goals can even come close in comparison to landing on the moon, but I like the mindset.

in my own words

I am not learning Clojure because it is easy.. but because it is hard.
I want and need to challenge myself.

I am not saying Clojure is hard, but was so different than the procedural code I wrote as a kid in Basic and C then OOP when I got to college. Functional blows my mind sometimes but I love it.

A few months ago I decided I am All In on Clojure. I stopped playing around with other languages unless I needed it for my billable work (currently node). Clojure is my focus now.

I have lots of support on my journey learning Clojure. Austin Clojure members Sam, Norman and Dar have been there for questions, screen sharing or meeting up at a coffee shop. Bridget Hillyer has lead a weekly online study group that has kept me doing at least some Clojure when I was uncertain about continuing. I installed Clojure on a chromebook and when I posted on Twitter, Justin Gehtland sent me some Clojure stickers and a handwritten note “Thanks for being so enthusiastic about Clojure” that I read when I feel like it is too hard.

Russ Olsen said at the end of his talk:

“You see, I’m familiar with the impossible. I saw it done on TV when I was a kid.”

I hope everyone has something hard to pursue. If there is nothing you are excited about, then find something hard to make possible.

I guess you could say, Clojure is my moon.

Using Records in Prismatic Schema

Previously I wrote about prismatic schema and records. Here I’m combining the two.

Add to project.clj

1
[prismatic/schema "1.0.5"]

Require in your file

1
[schema.core :as s]

Experimenting In the repl:

1
2
3
4
dev> (s/defrecord Recipe [name :- String  source :- String  url :- String  servings :- s/Int])

dev> (s/explain Recipe)
(record dev.Recipe {:name Str, :source Str, :url Str, :servings Int})

Using the function defrecord from schema, I created a record like in my last post on records. I added an additional field :serving to make it more interesting. Explain takes a record and does just that, explains it.

Validating a record is easy:

1
2
3
4
5
dev> (def tacos (->Recipe "Tacos" "Bob" "www.tacos.com" 4))
#'dev/tacos

dev> (s/validate Recipe tacos)
#dev.Recipe{:name "Tacos", :source "Bob", :url "www.tacos.com", :servings 4}

Doing an invalid record:

1
2
3
4
5
dev> (def pie (map->Recipe {:name "apple pie"}))
#'dev/pie

dev> (s/validate Recipe pie)
ExceptionInfo Value does not match schema: {:source (not (instance? java.lang.String nil)), :url (not (instance? java.lang.String nil)), :servings  (not (integer? nil))}  schema.core/validator/fn--11079 (core.clj:151)

I created the instance of the Pie recipe with just a name. With a regular record, it would have worked fine. But prismatic schema has these as required fields.

Also in addition to validate there is check, which returns nil if everything is good, or errors if not.

1
2
3
4
5
dev> (s/check Recipe (->Recipe "Choc Milk" "Charlie" "www.milk.com" 1))
nil

dev> (s/check Recipe (map->Recipe {:name "Apple Pie"}))
{:source (not (instance? java.lang.String nil)), :url (not (instance? java.lang.String

There is more things to learn with Prismatic Schema, but I am farther along now. :)

Testing Private Functions in Clojure

Unit Testing private functions/methods is controversial because you should test the public api and if the public api calls the private functions/methods then you are golden. However, I often like to test private methods/functions when developing. Sometimes I remove those after I have written a more comprehensive test later.

First, lets talk about how do you define a private function? I found three ways:

1
2
3
4
5
(def- my-private-function)

(def ^:private my-private-function)

(def ^{:private true} my-private-function)

Curious, I looked at the source of defn-

1
2
3
4
5
6
(defmacro defn-
  "same as defn, yielding non-public def"
  {:added "1.0"}
  [name & decls]
    (list* `defn (with-meta name (assoc (meta name) :private true)) decls))
nil

The first is just a macro that adds the metadata to the function, same as if you added the metadata yourself. However I like the second version ^:private, it is clear when reading the code.

Testing a private function in clojure is not so obvious at first:

The function I want to test:

1
2
3
4
(defn ^:private prepare-url [config url]
  (if (str/starts-with? url "http")
    url
    (str (:host config) url)))

To call the private function outside of the namespace where it is defined, you call it with the reader macro like this:

1
#'namespace/my-private-function

How I wrote the test with the private function:

1
2
3
4
5
6
7
8
(def test-config {:host "http://localhost:1111"})

(deftest test-prepare-url
  (let [private-function #'hawk/prepare-url]
    (is (= "http://localhost:1111/api/users" 
           (private-function test-config "/api/users")))
    (is (= "http://www.test.com/api/users" 
           (private-function test-config "http://www.test.com/api/users")))))

Although I could have used the reader macro to refer to the function in the test, I decided to use let to give it a nicer name. I called it private-function to remind myself when I read it later that I am calling a private function. I am pretty happy with this way to test private functions. I considered calling the function function-under-test but that is sort of wordy. I could shorten private-function to private-fn.

After coming up with that solution, I found this post that uses a macro to wrap your private tests making the functions available. That is cool. Downside of using my way of testing private functions if I later decided to make the function public, I’d have to rewrite a bit of the test (it would still pass, but reading the test would be confusing). If I used the macro described in that post, it would be easier, just removing the wrapping function.

Any other suggestions?

Edit: It has been pointed out to me that you probably shouldn’t ever want private functions in Clojure, it doesn’t have the need of encapsulation that OOP programming does. Ok, Cool. But should you need to access a private function, a reader macro is how you would do it…which is the point of this post. :)

Records in Clojure

I haven’t used records before in a project and when reading some code that used it, realize I really need to learn about records. It’s not complicated and actually kind of cool! Here’s my experimentation.

To start off, I re-read the first chapter in ClojureApplied where it talks about Modeling your Entities.

I took a familar problem, my recipe-api. This is a recipe api that uses liberator api reading recipes from a database. But, I can use the concept here.

1
2
user=> (defrecord Recipe [name link source])
user.Recipe

When you create a record, you get two factory functions for free to create instances of it. The first, a positional function:

1
2
user=> (def tacos (->Recipe "Tacos" "www.tacorecipes.com" "mom"))
#'user/tacos

You must put values for all the fields in or you get

1
2
3
user=> (def tacos (->Recipe "Tacos" "www.tacorecipes.com"))

CompilerException clojure.lang.ArityException: Wrong number of args (2) passed to: user/eval10000/->Recipe--10017, compiling:(form-init4856550892047924668.clj:1:12)

The other way you can create an instance of a record is using a map.

1
2
3
4
user=> (def pizza (map->Recipe {:name "Pizza" :source "dad"}))
#'user/pizza
user=> pizza
#user.Recipe{:name "Pizza", :link nil, :source "dad"}

We didn’t have a value for link in the map so it still let us create the instance and automatically assigned the value of nil. This might be a reason to use the map-> version over the positional version, if you don’t know all the values at that time. They work like maps, you can add in missing values later with assoc:

1
2
user=> (assoc pizza :link "www.pizzarecipes.com")
#user.Recipe{:name "Pizza", :link "www.pizzarecipes.com", :source "dad"}

Accessing the the values in a record is just like a map:

1
2
3
4
user=> (:name tacos)
"Tacos"
user=> (:source tacos)
"mom"

The chapter has a great section on why you would want to choose Records over Maps you should read! I won’t spoil the fun here, but there are some compelling reasons when to use records over a map.

Two Cool Tools for Clojure Development

I’ve recently used a couple tools that I want to write about because I think they are pretty useful.

1. Eastwood

Eastwood is a linter, which will check your syntax to see if it is valid. BTW I have actually made this mistake for real! The github page lists all the things it checks for and a lot of how to configure eastwood.

To get started, Install into your lein profile.clj:

1
{:user {:plugins [[jonase/eastwood "0.2.1"]]}}

My function, which by the way, is awesome:

1
2
3
(defn my-awesome-function [x y z]
"My function is awesome"
(+ x y z))

Running lein eastwood in the project shows me:

1
2
3
4
5
6
7
8
9
10
11
lein eastwood
== Eastwood 0.2.1 Clojure 1.7.0 JVM 1.8.0_60
Directories scanned for source files:
src test
== Linting blog-post.core ==
Entering directory `/Users/nolastowe/practicing/clojure/blog-post'
src/blog_post/core.clj:5:7: misplaced-docstrings: Possibly misplaced docstring, my-awesome-function
src/blog_post/core.clj:5:1: unused-ret-vals: Constant value is discarded: "My function is awesome"
== Linting blog-post.core-test ==
== Warnings: 2 (not including reflection warnings)  Exceptions thrown: 0
Subprocess failed

Doh. The docstring is in the wrong place, correcting that and eastwood is happy. That is an easy mistake to make!

1
2
3
4
5
6
7
lein eastwood
== Eastwood 0.2.1 Clojure 1.7.0 JVM 1.8.0_60
Directories scanned for source files:
src test
== Linting blog-post.core ==
== Linting blog-post.core-test ==
== Warnings: 0 (not including reflection warnings)  Exceptions thrown: 0

I’ve used eastwood a lot when I was developing amazon lambda functions, when I couldn’t see the result till I’ve uploaded to amazon. At first was frustrated only after all that to find an error which required me to do another compile to jar and upload. Now I use it before commit or when i’ve completed a unit of work or when I just can’t figure why something is not working..always a chance I did something stupid with syntax.

2. Criterium

This library is for benchmarking clojure and I was using it to compare different implementations when I was working on adding some string functions to Clojure. Initially I added it to my lein profiles (similar to how i did with eastwood) which works fine for testing in the repl. Ghadi saw me talking about using it in Clojurians Slack and gave me a bash script to start any version of clojure and criterium with a repl:

1
2
3
4
5
6
#!/bin/bash
M2=$HOME/.m2/repository
CLASSPATH=${M2}/criterium/criterium/0.4.3/criterium-0.4.3.jar:src
JOPTS='-Xmx4G -XX:+UseG1GC'
CLOJURE_JAR=$M2/org/clojure/clojure/${CLOJURE_VERSION}/clojure-${CLOJURE_VERSION}.jar
java $JOPTS -cp ${CLASSPATH}:${CLOJURE_JAR} clojure.main

Then save it as repl-version.sh and run with CLOJURE_VERSION=1.7.0 repl-version.sh to open a repl, then you can test away. I see this being useful when you are working on clojure itself or want to compare things in different versions of clojure. I used it when I was working on a patch for Clojure.

One of the things I used it for was figuring wanted to compare char? and instance? which should I use.. this is a true story! I was trying to figure this out one day..

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
user=> (quick-bench (char? \c))
Evaluation count : 39012534 in 6 samples of 6502089 calls.
Execution time mean : 5.188821 ns
Execution time std-deviation : 0.351751 ns
Execution time lower quantile : 4.755146 ns ( 2.5%)
Execution time upper quantile : 5.544008 ns (97.5%)
Overhead used : 10.020671 ns

user=> (quick-bench (instance? Character  \c))
Evaluation count : 51496848 in 6 samples of 8582808 calls.
Execution time mean : 2.510016 ns
Execution time std-deviation : 0.701679 ns
Execution time lower quantile : 1.693617 ns ( 2.5%)
Execution time upper quantile : 3.375066 ns (97.5%)
Overhead used : 9.812462 ns
nil

I wondered, how does char? work? So I looked at the source:

1
2
3
4
5
6
7
8
user=> (source char?)
(def
^{:arglists '([x])
:doc "Return true if x is a Character"
:added "1.0"
:static true}
char? (fn ^:static char? [x] (instance? Character x)))
nil

Ahh, I see, it just checked the instance? .. so I look what that function does:

1
2
3
4
5
6
7
8
user=> (source instance?)
(def
^{:arglists '([^Class c x])
:doc "Evaluates x and tests if it is an instance of the class
c. Returns true or false"
:added "1.0"}
instance? (fn instance? [^Class c x] (. c (isInstance x))))
nil

So of course, instance will be faster than char …. If you know your class of object, it looks like instance? will always be a little faster. But there ya have benchmarking in Clojure with Criterium. :)

When reviewing this post, Ghadi said:

One small note, instance? is intrinsified by the compiler. It turns into a bytecode instruction. Which you can see in this clojure source. It’s faster than a function call.

Book Review - Clojure Applied

Written by Alex Miller and Ben Vangrift published by Pragmatic Programmers.

I am a junkie for books and Clojure books are no exception. I have been doing Clojure since Jan 2014 and feel like I kinda got the hang of it but now what? When I saw the Clojure Applied “From Practice to Practitioner” was being written I kept a close eye on Pragmatic Programmer’s coming soon list and an eye on twitter. The day I heard the beta book was available I bought it. When I was offered a free copy to do this review, I said thanks but I already have it and I’ll still be happy to write a review. :)

I was not disappointed as I dived into the book, this is just the thing I needed. Here’s is how to do clojure for the real world with real application of clojure’s concepts applied.

This book is in 3 parts:

  • Foundations
  • Applications
  • Practices

Foundations

Chapter 1: Modeling your Domain

This section covers aspects of your application including modeling your domain using maps and record and when to use each of those. Getting your mind around immutable data is one of the hardest parts for me in getting into clojure and the discussion on entities here really helped solidify the concepts. Once your data is modeled you want to do operations, in OOP languages you have polymorphism. In clojure there are multi methods and protocols. Here are examples of each and when you would use them.

Chapter 2: Collecting and organizing your data.

This chapter talks about collections and which one to use and how to write a custom sorting function. Then it goes on to talk about updating data – wait I thought data was immutable in clojure? It is, so when you update a collection it returns a new copy with the data updated. Its typical of mutable functions to end in ! (danger Will Robinson!), This chapter closes with how to define a “new” datatype and what is involved there.

Chapter 3: Processing Sequential Data

This chapter starts with explaining how sequences are an abstraction connecting collections and the functions that act on those collection. This is the really beauty of clojure and at first mind blowing then once it sinks in, you can imagine how you could have thought differently about the concept. Reducing, Filtering, Selection, Sorting, Removing Duplicates and grouping are all operations you will want to do on your collection once you have it.

Applications

Chapter 4: State, Identity, and Change

Change, if our applications couldn’t change they would be boring, probably even useless. If you need more than one change at a time, you should use a transaction. Different data types (atom, ref, etc) have different functions to update and this goes over each of them in detail.

Chapter 5: Use your cores

This chapter talks about threads and introduces the concept of agents and promises to transfer a value from one state to the other. The talk about using reducers to process data in parallel and the performance considerations. Ending with with discussion about thinking in process using channels in core.async to create a sort of a pipeline for processing data. More meat to this chapter than I can summarize here.

Chapter 6: Creating Components

One of my favorite chapters. Now that you have your structure and processing down, how to combine it into something we can put in a package with a nice bow? There are a few ways to organize this and the first is a namespace. You want to use one or more for your code so it is easily read and comprehended. The chapter goes over some common namespaces used in project and what might go into those. You can connect your components with channels and structure their lifecycle with a record and functions to make start and stop. Leading to the next chapter…

Chapter 7: Compose your Application

Composing your application! Early in the chapter it starts talking about Stuart Sierra’s library name precisely that. Component. It’s like the previous chapter laid the foundation of why, and now this is how. Then putting those into a system and using profiles and configs to make your code operate in different environments (dev, staging etc).

Practice

Chapter 8: Testing Clojure

Ok maybe this is my favorite chapter. My love of testing has been equated with “sickness” for loving tests so much. This chapter kicks off with repl-based testing, by developing your function and calling it in the repl to work out the inputs/outputs. Then moving on to example based testing which I know as unit testing. The new part to me is in Clojure are “are” test function which can easily test a lot of short tests. Testing exceptions is also a thing which is mentioned briefly. Then some other features of clojure test are fixtures, a handy way to provide baseline data for testing. The section ends with a mention of ‘Expections’ test library which takes a different approach than ‘clojure.test’. This chapter ends with talking about property based testing and generators, which if you don’t know about them will blow your mind and leaving you wondering why we haven’t been testing this way all along, :)

Chapter 9: Formatting Data

This chapter seems shorter compared to the other chapters, talking about edn, son and transit. The first two I get and using them in clojure is awesome. But Transit, the authors follow the explanation with a concrete example, I think I get it but I will need to use it in a project for it to really sink in.

Chapter 10: Getting out the Door

This is where it talks about publishing your code and deployment. A few things to think about when publishing your code is to choose the collaboration platform (Github, Bitbucket etc), the contributing agreements and licensing and the minimum files you should have in your repo. When your code is ready to be published as a library most people use Clojars, Maven Central Repository or a self hosted Maven repo. The last part of this chapter is deploying your application to heroku or provisioning your own server by running a jar or to deploying to an application server.

Appendix

This contains two sections: Roots which talks about some of the concepts and Thinking in Clojure which describes the mindset of how clojure is written and what I think community strives to be like.

Overall this is a fantastic book and I’ve tried to summarize the key points to help you make the decision if this book would help you in your clojure career. I bet it will! :)

Liberator: As easy as it gets

When I first started using liberator I had a hard time figuring out what was what….and at the same time learning clojure and trying to change my OOP brain into understanding functional code. I want to talk about liberator from that perspective and maybe it will help others too. :)

Liberator is a clojure library to create web APIs. You create your endpoints by defining resources. Liberator needs a routing library to direct traffic to ring (Ring is something like Rack in ruby), Compojure is popular (there are others – liberator’s docs mention bidi … I am going to use compojure today (but bidi looks interesting!). Lets start with a plain compojure app, type:

1
lein new compojure simpleway

It creates a boilerplate web application with ring as the server. Lets try it out:

1
lein ring server

It (usually, unless you have something else running) starts on port 3000, but watch the screen when it starts up. It will say where it started. Then hit http://localhost:3000 to see a hello world message. Ok, it works! :)

Now lets add liberator, you can find out the current version on the project homepage which at this time is 0.13.

add

1
[liberator "0.13"]

To project.clj. Be sure to use double quotes (I have to break this habit of single quoted strings I inherited from years of ruby) and put it anywhere in the list of dependencies. So it looks like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
(defproject simpleway "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :min-lein-version "2.0.0"
  :dependencies [[org.clojure/clojure "1.6.0"]
                 [compojure "1.3.1"]
                 [liberator "0.13"]
                 [ring/ring-defaults "0.1.2"]]
  :plugins [[lein-ring "0.8.13"]]
  :ring {:handler simpleway.handler/app}
  :profiles
    {:dev {:dependencies [[javax.servlet/servlet-api "2.5"]
                          [ring-mock "0.1.5"]]}})

Ok thats all we need to change there. Open src/simpleway/handler.clj and add the following to the ns at top:

1
[liberator.core :refer [defresource]]

I add it as the second to last thing in the :require statement. It looks like this:

1
2
3
4
5
(ns simpleway.handler
  (:require [compojure.core :refer :all]
            [compojure.route :as route]
            [liberator.core :refer [defresource]]
            [ring.middleware.defaults :refer [wrap-defaults site-defaults]]))

Ok, lets make our first resource:

add this to your file

1
2
(defresource hello-world-resource
  :handle-ok "Hello Clojure World!")

Add it before defroutes app-routes

then change that to read like this:

1
2
3
(defroutes app-routes
  (GET "/" [] hello-world-resource)
  (route/not-found "Not Found"))

changing the GET function to use your resource instead of just spitting out a string. Uhoh .. when you visit http://localhost:3000 you’ll see

1
No acceptable resource available.

Let’s tell the hello-world-resource what media types are available

1
2
3
(defresource hello-world-resource
  :available-media-types ["text/plain"]
  :handle-ok "Hello Clojure World!")

This is something I didn’t realize at first.. in a defresource each thing (or pairs of things, always in a pair) falls into one of four categories as best I can figure out:

  • representation (either as :available-media-types (vector of strings) or :as-response (fn )
  • decision points (ends in ? or starts with [accept | if | is | method | post | put]
  • handler (starts with handle if a redirect also defines :location [url])
  • actions (:get!, :post!, :delete!, :patch!)

So now you have the simplest liberator resource which just returns text.

My thanks to Dar for his talk at AustinClojure which helped solidify some of these things :)

Turning a Chromebook into a clojurebook

My husband surprised me with a Acer Chromebook, 11.6-Inch, CB3-111-C670 (Intel Celeron , 2GB, 16GB SSD, White) .. yeah ok, I have an affiliate link in there, if you don’t want to throw me a bone then just search for it. :) So far I’ve earned about $1.35 :)

At first I was like … I don’t need yet another device … but I thought maybe chromebooks are good for those that just want to check email, facebook, so I should check it out so I could talk about it intelligently. ChromeOS is actually pretty good. All google’s sites work good and the UI was plesant. But could I develop on it? Maybe I said. I could, of course, ssh to a machine at AWS or DigitalOcean. But I could do that from my ipad.

Then I remembered Nitrous.io which I used awhile back. I was doing Ruby/Rails then and liked the collaborating features. I checked now and there was a java container so I fired up one of those. I installed lein (not the package, use wget and the url here to get the most up to date version.

Then I ran:

1
2
lein new compojure chromebook
lein ring server

Boom, a compojure server. In your Nitrous config, go to ports, then add port 3000. Then you can go to your UI for nitrous and go to Preview, and you’ll see your web interface.

Ok cool but what about emacs?

Open the terminal view full screen, type emacs and entering emacs. Cool, already installed! I didn’t check what emacs libraries were isntalled but I bet you can add what you need.

However all that depends on an internet connection and I am fixing to fly to New York on client business…so I am going to have alot of downtime and maybe not always with decent internet. Could I install linux locally and get clojure working?

Following these directions, I got it working. I used Unity and I was afraid it would be slow. But it really wasn’t to my surprise. The default terminal is crappy so I installed Tilda and it is minimal but works, I set the size to be 85%/85% and center horizontally. I don’t expect to do any side by side editing so just putting it in the middle was good.

I installed firefox default-jre and then lein (same method as above) … then I was able to get a compojure app running like before and go to port 3000 to view it.

I installed emacs with the update directions and then to get a decent set of emacs configuration I use brave clojure’s configs. With vim I always kept my configs on github but with emacs its pretty easy to just grab one of these setups like Brave Clojure has and build on it.

Pretty cool, battery life is amazing..11+ hours, keyboard is decent. Touchpad not annoying, though the slide to scroll doesn’t work but I am good with that. For $169 (at the time) it is pretty dang cool. And it doesn’t delete the the ChromeOS so I can always go back. :)

Closing it to put it to sleep works great and it wakes up with no problem. That always has been a problem when I’ve put ubuntu in a vm on mac or pc.

I’ll see how it performs on my flight (multiple lay overs… but some are in Chicago so I can grab an italian sausage at least!) but I think this is pretty awesome.

Now to just get some clojure stickers for the back of my clojurebook…

Clojure is copyright by Rich Hickey.

ChromeBook is probably copyright by Google

clojurebook is something I just made up :)

Simple API Backend for Development with Atoms

When playing with liberator I wished I had a simple way to skip over having a database for playing around with data and I also wanted to write about Liberator but not have to get bogged down with Database stuff. I wrote a simple API backend that uses atoms and I’ll write about this first, and this will be followed with some posts on liberator and what is the simplest setup and gradually building on.

As you may know, Atoms are a way to maintain state, a placeholder for changeable data. In my simple-api I defined two atoms, one for the data, an empty map and another for the auto_increment number.

models.clj
1
2
(defonce id-seq (atom 0))
(defonce recipes (atom (array-map)))

Since I only want to define them once (and they will reset everytime this file is reloaded) I used defonce. Once I run that, I can see the values:

repl
1
2
3
4
simple-api.models> @recipes
{}
simple-api.models> @id-seq
0

The @ is a reader macro that de-references the atom. Now we need a function to add a new recipe. For fun I used Prismatic Schema which I wrote about here.

models.clj
1
2
3
4
5
6
(defn add! [new-recipe]
  (let [id (swap! id-seq inc)
        new-recipe-with-id (assoc new-recipe :id id)
        recipe (coerce! Recipe new-recipe-with-id)]
    (swap! recipes assoc id recipe)
      recipe))

Here’s what each line of the function is doing:

1: Incoming map of recipe data comes in as new-recipe.

2: Change the value of the id-seq atom using swap!. Swap! has always felt a bit strange, you have to pass in a function to change the value… you can’t just pass in say 5 (5 is not a function..derp, to just replace the value use reset!). In this case inc works fine and is what we need.

3: Create a new-recipe-with-id by adding in the id as a key/value pair by using assoc.

4: Take the new-recipe-with-id and use Coerce! returns a validated data structure or throws an exception if it is not matching structure and type.

5: Finally take the recipe and add to our recipes atom as a key/value pair of id/recipe.

6: Return the recipe.

Lets see it in action:

repl
1
2
3
4
5
6
7
8
9
10
11
simple-api.models> (add! {:name "All American Beef Taco" :url "http://www.foodnetwork.com/recipes/alton-brown/all-american-beef-taco-recipe.html" :source {:name "Food\
Network" :url "http://www.foodnetwork.com"}})
{:source {:url "http://www.foodnetwork.com", :name "FoodNetwork"}, :name "All American Beef Taco", :url "http://www.foodnetwork.com/recipes/alton-brown/all-american-b\
eef-taco-recipe.html", :id 1}

simple-api.models> @recipes
{1 {:source {:url "http://www.foodnetwork.com", :name "FoodNetwork"}, :name "All American Beef Taco", :url "http://www.foodnetwork.com/recipes/alton-brown/all-america\
n-beef-taco-recipe.html", :id 1}}

simple-api.models> @id-seq
1

We add a recipe with all valid data. Then examine the two atoms: @recipes and @id-seq and both are correct.

Lets add another recipe:

repl
1
2
3
4
5
6
7
8
9
10
11
12
simple-api.models> (add! {:name "Mexican Grilled corn" :url "http://www.foodnetwork.com/recipes/tyler-florence/mexican-grilled-corn-recipe.html" :source {:name "FoodN\
etwork" :url "http://www.foodnetwork.com"}})
{:source {:url "http://www.foodnetwork.com", :name "FoodNetwork"}, :name "Mexican Grilled corn", :url "http://www.foodnetwork.com/recipes/tyler-florence/mexican-grill\
ed-corn-recipe.html", :id 2}

simple-api.models> @recipes
{2 {:source {:url "http://www.foodnetwork.com", :name "FoodNetwork"}, :name "Mexican Grilled corn", :url "http://www.foodnetwork.com/recipes/tyler-florence/mexican-gr\
illed-corn-recipe.html", :id 2}, 1 {:source {:url "http://www.foodnetwork.com", :name "FoodNetwork"}, :name "All American Beef Taco", :url "http://www.foodnetwork.com\
/recipes/alton-brown/all-american-beef-taco-recipe.html", :id 1}}

simple-api.models> @id-seq
2

Here’s the tests I wrote for this function:

models_test
1
2
3
4
5
6
7
8
9
10
11
(use-fixtures :each
  (fn [tests]
    (add! {:name "test" :url "test.com" :source {:name "mom" :url "mom.com"}})
  (tests)))


(deftest test-repository
  (testing "adding recipe"
    (let [recipe {:name "test more" :url "testmore.com" :source {:name "mom" :url "mom.com"}}
          r (add! recipe)]
    (is (= 2 (recipe-count))))))

The test fixture adds one in, and then my test adds another making it count to be equal to 2. I’m not really liking how I wrote this test…but it works. Suggestions?

BTW, here’s the the recipe-count function:

models.clj
1
2
(defn recipe-count []
  (count @recipes))

Now lets write a function to return a vector of recipe maps:

models.clj
1
2
3
(defn get-recipes [] (-> @recipes
                         vals
                         reverse))

In the code I found as inspiration for this the author doesn’t use the @ but passes it to deref … I think I like using the spiral as it results in shorter code. Any reason why one is better than the other?

Now a function to return just one recipe by id:

1
(defn get-recipe [id] (@recipes id))

Since the key is just the id it is a simple task to look up by id.

Now, the function to update a recipe:

1
2
3
4
(defn update! [recipe]
  (let [recipe (coerce! Recipe recipe)]
    (swap! recipes assoc (:id recipe) recipe)
    (get-recipe (:id recipe))))

Swap here just takes the recipe given and puts it in place according to id. Not too complicated here.

Finally, deleting a recipe by id:

1
(defn delete! [id] (swap! recipes dissoc id) nil)

Disassociate the value at id and return nil.

I was pretty happy with how these functions turned out, and I could easily replace them with korma or honeysql when I was ready. I debated calling get-recipe get! and get-recipes get-all! but since those are not desructive like add, update, delete I thought it was fine to leave them.

I’m still learning best and idomatic clojure, so please speak up if you have suggestions! I will be using this simple-api in future blog post as I start to write about liberator and see my tests for the rest of the functions.

Using lein-try to learn Prismatic Schema

When I first heard of lein-try on the Cognitect Podcast (and then again in the Clojure Cookbook I thought wow! How cool is that! I tried it with following the example in the readme clj-time to get familar with using it.

First add lein-try to your ~/.lein/profiles plugins vector, run lein deps to get it installed.

1
{:user {:plugins [lein-try "0.4.3"]}}

I wanted to understand Prismatic’s Schema and wanted to try it. I wanted to start simple so I could understand it.

1
lein try prismatic/schema

A repl starts, then require the library and give it an alias

1
(require '[schema.core :as s])

Ok now ready to play!

First we need to define a schema:

1
2
user=> (def Recipe "a recipe" {:name s/Str})
#'user/Recipe

Then we can use validate on a data structure to make sure it matches

1
2
user=> (s/validate Recipe {:name "Chicken and Rice"})
{:name "Chicken and Rice"}

On success we get back the same structure. However, if we pass an invalid structure

1
user=> (s/validate Recipe {:name 42})

Wait a minute. What is this? an actually helpful error message in Clojure?!?!

1
ExceptionInfo Value does not match schema: {:name (not (instance? java.lang.String 42))} schema.core/validate (core.clj:161)

It tells you the keyword of the invalid value and what value was passed in. Nice.

We can validate, but what does check do?

1
2
3
4
user=> (s/check Recipe {:name "Chicken and Rice"})
nil
user=> (s/check Recipe {:name 42})
{:name (not (instance? java.lang.String 42))}

Looks like you’d use check when you are prepared to deal with nil, and validate when a representation of the data structure is needed.

Now lets try a more complex example.

1
2
3
4
user=> (def Source {:name s/Str :url s/Str :year s/Int})
#'user/Source
user=> (s/validate Source {:name "Food Network" :url "http://www.foodnetwork.com" :year 2015})
{:name "Food Network", :year 2015, :url "http://www.foodnetwork.com"}

This example is Source which consists of 3 elements. Lets try leaving one off:

1
2
3
user=> (s/validate Source {:name "Food Network" :url "http://www.foodnetwork.com" })

ExceptionInfo Value does not match schema: {:year missing-required-key} schema.core/validate (core.clj:161)

This makes me think maybe there is a way to indicate an optional value? Reading down farther on the documentation I see how to make keys optional. Lets redefine our Source with year as optional:

1
2
3
4
5
user=> (def Source {:name s/Str :url s/Str (s/optional-key :year) s/Int})
#'user/Source

user=> (s/validate Source {:name "Food Network" :url "http://www.foodnetwork.com" })
{:name "Food Network", :url "http://www.foodnetwork.com"}

Sweet! I also read that since Schemas are just data structures they are composable. Lets add Source to Recipe to make a more complex structure.

1
2
3
4
5
6
7
user=> (def Recipe "a recipe with source" {:name s/Str :source Source})
#'user/Recipe
user=> Recipe
{:name java.lang.String, :source {:name java.lang.String, :url java.lang.String, #schema.core.OptionalKey{:k :year} Int}}

user=> (s/validate Recipe {:name "Chicken and Rice" :source {:name "FoodNetwork" :url "www.foodnetwork.com"}})
{:name "Chicken and Rice", :source {:name "FoodNetwork", :url "www.foodnetwork.com"}}

Nice!! Since I’m a ruby developer by day, I think how could I do this in ruby? Well, in Rails, we have validations on models and once you set the attributes you could call model.valid? which would return boolean. If it’s not valid, it populates a errors attribute on the model with a nested hash of key/error messages. Composing two together is not as straightforward. You might be able to with the accepts_nested_attributes_for on a Recipe model, but I conclude its not going to be as elegant as using Clojure and Schema :)

Using lein-try made it easy to experiement with a library and poke around to practice :)