Intro to Concurrency

in Clojure

Arnold Matyasi

github.com/loganisarn

Clojure Budapest meetup at
Prezi - House of Ideas
clj-bp - Jun 10. 2014

What is Concurrency?

  • Concurrency:
    • the coincidence of events or space (different things happening at the same time)
  • Parallelism:
    • the execution of operations concurrently by separate parts of a computer or by separate multi-core nodes
  • Thread:
    • the smallest sequence of programmed instructions that can be managed independently
    l

Where is Data?

  • Places:
    • memory, registers, storage
  • Concurrency:
    • Multiple threads on One Core / Multiple Cores / Multiple Nodes
  • Context:
    • Read / Write

1970s vs 2010s

  • Attitude of 70s-80s:
    • memory is small and expensive
    • storage is small and expensive
    • machines are precious, dedicated resources
  • A different approach:
    • new memory use new places
    • new records use new places
    • "in-place" changes encapsulated by constructors
  • Garbage collected languages:
    • stop referring to old unused records

Clojure values

  • Immutable - never change
  • Read-able - read lock is unknown
  • Maybe lazy - 1 TB of values
  • Cacheable (forever!)
  • Can be arbitrarily large (add 1 record to 1 million record address book)

What can be a value?

References or Refs

  • Refer to values (or other references)
  • Permit atomic, functional succession
  • Can model time and identity
  • Compatible with a wide variety of update semantics
  • Changes from one value to another: My weight
  • More on Refs

Epochal Time Model

  • Observers do not coordinate

Example i=i+1

(def i (atom 0))
; reference constructor
; syncronous, uncoordinated change
		
(swap! i + 1)
; +: pure function, 1: args
; swap!: atomic succession of changing values
		

More Examples

(def person (atom {:id 1}))
(swap! person assoc :name "Rich")
(deref person)
@person
		
(def number-later (promise)) ;promise is a kind of ref
(realized? number-later)
(deliver number-later 42)
@number-later
		

Key terms

|            | Async     | Sync   |
|------------+-----------+--------|
|Coordinated |           | Ref    |
|Independent | Agent     | Atom   |
		

Agent

Asynchronous single object store

(def i (agent 0))
(send i inc)
@i ; 1

(def person (agent {}))
(send person assoc :name "Bond")
@person; {:name "Bond"}  		  
		

Atom

Synchronous non transactional storage

(def i (atom 0))
(compare-and-set! i 1 2) ; false                                                
(compare-and-set! i 0 1) ; true                                                 
@i ; 1
(swap! i + 1) ; 2
		

Ref

Synchronous STM managed coordinated change

(def i (ref 0 :validator integer?))
@i; 0                                                                           
(dosync (ref-set i 1))
(-> (ref-set i 1) dosync)
(-> (ref-set i "a") dosync) ; Invalid reference state                           
; see: alter and commute 
 		

Distributed Atoms and Refs

Avout provides a distributed version of Clojure's MVCC STM.

AVOUT Atom

(use 'avout.core)
(def client (connect "127.0.0.1"))
(def a0 (zk-atom client "/a0" 0))
@a0
(swap!! a0 inc)
 		

AVOUT Ref

(def r0 (zk-ref client "/r0" 0))
(def r1 (zk-ref client "/r1" []))
(dosync!! client
  (alter!! r0 inc)
  (alter!! r1 conj @r0))
@r0
@r1
 		

Babel of Languages

  • Which programming language should I use?
    • Who uses any fortan library?
    • Who uses MATLAB, Mathematica, NumPy, SciKit-learn, Incanter or R?
    • What languages was used to write JVM (HotSpot)?
    • What language was used to write Python, CPython, Jython, Iron Python, PyPy?

The right tool for the right job

  • Competency of a language in a problem domain like concurrency or parallelism?
  • Your own competency in a language and its libraries
  • Number of competent people in your team
  • Number of lines needed to express a solution for a certain problem

Summary

  • No read-locks : thanks to immutable data structure
  • Writes are new immutable values
  • Single core, multi core and multi node concurrency paradigm works

Rich Hickey's Ant Simulator

Concurrent Ants Video

Thank you!

Credits: Rich Hickey, Stuart Halloway, David Edgar Liebke and Clojure Community