Introductory Examples

(CMS revise version for Julia v1.3)

Contents

This is lecture is a slightly modified version of https://lectures.quantecon.org/jl/julia_by_example.html Thank you to the amazing Quantecon.org team!

Level

We are now ready to start learning Julia!

Our approach is aimed at those who already have at least some knowledge of programming — perhaps experience with Python, MATLAB, Fortran, C or similar

In particular, we assume you have some familiarity with fundamental programming concepts such as

  • variables
  • arrays or vectors
  • loops
  • conditionals (if/else)

Approach

In this lecture we will write and then pick apart small Julia programs

At this stage the objective is to introduce you to basic syntax and data structures

Deeper concepts—how things work—will be covered in later lectures

Since we are looking for simplicity the examples are a little contrived

In this lecture, we will often start with a direct MATLAB/FORTRAN approach which often is poor coding style in Julia, but then move towards more elegant code which is tightly connected to the mathematics

Set Up

We assume that you’ve worked your way through our getting started lecture already

In particular, the easiest way to install and precompile all the Julia packages used in QuantEcon notes is to type ] add InstantiateFromURL and then work in a Jupyter notebook, as described here

Other References

The definitive reference is Julia’s own documentation

The manual is thoughtfully written but is also quite dense (and somewhat evangelical)

The presentation in this and our remaining lectures is more of a tutorial style based around examples

Example: Plotting a White Noise Process

To begin, let’s suppose that we want to simulate and plot the white noise process $ \epsilon_0, \epsilon_1, \ldots, \epsilon_T $, where each draw $ \epsilon_t $ is independent standard normal

Introduction to Packages

The first step is to activate a project environment, which is encapsulated by Project.toml and Manifest.toml files

There are three ways to install packages and versions (where the first two methods are discouraged, since they may lead to package versions out-of-sync with the notes)

  1. add the packages directly into your global installation (e.g. Pkg.add("MyPackage") or ] add MyPackage)
  2. download an Project.toml and Manifest.toml file in the same directory as the notebook (i.e. from the @__DIR__ argument), and then call using Pkg; Pkg.activate(@__DIR__);
  3. use the InstantiateFromURL package
In [1]:
#using InstantiateFromURL

# activate the QuantEcon environment
# activate_github("QuantEcon/QuantEconLecturePackages", tag = "v0.9.5");

If you have never run this code on a particular computer, it is likely to take a long time as it downloads, installs, and compiles all dependent packages

This code will download and install project files from GitHub, QuantEcon/QuantEconLecturePackages

We will discuss it more in Tools and Editors, but these files provide a listing of packages and versions used by the code

This ensures that an environment for running code is reproducible, so that anyone can replicate the precise set of package and versions used in construction

The careful selection of package versions is crucial for reproducibility, as otherwise your code can be broken by changes to packages out of your control

After the installation and activation, using provides a way to say that a particular code or notebook will use the package

In [2]:
using LinearAlgebra, Statistics, Compat

Using Functions from a Package

Some functions are built into the base Julia, such as randn, which returns a single draw from a normal distibution with mean 0 and variance 1 if given no parameters

In [3]:
randn()
Out[3]:
-1.1487888956197125
In [4]:
# Other functions require importing all of the names from an external library
using Plots
gr()
n = 100
ϵ = randn(n)
plot(1:n, ϵ)
Out[4]:
0 25 50 75 100 -1 0 1 2 y1

Let’s break this down and see how it works:

  • The effect of the statement using Plots is to make all the names exported by the Plots module available
  • Because we used Pkg.activate previously, it will use whatever version of Plots.jl that was specified in the Project.toml and Manifest.toml files
  • The other packages LinearAlgebra and Statistics are base Julia libraries, but require an explicit using
  • The arguments to plot are the numbers 1,2, ..., n for the x-axis, a vector ϵ for the y-axis, and (optional) settings
  • The function randn(n) returns a column vector n random draws from a normal distribution with mean 0 and variance 1

Arrays

As a language intended for mathematical and scientific computing, Julia has strong support for using unicode characters

In the above case, the ϵ and many other symbols can be typed in most Julia editor by providing the LaTeX and <TAB>, i.e. \epsilon<TAB>

The return type is one of the most fundamental Julia data types: an array

In [5]:
typeof(ϵ)
Out[5]:
Array{Float64,1}
In [6]:
ϵ[1:5]
Out[6]:
5-element Array{Float64,1}:
 -0.3608635828288956 
 -0.08710774991589583
 -1.416975967297102  
 -1.706498605491414  
  0.31536896454230956

The information from typeof() tells us that ϵ is an array of 64 bit floating point values, of dimension 1

In Julia, one-dimensional arrays are interpreted as column vectors for purposes of linear algebra

The ϵ[1:5] returns an array of the first 5 elements of ϵ

Notice from the above that

  • array indices start at 1 (like MATLAB and Fortran, but unlike Python and C)
  • array elements are referenced using square brackets (unlike MATLAB and Fortran)

To get help and examples in Jupyter or other julia editor, use the ? before a function name or syntax

?typeof

search: typeof typejoin TypeError

Get the concrete type of x.

Examples

julia> a = 1//2;

julia> typeof(a)
Rational{Int64}

julia> M = [1 2; 3.5 4];

julia> typeof(M)
Array{Float64,2}

For Loops

Although there’s no need in terms of what we wanted to achieve with our program, for the sake of learning syntax let’s rewrite our program to use a for loop for generating the data

Note

In the current version of Julia v1.0, the rules for variables accessed in for and while loops can be sensitive to how they are used (and variables can sometimes require a global as part of the declaration). We strongly advise you to avoid top level (i.e. in the REPL or outside of functions) for and while loops outside of Jupyter notebooks. This issue does not apply when used within functions

Starting with the most direct version, and pretending we are in a world where randn can only return a single value

In [7]:
# poor style
n = 100
ϵ = zeros(n)
for i in 1:n
    ϵ[i] = randn()
end
println(ϵ)
[-0.30815435083312737, 0.02426292233857655, 0.615253151696944, -1.3940291429800995, 0.13440227479242972, 0.43225591874258806, -0.5136366171206908, 0.05292697602824824, -0.5661727951362692, 1.1151780914464418, -0.24784028296528818, -2.4928402873471045, 0.4195432753539271, -1.4533155201932828, 1.2934205531380938, 0.6781271486730983, 0.091961670765624, 0.8381454649148846, 0.7796148180458423, -0.3097041385682623, 0.2542285728712798, -1.1623932342612007, -1.2746228067750793, -1.4362990903921014, 0.2152814689091812, 0.15831116090317562, 0.8269208655502857, -0.7702562898208652, -0.5223888852444695, -1.293257805072937, -0.9864815780698514, -0.3606510500453297, 1.2706126399292148, 0.016821797744042768, -0.03154975083701419, 1.1300558967898306, -0.5553053936173132, -0.1562771615256552, 0.10744395568893617, -0.7068325303217645, -0.052773185345403914, 0.17109957623140293, -1.0721611793825105, -1.7204809429220262, -0.8230949892176541, 1.8894642813447806, 0.629646329239019, -0.17917011213077055, 2.067543229260119, -0.14073553504598021, -0.14412631663957468, -0.8255006142467441, 0.9798789046791085, 1.202289391890049, 0.32845623308359456, -0.912555306577757, -1.4903322531481957, -0.2674238598718943, -1.1136017208556734, -0.273138409696701, -1.1890734340651443, -1.3463843811015472, 0.1122338998116031, -0.35550573729571877, 0.15687004158430135, 0.10851510093133586, 0.8216627934140627, -2.059847576149282, 0.014944641681499001, 0.6924289056810071, -2.6797616657501058, -2.266763176101336, 0.7093273307352757, 1.1501741540328998, 0.8000973261988643, -0.6106279751737539, -1.6802036414756243, -2.0519977494569535, -0.2782802711038282, 1.2669316727854183, 1.1293697189646439, -1.5914712736164145, 0.28451366918618587, -1.299792617021147, 0.13104551129014705, 1.26215661847093, -1.4412782973788953, -0.17987191381467296, -0.5293275830044714, -0.4490952532659069, 0.7104601298030679, 0.5508697756167947, 0.9367819760839153, 2.1516059569244055, -0.6408024135475817, 1.1916860356800603, 0.9607329699967216, -0.48498499892630814, 0.6300526563348312, -0.0520491413853623]
  • Here we first declared ϵ to be a vector of n numbers, initialized by the floating point 0.0
  • The for loop then populates this array by successive calls to randn()
  • Like all code blocks in Julia, the end of the for loop code block (which is just one line here) is indicated by the keyword end
  • The word in from the for loop can be replaced by either or =
  • The index variable is looped over for all integers from 1:n – but this does not actually create a vector of those indices
  • Instead, it creates an iterator that is looped over – in this case the range of integers from 1 to n
  • While this example successfully fills in ϵ with the correct values, it is very indirect as the connection between the index i and the ϵ vector is unclear
  • To fix this, use eachindex
In [8]:
# better style
n = 100
ϵ = zeros(n)
for i in eachindex(ϵ)
    ϵ[i] = randn()
end
println(ϵ)
[0.4646291730126219, -0.8432503784771712, 0.2738409491054763, 0.343594848937201, 0.01911117336446484, 1.4753932055039753, -0.933730472578213, -2.089211862250145, -1.4315283589419971, 0.5612118122735503, 1.7453137831634533, 1.899401319712275, 0.09883108222064778, -1.87446683328077, 1.0581432045707448, 2.178180486195132, 1.502687071792402, 0.8542202368512711, 0.5891523802478491, 0.40103855393505494, -0.31288965774587557, 1.3500053607878557, -1.1261415853650643, 0.2367034675701962, 0.08957852925980897, -1.6611245725584387, -0.42389549311633395, -1.6658924227481653, 1.0423613604729915, -1.2504775965219757, -0.260892546789619, 1.1780567396823798, 0.7059868361513896, -0.5270459372643019, -1.0052824256727833, 0.1791282531510668, 1.3701645396538895, 0.7962035880861759, -0.23640169483015522, -0.6801932851436712, 0.499162232751448, 1.3991310503368248, 0.7814001993765615, 0.9934475519182419, -0.801572084059252, -1.8685297203037903, -0.1854108723663389, -2.176283939564053, 0.24776883434157762, 1.170611373873387, -1.6250363691401835, -0.7087033908862421, -1.123867292066476, -1.2931341634642342, 1.8045764306451162, 0.5634761436117647, 0.23139888107058162, -0.07113217546181684, 0.6136810585741377, 1.471913857015736, -2.243363242649479, 1.1623853420782386, -0.684343746010731, 0.11977547949335807, -0.35046775649318895, 1.2457094193988276, 1.2572168870099334, -0.13526408727783906, 0.3275701578717554, -0.35320039192167385, 0.2641115192211678, -0.154375769534323, -0.8142865187150424, 0.46103868897177747, 1.431304297938316, 1.2111836660241266, -0.8294794363575057, -0.5538803083880655, -1.3840063242603688, -1.1801186769472658, 0.6007258545371206, 0.3184721815558555, 0.3921489008560035, -0.3267435066415791, -0.6948725273285367, 1.1974857859243053, -1.378027413195794, -1.1602887039926748, 0.7380926651078488, -0.010157030121777762, 0.1273834377002879, 1.2251104556771781, -0.27089152760010726, -0.4614746831700812, 0.4955730545105568, -0.16899784068618706, 1.4396550980299236, 0.18615628637130835, 1.2218418038959982, -0.58438835992166]

Here, eachindex(ϵ) returns an iterator of indices which can be used to access ϵ

While iterators are memory efficient because the elements are generated on the fly rather than stored in memory, the main benefit is (1) it can lead to code which is clearer and less prone to typos; and (2) it allows the compiler flexibility to creatively generate fast code

In Julia you can also loop directly over arrays themselves, like so

In [9]:
ϵ_sum = 0.0 # careful to use 0.0 here, instead of 0
m = 5
for ϵ_val in ϵ[1:m]
    ϵ_sum = ϵ_sum + ϵ_val
end
ϵ_mean = ϵ_sum / m
Out[9]:
0.051585153188518564

where ϵ[1:m] returns the elements of the vector at indices 1 to m

Of course, in Julia there are built in functions to perform this calculation which we can compare against

In [10]:
ϵ_mean  mean(ϵ[1:m])
ϵ_mean  sum(ϵ[1:m]) / m
Out[10]:
true

In these examples, note the use of to test equality, rather than ==, which is appropriate for integers and other types

Approximately equal, typed with \approx<TAB>, is the appropriate way to compare any floating point numbers due to the standard issues of floating point math

User-Defined Functions

For the sake of the exercise, let’s go back to the for loop but restructure our program so that generation of random variables takes place within a user-defined function

To make things more interesting, instead of directly plotting the draws from the distribution, let’s plot the squares of these draws

In [11]:
# poor style
function generatedata(n)
    ϵ = zeros(n)
    for i in eachindex(ϵ)
        ϵ[i] = (randn())^2 # squaring the result
    end
    return ϵ
end

data = generatedata(10)
plot(data)
Out[11]:
2 4 6 8 10 1 2 3 4 5 y1

Here

  • function is a Julia keyword that indicates the start of a function definition
  • generatedata is an arbitrary name for the function
  • return is a keyword indicating the return value, as is often unnecessary

Let us make this example slightly better by “remembering” that randn can return a vectors

In [12]:
# still poor style
function generatedata(n)
    ϵ = randn(n) # use built in function

    for i in eachindex(ϵ)
        ϵ[i] = ϵ[i]^2 # squaring the result
    end

    return ϵ
end
data = generatedata(5)
Out[12]:
5-element Array{Float64,1}:
 1.9986442319952185 
 0.8433530161503876 
 0.14166697469756018
 0.59667975011095   
 0.29684959584880855

Broadcasting: the dot .

While better, the looping over the i index to square the results is difficult to read

Instead of looping, we can broadcast the ^2 square function over a vector using a .

To be clear, unlike Python, R, and MATLAB (to a lesser extent), the reason to drop the for is not for performance reasons, but rather because of code clarity

Loops of this sort are at least as efficient as vectorized approach in compiled languages like Julia, so use a for loop if you think it makes the code more clear

In [13]:
# better style
function generatedata(n)
    ϵ = randn(n) # use built in function
    return ϵ.^2
 end
data = generatedata(5)
Out[13]:
5-element Array{Float64,1}:
 0.0014355606070317545
 0.29925112774872886  
 1.6533684000720275   
 0.6459309576099332   
 0.4912607393556788   

We can even drop the function if we define it on a single line

In [14]:
# We can even drop the `function` if we define it on a single line
# good style
generatedata(n) = randn(n).^2
data = generatedata(5)
Out[14]:
5-element Array{Float64,1}:
 0.22419493627733902 
 4.129697046085374   
 0.6725354362570525  
 0.026153036135931163
 0.373313418891394   
  • Finally, we can broadcast any function!
  • The function ^() is only one case.
  • We can broadcast your own user-defined functions as well.
In [15]:
# good style
f(x) = x^2 # our own simple square function
generatedata(n) = f.(randn(n)) # uses broadcast for some function `f`
data = generatedata(5)
Out[15]:
5-element Array{Float64,1}:
 0.0008360824228594497
 0.3010592827261743   
 0.7190318869858531   
 0.21982317447858513  
 0.1933729185766364   

As a final – abstract – approach, we can make the generatedata function able to generically apply to a function

In [16]:
generatedata(n, gen) = gen.(randn(n)) # uses broadcast for some function `gen`

f(x) = x^2 # simple square function
data = generatedata(5, f) # applies f
Out[16]:
5-element Array{Float64,1}:
 0.004357321040685262
 0.7767288203071202  
 0.005501136521339242
 1.190400896663905   
 1.624167979315186   

Whether this example is better or worse than the previous version depends on how it is used

High degrees of abstraction and generality, e.g. passing in a function f in this case, can make code either clearer or more confusing, but Julia enables you to use these techniques with no performance overhead

For this particular case, the clearest and most general solution is probably the simplest

In [17]:
# direct solution with broadcasting, and small user-defined function
n = 100
f(x) = x^2

x = randn(n)
plot(f.(x), label="x^2")
plot!(x, label="x") # layer on the same plot
Out[17]:
0 25 50 75 100 -2 0 2 4 6 x^2 x

While broadcasting above superficially looks like vectorizing functions in MATLAB, or Python ufuncs, it is much richer and built on core foundations of the language

The other additional function plot! adds a graph to the existing plot

This follows a general convention in Julia, where a function that modifies the arguments or a global state has a ! at the end of its name

A Slightly More Useful Function

Let’s make a slightly more useful function

This function will be passed in a choice of probability distribution and respond by plotting a histogram of observations

In doing so we’ll make use of the Distributions package, which we assume was instantiated above with the project

Here’s the code

In [18]:
using Distributions

function plothistogram(distribution, n)
    ϵ = rand(distribution, n)  # n draws from distribution
    histogram(ϵ)
end

lp = Laplace()  # an "instance" of a Distribution type. More on this later!
plothistogram(lp, 500)
Out[18]:
-4 -2 0 2 4 0 25 50 75 100 y1

Let’s have a casual discussion of how all this works while leaving technical details for later in the lectures:

  1. lp = Laplace() creates an instance of a data type defined in the Distributions module that represents the Laplace distribution
  2. The name lp is bound to this value
  3. When we make the function call plothistogram(lp, 500) the code in the body of the function plothistogram is run with
    • the name distribution bound to the same value as lp
    • the name n bound to the integer 500

Here's a Mystery

Now consider the function call rand(distribution, n):

  • This looks like something of a mystery, doesn't it?
  • The function rand() is defined in the base library such that rand(n) returns n uniform random variables on $ [0, 1) $:
In [19]:
rand(3)
Out[19]:
3-element Array{Float64,1}:
 0.427975337718536  
 0.9044423497165281 
 0.27749338038178384

On the other hand, we just heard that distribution points to a data type representing the Laplace distribution that has been defined in a third party package.

So how can it be that rand() is able to take this kind of value as an argument and return the output that we want?

The answer in a nutshell is multiple dispatch, which Julia uses to implement generic programming.

  • This refers to the idea that functions in Julia can have different behavior depending on the particular arguments that they’re passed.
  • Hence in Julia we can take an existing function and give it a new behavior by defining how it acts on a new type of value
  • The compiler knows which function definition to apply to in a given setting by looking at the types of the values the function is called on
  • In Julia these alternative versions of a function are called methods

Example: Variations on Fixed Points

Take a mapping $ f : X \to X $ for some set $ X $

If there exists an $ x^* \in X $ such that $ f(x^*) = x^* $, then $ x^* $: is called a “fixed point” of $ f $

For our second example, we will start with a simple example of determining fixed points of a function

The goal is to start with code in a MATLAB style, and move towards a more Julian style with high mathematical clarity

Fixed Point Maps

Consider the simple equation, where the scalars $ p,\beta $ are given, and $ v $ is the scalar we wish to solve for

$$ v = p + \beta v $$

Of course, in this simple example, with parameter restrictions this can be solved as $ v = p/(1 - \beta) $

Rearrange the equation in terms of a map $ f(x) : \mathbb R \to \mathbb R $

$$ v = f(v) \tag{1} $$

where

$$ f(v) := p + \beta v $$

Therefore, a fixed point $ v^* $ of $ f(\cdot) $ is a solution to the above problem, i.e. the value $v^*$ which makes this equation hold.

While Loops

One approach to finding a fixed point of (1) is to start with an initial value, and iterate the map

$$ v^{n+1} = f(v^n) \tag{2} $$

For this exact f function, we can see the convergence to $ v = p/(1-\beta) $ when $ |\beta| < 1 $ by iterating backwards and taking $ n\to\infty $

$$ v^{n+1} = p + \beta v^n = p + \beta p + \beta^2 v^{n-1} = p \sum_{i=0}^{n-1} \beta^i + \beta^n v_0 $$

To implement the iteration in (2), we start by solving this problem with a while loop

The syntax for the while loop contains no surprises, and looks nearly identical to a MATLAB implementation:

In [20]:
# poor style
p = 1.0 # note 1.0 rather than 1
β = 0.9
maxiter = 1000
tolerance = 1.0E-7
v_iv = 0.8 # initial condition

# setup the algorithm
v_old = v_iv
normdiff = Inf
iter = 1
while normdiff > tolerance && iter <= maxiter
    v_new = p + β * v_old # the f(v) map
    normdiff = norm(v_new - v_old)  # "size" of a vector in some space

    # replace and continue
    v_old = v_new
    iter = iter + 1
end
println("Fixed point = $v_old, and |f(x) - x| = $normdiff in $iter iterations")
Fixed point = 9.999999173706609, and |f(x) - x| = 9.181037796679448e-8 in 155 iterations

The while loop, like the for loop should only be used directly in Jupyter or the inside of a function

Here, we have used the norm function (from the LinearAlgebra base library) to compare the values

The other new function is the println with the string interpolation, which splices the value of an expression or variable prefixed by $ into a string

An alternative approach is to use a for loop, and check for convergence in each iteration

In [21]:
# setup the algorithm
v_old = v_iv
normdiff = Inf
iter = 1
for i in 1:maxiter
    v_new = p + β * v_old # the f(v) map
    normdiff = norm(v_new - v_old)
    if normdiff < tolerance # check convergence
        iter = i
        break # converged, exit loop
    end
    # replace and continue
    v_old = v_new
end
println("Fixed point = $v_old, and |f(x) - x| = $normdiff in $iter iterations")
Fixed point = 9.999999081896231, and |f(x) - x| = 9.181037796679448e-8 in 154 iterations

Using a Function

The first problem with this setup is that it depends on being sequentially run – which can be easily remedied with a function

In [25]:
# better, but still poor style
function v_fp(β, ρ, v_iv, tolerance, maxiter)
    # setup the algorithm
    v_old = v_iv
    normdiff = Inf
    iter = 1
    while normdiff > tolerance && iter <= maxiter
        v_new = p + β * v_old # the f(v) map
        normdiff = norm(v_new - v_old)

        # replace and continue
        v_old = v_new
        iter = iter + 1
    end
    return (v_old, normdiff, iter) # returns a tuple
end

# some values
p = 1.0 # note 1.0 rather than 1
β = 0.9
maxiter = 1000
tolerance = 1.0E-7
v_initial = 0.8 # initial condition

v_star, normdiff, iter = v_fp(β, p, v_initial, tolerance, maxiter)
println("Fixed point = $v_star, and |f(x) - x| = $normdiff in $iter iterations")
Fixed point = 9.999999173706609, and |f(x) - x| = 9.181037796679448e-8 in 155 iterations

Passing a Function

The chief issue is that the algorithm (finding a fixed point) is reusable and generic, while the function we calculate p + β * v is specific to our problem

A key feature of languages like Julia, is the ability to efficiently handle functions passed to other functions

In [26]:
# better style
function fixedpointmap(f, iv, tolerance, maxiter)
    # setup the algorithm
    x_old = iv
    normdiff = Inf
    iter = 1
    while normdiff > tolerance && iter <= maxiter
        x_new = f(x_old) # use the passed in map
        normdiff = norm(x_new - x_old)
        x_old = x_new
        iter = iter + 1
    end
    return (x_old, normdiff, iter)
end

# define a map and parameters
p = 1.0
β = 0.9
f(v) = p + β * v # note that p and β are used in the function!

maxiter = 1000
tolerance = 1.0E-7
v_initial = 0.8 # initial condition

v_star, normdiff, iter = fixedpointmap(f, v_initial, tolerance, maxiter)
println("Fixed point = $v_star, and |f(x) - x| = $normdiff in $iter iterations")
Fixed point = 9.999999173706609, and |f(x) - x| = 9.181037796679448e-8 in 155 iterations

Much better (closer to math), but there are still hidden bugs if the user orders the settings or returns types wrong

Named Arguments and Return Values

To enable this, Julia has two features: named function parameters, and named tuples

In [27]:
# good style
function fixedpointmap(f; iv, tolerance=1E-7, maxiter=1000)
    # setup the algorithm
    x_old = iv
    normdiff = Inf
    iter = 1
    while normdiff > tolerance && iter <= maxiter
        x_new = f(x_old) # use the passed in map
        normdiff = norm(x_new - x_old)
        x_old = x_new
        iter = iter + 1
    end
    return (value = x_old, normdiff=normdiff, iter=iter) # A named tuple
end

# define a map and parameters
p = 1.0
β = 0.9
f(v) = p + β * v # note that p and β are used in the function!

sol = fixedpointmap(f, iv=0.8, tolerance=1.0E-8) # don't need to pass
println("Fixed point = $(sol.value), and |f(x) - x| = $(sol.normdiff) in $(sol.iter)"*
        " iterations")
Fixed point = 9.999999918629035, and |f(x) - x| = 9.041219328764782e-9 in 177 iterations

In this example, all function parameters after the ; in the list, must be called by name.

Furthermore, a default value may be enabled – so the named parameter iv is required while tolerance and maxiter have default values.

The return type of the function also has named fields, value, normdiff, and iter – all accessed intuitively using .

To show the flexibilty of this code, we can use it to find a fixed point of the non-linear logistic equation, $ x = f(x) $ where $ f(x) := r x (1-x) $

In [28]:
r = 2.0
f(x) = r * x * (1 - x)

sol = fixedpointmap(f, iv=0.8)
println("Fixed point = $(sol.value), and |f(x) - x| = $(sol.normdiff) in $(sol.iter) iterations")
Fixed point = 0.4999999999999968, and |f(x) - x| = 3.979330237546819e-8 in 7 iterations

Using a Package

But best of all is to avoid writing code altogether

In [30]:
# best style
using NLsolve

p = 1.0
β = 0.9
f(v) = p .+ β * v # broadcast the +
sol = fixedpoint(f, [0.8])
println("Fixed point = $(sol.zero), and |f(x) - x| = $(norm(f(sol.zero) - sol.zero)) in " *
        "$(sol.iterations) iterations")
Fixed point = [9.999999999999972], and |f(x) - x| = 3.552713678800501e-15 in 3 iterations

The fixedpoint function from the NLsolve.jl library implements the simple fixed point iteration scheme above

Since the NLsolve library only accepts vector based inputs, we needed to make the f(v) function broadcast on the + sign, and pass in the initial condition as a vector of length 1 with [0.8]

While a key benefit of using a package is that the code is clearer, and the implementation is tested, by using an orthogonal library we also enable performance improvements

In [31]:
# best style
p = 1.0
β = 0.9
iv = [0.8]
sol = fixedpoint(v -> p .+ β * v, iv)
println("Fixed point = $(sol.zero), and |f(x) - x| = $(norm(f(sol.zero) - sol.zero)) in " *
        "$(sol.iterations) iterations")
Fixed point = [9.999999999999972], and |f(x) - x| = 3.552713678800501e-15 in 3 iterations

Note that this completes in 3 iterations vs 177 for the naive fixed point iteration algorithm

Since Anderson iteration is doing more calculations in an iteration, whether it is faster or not would depend on the complexity of the f function

But this demonstrates the value of keeping the math separate from the algorithm, since by decoupling the mathematical definition of the fixed point from the implementation in (2), we were able to exploit new algorithms for finding a fixed point

Anonymous Functions

The only other change in this function is the move from directly defining f(v) and using an anonymous function

Similar to anonymous functions in MATLAB, and lambda functions in Python, Julia enables the creation of small functions without any names

The code v -> p .+ β * v defines a function of a dummy argument, v with the same body as our f(x)

Composing Packages

A key benefit of using Julia is that you can compose various packages, types, and techniques, without making changes to your underlying source

As an example, consider if we want to solve the model with a higher-precision, as floating points cannot be distinguished beyond the machine epsilon for that type (recall that computers approximate real numbers to the nearest binary of a given precision; the machine epsilon is the smallest nonzero magnitude)

In Julia, this number can be calculated as

In [32]:
eps()
Out[32]:
2.220446049250313e-16

For many cases, this is sufficient precision – but consider that in iterative algorithms applied millions of times, those small differences can add up

The only change we will need to our model in order to use a different floating point type is to call the function with an arbitrary precision floating point, BigFloat, for the initial value

In [33]:
# use arbitrary precision floating points
p = 1.0
β = 0.9
iv = [BigFloat(0.8)] # higher precision

# otherwise identical
sol = fixedpoint(v -> p .+ β * v, iv)
println("Fixed point = $(sol.zero), and |f(x) - x| = $(norm(f(sol.zero) - sol.zero)) in " *
        "$(sol.iterations) iterations")
Fixed point = BigFloat[10.00000000000000222044604925031357388532909931412848377287867809936811555686155], and |f(x) - x| = 0.0 in 3 iterations

Here, the literal BigFloat(0.8) takes the number 0.8 and changes it to an arbitrary precision number

The result is that the residual is now exactly 0.0 since it is able to use arbitrary precision in the calculations, and the solution has a finite-precision solution with those parameters

Multivariate Fixed Point Maps

The above example can be extended to multivariate maps without any modifications to the fixed point iteration code

Using our own, homegrown iteration and simply passing in a bivariate map:

In [34]:
p = [1.0, 2.0]
β = 0.9
iv = [0.8, 2.0]
f(v) = p .+ β * v # note that p and β are used in the function!

sol = fixedpointmap(f, iv = iv, tolerance = 1.0E-8)
println("Fixed point = $(sol.value), and |f(x) - x| = $(sol.normdiff) in $(sol.iter)"*
"iterations")
Fixed point = [9.999999961080519, 19.999999923853192], and |f(x) - x| = 9.501826248250528e-9 in 184iterations

This also works without any modifications with the fixedpoint library function

In [35]:
using NLsolve

p = [1.0, 2.0, 0.1]
β = 0.9
iv =[0.8, 2.0, 51.0]
f(v) = p .+ β * v

sol = fixedpoint(v -> p .+ β * v, iv)
println("Fixed point = $(sol.zero), and |f(x) - x| = $(norm(f(sol.zero) - sol.zero)) in " *
        "$(sol.iterations) iterations")
Fixed point = [9.999999999999998, 20.0, 1.0], and |f(x) - x| = 0.0 in 3 iterations

Finally, to demonstrate the importance of composing different libraries, use a StaticArrays.jl type, which provides an efficient implementation for small arrays and matrices

In [36]:
using NLsolve, StaticArrays
p = @SVector [1.0, 2.0, 0.1]
β = 0.9
iv = @SVector  [0.8, 2.0, 51.0]
f(v) = p .+ β * v

sol = fixedpoint(v -> p .+ β * v, iv)
println("Fixed point = $(sol.zero), and |f(x) - x| = $(norm(f(sol.zero) - sol.zero)) in " *
        "$(sol.iterations) iterations")
Fixed point = [9.999999999999998, 20.0, 1.0], and |f(x) - x| = 0.0 in 3 iterations

The @SVector in front of the [1.0, 2.0, 0.1] is a macro for turning a vector literal into a static vector

All macros in Julia are prefixed by @ in the name, and manipulate the code prior to compilation

We will see a variety of macros, and discuss the “metaprogramming” behind them in a later lecture