Year in books 2014

I enjoyed a bunch of books in 2014 (46 in total). My selection process for picking books to read is rather random but this year I could lift a few themes. I feature my favorite books of the year in this post.

Bird by Bird: Some Instructions on Writing and Life by Anne Lamott

Bird by Bird

This is a book about writing and writer's life. Even though I don't write that much I could find plenty of good advice. Encouraging and inspiring (even though some people might find the honesty off-putting) read that also reminded me of the importance of the shitty first draft.

The Obstacle Is the Way: The Timeless Art of Turning Trials into Triumph by Ryan Holiday

The Obstacle Is the Way

Meditations by Marcus Aurelius

Meditations

I've been interested about Stoicism and read two books related to it. Great advice for the rough times and one-liners to spice up your day. I already bought couple other Stoic books so will read more during 2015.

Foundation and Empire & Second Foundation by Isaac Asimov

Foundation and Empire

Second Foundation

On the fiction front I read the second and third installment of the science fiction series Foundation. I enjoy the dry writing style of Asimov. The books are low in action and very dialogue heavy so not for everyone. I love the plots and uncovering the mysteries of psychohistory. I will continue reading the series in 2015.

World War Z by Max Brooks

World War Z

Another fiction pick is World War Z. It is a fictional collection of stories about lives of people after the Zombie war. The stories encompass various nationalities in different life situations told as dialogues with the interviewer of the book. Stories blend together social, economic, religious, political, and environmental themes and there is a clear critique and commentary on the current state of the world.

After reading the book I returned back to the stories in my head and marvelled on all the view points the book included. Event if you don't like zombies, I would recommend it. It's sometimes categorized as horror but I wouldn't label it as such.

Functional JavaScript: Introducing Functional Programming with Underscore.js by Michal Fogus

Functional JavaScript

I didn't read that many technical books this year but from the ones I did, this was my favorite. I've being dabbling with Clojure and this book linked many of those functional programming ideas to JavaScript.

Waking Up: A Guide to Spirituality Without Religion by Sam Harris

Waking Up

Still the Mind by Alan Watts

Still the Mind

On the spirituality front these books were my favorites.

The Innovators: How a Group of Hackers, Geniuses, and Geeks Created the Digital Revolution by Walter Isaacson

The Innovators

This is the book I have talked about the most this year, so I guess that makes it my favorite. It tells the stories of people and teams that have invented the ideas and built the technologies we rely on in today's digital world: programming, microprocessor, personal computer, software, internet and so on. It also observes what are the commonalities of the innovators and tries to lift themes from their stories. While reading the book I also read some of the papers written by the early pioneers. Reading papers written in the 60's and 70's you see how so many parts of the technology puzzle took decades to go from idea to wide usage.

I also found it interesting that when the first interactive computers were invented, the first user interface designers were psychologists. The teams building these interactive machines thought of it as a man-machine symbiosis so it only made sense to put engineers, the experts of the machine, and psychologists, the experts of the human, to the same team to work on the machines.

Here is a trailer of the book:


You can view all the books I reading in 2014 on my GoodReads profile. I wrote similar post also a year ago so you can see what I enjoyed in 2013.

My plan of writing notes or mini-reviews on the books I read failed miserably so I'm doing it properly this in 2015. See you in a year!

Comments

ActiveRecord migrations best practices

There are certain guidelines I follow when writing migrations using the ActiveRecord migrations library. The guidelines have formed during the years I've been working with Ruby on Rails.

Here is an example migration class that follows all the guidelines reviewed in this post:

class AddAreaToPlots < ActiveRecord::Migration
  class Plot < ActiveRecord::Base; end

  def up
    add_column :plots, :area, :float

    Plot.find_each do |plot|
      plot.area = plot.width * plot.height
      plot.save!
    end
  end

  def down
    remove_column :plots, :area
  end
end

Redefine ActiveRecord classes

If possible, ActiveRecord models should not be used in migrations. Migrations should be simple and failproof transformations of database schema and data. Using complex model objects that complicate the migration should be avoided.

If you have strong SQL skills, you can use it for data transformations. Well written SQL is often faster than using ActiveRecord. For schema changes I would use the ActiveRecord methods which are readable and give extra features provided by the framework (most notably the migration/rollback support talked later in this post).

If you want to use the familiar ActiveRecord API to do the data transformations, the ActiveRecord classes should be redefined. As shown in the example, the Plot class has been redefined inside the migration class. The redefinition overrides the model class from the application code and makes it a bare ActiveRecord class. This means it has no validations, associations or callbacks. If some of the callbacks should be called in the migration, those can be reimplemented in the redefined class. Same goes for methods in the model class.

Why go through this trouble? When the application changes, the models change with it. It's possible that the models change in a way that break the migrations. Or the models can be renamed or removed which will also break the migrations.

It might feel weird but it just works, makes your migrations more robust, and protects you from trouble in the future.

Use find_each

When modifying records, Model.find_each should be used to fetch the records in batches. If Model.all is used on a big database table, ActiveRecord will instantiate objects for all rows in the table which will consume all the available memory and the process starts swapping memory like crazy slowing the migration.

By default find_each loads the records in batches of 1000 but that can be configured. Only drawback of find_each is that you can't use order, offset or limit. This is because find_each orders the records by id and then uses offset and limit to control the loading of the batches. Other than these caveats, there are no drawbacks on using find_each. Filtering the records using where works just fine.

Don't change the data using the change method

The change method for migration classes was introduced in Rails 3.1 to make the writing of migrations simpler. Using change and methods that support inversing themselves, you can write migrations like this:

class AddAreaToPlots < ActiveRecord::Migration def change add_column :plots, :area, :float end end

When the migration are run by ActiveRecord, it executes the "up" version of the command adding the area column. On rollback it runs the code and executes the "down" version of the command for all the methods that have it. In the case of adding a column, rollback would remove it.

If you modify your data within the change method, ActiveRecord will run that code on both migration and rollback which is probably not what is wanted. Here is an example:

class AddAreaToPlots < ActiveRecord::Migration
  class Plot < ActiveRecord::Base; end

  def change
    add_column :plots, :area, :float

    Plot.find_each do |plot|
      plot.area = plot.width * plot.height
      plot.save!
    end
  end
end

On rollback the migration would first remove the column and then try to populate the column which would of course fail due to the missing column breaking the rollback.


In addition to the following these guideline it doesn't hurt to understand how the ActiveRecord migrations work. Read the official Active Record Migrations guide to get more information.

Do you have any best practices you follow to make your migrations better? I'd be interested hear about those in the comments.

Addition: Migrations should not contain seed data

Seed data refers to the initial data the database must have for the application to function correctly at all. Rails provides a [mechanism for populating the database with seed data] which should be used. Migrations are meant only to provide the structure of the database, not the contents.

If migrations are used for seed data, there is a chance that a database is created without the seed data. When the incorrect migrations get older and a new database is bootstrapped using rake db:schema:load it uses the db/schema.rb file to get the right structure, but the data from the migrations won't be created in the database.

Thanks for Christian Hellsten for the addition!

Comments

Keep track of test suite run duration with Travis CI

I wanted to view how the test suite run duration has been changing over time on a few projects. The tests are run using the pro version of Travis CI. Travis CI is a hosted continuous integration platform which provides easy to configure way to get a continuous integration server for your projects. Luckily Travis also has an API that exposes the test build information.

So why should you be interested about test duration?

Test duration is important because if running your tests takes too much time it builds friction and you run the tests less often. When you don't run your tests regressions get committed to version control which slows down your other team members. Also if you do test-driven development (TDD) the feedback loop provided by the tests gets too slow to be useful.

If you do the test suite duration comparison often enough you can pinpoint tests that slow the suite down disproportionately to the benefit the added test provides. When you know certain test is especially slow you can think of ways to improving the speed of the test or find a way to test the functionality with another, faster way.

Pulling test suite durations from Travis

Fetching Travis data is easy using Ruby and the official Travis Ruby library. These instructions are for the pro version of Travis which is meant for private repositories. You can also use the same instructions for the open source version of Travis but you need to change the commands and library usage to remove the references to pro.

First install the Travis RubyGem:

gem install travis

Then login to Travis using it:

travis login --pro --auto

Here is the Ruby file for fetching the data:

require 'travis/pro/auto_login'

repo_name = "your-organization/your-project"

builds = Travis::Pro.repos.find(repo_name).first.builds
finished_builds = builds.select(&:finished?)
durations = finished_builds.map {|build|
  [build.started_at, build.duration, build.commit.sha]
}
File.open("stats.txt", "w") {|f|
  durations.each do |duration|
    f.puts duration.join("\t")
  end
}

The program fetches all finished builds for a repository, gets the durations and writes those to a file stats.txt separated by tabs.

To make the data a bit more readable for large projects, lets skip some data points to get to less than 100:

require 'travis/pro/auto_login'

repo_name = "your-organization/your-project"

builds = Travis::Pro.repos.find(repo_name).first.builds

finished_builds = builds.select(&:finished?)
every_n_build = (finished_builds.size / 100.to_f).ceil

smaller_dataset = finished_builds.each_slice(every_n_build).map(&:first)
durations = smaller_dataset.map {|build|
  [build.started_at, build.duration, build.commit.sha]
}

File.open("stats.txt", "w") {|f|
  durations.each do |duration|
    f.puts duration.join("\t")
  end
}

Now you can use your favorite charting tool to get something visual.

Here is a screenshot of a data from one of my apps created using Numbers.app. It was created by opening the file in Numbers.app, selecting the data and selecting bar chart from the Chart menu.

Travis CI build statistics

Comments

Guest at the Frontend Friday podcast

If you understand Finnish, you can enjoy an episode of the Frontend Friday podcast with the theme of Hypermedia APIs. I was happy to be invited to discuss about what Hypermedia APIs are, why I find them interesting, how would you start implementing them and other Hypermedia API related topics.

You can download the episode and view the shownotes at the Frontend Friday website.

Comments

Exercise is a canary of a coal mine for the body

I've noticed that for me exercise works as a canary of a coal mine for the body. When the body is getting overstressed, it is first noticeable during exercise.

Things like flu, small aches and stiffness in the body are normal in human life. Urban life and stationary jobs strain the body. Usually these smaller things pass on their own and you don't pay too much attention on them. But sometimes those are a signal of something bigger. A disease that needs treatment or overstress that you should react to.

For me exercise works great on separating the signals that are meaningful and need to be reacted to. Exercise stresses the body in different way than home and work life. I do CrossFit exercise twice a week and have been doing for three years. If for many workouts in succession I don't feel normal, it is usually a sign that it is not the normal everyday things going on. I notice it from being out of breath easily and muscle pain after workouts being greater and taking longer to pass.

I also follow my workout results (using our own workout journal app WODconnect). My memory for results is horrible so having a place for the results that makes them comparable over time is very valuable. Sometimes not being able to get the results I have been previously been capable of is also a sign of something more severe (sometimes it is just being tired from previous workouts or something else).

Listening to your body is never stupid but it is especially important when exercising.

Comments

More in the archive...