Wednesday, April 3, 2013

blazing fast epeg photo resize on raspberry pi

I posted about speed tests and optimizations in imagemagick resize speed on raspberry pi. And I asked for suggestions for improvements or alternatives on raspberrypi stackexchange.

blazing EPEG

Krzysztof Adamski suggested I try out epeg on github, which uses libjpeg, and is touted as being "An IMMENSELY FAST JPEG thumbnailer library API".

I just did a few tests, and it does indeed seem insane-fast: I'm seeing ~1.1 second to generate a good looking 16KB image at the same 10% target size first attempt; In comparison to the fastest imagemagick option, "sample", the result is 6 times faster, is around 1/3 of the filesize (60KB) and looks about the same. The quality can be reduced further, eg to "50" using the quality parameter, which executes in the same amount of time (~1 second) but makes for a significantly smaller image (16KB) which looks similar, albeit less vibrant. Amazing results!

take a look

... with resize.jpg taking ~25.5s, scale.jpg ~5.7s and the epeg version taking just ~1.1 second (average of 3 iterations).

epeg & libjpeg setup/compile/install

There's no binary, so needed to compile. Here're the relevant steps/resources:


  • epeg is Fantastic Fast :)
  • stackexchange's raspberrypi forum is great! (as is Krzysztof Adamski)

Monday, April 1, 2013

imagemagick resize speed on raspberry pi

update: alternative approach using library epeg described in later post: blazing fast epeg photo resize on raspberry pi.


  • Looking to simplify and improve the way photos are managed, especially in a scenario without a laptop/desktop computer in the home (eg, mobile devices only)
  • Part of the solution is to upload all images to Flickr, but uploading full-res images takes time;  100 photos @ 8 MB's is 800 MB's ... taking ~110 minutes / 2 hours.
  • Ideally, we'd upload small-size previews first and then upload the high-res asynchronously assets overnight.

Fastest Possible Resize

  • I'm looking primarily for speed; need preview of ~100KB from an original of about 8MB
  • quality is less important; I'll upload the preview to Flickr straight away, but asynchronously I'll replace the preview with the high-res version later/overnight
  • on a Mac Pro laptop, imagemagick's normal "resize" command took about a second - on the Rasperry Pi it took about 32 seconds. Over 100 photos, that's 53 minutes ... at which point, we might as well just wait and upload the high-res assets.

Changing the variables

I ran a lot of small tests, changing a range of variables I hoped would make a difference. Here's what I found

What Really Matters

Here's the short version of what matters, in order of importance
  • the size of the original photo (obviously) matters;
    • "resize" on a 8.1 MB photo was ~44 secs and and 2.4 MB photo just 8.5 secs. a size increase by 340% resulted in a duration increase of 520%
  • the specific ImageMagick resizing command matters;
    • "sample" and "adaptive-resize" both took only about 25% the duration of "resize", with "scale" being in the middle
  • the target size matters, and not just the smaller the faster;
    • it seems the relationship is non-linear: a 10% "resize" (31.7s) was faster than both 2% (49s) and 50% (44s). Though "sample" was ~7s for both 2% and 10%.
  • overclocking the Pi will indeed give you more cycles;
    • The default is 700 MHz, and moving up to "High/950MHz" reduced the run from 7.3 secs to 5.9 secs

What Does Not Matter

  • the "-quality" variable; indeed a low value creates poor JPG's, but the duration is about the same. There is a difference in the size of the output (btw 30KB and 160KB) but the quality is very poor at 30 KB, and would be preferably to use a smaller target size.
  • location of the image; tried it on both the SDCard which boots RPI, and on a USB stick
  • GPU/CPU split; tried both 64MB and 256MB for the GPU
  • whether desktop is running (tried disabling booting direct into desktop)
  • running as "root" instead of "pi"


Here's a resized version of the photo used (resized by Blogger)

The ImageMagick resizing options
(8.1 MB JPG, to 10% size)
(resize: 136KB / scale: 134KB / sample: 167KB / Thumb: 109KB / Adaptive: 164KB)

Relatively the methods behave the same on Mac/Raspberry Pi

Image size & Duration

Target size matters, and not linearly

Overclocking works - from the default 700 MHz to the next-highest 950MHz

Quality option does not matter for speed

... but quality option does matter for size

Does not matter if user is "pi" or "root"

And does not matter is desktop is running or not

And it matter very little how the GPU / CPU split is setup

Interestingly, the medium didn't matter; whether (fast) USB stick or (fast) boot SDCard


  • tests done via a ruby script which executed each command 3 or 5 times and calculated the average speed. Running a few commands directly on command line, with "time" command suggested that ruby wrapper did not add noticeable overhead.


Friday, March 29, 2013

Gigabit & 5 GHz

Old router was getting flaky ... Good occasion to upgrade to gigabit (should help with Iomega NAS).

Sunday, August 5, 2012

Test Automation with auto cropped iOS screenshots in Frank/Cucumber HTML reports

The ability for anyone, at any time, to inspect the logic of any test and visually inspect the results gives confidence; seeing the iOS screenshots along with the nicely readable Gherkin steps provides additional validation of succeeding tests, and provides a helpful tool for diagnostics for failing ones.

Frank's Step Definition Compendium has long had a simple step which supports doing screenshots, but the shots include the whole screen, and if something else happens to be in the foreground while your test-suite is running then you're not getting the screenshots you were hoping for.

Cropper with Frank HTML reports

Oliver Fürniß created a great little Mac tool; iOS-Simulator Cropper allows you to easily and quickly take screenshots without the need for manual cropping. And it takes the correct screenshot even if the simulator is not in the foreground on the screen. Fortunately, Oliver was also kind enough to add a little command line interface.

Add a bit of code and you have a solution which addresses three problems with the current step definition for screenshots:
  • screenshots are automatically cropped to only include the simulator
  • screenshots work even if the simulator is not in the foreground (another window is "on top")
  • thumbnail images are added to Cucumber HTML test reports, and allows click-through to a full-size image

Build- and Test-automation with Pete's Olympics project

Pete Hodgson's did a great worked example on how to hook up an iOS project to Jenkins with Frank/Cucumber using Bundler and Rake. I extended it to include Oliver's Cropper tool with a new implementation of the screenshot step definition as well as some additional steps in the Rake build script.

The output

Here are a few shots of the output:

When you click through on a thumbnail, you get a full-size image:

And it's all available from within Jenkins' artifacts:

Step definition

The step definition syntax is the same as the original one from the Compendium; that allows for easy replacement. The code will use the Cropper tool if it's available, and will fall back to the standard screenshot tool if not (either way, the screenshot will be included in the report).

For the Cropper tool to work, you'll need the Cropper .app the directory you're executing your tests from (typically, "Frank" if you're executing command line without CI).

I've forked Pete's project, and updated the two features to take screenshots, added the step definition (screenshot_steps.rb), and also added the Cropper tool. Cropper is included as a .zip, so all you need to do is unzip it.

My fork also fixes a build issue I experienced, seemingly as a result of the source-code directory including a " " space; the master fork is just fixing that problem. Whereas the "screenshots" branch on that fork includes the enhancements for screenshots.
... here's a direct link to the screenshot_steps.rb file is here: Frank/features/step_definitions/screenshot_steps.rb

Jenkins CI

I also forked Pete's CI project, so if you have a CI setup you can just check out the "screenshots" branch of my fork.
It has the project source-code (and Frank/Cucumber) as a submodule; it points to my forked screenshot branch, so hooking that up to Jenkins should be straightforward.

I extended the rake file by adding two steps:

  • extract the .zip which contains the Cropper tool (having it as a .app subdirectory in SCM felt wrong; too many files)
  • move the screenshots into the ci_artifacts directory after the run.

The extract is a bit messy; it copies it to two places, in order to make it available to executions of Rake and via Cucumber (in app/Frank subfolder). Getting the Cropper command line to accept absolute paths proved tricky, so instead it's always called from the working directory, which is why it's being extracted to two locations. Admittedly, that Rake file is not pretty - but it proves the concept.


  • Would be great to have a cleaner Rake file
  • A proper wrapper on top of the Cropper tool would be nice - and it'd be nice to be able to manage it as a dependency, instead of having it in SCM
  • Figure out how to run the Cropper tool from other location than the current working directory; perhaps it's related to the spaces included in the tools name (and Contents of it's .app dir) - or perhaps there's a better way to call it from Ruby
  • Consider how to more appropriately add the screenshots to the report; at the moment it's done via "puts" which seems hackish; maybe a custom formatter. Ideally something that would also be compatible with junit output etc.


here's the source, copy/paste from github (Aug 5, 2012):

Sunday, December 4, 2011

15km run today: my longest so far

I did my first 15 km run today :)

No record-setting pace; I got through it by keeping the pace low; took me a total of 1h and 40m to get back.

I borrowed Ida's heart-rate monitor for the run, making sure I wasn't over-stretching myself. Wanted to share the Nike+ Maps illustration of the run.

Thanks to Matt, Bobby, Dale, Diptamay, Dan, Lambert and Tim for getting that stuff live in the first place :)

Route & top-level stats

Pace, elevationand heart-rate stats

Monday, August 1, 2011

PMI is catching up with Agile

The PMI is regaining some relevance by embracing Agile as part of it's official certification process; the new buzz acronym is "PMI-ACP", and while in beta/pilot stage the fully-spelled-out-term is "PMI-Agile Certified Practitioner (PMI-ACP) Pilot Program.

The first certifications are due out in Q4 this year, with the  examination becoming available in September.

New Scrum rule book is out; July 2011 edition

Ken Schwaber & Jeff Sutherland released an updated Official Scrum guide in July. It now only contains the core rules of the game; the "best practices" have been taken out, allowing the many varieties of adoption discussed in books/blogs.

The benefit is an increased focus on what is required in order to be allowed to use the phrase "Scrum" for what you're doing. At least according to Schwaber & Sutherland