I can see the moon from here

I moved to a new house recently. There’s much to love about it and many horror stories about how long it took to get here. I might write about it, another day.

But tonight, I was going upstairs and I saw the moon through the roof windows.


Whistler before sunrise

I’m in Whistler for the week and the jet lag has me waking up at odd hours, so I went out for a walk and took some pictures.


Last week I spent some days with my team in Rome. Now I want to go back with a better camera. This is what I was able to capture with my phone.

Attaching patches to Pull Requests

This might sound strange, but sometimes I prefer patches to pull requests. The main scenario is when I’m reviewing someone else’s code and I want to propose an alternative implementation.

I could just create a new branch and pull request with my change, but then the conversation is split between two PRs, and there’s a new branch that you have to clean up.

When the change is small enough, or I’m not sure if it will be accepted, I’d rather send a patch. So far I’ve been doing git diff, uploading the result to gist, and posting the link as a comment in the PR. This has a few shortcomings:

  • No binary support.
  • If the original author wants to use it, authorship is usually lost, unless they use the --author option for git commit, and even then there’s room for typos.

I know there’s a better way, as Git was originally designed to share patches, not pull requests. I think I’ve been avoiding it because it’s not as common and the original author might not know what to do with the patch. So I’m writing this as a quick tutorial.

Creating a patch

Before creating a patch, you have to commit your changes. git format-patch will create a patch file for each commit, so your history can be preserved. Once you have a commit, your branch is ahead of origin, so we can use that to tell format-patch which commits to pick

branch=git rev-parse --abbrev-ref HEAD
git format-patch $origin

This will leave one or more .patch files in your project directory:

$ ls *.patch

Upload those to Gist and leave a comment with the link on the PR:

$ gist -co *.patch

Applying a patch

For a single patch, you can copy the Raw link in the Gist and download it

$ curl -sLO https://gist.github.com/koke/1b30d861e6bb9d366f69bc186d0e9525/raw/8cc27f3e589a7823b2e9f1746aa921b92da14187/0001-Store-relative-paths-for-reader-topics.patch

If there are multiple files, make sure you use the Download Zip link (or download all the files one by one):

$ curl -sLo patches.zip https://gist.github.com/koke/ab100907c17c4ef6a977350494679091/archive/3fb0136a21a6bc499bff2511750c62ae6dc41630.zip
$ unzip -j patches.zip 
Archive:  patches.zip
  inflating: 0001-Store-relative-paths-for-reader-topics.patch  
  inflating: 0002-Whitespace-changes.patch  

Once you have the patch file(s) in your project directory, just run git am -s *.patch:

$ git am -s *.patch
Applying: Store relative paths for reader topics
Applying: Whitespace changes

Review the changes, and if you’re happy with them, git push them. Otherwise, you can reset your branch to point at the pushed changes:

branch=git rev-parse --abbrev-ref HEAD
git reset --hard $origin

Finally, run git clean -df, or manually remove the downloaded files.

Composing operations in Swift

Continuing on From traditional to reactive, the problem I’m solving today is refactoring our image downloading system(s).

If I remember it correctly, it all started a long time ago when we switched to AFNetworking, and started using its UIImageView.setImageWithURL methods. To this day I still feel that there’s something terribly wrong in a view calling networking code directly, but that’s not today’s problem. Instead it’s how this system grew by adapting to each use case with ad-hoc solutions.

If I haven’t missed anything, we have:

  • AFNetworking’s setImageWIthURL: you pass a URL and maybe a placeholder, and it eventually sets the image. It uses its own caching.
  • UIImageView.downloadImage(_:placeholderImage:), which is basically a wrapper for AFNetworking’s method which sets a few extra headers. Although it’s not very obvious why it’s even there.
  • A newer UIImageView.downloadImage(_), which skips AFNetworking and uses NSURLSession directly. This was recently developed for the sharing extension to remove the AFNetworking dependency. It creates the request, does the networking, sets the result, and it adds caching. It also cancels any previous requests if you reset the image view’s URL.
  • Then we have downloadGravatar and downloadBlavatar which are just wrappers that take an email/hash or hostname and download the right Gravatar. At least these were refactored not long ago to move the URL generation into a separate Gravatar type. Although it seems the old methods are also still there and used in a few places.
  • WPImageSource: basically all it does is prevent duplicate requests. Imagine you just loaded a table to display some comments and you need to load the gravatars. Several of the rows are for the same commenter, so the naive approach would request the same gravatar URL several times. This coalesces all of the requests into one. This doesn’t really have anything to do with images in theory, and could be more generic, although it also handles adding authentication headers for private blogs.
  • WPTableImageSource: this one does quite a few things. The reason behind it was to load images for table view cells without sacrificing scrolling performance. So it uses Photon to download images with the size we need. If Photon doesn’t give us what we want (if the original image is smaller than the target size, we don’t want to waste bandwidth) we resize locally. We cache both original and resized images. It also keeps track of the index path that corresponds to an image so we can call the delegate when it’s ready, and it supports invalidating the stored index paths when the table view contents have changed, so we don’t set the images in the wrong cell.

These are all variations of the same problem, built in slightly different ways. What I want to do is take all of this into composable units, each with a single responsibility.

So I built an example for one of the simple cases: implement a UIImageView.setImageWithURL(_) that uses Photon to resize the image to the image view’s size, downloads the image and sets it. No caching, placeholders or error handling, but it customizes the request a bit.

This is what the original code looked like approximately (changed a bit so the example is easier to follow):

The problem I encountered is that Photon has an issue resizing transparent images, so for this I want to skip photon and do the resizing on the device. If you want to do that with the original code I see two options: duplicate it, or start adding options to the function to change its behavior. I’m not sure which option is worse.

The first step is to break the existing method into steps:

  1. “Photonize” the URL so we are asking for a resized image
  2. Build the request
  3. Fetch the data
  4. Turn it into an image
  5. Make sure it’s an image, or stop
  6. Set the image view’s image to that

So I’ve extracted most of those steps into functions and this is what I got:

This is much more functional, even if it looks sequential. It’s written that way because I find it’s easier to read, but you could write the same thing using nested functions.

If you’re into Lisp, you might find this version more pleasing, but in Swift this feels harder to follow than using intermediate variables. This is why other languages have things like the compose operator which you can use to compose functions without all that nesting. When I followed this route, I hit some limitations on Swift generics, and it also looks very foreign. This might work well in Haskell where all functions are curried, but it’s not so much in Swift.

Also note that the pattern of nesting breaks on fetchData as we don’t have a result yet. I would hardly call that callback hell, but we can start moving in that direction easily if we say we want all the image processing to happen in a background queue. Or if image resizing was an asynchronous operation.

Then I tried with RxSwift. All I wanted for this could be done with a much simpler Future type, since I don’t need side effects, cancellation, or any operator other than map/flatMap. But I already had RxSwift on the project so I’m using it as an example of the syntax. I also added a second version with the transform chain grouped into smaller pieces to improve readability.

What I haven’t tried yet is a solution based on NSOperation, but I have the feeling that it would add a lot of boilerplate, and wouldn’t feel completely right in Swift.

Finally, what I think I’d build is something based on the traditional version with customization points. I’d love to flatten the data pipeline and be able to just keep mapping over asynchronous values without depending on an external framework. For this example though, it seems that callbacks don’t complicate things much, at least yet. So I think I’ll start from something like this.

Here’s a gist with all the examples and the helper functions: ImageServices.swift

If you have a better design, I’d love to hear about it.

From traditional to reactive

Brent Simmons has a series of posts about Reactive vs “Traditional” which sound a lot like the arguments we’re having internally as a team lately. I’ve gotten into RxSwift in the past months and believe it’s a great solution to a number of problems, but it is indeed a big dependency and has a steep learning curve.

I find RxSwift declarative style much easier to read and follow than the imperative counterparts, but it took a while to get there. I remember struggling for months learning to think in terms of streams, and I was really motivated to learn it.

But I thought I’d take some time to take the example in Comparing Reactive and Traditional, as I don’t think it’s a fair comparison (and not in the way you might expect). You can download all the steps in a playground: Traditional to Reactive.playground.

My first “refactor” was just to clean up the code enough to compiled without errors, so I can put it on a playground.

Then I addressed what I found most confusing about the traditional implementation, which is all that throttling logic scattered around the view controller. There’s no need to go reactive there, we just need to contain all that logic and state in one place: Throttle. This is a generic class that takes a timeout and a callback, and you can send new values or cancel. It will call callback after timeout seconds since the last input unless a new input happens. Note that this class is generic and doesn’t need to know anything about the values. It is also easily testable.

This leaves us with a simpler view controller, but we still have some logic in there that’s not really specific and could be extracted: keeping only the last request and canceling the previous one. I’ve called this SwitchToLatest, and it couldn’t be simpler: it just has a switchTo method that keeps a reference to the task, and calls cancel on it whenever there’s a new one. It uses a new Cancelable protocol since we don’t really care about what the task is, we just need to know that we can cancel it. Also really easy to test.

And this is how the final ViewController looks:

Here’s where I think the fair comparison starts. The traditional version has all the behaviors we want in units with a single responsibility, and the view controller takes these building blocks to do what it needs.

What RxSwift brings to the table is the ability to compose these units and a common pattern for cancellation. What you can’t do here is take Throttle and pipe its output into SwitchToLatest. But in this case, that’s fine.

The live search example is typically a nice example of reactive code as it showcases all of it’s features: streams of values, composition of operators, side effects on subscription, and disposables/cancellation. But as we’ve seen, it doesn’t work as a way for reactive code to prove its value, as the traditional counterpart is simple enough.

Compare this to the thing I was trying to write in RxSwift.

  • We have some data that needs to be loaded from the API. That data might be used by a few different view controllers.
  • When a request fails because of network conditions, it should be automatically retried a few times before we give up.
  • We want to poll for new data every minute, counting since the last successful refresh.
  • If a request is taking more than a few seconds, the UI should display a message indicating that “this is taking longer than expected”.
  • If the request fails because an “unrecoverable” error or we give up on retrying, the UI should display an error and we shouldn’t try to poll for data until the user manually refreshes again.
  • The UI should show a message when it detects it’s offline, and stop any polling attempts. When we detect we’re online we should resume polling. (I later learned that Apple doesn’t recommend using Reachability to prevent network requests unless they’ve already failed).
  • If a second VC wants the same data, there should not be a second request going out. Not just caching, but reusing in-progress requests.

When written in RxSwift it was already a complex logic to follow, and I even had to write a couple of custom operators (pausable and retryIf). There’s no silver bullet, and that’s also true for RxSwift, but I believe it can help. I can’t imagine reasoning about all the possible states in a more traditional design.

I also would argue (although someone might disagree) that any time you use NSNotificationCenter, NSFetchedResultsController, or KVO, you are already doing some sort of reactive programming, just not in a declarative way.

Apple on reusing

There was an Apple Event today for the press where they announced new iPads and iPhones.

An interesting thing about the event is that they spent around ten minutes (out of sixty) talking about the environment, how they use renewable energy, and how they work on reusing and recycling their products. They mentioned that the vast majority of iPhones that come back in, get reused in some way.

Everything was awesome and Apple looked like it did a fantastic job in the green category. For a second I almost forgot about planned obsolescence, or how their business is based on people buying new stuff all the time.

Fast forward a bit, and here’s Phil Schiller on stage talking about why they chose the size for the new iPad to target existing PC users, and he says this:

There are over 600 million PCs in use today that are over 5 years old. This is really sad.

Phil Schiller on Stage

I recently replaced a 4 year old MacBook Pro only because I needed all the performance I could get for development work, but it was perfectly functional for less intensive tasks. My previous 5 year old iMac was sold to a friend who still uses it frequently. We also just sold a 6 year old MacBook Air that also worked fine. I even keep a 12 year old iBook that would work fine if it weren’t for the broken AC adapter. My mother uses my 6 year old iPhone 4. I fail to see how any of this is sad other than it means less money for Apple.

Learn me a Haskell

Lately I’ve become interested in functional programming, and Haskell always felt like the thing I should learn, so this week I finally started reading  Learn You a Haskell. So far I’ve been through the first six chapters and they do a great job at indroducing many concepts.

Pattern matching got me hooked. I had read the Patterns chapter in the swift book, and a couple blog posts on the subject, but they always felt like little more than “switch with ranges”. After reading the pattern matching chapter in Learn You a Haskell, suddenly Patterns appeared much more useful, even in Swift.

And then I read some of the concepts around higher order functions, how all functions are curried, how trivial partial application is, and how concise and clear everything reads once you get past the syntax choices. My mind was blown time after time.


One of those days…

I wake up early, ready to start the week with a fresh mind and debug that issue that drove me crazy on Friday. I pick up from were I left in Xcode, and hit run.

It slowly builds and installs the app on my iPad, then promptly shows the launch screen, then more launch screen. I try again, change some things, reconnect the iPad. One hour later I manage to run the app.

I set a few breakpoints in the debugger to figure out what’s going on. When I hit one, I inspect the call trace and Xcode crashes. Relaunch, debug, inspect, crash again. Repeat 3-4 times.

I yell at the screen, and decide to take a deep breath and walk for a bit to relax. I come back, five minutes later, to a dead mouse that doesn’t seem to work. I change the batteries, I try all the batteries in the house. I clean the contacts. It seems dead.

I try some of those tricks where you shut down your computer, press ctrl-option-shift-power and then start. Nothing. Also, my MacBook Pro won’t even start if the Thunderbolt Ethernet Adapter is plugged in.

I disconnect that, restart, and reconnect. I’m greeted by a couple dialogs asking for permissions, and an app wants to install an update. Nope, not now.

I give up on the Magic Mouse, and go back to the G500 which still works, although it sometimes double-clicks when you click.

I started working 4 hours ago, and so far I got nothing done, for reasons beyond my control.

Some days, I really hate technology.