June was a challenging month. Which is probably normal for this stage of an ambitious project. When you start a new project it’s exciting, and when you finish a product it’s satisfying, and during both of those times you tend to get a lot of positive outside reinforcement from other people who are interested in what you’re doing. But the middle is the hard part. The middle is where all the work happens, and you have to keep grinding away and pushing through obstacles even when it turns out to take longer and be more work than you thought it would. Hello?
I would describe FilmLab as being toward the end of the middle, and last month I experienced plenty of the end-of-the-middle blues. I got sick and lost most of a work week, and I spent part of the month time working on other projects for clients (I’m doing part-time contract work to keep the bills paid until FilmLab ships), which slowed down progress. Then, when I started to do QA testing on frame detection system I’d been working on since April, I discovered some problems that forced me to rethink the design.
Meanwhile, there are a lot of people waiting for FilmLab to be done, with varying degrees of (im)patience. When the work is difficult and there’s a pile of messages asking why it’s not done already, it can be hard to stay positive, especially when you’re working by yourself.
But those things come with the territory. My job right now is to stay focused on the end goal, and continue to push through the list of things that need to get done to get there. So that’s what I tried to do in June.
Reliable frame detection is hard
Back in April I wrote about my decision to use traditional hand-coded algorithms for identifying film frames, instead of state-of-the-art machine learning. My initial results (using a small collection of test images of my own film and slides) made it seem like this approach was going to be acceptably accurate. I confidently went ahead with the work of getting the code running on mobile, integrated with my real time tracking system, and optimized for speed. But after all that, I found that it did poorly in real-world use. I could get it working well enough on some rolls of film, but then it would do badly on others, and trying to fix it was like playing whack-a-mole: every fix seemed to create a new problem.
It turns out I made the wrong call here. This kind of complex problem, where it’s very difficult to describe a set of rules that will work perfectly in every situation, is exactly what machine learning is good at. By trying to do it the old way, I picked the wrong approach. So I’m making a change, and replacing the frame-detection part of FilmLab with a trained neural network. This should work better then my hand-coded algorithms, and continue to work better over time as I train it with more data.
Preview 0.4.0 for iOS
Despite not having a the new frame detection engine ready, I went ahead and released version 0.4.0 for iOS last week. I wanted to give testers a glimpse of where the FilmLab UI is going. The big changes you’ll notice is that FilmLab now has a capture button, like most camera apps, and has a crop overlay that you can manipulate in real time to control what you’re going to capture. Here’s how it works in action:
Although the automatic frame detection isn’t there yet, it’s nice to have eliminated one of the major pain points of using FilmLab, which is that you never quite knew what you were going to capture when you tapped the screen. Now you get a real-time, accurate preview of what the crop is going to be, and you can correct problems yourself ahead of time.
Design update
Building the 0.4.0 iOS release helped me clarify an important part of the user interface design for this app. In the original proof-of-concept builds of FilmLab, which I demoed in the Kickstarter video, there were three different states the UI could be in:
- Live view of everything the camera saw, with no controls
- Live view zoomed in on a single frame, with a manual focus helper and a capture button
- View of a captured image, with export button
For the 0.1 release and subsequent preview releases, I tried to simplify this by going to two screens:
- Live view of everything the camera sees, cropped to fill the screen of your device
- View of a captured image, with edit controls and export button.
But in practice, this simplified design doesn’t feel very good. It’s slow and heavy to have to switch over to the full editor just to be able to fill the screen with one frame, and in current builds there’s an annoying blackout while you wait for the editor to load. So I’ve been kicking around ideas for a better UI for a while.
When I started testing the 0.4.0 build with the live crop tool, the lightbulb went off: The live view and editor don’t have to be separate screens at all! There really only needs to be one screen, with all the controls at your fingertips, whether you’re viewing a live camera feed or a saved image. Transitioning from the video feed to a captured image should be instant, with the only difference being that the resolution increases. I’ve been working through the designs for this, and I think it solves a lot of the big UI problems I had been grappling with. I’ll starting to implement this design in the 0.4.1 release for iOS soon. Please let me know what you think once you get a chance to try it.
Android status
UI work takes a lot of time, so I’m waiting to get a few more things figured out using iOS as my testbed before I build the new UI for Android. All my current work is being done with Android in mind though: I’m building all the core technology with cross platform code, and planning ahead for the Android implementation. In the end, this approach is going to result in both the iOS and Android versions getting done more quickly than if I was taking time to build out the UI on both platforms for every experimental change. But Android users, I’m sorry that it’s taking so long to get you a fresh build.
That’s it for now. Thanks for your continued interest in and support of FilmLab. I also want to say how much I truly appreciate it when you guys take time to leave an encouraging comment. Your support really helps me stay positive and keep going!
-Abe