Author Archive for LeoJweda

Complexity in Software

When it comes to complexity, somewhere between Hello World and Google exists every other piece of software ever written. What makes one piece of software more complex than another? Lines of code might be an obvious answer especially considering the link above but I believe that complexity causes an increase in the number of lines of code not the other way around.

The number of components that need to play well together is directly proportional to complexity. Web applications need a database, a back end, and a front end. The front end is rendered, it asks the back end for data, the back end retrieves the data from the database, formats it, and sends it to the front end. Let’s think of software in terms of MVC. Even though the front end and the back end both seem to utilize a variation of MVC, for our purposes, we’ll consider the database to be the model, the front end to be the view, and the back end to be the controller. Most features involve changing all 3 components and they all need to work well, and work well together, for the feature to work. This adds complexity. Not only because there’s code whose sole purpose is to communicate with the other layers, but also because you need to test all three components together.

This applies to all software of the same category though but some pieces of software are inherently more complex than others. Let’s take the current side project I’m working on. It is, in its core, a very CRUDy application. Users are presented with things, they can save said things, and the things they saved are presented back to them. Simple, right? Well, sure, but that’s just the techy elevator pitch. The things need to be presented nicely, they need to be laid out in a meaningful way, the pages need to make sense, the things need to have other meta data associated with them, the things need to be imported from 3rd parties, and there needs to be a way to keep the things up to date. The list goes on, and on, and on.

The number of features in a piece of software is an indicator of complexity but it’s not enough on its own since features have different complexities. A list of featured items is not as complex as a recommendation engine. In the former, data needs to be retrieved from the database, aggregated, and displayed. A recommendation engine, assuming one is written from scratch, requires certain usage metrics to be saved and an algorithm to be written that determines what people are interested in with a lot of planning involved.

If lines of code are caused by complexity, and features don’t correlate 1:1 with complexity, then what determines complexity in a piece of software? Requirements do. Features have multiple requirements. Generally speaking, the more complex a feature is, the more requirements it has and/or the more complex each requirement is.

Let’s take the two example features above as an example. Both features display a set of things to the user. The first, a list of featured things, might have the following requirements:

  • Featured things must refresh every 10 minutes.
  • Featured things are things that are saved most recently by users.

Now let’s look at the recommendation engine’s requirements:

  • Each user sees a recommendation based on their own activity.
  • Similar things are determined based on what users save together.

They both have two requirements, how can we say that one is more complex than the other? That’s because requirements don’t map directly to code. The second requirement here involves creating new tables, code to save user activity, and a relatively-complex query to determine which items are saved together. More granular requirements and subtasks of a feature provide these details.

Once we get to very granular details like “create users table”, “create API endpoint to sign in”, “create signin page”, we have a pretty good idea of what the complexity will be but almost no one bothers to create such granular subtasks beforehand and that’s why when a developer sets out to build a “simple” application, they underestimate the complexity involved. This is also what makes estimating so hard but that’s another topic for another day.

Shiny New Things

A conversation with a coworker today made me realize that it’s been 6 months since my last post so I figured I should give an update.

Outside Tasks is still something I’m interested in doing and would like to get to. However, for the past few months I’ve been working on another project that got me excited. Initially I thought it was going to be a quick project that would serve two purposes:

  1. Help me (re)learn Rails.
  2. Fill a gap that I’ve identified in a certain category of websites.

I still think it is a relatively-simple project. However, I ended up switching the front end from using plain Rails CoffeeScript, to using React once Rails 5.2, to Vue.js once I realized that React doesn’t support two-way binding and their argument for it wasn’t convincing. The benefit of all the switching is that I got exposure to frameworks that I wouldn’t have been exposed to otherwise.

Just like individual features that makes them take longer than estimated, when it comes to developing a simple application, the devil is in the details. The reason I think it’s simple is because of functionality I got out of so little code. One could argue that that’s a perk of frameworks but I maintain that it is a measure of simplicity. Every developer thinks they can develop a Twitter clone in a weekend, and they probably could. However, after they develop the core functionality, they would spend a lot of time making it as functional, feature rich, and secure as Twitter.

The reason why I haven’t blogged about the new project is because, so far, the only time someone commented on blog was in an interview and the new project is NSFW in nature. I don’t mind discussing it in public here but given my website is on my resume, I don’t want to risk it becoming unprofessional.

When it comes to the future of this blog, I need to figuring out its goals and audience. However, more immediately, I need to decide on the content I will post since I don’t want to abandon it. As I see it, I have three options:

  1. Dedicate some time to work on Outside Tasks and continue my posts about it. This will make me progress slowly in both projects but it will keep both projects active, I will be making progress on both, and, if I lose interest in one, I can continue to focus on the other.
  2. Give my new project a code name and talk about it in ambiguous terms. This would allow me to discuss the details of the technical challenges I face and resolve.
  3. Post about the bigger picture challenges I face that pertain to software development and steer away from the detailed technical discussions.

I am leaning towards option #3. Also, going forward, I would like to post on a weekly basis instead of the more frequent daily posts I attempted last time since those became a chore really fast.

Preventing Feature Creep

As we developers work on our software, we get ambitious. We want to add features that are cool that will make life easier for our users. This may seem like a good thing at first, but, in the words of Admiral Ackbar, it’s a trap.

I have come to such point working on Outside Tasks. Having added dependent tasks, I had two tasks that depend on each other. The first task was to ask someone for some information, and the second task was to use the information. The problem is that, in keeping with my original vision for Outside Tasks, the second task shouldn’t be available until it is relevant. However, the second task is not relevant until you receive the information. The challenge is knowing when this task is blocked blocked and when it is no longer blocked.

Currently, Outside Tasks has two ways to defer a task:

  • Defer by date. An example of this is my recent interaction with customer service where they said they only do certain things on Tuesdays and Thursdays which meant that my task was deferred until Tuesday.
  • Defer by task. This is how dependent tasks are created. If task A is blocked by task B then task A won’t be available until task B is done.

The problem is that neither one of these methods of deferring a task is sufficient.

There are a few solutions to deferring the task of acting on the information:

  • Add a manual task is deferred flag. This allows users to mark the task as deferred and then unmark it when they receive the information. However, this is problematic because the task is no longer in their relevant tasks so they might completely forget about it.
  • User defer by date. Deferring the task by a few days will do the trick but it is far from perfect. You may still not have the information by the chosen date in which case you have to defer it again. You may get the information before the chosen date in which case we are back to the problem of the first solution.
  • Defer by person. This solution involves determining if the person has been communicated with. This provides the most accurate results but the most amount of work. Imagine having to interact with APIs for all popular methods of communication and supporting new ones. We’re talking email, Jabber, Discord, WhatsApp, Viber, SMS if an API is available, Facebook Messenger, Snapchat, Instagram, etc… But wait, there’s more! You don’t want the task to be unblocked by the first message that the user receives from that person; they could have a conversation about something entirely different, or the person you are waiting for may ask for clarification. This means that we need some kind of trigger word that marks the task as unblocked. What if the person doesn’t use that word?

Unfortunately, I haven’t figured out how to best solve this issue. My best inclination is to use a combination of defer by date and some kind of view that allows users to quickly see which tasks are blocked and unblock those when appropriate. Additionally, GTD involves performing weekly reviews. My initial goal was to reduce the need for weekly reviews. That is why I created dependent tasks since they allow for planning tasks ahead of time. I believe that, as a compromise, that solution will have to do for now.

Decisions like this slow down development, however, they do not slow it as much as implementing a cool feature just because it is the best one that provides the most convenience to the user.

Since the last update I have added dependent and sub tasks which are displayed in a tab that only shows the tasks that are relevant right now. I have also added projects to the app to help organize tasks a bit more. It is still a long way to go until I have something I am comfortable releasing but I should be able to have a stable alpha version in the next month or so.

DMCA, Easylist, Git, and functionalclam: A Solution

Recently there was a post on /r/programming entitled Ad blockers under attack: DRM through the DMCA.

The gist is that a DMCA claim, usually reserved for copyright infringement, was used to take a domain name (functionalclam.com) out of a blacklist that adblockers use to know which domain names to block. The maintainers of the git repo were forced to remove the domain name from the list.

The thing is that instead of removing the domain from the git history, the repo maintainers created another commit to remove the domain name from the list. The commit had the commit message “M: Removed due to DMCA takedown request”. Any commit in git can be reverted. Unless the repo owners are forced to remove the commit from the git history, users can always revert the commits that remove domain names because of DMCA claims and they will have the complete list of blocked domain names. If all DMCA takedown requests have the same commit message then it will be trivial to find and revert all the commits whose purpose is to remove entries based on DMCA takedown requests.

I went ahead and created a general-purpose shell script that searches for all commits with a particular commit message and revert those commits.

Change the permissions for the shell script by running

 

chmod a+x revert_commits.sh

You can run the script like so:

 

revert_commits.sh "M: Removed due to DMCA takedown request"

Let’s break the command down:

 

git log --pretty=online --grep"$1"

This part gives us a list of all the commits whose commit messages match a particular string. In this case, that string is the first argument to the command. In the example above, that string is “M: Removed due to DMCA takedown request”.

 

awk '{print $1}'

Each line contains both the commit hash and the commit message but we only want the commit hash. This command gives us only the hash.

 

xargs git revert

Finally, we pass the hashes as parameters to git revert which reverts each commit.

I don’t expect this to be used practically by anyone since most ad blockers download their lists automatically and overriding that list manually, if possible at all, is inconvenient to say the least.

PS: This is a general-purpose shell script. I do not condone piracy.

The Downside of Following Tutorials

Since I am learning iOS development as I go, often times I follow tutorials to get things done quickly instead of digging through documentation to figure out how to use a certain class in the SDK. The problem is that tutorials are meant to get you from not having a feature to having it with little regard to code organization and maintainability.

This is fine if you follow tutorials occasionally but I followed quite a few which quickly made me unhappy with the state of certain files. Comparing strings that are displayed on the UI is just bad design because as soon as you introduce internationalization all hell breaks loose.

Should we not follow tutorials? Not at all. However, instead of following the tutorial exactly, one could try to improve on the code as it is written. Alternatively, after the feature is complete but before it is committed one could look at the code and try to optimize it. Of course there’s always the option of not using tutorials and digging through documentation but who has time for that? I guess people that write tutorials.

It’s been two weeks since my last update and I just linked to my blog on reddit so I should give an update on the status of Outside Tasks.

I think I’ve been focusing on the smaller details instead of the big features since progress feels slow even though I am working on it regularly.

Tasks can be created, searched, edited, associated with contexts, and assigned urgency, importance, and effort. Contexts can be created, searched, edited, and assigned a method of activation (a way to activate automatically based on time, location, and device). You have the option to see only tasks that are currently relevant.

What I haven’t done yet: projects, actually implementing filtering by the activation methods, and finding a freaking checkbox because swiping to mark tasks as complete simply won’t work in the widget.

I really need to focus more on big features over little details even though perfecting the app is satisfying. I guess the old adage “Just ship it” is more than just a cliche.

Usability vs Functionality

When developers work on an application, we tend to want to make it do everything that is technically feasible. We want to make everything to be customizable, update in real-time, etc… If you don’t control yourself and try to minimize nice-to-have features, you need really good UX to keep the user from getting confused or overwhelmed. Cramming every feature imaginable in an app without regard to keeping things simple is how you end up with Android. On the other hand, you are not doing your users any favours by keeping your application as simple as possible. Simplicity means fewer features. Features that would make your users’ lives easier.

There needs to be a balance between usability and functionality. I recently faced this decision while working on Outside Tasks. I want to make contexts automatically detect when they are active. The Home context is active when you are at home, the Calls context is active when you have your phone on you, etc… At first it seemed like it is a choice between letting users choose from preset contexts which will provide this functionality and letting users create their own which would let users create any contexts they want but they won’t have the automatic detection functionality. Then I thought what if I let users choose from existing ones which are not customizable (Home is always called Home) and let them create additional customizable ones. This is what Google’s Inbox does with bundles.

Finally, I had an epiphany. What if users can create whatever contexts they want and then they can customize when each context is considered active. After all, so far I only have two methods of detecting when a context is active: location, and device. Once the user chooses which method they want to use they can choose the specific location or device. This solution offers more power to users than either of the other solutions but the options seem simple enough that it won’t be confusing.

Is this the best solution? I don’t know yet. Good thing I’m dogfooding Outside Tasks and will know soon enough if there are any problems with my solution.

Do Not Copy and Paste Code!

If my coworkers read the title they’d think my blog got hacked. I can almost be called a proponent for copying and pasting code. I don’t encourage it but I do do it quite often. My philosophy is: copy and paste code, pay attention, and fix whatever problems you introduce later That is until now. So far, the worst thing copying and pasting code has done is introduce bugs. Today, copying an pasting code caused data loss.

I wanted to take the logic I have in the task classes and use it for contexts. I decided to create corresponding context classes and copy functions from the the task classes to the new context classes. It worked. However, I missed replacing one occurrence of task with context. Probably the most important and impactful one. When persisting data in an iOS app you specify the file path where you want the data to reside. I forgot to replace the word “Task” with “Context” which caused the list of contexts to be written where tasks are. Good thing I have my tasks saved somewhere else. This mistake has taught me a valuable lesson. Fortunately it happened in a personal project and not a work one.

Will I stop copying and pasting code? Probably not entirely, no. I will, however, be more careful, do it more sparingly, and probably not do it at all in critical areas.

Goals for tomorrow are: Add the ability to link tasks to contexts, and finally get around to storing urgency, importance, and effort.

UX and Dogfooding

I woke up this morning thinking “I need to focus the task text box when opening the new task screen”. It’s minor, it’s not significant, and I’m not sure it would annoy many users if that feature wasn’t there. The reason why it matters though is that part of Getting Things Done® (GTD®) is called a brain dump where you are encouraged to dump everything on your mind into your system to be reviewed later. Brain dumps involve adding multiple items in a row; saving users a click per item could make the whole experience a lot smoother.

I didn’t sit down and think “OK, how can the UX be improved?”, it simply popped in my mind. The reason I think is that I use the app a lot while developing it. If you do something over and over again, eventually the little annoyances start bugging you which makes them surface to your consciousness.

This feature took me less than 5 minutes to implement but I believe it will have a significant impact to users. In addition to that, I managed to meet my goals for today: Outside Tasks now allows you to add, edit, and swipe to delete tasks. Additionally, the tasks are now persisted across sessions! I didn’t get around to implementing urgency, importance, and effort editing though.

My goals for tomorrow are: implement urgency, importance, and effort; allow users to swipe right to mark tasks as completed; and allow users to add, edit, and swipe to delete contexts.

I don’t know if the app will have every feature I want it to have by Friday but it will be in a state that allows me to move my tasks to Outside Tasks before my OmniFocus trial expires. I will of course keep a back up somewhere in case I mess something up.

For some reason, Apple’s Start Developing iOS Apps (Swift) guide seems to be wrong. This answer on Stack Overflow provided me with the correct way to implement the cancel button.

GTD and the Eisenhower Matrix

Woohoo! First project to get a post other than the initial post introducing it!

After a day’s worth of work I am able to add tasks to a list of tasks in the Inbox tab. Doing a bit of research about what attributes I should have for each attribute I decided on urgency, importance, and effort. I am contemplating suggesting priority based on these 3 attributes but I can’t find any sources that provide a deterministic way of calculating priority. The Eisenhower Matrix determines what to do about each task based on its urgency and importance which seems to fit the Getting Things Done® (GTD®) model.

  • High importance and high urgency: Do now. These become the next actions.
  • High importance and low urgency: Schedule. These can be scheduled in the calendar.
  • Low importance and high urgency: Delegate. These are delegated and then move to the Waiting For section.
  • Low importance and low urgency: Do later. These go in the Someday/Maybe section.

This system is great but there are two flaws in it:

  1. It only supports low or high urgency/importance. I would like to have the ability to leave tasks on medium urgency/importance if they really are neutral.
  2. It does not take effort into account.

I will think about this some more but it is going to be one of the last things I do.

My focus for tomorrow will be: storing the urgency, importance and effort for each task along with its name; editing and deleting tasks after they’ve been created; and possibly data persistence.

This tutorial was very helpful: Storyboards Tutorial in iOS 9.

Project: Outside Tasks

When I was an undergrad at Trent University I used Google Tasks to organize my to dos including my assignments, quizzes, and exams. Since then, I read Getting Things Done (non-referral) by David Allen which I really liked and I started following the philosophy in organizing my to dos. I also followed the official Evernote for Mac Setup Guide and used Evernote for my to do for years.

Recently, I started looking for a more sophisticated solution for my to dos. In particular, one that is compatible with the Getting Things Done® (GTD®) system, one that has task dependencies, and one that allows me to check tasks off through a Today widget on my iPhone.

I explored multiple options including OmniFocus, Nirvana, and Asana. Sadly, each one had something missing from my requirements list. Either they don’t support task dependencies, or they don’t have a Today widget, or they had everything but they looked terrible and had a really bad UX. That is why I set out to build my own simple to do iOS app!

Before discussing what the app does, here’s a simple introduction to the GTD system. It’s very simple, at least how I use it.

Getting Things Done® (GTD®)

To start with GTD, try to think of everything you have in the back of your mind, if it is something you can act on and it’s something that you can do in less than 2 minutes, then do it right now. If it would take longer than 2 minutes, then either delegate it to someone else and it goes in a Waiting For list, or you defer it. If you defer it then it’s either something you schedule for later (just use your calendar), or you put it in a Next Actions list that can be organized by context (phone call, computer, home, office, etc…) so that whenever you have time to get some things done you go to the context that applies to you now and perform some of the actions that are under that context. You can also stash things in a Someday/Maybe list for things that you want to do someday but are not immediately relevant. Finally, you can store things in the Reference list for things that you simply want to store for later.

You can also organize actions into projects. Projects are outcomes that require multiple actions to complete. GTD recommends reviewing things periodically and updating projects with the next applicable action. I would like to avoid having to do that by planning a project ahead and enqueuing tasks by deferring them until a prerequisite task is done.

Outside Tasks

Here’s my vision for Outside Tasks for now:

  • Data should persist across sessions (duh!).
  • Tasks will be used for actions..
  • Projects will simply be tasks that have been marked as projects and/or tasks that have subtasks in them.
  • Contexts will be tags on tasks.
  • Tasks can be deferred until a certain date.
  • Dependent tasks will be implemented by allowing for tasks to be deferred until another task is complete.
  • The Today widget will show all tasks that are currently actionable.
  • Tasks can be checked off through the Today widget.

Bonus feature:

  • Tasks can be linked to Evernote notes.

Here’s the kicker: I’ve already put all my to dos in OmniFocus before realizing it doesn’t support task dependencies and my free trial expires in 7 days which means that I have 7 days to get this app to a usable state!

Next steps: I have to finish reading Start Developing iOS Apps (Swift). Right now I’m on Implement a Custom Control.