How I wrote DroPub in two days

Yesterday I wrote DroPub – a simple but powerful little OS X application which transparently handles file transfers “from the desktop”.

Even though it has a lot of features, have been tested, updates itself and so on, I only spent about two days on the whole project – for me, this is the essence of Cocoa.

DroPub is heavily based on NSOperations and uses a hierarchy model for structuring operations. NSOperation hierarchies are powerful means for writing most types of “service” applications. The code can easily be followed by a Cocoa programmer and the operating system frameworks and libraries can give really good performance.

Practical implementation using NSOperationQueue

In almost all cases one process-global NSOperationQueue is enough and makes things much easier for you. In prefix.pch we declare the instance and allocate it in main.m:

NSOperationQueue *g_opq;

int main(int argc, const char *argv[]) {
  g_opq = [[NSOperationQueue alloc] init];
  NSApplicationMain(argc, argv);
  return 0;
}

Note that the code snippets presented here are condensed versions of the actual code, for illustrative purposes

DroPub operation hierarchy

For each folder which is watched in DroPub, there is one NSOperation called DPSupervisor. Whenever the main “thread” detects that a folder should be watched (might have been added to the configuration or changed location), it calls startSupervising: which starts a new DPSupervisor:

DPSupervisor *sv = [[DPSupervisor alloc] initWithApp:self conf:conf];
sv.delegate = self;
[g_opq addOperation:sv];

Note here how g_opq refers to the global NSOperationQueue mentioned earlier. The conf argument is simply the per-watched folder configuration containing path, remote host, and so on.

The DPSupervisors main method then looks at the designated folder for new files to appear:

- (void)main {
  while ( !self.isCancelled && conf ) {
    // [sets up a NSDirectoryEnumerator here]
    while (filename = [dirEnum nextObject]) {
      // [continues if the file matches certain criteria (isn't hidden etc)]
      if (![filesInTransit containsObject:path])
        [self sendFile:path name:filename];
    }
    sleep(1);
  }
}

The supervisor uses a NSMutableSet (filesInTransit in the code above) to keep track of which files are currently in transit. Here the question of robustness might occur — yes, this is actually a very robust construction. Since the nature of the application is to atomically transfer (to a temporary location then mv once completed successfully) files, so if a operation crashes or if the whole app crashes (oh noes!) the file will simply be transferred again a few seconds later or when the application is restarted. The only danger here is if we mess with our NSMutableSet of files in transit, then the worst case scenario is probably corrupt files, so let’s not mess with it.

Next step is dispatching yet another NSOperation, subordinate to the DPSupervisor, which in DroPub is called DPSendFileOp. This is done in DPSupervisors sendFile:name:

- (void)sendFile:(NSString *)path name:(NSString *)name {
   // [make sure we can get an exclusive lock on the file here, 
  //  otherwise try again later]
  [filesInTransit addObject:path];
  DPSendFileOp *op = [[DPSendFileOp alloc] initWithPath:path name:name conf:conf];
  op.delegate = self;
  [g_opq addOperation:op];
}

As you might have figured out, DPSendFileOp takes care of the actual transmission and reports back to it’s parent (technically its delegate) DPSupervisor.

The main method of DPSendFileOp is too comprehensive for pasting here in this article but this is a summary of what it does:

1. Starts a new subordinate operation which watches the file being sent for deletion (called DSFileExistenceMonitor).
2. Constructs a temporary filename for the transfer -- basically the original name prefixed with ".dpupload_".
3. Constructs the SCP program invocation arguments.
4. Executes the SCP program and supervises the process I/O and status -- this is where the actual transmission of the file is taking place and this step might take a long time.
5. The DSFileExistenceMonitor is cancelled.
6. If the transfer was successful (i.e. not interrupted or corrupted) a remote mv is done over standard SSH.

If the file is removed while being transferred DSFileExistenceMonitor will notify DPSendFileOp which will interrupt (by signalling) SCP and then notify it’s delegate through fileTransmission:didAbortForPath:. On the other hand, if the transfer is successful the DPSendFileOps delegate is notified through fileTransmission:didSucceedForPath:remoteURI: in which case the parent DPSupervisor will move the successfully uploaded file to Trash.

Cancellation

Since we use a hierarchy of operations – running in parallel, thus we can not use the dependency system of NSOperation – we need to take care of cancelling child operations. We do this by keeping an NSMutableArray in each parent in which we store references to any subordinate tasks which have started and not yet exited. When the parents cancel method is invoked, we simply propagate the message to our children:

- (void)cancel {
  for (NSOperation *op in childOperations)
    [op cancel];
  [super cancel];
}

User interface

I’m not going to talk much about the user interface in this article, but to sum it up DroPub uses bindings, key-value coding and key-value observation to communicate with the “background” parts.

The interface was built almost completely in Interface Builder.