Update: Otto has been updated and is now available again. Go grab Remote and Antenna!

Over the weekend, I released a minor update to Otto’s Remote which fixed a crash that, while a bit off the beaten path, was probable enough that at least one of my users encountered it¹. There was one change to Remote: a one line fix for the crash. Before submitting, I tested the same binary destined for the App Store, as I always do. At the time, it worked. Unfortunately, something happened between submitting the update and it being approved.

The most likely change being iOS 7.0.3, which came out in that time period. Regardless of the actual cause, when Remote 1.3.1 was released, I immediately began receiving reports that the app was complaining that users were not logged in to iCloud. I gave the usual advice of logging out and logging back in to iCloud. It is a pain, but in the past it has solved 90% of the problems that have been reported to me. I also tried it myself, and was distressed to discover that, this time, it did not help.

I immediately removed Otto’s Remote and Otto’s Antenna from the App Stores. I have no interest in selling a broken product.

Then I set about to figure out what went wrong. My first step was to test the binary I had submitted to the App Store, signed with an ad-hoc certificate. This is the same process I went through before submission, in fact I didn’t even recreate the binary but used the copy I keep archived of each release. This same binary which was working fine a week before could no longer access iCloud. This is what makes me believe the problem is new to iOS 7.0.3.

Regardless of where the problem came from, though, I needed to fix it. A look at my iPhone’s console log revealed that the app’s request to use iCloud was being denied due to not having the proper entitlements. But the entitlements were fine, and were of course unchanged from before. I tried a number of things, regenerated provisioning profiles for example, but nothing seemed to make a difference.

Then, on a whim, I changed the app’s entitlements to hard code my team identifier prefix instead of letting Xcode fill it in for me². Suddenly, Remote began to work again. This made very little sense to me, so I dug in to see what was different between the working and non-working app bundles. Unsurprisingly, the embedded provisioning profile was identical. There was no sign that anything related to my team identifier prefix was any different.

However, CodeResources inside _CodeSignature had changed quite a bit. Of course there were usual differences in the checksums for xib files, which is to be expected. But there were also two whole new sections, files2 and rules2, which hadn’t been there before. There is a brief mention of these two new sections in a thread in the developer forums (here). The gist is that these two sections form a new resource seal. The Apple engineer providing that information specifies that this is for OS X 10.9, though, and there is no mention of iOS 7.

Regardless, I had a working build, so it has been submitted and is awaiting approval. As soon as Apple approves it, I will release the update and make both apps available on the store again.

I have not seen any other reports of apps failing to access iCloud on iOS 7.0.3 and it is unclear what, if anything, I’ve been doing wrong with Otto’s Remote. I do not think very many apps depend on the iCloud key-value store as heavily as Remote does. Perhaps apps are failing to access the kv-store, but people aren’t noticing? Or perhaps most people hard code their team identifier prefix in their entitlements file, and have been getting these new CodeResources sections already. Unfortunately I’m not sure that I will ever know.

A possible lesson here: If Apple releases an update to their OS between when you submit your app for approval and when it is ultimately approved, even a sub-minor update like 7.0.3, it is probably a good idea to hold off on pressing that “release” button until you’ve verified it still works. In this case, I had set things up to automatically release the update on approval. I won’t do that again.

Since version 1.3 still works, I attempted to use Apple’s new iCloud download version settings to make 1.3 available to current customers who had already updated to 1.3.1. The tool works by providing developers a list of all versions of their app, with a checkbox next to each one. Versions with the checkbox marked can never be downloaded by end users, while any version without the checkbox marked is eligible. When the user goes to download the app from the Purchases section in the App Store app, the theory goes, the App Store app will download the most recent unchecked version of the app that can run on the current device. It is not possible to mark the version which is currently available in the store as ineligible for download. It is, however, possible to do so if the app has been removed from the App Store, as is the case with Otto’s Remote. When I discovered that, I had hoped the App Store app would allow current customers to download version 1.3. After several hours, the App Store app did stop letting me download 1.3.1 but instead of offering me 1.3, it simply stated that the app was not available anymore. I still had some hope that this was a data propagation issue, and 1.3 would eventually become available. Unfortunately it has now been more than two days and nothing has changed. This feature did not work as I had hoped and I filed a bug with Apple. It is unlikely it will even get looked at before 1.3.2 is released, but hopefully they will see how useful the behavior I was looking for would be and fix it at some point in the future.

To my existing customers who downloaded 1.3.1 and are now unable to use the app, I sincerely apologize. I am not trying to point fingers here, though it probably reads that way. I accept that there may have been more I could have done to prevent this issue from happening. But I wanted to share this information, both so my customers know what is going on and hopefully to help any other developers who may encounter a similar issue. I will do my best to try to prevent something like this from happening again.

Thank you for your patience.