This story originally appeared in The Loop Magazine Issue 18

John Gruber recently suggested that users who are annoyed by “Please Rate This App” panes should leave one-star reviews.

While lots of users rightfully hate these panes, this incendiary suggestion naturally resulted in fallout from the developers I know. For those who make our living on the App Store, a one-star rating is an existential threat. The equivalent would be for someone like Justin Bieber to suggest that people bothered by seeing ads on tech blogs like Gruber’s boycott their advertisers.

Small OS X and iOS software companies live and die by their App Store ratings. If an app is rated one star, nobody will look at it or buy it. If it’s rated five stars, the company will make a decent living.

This system seems fair and good, but there are three big problems with the current rating system. The first is that Apple hasn’t built bug reporting into the App Stores, so customers use one-star ratings as a way of reporting bugs.

“One star: didn’t install for me.”

What can we do for this customer? Well, I can’t contact the people who post ratings, so there is no way for me to actually help the customer with the problem. No, “Okay, sir, I’m sorry you’re having an issue. Is there a chance it’s something with your setup?” just an unhappy customer who feels ignored.

The second problem is that people are much more likely to complain than to praise. If you have an app that crashes for 0.01% of your users, those users are generally not going to quietly ask Apple for their money back. They’re going to post a negative review on the App Store. However, the 99.99% of people for whom it works great are not going to post five-star ratings for the software working as expected. You don’t write a letter to Starbucks every time they don’t burn your coffee.

Now, if an app has 100,000 users, and even 0.01% of them have a problem, the app ends up with ten one-star reviews. Is ten a big deal? Yes, it turns out, because of the third problem: the App Stores only show the ratings given to the latest version of an app. Every time a minor update is released, the app’s ratings history is effectively wiped out. If you look at the reviews of Delicious Library 2 right now, you can see that our latest version has four one-star and one two-star reviews. Since Apple resets the rating of apps with every minor version, our composite rating is now one and a half stars.

One and a half stars means starvation for us developers.

The total number of users who’ve rated this particular version is only six. Never mind that 113 people have rated our app before—if you look at the “all versions” rating, our rating is a much more acceptable four stars. But the “all versions” rating is hidden below the “current version” one. The “all versions” rating isn’t the one shown in the results matrix when you search the App Store. Nobody is ever going to click through to an app that’s showing one and a half stars to discover that its real rating is four stars.

Five ratings can be death for a company.

The App Store’s version of Delicious Library 2 did indeed have a crasher we introduced as we tried to make it sandbox-compliant. So three of those reviews appear to be on the money. Again, it only happens for a handful of people, we’ve fixed it, and the new version has been submitted to Apple and is currently on our website.

But those one-star reviews aren’t going anywhere.

And what if the review is just wrong? If you look at the topmost review on the Delicious Library 2 page (as of right now), you can see we didn’t cause the crasher—our customer put his SQLite database file in Dropbox and eventually it got corrupted. Of course he’s upset. But this is the number one review, and it’s not actually our fault. We recommend against sharing our data files because it’s a crapshoot—sharing a database that you are actively modifying without any locking protocol sometimes works, but sometimes leads to file corruption. You have to have some technical knowledge to even get Delicious Library 2 to look in the Dropbox folder. Unfortunately, when it went wrong this user blamed us.

And we get one star.

Further, we can’t contact the user to tell him that he’s corrupted his file, and he needs to go to his backup, because the Mac App Store doesn’t allow us to respond to reviews.

We have a “contact us” menu item under our Help menu, but if users decide to post reviews instead, we cannot provide any tech support. So we also can’t tell the other four people that their crashers should be fixed now, and they should update to the version on our website (or wait for the App Store approval). Apple’s customers are Apple’s customers, except we’re supposed to support them, but we aren’t provided the tools by Apple to do so.

From four stars to one and a half stars because of five users whose problems we really want to fix (or have already fixed).

But we can’t contact them.

Luckily for us, Delicious Library 2 isn’t where we’re making most of our money —we launched Delicious Library 3 this year. Except: when we first released it, it had the same rating problems. Our earliest customers were unsurprisingly the users of version 2, and a percentage of them liked the look of 2 better. Now it’s no insult to me if you like my old work better. But several of these customers bought Delicious Library 3 without trying the free version that’s on our website. This was easy to do, since we aren’t allowed to have trial versions directly on the App Store.

Then, after buying from the App Store, these customers wanted to go back to version 2. Which would be fine if they’d bought directly from us, or if they’d used the trial version. In the App Store case, we’re in an awkward situation. While we happily refund customers’ money—in this case we only made $17 from the $25 they spent. If we refund $25 directly to the customer, we’re actually losing $8. Furthermore, since Apple’s customers are Apple’s, we don’t have a list of who has actually bought our software; we have no way of knowing if someone complaining is actually our customer, or just someone who heard he can get $25 from us by writing us saying, “I want to go back to version 2.”

Photo: Tupungato / Shutterstock.com

Many of these customers don’t even talk to us, or ask the App Store for a refund. They just post one-star reviews: “I don’t like the look, preferred version 2.” We ended up with a rating that was around three and a quarter stars. We were losing money.

In desperation, I stuck in a rating pane when we did version 3.1. It reads:

Ok, we hate these stupid “Please rate our app” panels as much as you do, but since we’re in the Mac App Store we live and die by your ratings, and of course you can’t rate an app the moment you buy it, since you don’t know what it’s like yet.

So, in the politest way possible, we’d like to ask—just this once—that if you like our app you go back and rate it, so we can stay in business and continue making cool software.

The panel only kicks in for people who’ve been using Delicious Library 3 for a while and have scanned a number of items. It defaults to never showing itself again if you dismiss it, and (because it’s on OS X) it doesn’t have to quit Library to open the App Store if you do say yes.

Our rating shot up to five stars within a week of releasing this version. They’ve now settled at a very respectable four and a half stars. We’re making money, and we’ve started work on our next app.

None of this is to say I like being bothered to rate an app—I resent it strongly when it feels like coercion. But, as Daring Fireball itself noted in followups, a lot of this is in how you ask, not that you are asking.

Part of the problem is that once developers get it in their head they should ask users for ratings, they lose perspective on what’s polite.

Some ask specifically for five star ratings, which I think is incredibly presumptuous. Some ask multiple times, and have confusing (or missing) options to opt out of them asking again. And some ask even after you’ve rated the app. For iOS apps, all of them require that you leave the app to rate it, which is kind of a pain. You obviously launched the app because you want to, like, actually use it. All of this creates enmity in the users, and, yes, makes us want to rate their apps with one star just to shut them up.

I assume, by and large, developers have never been waiters. They don’t know that if they go to a table and say, “Hey, I’d really appreciate it if you guys would tip me 20%, but, also, you can’t do it here—can you stop eating for a minute, come out back, and give me a tip now, before your meal is over?” they’re just going to end up with customers that resent them. If they keep doing it, every time they bring another course? If, after the customer has paid the check, they still do it? Well, they’re fired.

So good sense is required here. Part of the problem is that Apple hasn’t provided developers with any APIs to tell if users have rated our apps, or to make it possible for users to rate apps inside the app itself. This would obviously make it less intrusive for users.

Since Apple resets apps’ ratings with every tiny release, developers find themselves wanting to ask users over and over for ratings, since they get wiped out all the time. But no customer should be asked to rate an app twice—it’s like that clueless waiter asking a customer to tip twice.

With the App Stores, Apple has created a micro-economy where ratings are king. However, as with any tightly controlled economy, tiny flaws can blossom into huge problems. With App Store ratings getting reset on every minor version, we are actively discouraged from updating our apps once they have good ratings. I’ve just submitted a new version of Delicious Library 3, and I’m scared out of my head that the first two or three people who review it won’t like it, which will tank it to the extent where nobody discovers it any more, so there won’t be any positive reviews to balance them out.

This system also sucks for customers, who put the time into reviewing an app that—as soon as a new minor version comes out—is thrown away. Only the people who rate the very latest version of any app have their vote count.

Both the Mac App Store and iOS App Store need to allow secondary links to trial versions, show the “All versions” rating on search results, give us a way to engage customers who are having problems (even if it’s totally anonymously), and make it clear how customers can get refunds on software so we don’t get one-star ratings instead. For the iOS App Store, Apple also needs to provide a standard way for users to rate an app without leaving the app, and let developers know if their app has been rated so they won’t bug the user again.

In the meantime, developers need to treat their customers with respect: ask for a rating only once, don’t suggest that five stars are good (I mean, duh?), apologize for being gauche, and don’t ask until the user has used the app enough to get a feel for it.

Wil is the Chief Monster at Delicious Monster, creators of Delicious Library. He started programming when he was 12, taking college courses in BASIC and Pascal and generally making himself unpopular with the undergrads. He founded The Omni Group while a student at the University of Washington, and helped design and write OmniWeb, OmniPDF, and OmniGraffle among others. Under his leadership Omni’s products won a record five Apple Design Awards, and Delicious Monster has won another three.

Wil’s Website | Wil’s Twitter