Matthew Panzarino:
Parsons confirmed the percentage, but said that the differential between the prices of the iPad and the prices of the Samsung tablets that were included in the survey was large enough to “more than offset” the score in the other four categories. Parsons says that the price category contributed to a full two-point difference between Apple and Samsung.
B U L L S H I T!
There’s no way it explains the tablet numbers.
It’s pretty clear Samsung wrote a check – maybe not directly, but someone got paid to make this happen.
See http://techpinions.com/j-d-powers-explains-its-inscrutable-tablet-rankings/24687. It actually gets worse.
Yes.
J D Power is a business, and the business it is in is not ranking things for consumers. The business they are in is ranking things for manufacturers. Here it is:
http://www.autoblog.com/2012/04/06/who-exactly-is-j-d-power/
AMEN Brother!
That’s absolute bullshit. So out 1000 total points, cost was 16% or 160 points. Samsung wasn’t on top, even on cost, so let’s assume a maximum of 159. For Apple to be more than 100 points lower, they would have had to have gotten a score of 59 or LOWER. Somebody was paid, plain and simple.
They didn’t even test the ability to get updates, did they? Let’s see what OS each tablet is running in 3 years. If the Samsung can last 3 years, that is.
(I made the same post on TechCrunch in response to the article, apologies if it feels like I’m spamming)
I think this is an Occam’s Razor situation. The simplest answer, to me, is that JD Power has a ranking system, it’s more complex than is represented here but is kept secret (as it should be). The situation appears to be an edge case where the simplicity of the representation is dissonant with the complexity of the real math underneath, and that no one is taking bribes or unduly influencing the outcome.
Personal anecdote alert:
In a former life I worked for a website that covered college football recruiting. This was in the early 2000’s, the recruiting journalism industry was still early in the jump to putting everything online. But for years the output was in print. Recruiting experts would often offer their ratings of recruiting classes, which methodology was the product of years and years of evaluating recruits and recruiting classes. It was a rating, though, not a ranking: the recruiting expert’s subjective “feel” for the quality of the class.
Most all recruiting experts at the time operated regionally, few were able to travel nationwide, but some did post national ratings. Invariably they were derided as favoring their local areas (Bobby Burton would be accused of favoring Texas; Bill Kurelic of favoring Michigan and Ohio State, Jaimie Newberg would favor the SEC, and so on).
When we assembled these recruiting experts onto our website network, one of the natural things to do was to turn the ratings into rankings, based on inputs that the experts tended to use in their ratings. That way we could rank all the NCAA schools instead of just the top 20. Our first rankings came out and, as one would expect, the recruiting editors immediately asked for tweaks in the equation. They were managing the bottom line: based on what their experience told them, the rankings looked wrong enough to not be a good representation.
Each year we went through this, nudging and massaging the formula. Until one of us (hint: it was me) decided that we had to be going about it all wrong. I did some work on a new formula, a completely different means of evaluating the same inputs. It was more heavy, abstract math than they were used to (though probably still very simple to experienced mathematicians or statisticians) but the important thing was: it worked. I ran a few years’ worth of data through the formula, showed the results to the experts, and after just a little massaging, got the enthusiastic thumbs-up.
The thing was, because of the specifics of the formula, the same recruit would be “worth” a different amount of points to different schools. And the marginal value of a recruit early in the recruiting season was different than his marginal value later in the season. The system was designed to compare entire classes, not individual recruits. But that’s not what the fans see: they see their school sign a four-star defensive back, and scream bloody murder that their total points didn’t rise as much as some other school’s did when they also signed a 4-star defensive back. So of course the fans on our different network sites were POSITIVE that it was all a sham, it was all just a rating again where the editorial staff was deciding who would win.
Sometimes I would be asked to go onto team message boards and explain things as best I could without revealing the exact formula. But again, the math was too abstract. Fans who didn’t understand the math thought we were crazy to not just assign a point value to a recruit and add them all up. Fans who DID understand the math knew I was keeping a lot of detail secret, which I was. They all were sure that we were just out to get them, that we wanted [some other school] to win.
But that was most definitely not the case. I can attest to it. I was literally the only one on the editorial staff that knew the formula, that would have been asked to change it or alter the website such that it could be worked around. And I was never asked to do so, nor was the engineer who did the actual coding. But, hey, your justifiable expectation was X and you got Y: so someone must be pulling the strings and you don’t want to hear otherwise.
So I’m sympathetic to JD Powers’ predicament, and I’m comfortable with their promises that they’re not doing anything untoward.
EDITED: It’s still a good complaint that the representation is misleading in what feels like an edge case. And it’s certainly reasonable to argue that you don’t agree with the outcome. But I’m pretty confident in taking JD Power at their word that this is the legitimate outcome of their formula, and not the result of someone writing a check.
Samsung (Circles,Highest%,Weight,Result) 3 60 26 15.6 3 60 22 13.2 4 80 19 15.2 4 80 17 13.6 4 80 16 12.8 100 70.4 Apple (Circles,Lowest%,Weight,Result) 5 81 26 21.06 5 81 22 17.82 5 81 19 15.39 5 81 17 13.77 2 21 16 3.36 100 71.4I second your bullshit, Jim. If we allocate percentages to each yellow circle, AND we give Samsung the HIGHEST each circle can possibly represent, AND we give Apple the LOWEST each circle can represent, Apple STILL beats Samsung by a point!
The only way to make this work is to let the circles represent uneven divisions–and even then I’m not sure, but I’m too lazy to test the possibilities.
I know of no evidence for money changing hands, but J.D. Power does not have the most sterling reputation. I worked for BusinessWeek when McGraw-Hill bought Power and we kept looking for ways to do business with out sister company. Very little came of it, largely because we couldn’t make sense of their rankings.
Not sure if this way of aquiring a tablet has also been covered by this study, but it’s not hard to lead in the price comparison if you give give free tablets when people preorder a Galaxy S4, Galaxy S4 Mini or even televisions and refrigerators.
Stars is just a ranking of finish, but not arrived at as a function of gap: you can win by a mile and you’re still five stars and the next guy four stars. You can lose by a hair and you’re still just four stars and the other guy 5.
Factor 1:
A scores 800 (5 stars)
B scores 798 (4 stars)
C scores 794 (3 stars)
D scores 750 (2 stars)
E scores 742 (1 star)
Factor 2:
A scores 786 (5 stars)
B scores 782 (4 stars)
C scores 780 (3 stars)
D scores 740 (1 star)
E scores 745 (2 stars)
Factor 3:
A scores 756 (5 stars)
B scores 750 (2 stars)
C scores 740 (1 star)
D scores 752 (3.5 stars)
E scores 752 (3.5 stars)
Factor 4:
A scores 710 (2 stars)
B scores 760 (4 stars)
C scores 780 (5 stars)
D scores 715 (3 stars)
E scores 700 (1 star)
Total:
A total is 3052 (3rd in points, 17 stars)
B total is 3090 (2nd in points, 14 stars)
C total is 3094 (1st in points, 12 stars)
D total is 2957 (4th in points, 9.5 stars)
E total is 2639 (5th in points, 7.5 stars)
I’m sure in this example the data is much simpler and the formula much more complex, but here’s a situation where one competitor (A) gets five stars in three out of four, but still comes in third place overall.
Look at the timing, the day before the iPad Air launch. Smells fishy.
Whenever I read of Samsung almost winning some kind of ratings contest I cannot help but think of Samsung’s alleged $12 billion payola, oops!, marketing budget.
I see the inestimable Steve Wildstom as a commenter here. Steve, please don’t take me to task, again, as you did when I made the same kind of snarky comment about Samsung in a comment to an article you or one of esteemed colleagues wrote for techpinions.
Cheers!
You can only make this work if you make the bands numerically very narrow. You total span is 100 points. But then you can’t say you are scoring on at 1000 point scale.
That’s because it’s plainly initial satisfaction – they say it’s “less than a year” of ownership. This is absolutely fair, honestly, for the same reason there’s “initial quality” reporting on cars separate from long-term-ownership quality.
Most people seem to upgrade mobile devices on about a 2 year cycle, and honestly apart from geeks a lot of people don’t even care about updates very much.
Note that JDP explains how the circles work (thanks to the poster on the other thread who found that!).
Three circles is “Companies or models scoring between 10 percent of the range above the
industry or the segment and 20 percent below the industry or the segment
receive a rating of 3 Power Circles”
Four is over 10% above the average, but not the Best. And Five is The Best (depending; if the segment is “highly competitive” it might be within 10% of The Best).
So the difference between four and five can literally be one point – and depending on the shape of the curve, three and four can be VERY close.
Samsung got 4 stars in three categories, making it perhaps only very marginally lower than Apple in points – and in one of those is definitely BEAT Apple (and probably significantly) because Apple wasn’t the 5-star on price; Apple got hammered on the price category even though it was low-weighted.
In the two it got three-star rankings on, if might well be that Samsung was “nearly at 4 stars” (right at the edge of a 4, maybe at +9.9% of average).
Combine all that and I can believe Samsung really did win by two points, on price – and that JDP is terrible at explaining that you can’t just “Add stars and average it!” to get results.
It’s not like I find it hard to believe that initial satisfaction on the price-sensitive market segment might let Samsung narrowly win vs. the not-as-price-sensitive Apple market.
(ie. Cheapies and Android-rooting Geeks buy Samsungs, people who prefer UX to cheapness buy Apple.)
I find this kind of confusing. Do people really not understand how most marketing research companies work?
Here’s a hint: marketing research companies sell awards to companies. Companies use these awards to advertise their products to other businesses. Employees at these other businesses use these awards as external, third-party justification for their decisions, so they can avoid getting fired when things go wrong.
That’s what these awards are: anti-firing devices. If reporters write about these awards as if they were actual news, they are doing their jobs wrong.
It’d be nice if we had J.D. Powers’ actual numbers, and not have to rely on guesswork that reverse-engineers their dots.
agreed. thanks for the insight.
Boy, if you can’t trust JD Power, who can you trust? I bought a couple of appliances based on their recommendation, including a Samsung refrigerator…
Wrong. Look at Apples update percentages.
Much better and clearer than I’ve been putting it.
Even though many of us don’t trust Consumer Reports much either. We can see why CR fights not to have their reviews quoted in advertising. The thought of JDPower selling their positive review to Samsung comes to mind as an incentive for this.
I may give them some credit for subtlety. I don’t think that they sold the review outright.
I presume they license their trademark for use in ads (you always see the exact same award and logo when car companies cite JD Power). Since Apple is unlikely to ever use JD Power citations in their ads (Apple is more interested in citing their own brand and reputation as a statement of quality) , JD wasn’t ever going to profit from stating Apple as a winner in anything.
So, never mind the individual category scores, they put their thumb on the scales to make Samsung the winner. But incompetently made it so that the math doesn’t work out.
If this is the case, then JD Power is less likely to see any benefit, as the controversy might keep Samsung away from wanting to cite this.
Worse, if this story gets around beyond the Apple and general Tech press, the JD Power brand will be diminished. Who knows, this may mean JD Power awards start disappearing from automobile ads.
Sorry for replying on a 5 day old comment, but just to clarify: JD Powers explains that Samsung exceeded Apple’s score in cost by more than 100 points. Since the total score is 1000 and cost is 16%, the max score for cost is 160. Since even Samsung wasn’t the top in cost (Amazon was), the absolute maximum Samsung could have got was a 159. Apple was more than 100 points below that, so the maximum Apple could have got would be a 58. Absolutely crazy, considering the price points aren’t that different, especially when factoring in the difference between Samsung products and Kindle Fires.