A Canadian software company, Blaze Software, released a report today of what it calls the “largest ever research study of smart phone browser performance.” The problem is, the study is flawed.

[ad#Google Adsense 300×250 in story]According to the results of the study, Android was 52 percent faster than the iPhone loading Web pages. The company did 45,000 tests with the mobile operating systems to get their results.

One of the biggest surprises the Blaze team found was that “despite significant JavaScript performance gains in the latest Apple iOS 4.3 release and Google Android 2.3 releases, these improvement made no measurable improvement on the actual page load times of the sites tested.”

There is a good reason for this. According to Blaze’s own documentation the “measurement itself was done using the custom apps which use the platform’s embedded browser. This means WebView (based on Chrome) for Android, and UIWebView (based on Safari) for iPhone.”

The problem with using UIWebView is that, even though it’s based on Safari, it didn’t receive any of the updates that Safari did in iOS 4.3. Using an embedded browser is not the same as using the official browser.

Apple’s Safari Web browser included the Nitro JavaScript engine that Apple said runs JavaScript up to twice as fast as its predecessor. Since UIWebView didn’t include any of those enhancements, it’s kind of disingenuous to say that Android beat Safari.

“Their testing is flawed because they didn’t actually test the Safari web browser on the iPhone,” Apple spokesperson, Natalie Kerris, told The Loop. “Instead they only tested their own proprietary app which uses an embedded web viewer that doesn’t take advantage of Safari’s web performance optimizations. Despite this fundamental testing flaw, they still only found an average of a second difference in loading web pages.”

“Obviously someone is looking to make a mountain out of a molehill,” Gartner analyst, Michael Gartenberg, told The Loop. “It’s not an apples to apples test.”

Guy Podjarny, Blaze CTO and Co-Founder said that they “assumed that it would be a closer race and that the latest JavaScript speed improvements would have a more material impact on performance. The fact that Android beat iPhone by such a large margin was not expected.”

The problem of course, is that the embedded browser used in the tests had no JavaScript speed improvements.

While it’s not mentioned in Blaze’s press release, they did the time difference between Android and iPhone page loading in the documentation. It comes out to just over 1 second.

According to Blaze, Android’s median load time was 2.144 seconds vs. iPhone’s median load time of 3.254 seconds.

Blaze has now admitted in a comment to CNET that their tests were flawed. It doesn’t seem right for a company to test a product unless they know for sure what technology it’s using.

“This test leveraged the embedded browser which is the only available option for iPhone applications,” reads the Blaze statement. “Blaze was under the assumption that Apple would apply the same updates to their embedded browser as they would their regular browser. If this is not the case and according to Apple’s response, it’s certainly possible the embedded browser might produce different results. If Apple decides to apply their optimizations across their embedded browser as well, then we would be more than willing to create a new report with the new performance results.”

Update 2: Added the statement from Blaze.

Update: Added comment from Apple.