by Kyle Weems 3.April 2008 08:25
Like a slowly bubbling cauldron overseen by a benighted hag, the web developer blogosphere is beginning to boil over with putrid drama. Browser makers are drawing lines in the sand, designers are lobbing opinions out, and comment sections are filling up with bile. What could be causing such acid behavior?
Acid3. (Warning, the Acid3 test may cause certain browsers to crash while they try to jump its many hurdles.)
I've already briefly talked about it before, but in short the Acid3 test is the third in a series meant to serve as a benchmark for browser compliance in the areas of standards and emerging web technologies. Like its predecessors before it, the purpose of the test (in theory) is to encourage browser makers to improve their browsers to the point where they pass the test, and by doing so, make a better product. Think of it as an industry litmus test.
However, it's apparently not as successful in practice as it is in theory. Both the Webkit (Safari) and Opera teams have announced 100/100 scores on public builds for their browsers in regards to these tests. Awesome, right?! The question that seems to follow up on this is why is Mozilla so far behind? Don't they care about these things? (You'll notice there's not a lot of surprise that IE8 isn't coming along as quickly, but such is Microsoft's lot in the world of web browsers).
Mike Shaver spoke up on his blog a few days back discussing this very issue. He makes two major points worth consideration. The first should be obvious: Mozilla is in the endgame of development for Firefox 3, and as such need to focus on releasing as bug-free a browser as possible. Tweaking it for Acid 3 just before finishing up could introduce a number of flaws that would sour their effort. This is completely justifiable, and I think Mozilla should get their product out (I am already salivating) and then focus on improving its Acid 3 score.
The second point is more drama-filled, and at the center of the firestorm. He feels that at its core Acid3 is missing the point. To quote:
"Ian’s Acid 3, unlike its predecessors, is not about establishing a
baseline of useful web capabilities. It’s quite explicitly about making
browser developers jump — Ian specifically sought out
tests that were broken in WebKit, Opera, and Gecko, perhaps out of a
twisted attempt at fairness. But the Acid tests shouldn’t be fair to
browsers, they should be fair to the web; they should be based
on how good the web will be as a platform if all browsers conform, not
about how far any given browser has to stretch to get there"
He goes on in more detail, and I recommend reading through his post if you've got the time and inclination. Ian Hickson, one of the creators of the test, defends his position in a comment to Mike's post that seems to be summed up to the effect of "You're just upset because we released the test just as you're finishing your browser". I think that's a relevant issue, but I think he missed the overall point.
I'm not as knowledgeable as browser makers about how this whole browser testing thing works, so I'll do what any good blogger does: I quote smarter people.
Eric Meyer (a really smart guy) can be accused of being fairly dry when it comes to speaking, but the man gets to the heart of the issue when he talks about the recent Acid3 flareup. A key phrase of his post is:
"The real point here is that the Acid3 test isn’t a broad-spectrum
standards-support test. It’s a showpiece, and something of a Potemkin
village at that. Which is a shame, because what’s really needed right
now is exhaustive test suites for specifications– XHTML, CSS, DOM, SVG,
you name it." -- which digs to the very core of the issue.
The point of the Acid tests is not to pass them for the sake of passing them. That's about as useful as Washington's WASL tests (ask a teacher in the state about them some day and see how that goes). The entire goal is to make the web a better place by pushing browsers to improve. As Meyer also says:
"What I disagree with is the idea that if you cherry-pick enough
obscure and difficult corners of a bunch of different specifications
and mix them all together into a spicy meatball of difficulty, it
constitutes a useful test of the specifications you cherry-picked.
Because the one does not automatically follow from the other. For example, suppose I told you that WebKit had implemented just the
bits of SMIL-related SVG needed to pass the test, and that in doing so
they exposed a woefully incomplete SVG implementation, one that gets
something like 2% pass rates on actual SMIL/SVG tests. Laughable,
right? Yes, well." (emphasis mine)
How do the people involved in the Acid3 scorecard race respond? Opera's Anne van Kesteren writes in his blog:
"As for the complaining about the test. It’s certainly true that if
vendors don’t take it seriously, it won’t be relevant for the Web... complaining about it now some browsers are passing seems a bit lame." -- And, to show how strong he feels his position is, he's been locking comments on all his recent posts about the subject. Good show, ol' chap.
Ultimately, this whole horserace that Opera and Webkit are involved in is a pointless publicity stunt that demeans the purpose of the Acid test and what it's meant to do for the browser community. As Ian admitted himself in his comment to Mike Shaver, "With Acid2, the original “first cut” failed a lot in IE, Mozilla, and
Safari, but actually did pretty well in Opera. We (Håkon and I) then
went on a hunt for Opera bugs and made Opera fare much worse on the
test." (again, emphasis is mine)
Acid3 isn't going to be any different. Why go for a horserace for test compliance at the cost of introducing bugs that you will later have to fix (thereby lowering your initial fancy score)? I'd rather see all the browsers have low scores for a longer period of time if it meant that when they finally passed it was with stable, usable browsers that didn't have other flaws. Opera, Webkit, shame on you both. Stop acting like showboating highschoolers and get to work on passing the test legitimately without ignoring the gaping holes you're currently creating in the process.