Some time ago we started a little sub-site called "Test UmlCanvas". The goal was to gather more information about how UmlCanvas was performing on a variety of browsers, versions and operating systems. Remember, this is web-browser-land and to make things even worse, we're chasing a draft specification. Rendering differences lurk around every corner and things are never the same. With more browsers hitting the web every month, it's a sheer impossible task to test all of these browsers just with the few of us.
So, the basic idea was to have a small HTML page containing a reference UmlCanvas. Next we ask anyone on the web to spare us a minute (I doesn't even take that long) of their time to take a screenshot of that HTML page and upload it to our server.
We started out with a late revision of UmlCanvas 0.2 and upgraded the reference UmlCanvas to reflect the possibilities of UmlCanvas 0.3. The latter didn't change that much with respect to rendering, so both versions are valid candidates for analysis.
Because that's what we're now going to do: look at each screenshot and find ways to improve the rendering capabilities of UmlCanvas, so that the differences between the different implementations for the different browsers are less and less noticeable and all users have an equally good experience when viewing UmlCanvas diagrams on their personal browser/operating system combination.
This article takes a look at the current samples we've collected and tries to come up with a first set of conclusions. These conclusions will be targeting both Test UmlCanvas and the next release of UmlCanvas, 0.4.
Let's start off with some basic numbers: So far, we've received 57 submissions. It might not seem much, but it's more than we could have tested on our own. So a big thank you to everyone who contributed.
These submissions represent in fact 44 unique browser/version/operating system combinations. 25 of those were shots of UmlCanvas 0.2 and 27 of UmlCanvas 0.3. Euhm, wait a second, 25 and 27 makes 52 and not 44 ?! Correct, 8 combinations where tested on both versions. The remaining 17 and 19 combinations were only tested on one of the two releases.
57 submissions 44 unique browser/version/os submissions 25 UmlCanvas 0.2 27 UmlCanvas 0.3
So what browser has been tested most thoroughly and which one is on the verge of extinction ?
browser | count
---------------------+-------
Firefox 3.5 | 12
Safari 4.0 | 7
Unknown Browser | 7
Internet Explorer 7 | 6
Firefox 3.0 | 6
Chrome 3.0 | 5
Internet Explorer 8 | 4
Opera 9.6 | 3
Firefox 2.0 | 2
Chrome 4.0 | 2
Opera 10.* | 2
Konqueror 4.3 | 1
Based on these results, we must admit that FireFox rules "our" world. But we have to look at these numbers and take in account the number of operating systems this browser is available on. Taking that into account we get the following overview:
browser | tested | possible | pct
---------------------+--------+----------+-----
Chrome 3.0 | 3 | 3 | 100
Chrome 4.0 | 1 | 1 | 100
Konqueror 4.3 | 1 | 1 | 100
Internet Explorer 8 | 3 | 4 | 75
Firefox 3.0 | 4 | 6 | 66
Internet Explorer 7 | 2 | 3 | 66
Firefox 3.5 | 5 | 8 | 62
Safari 4.0 | 4 | 7 | 57
Opera 9.6 | 2 | 5 | 40
Opera 10.* | 1 | 5 | 20
Firefox 2.0 | 1 | 6 | 16
Ah, you just gotta love numbers. This looks a bit different. We wanted to test Chrome on respectively 3 and 1 operating system, and so we get a pretty good coverage. Firefox 3.5 is available on all 8 operating systems we want to test, but we only got reports of it on 5 operating systems.
But do we also actually learn something out of this last set of numbers? Yes we do: Opera and Firefox 2.0 don't even come close to a respectable coverage of the tests we wanted. The only thing we can conclude is that our current focus-group doesn't really use these browsers (anymore).
I think it's fair to say that if you are still using Firefox 2, you really should upgrade. 3.0 and 3.5 are already available for some time now and 3.6 is hitting the web as we speak. 3.0, 3.5 and 3.6 are three generations of Firefox and that's enough. We're going to drop Firefox 2.0 from our test set. Looking at the visitors of TheModelFactory, this seems to be a valid decision:
Then the Opera case. Given the fact that the tests for Opera were all submitted by myself (and 2 confirmed by an anonymous source), we currently don't really have a lot of benefit in focussing on Opera support. We're going to keep it in our test set, but the analysis of the shots will have to show if the differences to other browsers are big enough to justify dedicating some time on them, or if we can live with it for another release.
And where is Internet Explorer 6 ? Clearly on the way out, luckily. Well, there's a good reason why it probably isn't showing up in the stats: The front page of Test UmlCanvas contains some "rogue" Javascript that simply crashes IE6. We did receive one submission, probably by a die-hard fan of IE6 ;-) No, really, respect for still trying to get that shot to us. But we have bad news: we're not going to try and make IE6 look as pretty as on other browsers. We are going to fix it though, because some recent commit causes it to no longer display anything, but that's where it will end. It seems we're not the only one pulling the plug on IE6. Which is not something we like to do, because looking at the statistics, we still have a lot of visitors using IE6:
Sorry guys, but you really must move on. You're missing out on a lot these days, and you're making our lives pretty harsh trying to come up with solutions for such a dreadful browser.
Noticed the number 3 spot in the first list ? The Unknown Browser with 7 submissions. Luckily we also offered the chance to correct our detected useragent information. This is what you submitted, where we failed to identify your browser:
browser | os
------------------------+---------------------
Google Chrome beta | linux
Unknown Browser | linux
IE6 | xp
Chrome 4.0 | linux
Unknown Browser | osx10.5
android native browser | android 1.6 (linux)
Chromium | linux
Looks like there clearly is a new kid on the block. Chrome or Chromium is clearly on the rise. We clearly need to add more combinations for it in our test set. Both Unknown Browsers in this list are also beta's of Chrome, which leaves us with the famous IE6 report that had to be submitted through a different browser and a report that really deserves a big round of applause: Android. PJV adds: "It works, mostly. Touch interface adds some issues: selecting, multiple-selecting by dragging, dragging an object or rather scrolling the page? Slow but just workable.".
Okay, let's be honest about it. Android currently beats the s**t out of our own iPhones. Some time ago, our support on the iPhone was great, now ... well, you can make up your own mind. It's a shame, because our main development browser is Safari. I really need to set this straight ;-)
We initially decided to test browsers on different operating systems, to be sure that there were no issues there. Analysis of the screenshots will soon show us if this is still needed. If we can eliminate some operating systems, the total number of required tests goes down and we can achieve a better test coverage, focusing on the more important differences. Let's just take another look at the raw numbers first and see how the different operating systems do.
os | count
------------------------------+-------
Windows XP | 17
Windows Vista | 12
Unknown OS | 7
Linux | 6
Mac OS X 10.6 - Snow Leopard | 6
Mac OS X 10.5 - Leopard | 5
Mac OS X | 2
Windows 7 | 2
No worries, everything is under control, again, we need to take into account the number of possible browsers we can test on each platform:
os | tested | possible | pct
------------------------------+--------+----------+-----
Unknown OS | 7 | 1 | 700
Mac OS X 10.6 - Snow Leopard | 6 | 3 | 200
Windows XP | 17 | 13 | 130
Windows Vista | 12 | 12 | 100
Linux | 6 | 7 | 85
Mac OS X 10.5 - Leopard | 5 | 6 | 83
Mac OS X | 2 | 3 | 66
Windows 7 | 2 | 7 | 28
Now, that's more like it. Of course, we only expected 1 unknown browser on 1 unknown operating system, so when we receive 7 of them, ... But much more important is that Snow Leopard now is on top ;-) So no need to dig any further. Let's move on...
Let's move on to the real analysis. The number above a merely fun to look at (though they already provided us with enough feedback to drop at least 1 browser/version. Which is beyond any doubt a good thing for Test UmlCanvas itself, but we want to improve UmlCanvas.
There are two ways we need to look at the screenshots: differences between browsers and differences between operating systems. Now we know that some browsers share the same rendering engine underneath, so we might have to change the browser criteria into a render engine criteria. That's mainly the reason why we mentioned it on the main page of Test UmlCanvas. Let's see if we were right and if we can start eliminating browsers based on their equivalent render engine.
We know that Safari and Chrome use the webkit render engine. So do their shots confirm this?
Besides some small differences regarding the rendering of the textarea and the size of the actual window, both UmlCanvas diagrams are pretty identical. Great. Does this eliminate one or the other. No, because we never know if Google or Apple introduces changes in their own browsers. But, we can consider Chrome to be pretty much in line with the results we get from our main development browser: Safari.
Once upon a time, Webkit was forked from KHTML, the render engine of Konquerer. Wikipedia has a nice page describing the history and we learn that the Konquerer developers try their best to incorporate the changes in webkit back into KHTML. Does this mean that Konquerer too is at the same level as Safari and Chrome ?
Not quite, but wait a second, this looks a lot like the "issues" with Firefox 2:
How is that possible? Well, we're happy to tell you that both browsers are rendering "as designed". Both Konquerer 4 and Firefox 2 have limited support for the HTML5 Canvas element. They both lack support for rendering text. This was the case with most browsers when we started out, and UmlCanvas really needed text support.
So we rely on work by Jim Studt who wrote CanvasText a library to render text using basic line strokes. It implements only one font, which is of course different from the fonts we have access to on implementations that do implement text rendering. So UmlCanvas diagrams using this "manual" text rendering will look different, but they both look equally different, so that's a good thing. If we want to improve this, we need to align the "manually stroked" font to the default font on native text rendering implementations.
With Firefox 2.0 making its exit, the need to render text ourselves is leaving with it. Konquerer 4 is one of the last browsers still in need of help rendering text. Another one is Opera, but we already kind of decided that we were not really going to focus too much on it. It basically looks a lot like Firefox 2 and Konquerer:
Previously Safari also lacked this support, but in the mean time, we don't even have such an "old" release of Safari anymore and it seems that no one else does, because we only received reports for Safari 4. The visitor statistisc of TheModelFactory confirm this:
Versions starting with "53..." are Safari 4, so with over 90% it clearly shows that Safari users are following the current releases and we don't have to worry to much about our CanvasText implementation on older Safari browsers.
Unlike with Safari, the Mozilla developers didn't wait for a draft spec to be ready. They implemented their own text rendering functions. This was the case in Firefox up to version 3.1. Let's see if there is a significant difference between those text rendering functions and the final ones:
It almost makes me want to take a look at the commits, but something tells me, they reused their initial implementation when implementing the "official" API. Good for them. And us.
Now how does a modern Firefox compare to our main development platform, Safari? Let's put them side by side and see:
Now, this really makes us happy. It appears that Firefox' renders its text a fraction larger than Safari, which causes the little differences in position of the classes and one inheritance relationship to show a little twist, but overall this is pretty neat.
Finally, how does Internet Explorer perform ? Let's take a look shall we ? From left to right IE6, IE7 and IE8.
Let's get something straight first: What you see here is by all means NOT the result of Microsoft implementing the HTML5 canvas element. If it wasn't for Google, there would be no UmlCanvas on IE. Google has implemented explorercanvas, an emulation of the canvas element using VML, the IE-only graphical rendering extension. And they have done a great job, as you can see on the right hand side.
So, okay, we know we screwed up somewhere with some recent commit, because IE6 is doing all the hard work, but we're not getting any visual feedback anymore. We are going to look into that, and are going to fix it, but we're nog going to spend additional time on it to fix things that only make IE6 look better. If we improve the result on other browser, and IE6 benefits from it, so be it.
This seems harsh, but in fact it is not. Because ALL IE need explorercanvas, it is in fact explorercanvas that we're up against. If we improve the rendering on IE8, IE7 and IE6 will benefit also. That's pretty logical, right ?
Wrong. It must be bad karma, but even when someone else writes an emulation layer on top of Microsoft's own VML, they manage to make a mess out of it. The mere fact that IE6 isn't displaying anything and that there is a different in width of the note between IE7 and IE8 shows that although all three IE instances use the exact same UmlCanvas code and exact same explorercanvas code, there are still differences :-)
You might argue that it has something to do with different versions of VML. Okay, that would explain the differences between IE7 and IE8. But what about ...
The shot of IE7 above is also taken on Windows XP. Still, it renders the normally italic font at the top-left side, rather funky.
One final note on IE: If you look carefully at the shot of IE7 and IE8 you might notice that there is one element on de diagram that's drawn very differently than all other shots we've seen before. Indeed, there is a carriage return in the note, splitting the text on two separate lines. This is actually how it should be. At this moment, IE8 is the only browser that correctly renders that note. Oh, the irony.
So, we covered most of the browsers and their versions. Time to see what influence the operating system has on UmlCanvas. Let's take one browser and compare it on as many operating systems as we can.
It seems that Firefox renders UmlCanvas pretty much the same across operating systems. Which is, in a way what we would have expected. There are again pixel differences, but it seems that they are, again, mostly due to small differences in the font-sizes. Let's take the same look at Safari, which is now also available on Windows:
Here we clearly see that there is a difference between operating systems: Mac and Windows clearly deal with fonts in a different way. How much we would love to think of it as just a bunch of browsers, we must come to the crazy conclusion that the operating system actually is involved.
Time to come up with some conclusions and items we're going to address:
... because this article is clearly marked "Part 1". There is another story we can tell from these shots. We can learn how we can improve not only the technical aspects, but also how we can improve the rendering algorithms of UmlCanvas. By changing the way we deal with elements on the diagrams, we can avoid running into visually annoying differences and have diagrams looking great on different browsers, while not being pixel-perfect clones.
|
For Everyone |
For Developers |
Social Modeling |
News & Updates
|
