Browser VS Browser - How they scored

[Xd]

In my search for the best browser I selected 6 fairly common web sites and like many others decided to post my results for the benefit, if any, for those who are interested. My disclaimer is this, I tested each load with a timer and no bias or opinion of outcome and each browser was free of any cache prior. I have broadband cable and a download rate of 2871 kbps and an upload rate of 623 kbps. Not exact science but as close as I could get. Times are in the order as the sites to the hundredth second.


ESPN Discovery EOnline NatnlGeogra BBC Amazon TOTAL
Camino0.8b / 10.28 7.15 7.84 3.50 4.32 4.99 38.08
Safari1.2 / 8.55 7.27 8.48 4.47 4.69 4.31 37.77
Firefox0.8 / 16.24 6.56 7.78 12.61 4.83 4.90 52.92
Netscape7.1/ 14.5 9.24 9.17 4.16 4.51 5.23 46.81
IExplorer5.2/ 20.60 11.09 8.99 6.42 7.82 6.83 61.75
Mozilla1.6 / 16.87 8.96 7.27 4.80 3.74 4.44 46.08
Opera7.50 / 17.84 10.24 8.79 5.26 7.72 6.87 56.72


Summary: Loading the 6 websites from fastest to slowest Totals are as follows; Safari(fastest), Camino, Mozilla, Netscape, Firefox, Opera and the slowest was Internet Explorer(snail speed). However the results offer more than overall speed, not all browsers share the same compatibility as the host site requires. In choosing your browser decide which browser benefits the web pages you most frequent. Instead of asking which browser is fastest, I have learned it is more intelligent to ask which browser best suits your needs. In this scenario Safari is best suited for these sites and the results clearly support that. This is how I decide which browser is best for me. I hope my little study ie tutorial helps.

[wgscott]

Each one of these measurements has an associated error. Do you have an estimate. To be conservative, I would give each one an error of +/- one second. If that is the case, and the errors are additive (they should be), then any differences for the totals that are smaller than six seconds would not be significant. So it looks like Camino and Safari are in one group and the others are in a slower group. This makes some sense given the way they are designed.

I use safari as my primary web browser, use Camino for things safari messes up (some java scripts for example) and I use netscape primarily for Netscape Composer (a free and not bad WYSIWYG HTML editor).

[mervTormel]

not a help request. moving to reviews...

[Xd]

I think +/- 1 second margin of error is fair and accurate. Actually I use Safari/ Camino in similar fashion as my needs grow larger. I am going to check out the editor as well. Thx for the post.

[yellow]

Putting those times in code brackets will allow you to format them in a more reader friendly manner.

[mclbruce]

Quote: Originally Posted by Xd I think +/- 1 second margin of error is fair and accurate. Actually I use Safari/ Camino in similar fashion as my needs grow larger. I am going to check out the editor as well. Thx for the post.

Thank you for posting your work. By the way Mozilla has an HTML editor that should be very similar to Netscape's. it's called Mozilla Composer; just open the Composer window and your off and running.

By the way there are several hint on the main site for speeding up Safari and Camino (formerly called Chimera.)

Here's one for Safari:
http://www.macosxhints.com/article.p...40516220739506

And a couple for Camino which may be out of date:
http://www.macosxhints.com/article.p...02111306144098
http://www.macosxhints.com/article.p...30117071817851

Thanks again, -B.

[Xd]

Actually Yellow, I was thinking the same thing but, I will try to clean it up soon.
mclBruce very nice links I appreciate your post.

[mervTormel]

$ pbpaste  | sed 's/\///g' | column -t

prod/srvc     ESPN   Discovery  EOnline  NatnlGeogra  BBC   Amazon  TOTAL
Camino0.8b    10.28  7.15       7.84     3.50         4.32  4.99    38.08
Safari1.2     8.55   7.27       8.48     4.47         4.69  4.31    37.77
Firefox0.8    16.24  6.56       7.78     12.61        4.83  4.90    52.92
Netscape7.1   14.5   9.24       9.17     4.16         4.51  5.23    46.81
IExplorer5.2  20.60  11.09      8.99     6.42         7.82  6.83    61.75
Mozilla1.6    16.87  8.96       7.27     4.80         3.74  4.44    46.08
Opera7.50     17.84  10.24      8.79     5.26         7.72  6.87    56.72

[Xd]

Thx for the help merv.

[roncross@cox.net]

Although to a layman looking at the data, they would be inclined to say that Safari and camino are the fastest. However based on the data, I conclude that there is no statistical difference in the data.

To put it another way. 6 times the stddev is greater than any differences observed in the means of the data. This is due to small sample size and large stddev. This is also confirm withan analysis of variance test where it is revealed that there is no statistical differences due to small sample size and large variation.




Means and Std Deviations of browser test

Browser Num Mean StdDev Std Err Mean Lower 95% Upper 95%
Firefox 7 15.1200 17.1944 6.4989 -0.7821 31.022
IExplorer 7 17.6429 20.0473 7.5772 -0.8978 36.183
Mozilla 7 13.1657 15.1904 5.7414 -0.8830 27.214
Netscape 7 13.3743 15.1866 5.7400 -0.6709 27.420
Opera 7 16.2057 18.3195 6.9241 -0.7370 33.148
Safari 7 10.7914 12.0403 4.5508 -0.3440 21.927
camino 7 10.8800 12.2171 4.6176 -0.4189 22.179

When you exclude the outlyers, the data in the last column of the original test, the result are as shown below:



Means and Std Deviations of browser test

Level Num Mean StdDev StdErrMean Lower 95% Upper 95%
Firefox 6 8.8200 4.62377 1.8876 3.9677 13.672
IExplorer 6 10.2917 5.32307 2.1731 4.7054 15.878
Mozilla 6 7.6800 4.91077 2.0048 2.526 12.834
Netscape 6 7.8017 3.98796 1.6281 3.6166 11.987
Opera 6 9.4533 4.44201 1.8134 4.7917 14.115
Safari 6 6.2950 2.03257 0.8298 4.1619 8.428
camino 6 6.3467 2.54550 1.0392 3.6753 9.018

The sample size is to small to conclusively and statistically say that one browser is faster than any other. If you repeat the same test a couple of more times, that should bring down the variation and increase the sample size to make a conclusive determination.

thx
RLC

[roncross@cox.net]

Quote: Originally Posted by mervTormel $ pbpaste | sed 's/\///g' | column -t prod/srvc ESPN Discovery EOnline NatnlGeogra BBC Amazon TOTAL Camino0.8b 10.28 7.15 7.84 3.50 4.32 4.99 38.08 Safari1.2 8.55 7.27 8.48 4.47 4.69 4.31 37.77 Firefox0.8 16.24 6.56 7.78 12.61 4.83 4.90 52.92 Netscape7.1 14.5 9.24 9.17 4.16 4.51 5.23 46.81 IExplorer5.2 20.60 11.09 8.99 6.42 7.82 6.83 61.75 Mozilla1.6 16.87 8.96 7.27 4.80 3.74 4.44 46.08 Opera7.50 17.84 10.24 8.79 5.26 7.72 6.87 56.72

I just saw this data. Good table summary. However, in order to conclusively and statistically say that one browser is faster than any other, you show repeat the same test at least 2 or 3 more times under the same conditions as the first test. What we are really looking at in this statistical test is a two way anova. The factors in the test being the browsers and the website.

If you do this, it may yield some very interesting and unexpected results. I can help with the statistically analysis if you like.

thx
RLC

[Xd]

I actually intend 2 more studies at 8 hour intervals. I will post the results soon. Absolutely any statistical analysis is welcome. Stay tuned.

[roncross@cox.net]

Quote: Originally Posted by Xd I actually intend 2 more studies at 8 hour intervals. I will post the results soon. Absolutely any statistical analysis is welcome. Stay tuned.

Hi, thanks for the update. Will you provide more details as to how you are loading the browsers? Do you load them one at a time or simultaneously?

In your last test, do you remember the run order of the test? For example a simple run order may look like the following table below:


Column1 ESPN Discovery Eonline NatnGeog BBC
Camino 1 8 15 22 29
Safari 2 9 16 23 30
Firefox 3 10 17 24 31
Netscape 4 11 18 25 32
Iexplorer 5 12 19 26 33
Mozilla 6 13 20 27 34
Opera 7 14 21 28 35

The numbers in the table represent run order and NOT browser load test time. Let's discuss this a little before you proceed with the next test. The point here is that if you run each test exactly with the same run order, you may potentially be putting bias into the test results. Biasing makes it more difficult to draw a conclusion due to confounding the data. I would like to remove any possible residual effects in the analysis.


Notice, I am using the code ["mono"] stringofdata ["/mono"]. The data is still not displaying properly. Yellow, what code are you recommending that we use?

thx
RLC

[mervTormel]

Quote: Originally Posted by roncross@cox.net Notice, I am using the code ["mono"] stringofdata ["/mono"]. The data is still not displaying properly. Yellow, what code are you recommending that we use?

unfortunately, the only formatting that maintains whitespace is the <code> tag/s, i think.

Column1    ESPN  Discovery  Eonline  NatnGeog  BBC
Camino     1     8          15       22        29
Safari     2     9          16       23        30
Firefox    3     10         17       24        31
Netscape   4     11         18       25        32
Iexplorer  5     12         19       26        33
Mozilla    6     13         20       27        34
Opera      7     14         21       28        35

[Xd]

Lotta honey-do's right now. I loaded 1 browser at a time in the order I wrote them ie; espn,discovery,eonline,nationlgeo,bbc and amazon. First w/camino,safari,firefox,netscap,explore,mozilla and finally opera. I will post the 2 other results when my chores are finished tonight. Good to see the help and interest. will Post back in a few hours.

[roncross@cox.net]

Thanks for the run order data, I will analyze the the data when you post the results. I will also look at the residual with respect to run order as you described. Hopefully, this will not influence the data much, but there is no guarantee.

For the best best results with a test like this, it is best to randamize the run order so as to prevent confounding the results. At any rate, any data is better than no data at all. Thanks for providing the data. An example of good randomization would be something like the following:

 Column1   ESPN   Discovery   Eonline   NatnGeogr   BBC   Amazon
Camino	    31      4               1         33      36    28
Safari	    29      13             30        12      37    22
Firefox	    24      27             18         41     21    6         
Netscape    11      9              15          5      23    17   
Iexplorer    7       42             32         34      25    40
Mozilla	   26        3             2          19       20    38
Opera	    35        10           16         39       8     14

Note that there is no obvious pattern in the run order of the data. Therefore making any conclusions based on the data very solid and conclusive.

I am using the tag code as recommended. It does look a little bit better aligned using my firefox browser. Let me know if the table looks better on your end with respect to alignment. It not a perfect alignment, but it is a step in the right direction.


thx
RLC

[Xd]

Ok. The first study was done at 10 a.m. This study was done at 10 pm. The results are a bit interesting. My 3rd and final study tomorrow will be completely random. I think this will give us a broad base for our final statistics, 2 on increments of 12 hours and 1 completely random.

ESPN Discovery EOnline NatnlGeogra BBC Amazon TOTAL
Camino0.8b / 5.44 8.28 5.25 3.99 2.79 4.54 30.29
Safari1.2 / 9.73 10.44 7.84 4.97 4.30 5.94 43.22
Firefox0.8 / 19.94 12.07 6.85 5.39 4.45 5.39 54.09
Netscape7.1/ 12.11 8.70 6.95 4.33 4.08 3.27 39.44
IExplorer5.2/ 20.78 12.76 6.65 6.97 6.44 6.51 60.11
Mozilla1.6 / 17.27 7.16 7.05 4.06 2.90 3.78 42.22
Opera7.50 / 17.87 11.17 6.03 4.27 3.20 5.43 47.97

I lack the skill at present to make a chart(html) as yet. Someone else can help w/that, hopefully clean it up for all our efforts. Same order as b4 only 12 hours later, cache clean. From fastest to dirtnap this time it is Camino, netscape, mozilla, safari, opera, firefox, Explorer(once again snail speed). So far the only constant is mozilla remains at 3rd for good or bad and explorer is still in the dark ages.

[roncross@cox.net]

Thanks for the results. I look forward to seeing the rest of the data. In the meantime, I will go ahead and take a look at this data and give preliminary results when I have them.

Don't worry about making the charts. I can make the charts and post it via my webpage for all to view. However, since my webpage is not a permanent resting place, I would recommend that one of the site administrators on this forum to copy and post the charts to a more permanent location, so that the results can be readily available for all to review in the future.


thx
RLC

[Xd]

I think that is great. You should put your website in your sig for all to see. Alot of others do it. Its intersting and fun to chart this out and hopefully the final analysis will be clean and as someone mentioned, reader friendly. We can post the results as a read only for those interested. If the mod's approve our efforts. Good on ya m8.

[roncross@cox.net]

I am looking at all the raw data from both test. I look forward to the final data so that I can write the full report. Since this is only preliminary, there are no graphs. Final report will include links to graphs.

Preliminary results after the 2nd test supports the following:

1.) Safari and Camino are statistically faster than the rest of the browser for url ESPN - This is generally a slow loading page. Data for url ESPN is shown below. Netscape is the 3rd fastest and the other browsers are just statistically slower than the rest.

Means for Oneway Anova for ESPN url

 Level	         Number	Mean Std Error	   Lower 95%	Upper 95%
Firefox	         2	18.0900	1.2594	    15.112	21.068
IExplorer	2	20.6900	1.2594	   17.712	23.668
Mozilla	         2	17.0700	1.2594	    14.092	20.048
Netscape	2	13.3050	1.2594	  10.327	16.283
Opera	        2	17.8550	1.2594	   14.877	20.833
Safari	         2	9.1400	1.2594	     6.162	12.118
camino	       2	7.8600	1.2594	   4.882	10.838

2.) For all 6 urls in the test. Camino and Safari are statistically faster than the other browsers followed by netscape and mozilla. The other browser are statistically slower. This means that over time, you will load pages faster on average than other browsers. The data for all 6 urls is given below:

Means for Oneway Anova for total time.  This is really just a function of the slower urls such as ESPN.

Level	        Number	Mean Std Error Lower 95% Upper 95%
Firefox	        2	53.5050	2.9291	46.579	60.431
IExplorer	2	60.9300	2.9291	54.004	67.856
Mozilla	        2	44.1500	2.9291	37.224	51.076
Netscape	2	43.1250	2.9291	36.199	50.051
Opera	        2	52.3450	2.9291	45.419	59.271
Safari	        2	40.4950	2.9291	33.569	47.421
camino	        2	34.1850	2.9291	27.259	41.111

Std Error uses a pooled estimate of error variance

3.) The 3rd preliminary observation is that camino and safari appears to give a boost in load times when the url page is slow to load such as ESPN, but there appears to be no benefit when the url load relatively fast.

4.) In the analysis of variance. The load time is more depended on the url being downloaded than the browser being used except in cases where download time for the url may be slow. People should just design better urls

5.) Eonline appears to be a very well designed url. All of the browsers perform well when loading this page. There maybe something to learn from this website.


6.) Internet Explorer and firefox appears to perform poorer than the other browsers with respect to all urls involved.



Again, this is just a preliminary summary. Final report with graphs will be published when the data is available.


thx
RLC