Very Odd Web Page Loading Problem
 

Consider this page:

Thai Justice Web Board

This is a page from a forum on the Thai justice system. At the bottom of the page should be the sort of quick reply box that you see on many web forums.

I have four Macs: a G4 PowerBook, a G4 iMac, a G3 iBook and an Intel Mac Mini. The web page displays properly only on the G4 PowerBook. The quick reply box is missing on all three other machines. It doesn't matter what browser you're using. All three newer machines are using Safari Version 4 Public Beta (5528.17). The iBook is using Safari version 3.2.1. Safari Version 4 is configured identically on the three machines that are using it. (We went through the preferences on all three machines, item by item, they are identical.) None of the machines has "enhancers" or "extenders".

But, as I said, it doesn't matter what browser you're using. On the machines where the web page doesn't display properly, it fails equally with Safari, FireFox and Camino. On the Mac Mini it fails even with Konqueror (running on Kubuntu in VirtualBox) and FireFox (running on Windows XT in Virtual Box).

The following is the source code of the bottom of the page taken from the PowerBook, the only machine that displays the page properly:

Here's the source code of the bottom of the page taken from a machine that doesn't display the page properly:

As you can see, the entire form is missing along with four Javascript lines.

I can't understand why three machines fail to load this form data no matter what browser or even OS you're using while one machine loads the form correctly no matter what browser you're using.

My wife and I have wracked our brains on this for days now and end up without a clue. maybe a fresh eye can spot something?

The attachment shows how the page should display, quick reply box and all.

Won't get very far without also knowing what happens on the server.

All machines are set up the same? Same os / browser / ISP?

What happens when you load the page with curl?

In the Terminal:

curl http://website.com/pagename.html

All the machines are connected to the same ISP via the same router. The PowerBook, which loads the page correctly, is connected wirelessly, as is the MacMini, which fails to load the page. The iBook and iMac are wired. The two G4's and the Intel Mac Mini are all running 10.5.7. The G3 is running 10.4.11. The firewall settings on all the machines is the same: off.

As I noted in the OP. The page fails to load correctly on the MacMini even when running virtual OSs: Kubuntu (Konquerer and Firefox) and WindowsXP (Firefox).

This is an .asp web page. curl doesn't handle parameters. It just returns the page as if the parameters were not there.

1) curl doesn't care

2) curl does indeed handle GET and POST parameters, cookies and just about everything else. For GET variables just quote the URL ( 'http://example.com/somepage.xyz?a=2&b=whatever' ), for POST vars use the -d option followed by the info.

I've gotta second acme here: if the server is dynamically constructing the page, and something in the script is checking for different browser types and isn't recognizing the newer Mac browsers (or is throwing an error), then that would cause exactly the effect you're seeing. I can even see how it might happen: the server tries to load different javascript functions for different browsers/machines (understandable for some kinds of DOM manipulations); the designer hand-coded in browser/machine designators without anticipating revision changes; the server doesn't recognize your newer browsers/machines as any of the acceptable types; nothing gets outputed. most likely a quick email to the webmaster will get the problem fixed.

acme.mail.order: Thanks for the help. Curl fails to load the form that is named: wwbaaddfm. It fails to do so on the machine that properly displays the web page and on the machines that don't. But, I'm not very adept at curl, so would you please show me exactly what the query should look like? I tried:

curl 'http://www.thaijustice.com/webboard.asp?sub=0&id=1077633'

Here's the last few lines of file that curl returned. It's the same on the machine that works and the machine that doesn't.

tw: The machine that works and the machine that fails are both running the same version of the same browser and the same OS. I've tried changing user agents, but that doesn't work.

The web page is properly displayed when using Safari Version 4 Public Beta (5528.17) on a G4 PowerBook running 10.5.7.

The web page is not properly displayed when using Safari Version 4 Public Beta (5528.17) on a G4 iMac and an Intel Mac Mini, both running 10.5.7.

Afterthought and clarification:

.asp pages are, by definition, dynamically constructed on the server. Therefore we want to see what happens with a generic HTTP_USER_AGENT and HTTP_ACCEPT header.

I don't know how to do that.

the User Agent header that gets sent to the server tells all sorts of things, including the machine platform - see this. It just strikes me as far more likely that this is a programmer error than an operating system bug. could be wrong, of course, but for it to be a bug it would have to (somehow) screw up rendering consistently in multiple independent applications, which would be bizarre - it would imply a fairly significant but uncaught error in some core framework (but not webkit, since I don't think Konqueror or Firefox access webkit). Occam's razor, you know...

i see the form with curl, but there are no normal form controls, like submit. And I don't read thai, so the page just looks full of curly Ns.


All bets are off if the software says 'beta' anywhere.

It seems odd to me that only one machine in my stable can correctly display the web page and can do so with whatever browser you select (beta or not). It also seems odd that every other machine fails to display the web page no matter what browser you select (beta or not).

Here's the source of the page viewed with Camino Version 1.6.7 (1.8.1.21 2009032711):

The form is missing.

Here is is with Firefox 3.0.8:

The form is missing.

It's not the browser. It's the machines.

Question for acme.mail.order: Do you see the form when you load the page in a browser? How about you, TW?

I don't think it's the browser or the machines - I think it's the server-side script that builds the webpage. but no, I don't see the form in safari or firefox on my g4 iBook.

OK, then, what could the server-side script see as being different between a G4 iMac and a G4 PowerBook, both running the exact same browser configured exactly the same under the exact same OS?

Why does the script build the page correctly for the PowerBook and not the iMac? (For the sake of discussion, let's leave the other two machines out of it.)

Try going to some of the "browser privacy test" sites (e.g.: http://www.2privacy.com/cgi-bin/PC_B...rivacy_Test.pl
others at: http://www.computerbasicsandbeyond.c...acy_tests.html )
and see what the difference is in the info provided in the various cases.
In particular, look at what the USER_AGENT is in these different cases (the Macs that work and those that don't work with that form)

The results are identical, except for the screen resolution. I tried several of the tests. The results below are from 2privacy.com

The machine that doesn't work:
The machine that works:

Maybe try changing your Mac's screen resolution to see if the original form is affected by this.

I changed one of the machines that doesn't work to have a screen resolution identical to the screen of the machine that does. No joy.

Oddly, my wife just informed me that the page displays perfectly under Safari on her iPod Touch. The reply form is right there.

well, there is a slight difference (PPC vs Intel in the User Agent line). but that being said, I tried a more direct test. I cut and pasted the missing html bits (that you gave above) into a local html file. I tried it both as-written, and with links to the actual javascript files on the thai justice site, and in both cases the html loads just fine in the browser, even though they don't appear in the actual thai site for this browser. that tells me that the browser is perfectly capable of rendering these bits, and that there are no javascript errors that are impeding the page processing. what else can that mean except that the text in question is not getting sent to the browser by the server? the lines following that section are present, so it's not like a page rendering failure, and no errors at all are showing up in Safari's error log. I'll bet you dollars to doughnuts that this small form is tucked inside a conditional block in the asp code, and that something (a script error, or a badly designed bit of code, or an intentional but ambiguous restriction) is causing that conditional to crap out in repeatable but more-or-less meaningless cases. I'd have to see the code, though, to be sure.

ps - I have to add, whoever designed that site has an unfortunate love of flashing text. gives me a frigging headache. :)

Here's a browser test from one of the PPC machines that doesn't work:

So, it's not an Intel vs. PPC problem. The only machine that works is PPC.

I agree, but there has to be something about that one machine that makes it work and I ought to be able to find out what it is.

The reason I think it's machine specific is that the page is generated incorrectly even under virtual machines. Plus, I know that the page works fine when my wife views it under XP using FireFox at work. But when I run XP on a virtual machine on the Mac Mini, the page generation fails just as it does under the Mac OS on the same machine.

Could it be a network configuration issue?

Sadly, many Thai web pages are like that. Have a look here and keep a bottle of aspirin handy. This is one of the most popular web portals in Thailand:

Sanook

well, I don't doubt that it's machine-specific, but I think it's machine-specific on the server end, not the client end (i.e., something on the server is identifying different machines correctly but doing something incorrect with that identification). I'm not ASP programmer, but a brief look at the docs tells me that the ASP browser detection system is a bit of a bear to work with. apparently ASP has a whole set of profiles for determining user agents, and an extensible system for creating user-defined rules, all of which is just begging for something screwy to happen.

Microsoft... they refuse to comply with W3 standards that would make browser integration simplistic, and 'compensate' for the problem by creating the world's kludgiest sniffer. yeesh.

As a last resort I fired up XP under Virtual Box and tried Internet Explorer. Same failure. So, I went to the 2privacy.com site to see how the browser is identified:

So, even with the machine identified as an x86, the browser as IE6 and the OS as Windows XP, the server still fails to properly generate the page.

I guess it's time to give up and thank everyone for their help!

Since the problem even occurs in virtual machines, I think it is likely to be something in the networking setup of the afflicted Macs.
You could investigate this by looking at the low level packets being sent (e.g. with "WireShark").

Or just do an "archive & install" (checking the box to preserve users) from the OS X Install CD/DVD to get a fresher start. I say "fresher" rather than "fresh" since that checkbox to preserve users also preserves network settings, so it may preserve whatever (I presume) is causing the problem. In that case, a full "erase & install" might be called for.

except that it happens with my machine, too - do I share a network setup problem with some guy I've never met? I think the archive and install suggestion might be a bit premature...

The sort of problem I'm thinking of is some 3rd-party firewall or other network-related software. It's not that unlikely that you both have the same software installed.
One thing to try might be to do a Safe Boot (holding down the Shift key after the startup chime) and then see if the problem exists.

I tend to agree that it's a networking problem, but I can't imagine what it might be. All the machines are connected to the same router, a LinkSys WRT54G running Tomato version 1.23. All connect via DHCP and all have manually assigned IP addresses. None of the machines have third party firewall software installed and all have the native firewall turned off. The router firewall is on. One, and only one, of the afflicted machines has Little Snitch installed, but it is currently turned off.

The router has QoS enabled, but disabling it didn't help. I tried turning on DMZ for one of the afflicted machines, but that didn't help either.

One of the afflicted machines is a G3 iBook that does nothing but host a web cam. It had a clean install of Tiger (10.4) late last year and the only thing that has been installed on it since then is the web cam software and Apple software updates. In other words, it's about as pure as the driven snow as you can get....

At this point, I'm more interested in actually figuring this out than I am in actually eliminating the problem. It's become an interesting puzzle.

I'll give Wireshark a try. I'm an old retired geezer and no longer the sharpest kid on the block, but at least it will keep me occupied on what looks like a rainy day in the tropics.

It seems that for systems that it doesn't work on, no javascript is being sent at all. This makes me think that it's something to do with the HTTP_ACCEPT headers. See if they're different using this link which should show a little different information about your headers:
http://nerdlabs.org/tools/request_headers.php

If this is right then it is a problem with their ASP code.

Is this what you were after?

The ACCEPT headers look the same to me. Is there something else I should check?

Good News - Bad News
 

The bad news is that the site is no longer working with the PowerBook, where it used to work just fine.

The good news is that I don't have to worry about it anymore. No "good" machine to compare with now.

I wonder if they'd updated the page and you just had one system that was still using a cache. Does anyone still get the reply form?

I just tried to load that page again this morning and get this:

Service Unavailable

and I suspect there's a 'Web Help Wanted' ad circulating around the Thai Craigslist right about now. :rolleyes:

Solution
 

As noted above this web site was down for a while. It's now back up and one addition indicates a probable solution for the trouble I was having. The ThaiJustice web board main page now contains a bright red, scrolling notice that says that the web board policy is that you must have visited the site five days in order to reply to posts and seven days to post a new topic. This notice does not appear on the Thai Justice main page nor on the pages that display individual threads.

This policy is enforced using cookies.

This explains why only one of my machines could "see" the reply form. That's the machine my wife usually uses. And, she uses whatever account happens to be logged in at the time. That's why I could see the reply form on that machine from my account, but not on any other machine.

The crazy thing is that although they use the cookies to decide whether or not to display the reply form, they continue to display an anchor link to the reply form [<a href="#wwbaaddfm">คลิกที่นี่</a>] at the bottom of the page even when the reply form is not there.

Unfortunately this does not explain how acme.mail.order was able to download the page source, including the reply form, using curl.