How can we use JavaScript to get a web page between domains without the use of XMLHttpRequest? Is there a plug-in that could do that?


using JavaScript, it is much needed to get some pages from the web using without actually moving from the current page and hidden from the user's eyes.

To request a web page without showing it to the user, it is easy to use XMLHttpRequest but it has its own limitations most importantly it does not retrieve cross-domain pages very well. For security reasons the browsers (Mozilla FireFox 3.6+ in my case) retrieve a header from the target site and if the referrer's location is allowed access in that header, only then will the browser continue getting the target web page and JavaScript can only then parse the retrieved info.

This causes the XMLHttpRequest to work with some pages and not work with others if you are trying to access cross-domain pages. Of course it works well if you need to retrieve the information from the same location as the referrer page where the XMLHttpRequest is located.

This is a big problem, when security is not really no 1 priority. For example, imagine writing a script for retrieving live data from a statistics-producing web site or imagine a bot that needs to retrieve data from an online gaming web-site.

Now, how can JavaScript be used to get pages from other domains (cross-domain reference)?

I thought maybe we could find a plug-in that does the job (of course after installation upon user's permission) and then use its properties by JS and eliminate the need for XMLHttpRequest. Do you know any such plug-in or another roundabout for this problem? (ie get cross-domain data by JS without XMLHttpRequest) of course we cannot use XMLHttpRequest as we don't have any control over the target page headers and we obviously want to hide the whole process from the user

You’ll find that it’s the priority that the target site puts on their own security that is most important. If they're unconcerned about JavaScript on other sites accessing their site, they can set the HTTP Access Control headers for cross-domain XMLHTTPRequest, provide a crossdomain.xml file for Flash, provide a JSONP API, or provide some hooks for iframe monitoring.

The second solution is to make the requests to a server on your domain which proxies the request to the target site. In certain circumstances you may be able to use a third party server which supports cross-domain or JSONP requests, like Yahoo! Pipes.

If neither of these is feasible, you'll need to convince the user to allow you to run your own code on their PC. This could be via a signed Java applet which requests special permissions, or your own custom browser plugins or extensions.