It’s not so obvious how to test a scenario, where user clicks an export link and gets a CSV file download. Especially if the user is able to filter the generated CSV content before downloading.

Capybara’s Poltergeist engine is a wrapper for headless browser called PhantomJS. A basic Capybara spec to test the downloading might look like this:

1
2
3
4
5
6
7
8
9
10
11
12
13

it 'outputs valid csv with status' do
  login candy_user
  visit candy_invoices_path

  select 'sweets', :from => 'Type'
  click_button 'Apply filter'
  click_link 'Export Invoices'

  page.text.should eql(<<-CSV)
Customer;Month;Invoice #;Amount;Difference;Status;Type
Name 0;September 2002;INVOICE0;-;-;new;sweets
  CSV
end

We select a filter from a select box, apply the filter and download the CSV. The problem here is that Poltergeist starts a download when clicking the export link and the page.text contains the previous page, not the CSV one might expect. There seems to be no clean solution for this.

An easy way out is to register a different mime type for CSV files for the tests, so we can trick Poltergeist to render the CSV instead of downloading it:

1
2
3
4
5
6
7
8
9

before do
  Mime.send(:remove_const, :CSV)
  Mime::Type.register 'text/plain', :csv
end

after do
  Mime.send(:remove_const, :CSV)
  Mime::Type.register 'text/csv', :csv
end

And now our page.text contains the data we want.

Some of you have seen an error when installing debugger gem for a specific version of Ruby.

1
2
3
4
5
6
7
8
9
10
11
12
13

Using debugger-ruby_core_source (1.1.1) 
Installing debugger-linecache (1.1.1) with native extensions 
Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native
extension.

/home/pimeys/.rvm/rubies/ruby-1.9.3-p194/bin/ruby extconf.rb 
checking for vm_core.h... no
checking for vm_core.h... no
Makefile creation failed
**************************************************************************
No source for ruby-1.9.3-p194 provided with debugger-ruby_core_source
gem.
**************************************************************************

This happens for several Ruby patchlevels because the debugger-ruby_core_source gem contains only a handful of headers. Luckily this is pretty easy to fix. It is possible to install the missing headers with the provided rake task.

1
2
3
4
5
6

> gem contents debugger-ruby_core_source
/home/pimeys/.rvm/gems/ruby-1.9.3-p194@gemset_local/gems/debugger-ruby_core_source-1.1.1/CHANGELOG.md
...

> cd /home/pimeys/.rvm/gems/ruby-1.9.3-p194@gemset_local/gems/debugger-ruby_core_source-1.1.1/
> rake add_source VERSION=1.9.3-p194

Go back to your code directory and re-run bundle install. The debugger-linecache should now install correctly.

The problem is incredibly simple to explain and understand: you need to request or post data from/to a web page in a domain to a server on another domain with JavaScript, using the HTTP(S) protocol. As you’ll soon find out, a simple and straightforward method doesn’t exist (yet). You’ve hit the Same-origin policy wall. Once you dig into the various possibilities (and their limitations and problems) you’ll eventually develop the urge to run in a galaxy far far away.

First of all, let’s try to be precise here: when we speak about “domain” we are only saying part of the truth. The focal point is indeed the origin. Not only the hostname must match but also the protocol and the port. This is why there will be no joy between http://example.com/x.html and http://example.com:3000/ and not even between https://example.com/x.html and http://example.com. No way to touch http://www.example.com from http://api.example.com/x.html. Sorry.

Why?, you ask squinting. It’s all about security. It’s for your own good. Trust them.

Let’s briefly discuss which are the most common options you have to overcome this limitation.

• Cross-origin resource sharing (you’ll always use the much cooler CORS acronym for that). CORS is nothing more than the good old XMLHttpRequest, with an additional HTTP header: Access-Control-Allow-Origin sent by the server (or the application). With that header the server is telling the browser that he’d permit requests from whoever matches the value of that header (with an “*” meaning “everyone is welcome”). CORS is nice and simple, but you need control over the server (to send the header) and support from the browser (to apply the policy, parsing the new header). Needless to say, the problem with browsers is Internet Explorer: we can use CORS only with IE8 onward - and it also needs a special treatment. You might want to take a look at the advocating site for CORS for more details
• Flash based solutions. The Adobe Flash plugin can open sockets and use a variety of protocol to speak with our server (or others’). You can also easily interface your Flash movie with JavaScript exposing a set of API. You can then write a movie that wraps an HTTP communication layer for you JavaScript to use. There is still the same-origin policy in place (it’s all about security, remember that), but that limit can be weakened deploying a particular file on the server, namely crossdomain.xml. The file contains instructions to white-list which domains can access the resources from the server; more on this on Brightcove. Admittedly, this solution is fairly good: it’s cross browser, robust and Flash is - so far - widely available, but you still need control over the server (deploying the crossdomain.xml file). You should also probably want to use a ready to use library: I could suggest you flXHR, if only the project would still be actively maintained. Please note that there is another fairly important limitation: as far as I know, you cannot issue synchronous HTTP requests with Flash.
• JSONP. If you think for a moment to the cross domain problem you eventually realize an important thing: browser’s job is to fetch data to render your web page and this data (images, CSS, HTML, JavaScript) is naturally fetched from a whole lot of different origins already. There is no “same origin policy” for images. Or JavaScript files in your HEAD element. This is the trick: if you need to get some data from a server of yours, just use some JavaScript code that creates a new SCRIPT tag in the browser. The src attribute of the dynamically created script will point to a JavaScript snippet (server created) containing a function (the “callback”) which nicely wraps a JSON object (your data). Once the data is received, the function will be called and - voilà - the data will be available as a pure JavaScript object.

JSONP doesn’t need any particular plugin or support from the browser. It’s nothing more than a clever hack, plain and simple, but it works very well. You only still need control on the server (as you might have inferred, there is no way out from this server control thing, for any solution. You’ll never be able to communicate with “google.com” with JavaScript). Even if it’s relatively simple to implement, you want to use a library to add JSONP capabilities to your scripts. jQuery has JSONP support, of course, but the JSONP plugin might be a bit more robust regarding error control.

JSONP suffers of a big limitation, though. First of all, just focus on these points:

• JSONP is not properly Ajax. At least, it doesn’t use XMLHttpRequest at all
• JSONP will always use an asynchronous HTTP GET
• JSONP relies completely on how the browser (not your code) fetches the script from the server. There is no “standard” way, on this. I could write a browser that will not fire the “onload” event after a successful load of the script and you’ll be screwed (the callback will not be called)

The last point is the most important one: a corollary to it is that error management with JSONP is a problem. You’d like to get fine control over errors that could eventually arise trying to fetch important data from the server. With XMLHttpRequest this is a solved problem: you have the fine “error callback” that every library will give you for free… any HTTP status code that is not 200 will fire the callback and you’re safe: you access the error object with all the data you need to understand what is going on.

That’s not the case with JSONP. Let’s dig deeper into this mess, using the jQuery JSONP implementation:

• Same domain, HTTP Status 404 the request will be converted to a normal XHR call, so the error is raised and you get the full XHR object to play with
• Same domain, HTTP Status 500 the request will be converted to a normal XHR call, so the error is raised and you get the full XHR object to play with
• Cross domain, HTTP Status 404 the request fails silently in every browser (the error callback is not even called).
• Cross domain, HTTP Status 401, 403 the request fails silently in every browser EXCEPT Chrome. In Chrome the error callback is called with the parsererror error
• Cross domain, HTTP Status 500 the request fails silently in every browser EXCEPT Chrome. In Chrome the error callback is called with the parsererror error
• Cross domain, HTTP Status 200, bad JSON error callback is fired in every browser (additionally, a JS error is raised). Status is 200 and error is “parsererror”

This report is only partial, not accurate and it will possibly change in time, but the message is clear: if you need fine errors control over the communication, you just don’t want to use JSONP.

The long term solutions for cross domain communication are then CORS (solid, standard, future proof) or some kind of tecni que stack involving cross domain messaging using HTML5 postMessage and XMLHttpRequest (using an hidden iframe and some juggling).

But cross domain messaging is another big topic that surely deserves a post for its own.