Threaded XMLHttpRequest In Shoes

August 15th 17:36
by why

Threads can be tough and don’t suit beginners very well. And, well, Ruby threads can tie up the main app thread.

So, Shoes steals the underpinnings of Ajax to give you asynchronous downloads without needing to get into threading. Many of the young Sneakers are building Twitter and Flickr apps; it seemed the morally upright thing to do. In addition, I was able to use these HTTP threads to load remote images in the background. So, in Shoes, images loaded from the web will appear as they load.

Here’s the simple-downloader.rb from the samples that come with Shoes:

To achieve this, Shoes uses platform code for both threading and HTTP. On Windows, CreateThread and WinHTTP. On Linux, pthread and curl. And, on OS X, NSURLDownload — which does the threading for you.

Downloading is reduced to a single line:

Shoes.app { download "http://shoooes.net/shoes.png", :save => "shoes.png" }

This happens asynchronously, so shoes.png won’t be there yet when this method ends. It might be huge. It might appear an hour later. You can attach a finish event to be notified when the download is complete.

Shoes.app do
  download "http://shoooes.net/shoes.png", :save => "shoes.png" do |dl|
    alert "Scuse me. Your shoes.png has arrived."
  end
end

Omit the :save option and you can get back the download as a string.

Shoes.app do
  download "http://hacketyhack.net/pkg/osx/shoes" do |dl|
    alert "The latest OS X download is: #{dl.response.body}"
  end
end

You can also attach :method, :headers and :body options to the download, if you want to customize the request beyond that. I studied XMLHttpRequest closely and tried to be sure the same things could be done with this.

As for events, you get four of them: start, progress, finish and error. You can either pass proc objects in as options:

Shoes.app do
  url = "http://shoooes.net/dist/shoes-0.r905.exe"
  status = para "Downloading #{url}"

  download url, :save => "shoes.exe",
    :start => proc { |dl| status.text = "Connecting..." },
    :progress => proc { |dl| status.text = "#{dl.percent}% complete" },
    :finish => proc { |dl| status.text = "Download finished" },
    :error => proc { |dl, err| status.text = "Error: #{err}" }
end

Or, use the method syntax:

Shoes.app do
  url = "http://shoooes.net/dist/shoes-0.r905.exe"
  status = para "Downloading #{url}"

  get = download url, :save => "shoes.exe"
  get.start { |dl| status.text = "Connecting..." }
  get.progress { |dl| status.text = "#{dl.percent}% complete" }
  get.finish { |dl| status.text = "Download finished" }
  get.error { |dl, err| status.text = "Error: #{err}" }
end

The last thing I will mention is that every queued download is attached to the window containing it. When you close the window, the download stops. So, if you’re queueing a download from a temporary popup, be sure to queue it on the main app window.

12 comments

bonzo

said on August 15th 14:24

What happens when you use method syntax and the download finishes before you set any of the other events up?

_why

said on August 15th 16:45

Originally I was going to say that the download doesn’t actually start until the next repaint. But I checked the code and that isn’t so. Going to change that now, thanks, bonzo!

Jim

said on August 15th 17:13

How about uploading?

Semaphore Horse

said on August 15th 17:23

Ooh, that is rather fancy! I wish the interfaces in the browser were as nice and simple as this!

So, adding a download causes a repaint even if you have changed none of the actual UI? How delightfully hacky.

The one little niggling inkling that is bugging me, while this sure is powerful enough to do all your http doings as far as I can see, it provides no easy interface for file uploads. If you could add a :upload parameter which results in setting the request content-type to multipart, and doing all that mime stuff for us, that’d be great.

No no, wait, that’s a bad idea! Here’s a better one:

When :body.is_a?(Hash) we loop through all :body’s .values, if any of them are File’s, we change the default Content-Type to multipart/form-data, and do multipart encoding for the file uploads.

A convenience :params option would be funky too, which just translates a hash in to to_s’d key and value pairs and sets that as the URL’s query variable. Extra awesomeness points for decoding any existing params in the URL , and merging the new ones in with the existing ones. Half an awesomeness point for instead just adding a & to the end and appending the new stuff when a query string is already present.

None of this couldn’t be implemented in pure ruby on top of this thing, but there are some Super BonusMonkeyAwesomenessBall points in it for you if it’s implemented in such a way that file uploads are streamed out to the web server, and the file isn’t completely loaded in to system memory at any point for those lovely 20gig uploads the kids are so fond of these days.

Semaphore Horse

said on August 15th 17:25

@Jim: Well, you could do your multipart encoding, set the Content-Type header to multipart/form-data, set the method to POST , shove the encoded stuff in to @body, and glory in the amazing effort of doing it yourself.

_why

said on August 15th 17:32

I think upload will go like this:

upload "shoooes.png", :save => "http://shoooes.net/upload"

With all of the same other events and options. I have been meaning to scrutinize something like SWFUpload before going too far. Multiple file uploads? If you wanted to send a string do you pass in a StringIO?

loincloth

said on August 15th 17:39

vury kool, thx thx thx

Semaphore Horse

said on August 15th 18:00

So what if you pass in a StringIO? It’d get to_s’d like everything else that isn’t a File. Mind you, a StringIO might be a smart way to do file uploads from sources that aren’t files! I like my way, because it’s obvious how multiple file uploads work, also, your way may very well be good for PUT uploads, but nothing uses them. What we need is multipart POST uploads, like forms do on the web. These can have multiple files, all the files have a filename and a content type, as well as a variable name. Most importantly, your syntax needs to at least require the variable name, and also simply must provide a way to specify other variables to include along with it for authentication and the likes.

Does the cache work on the same rules as XMLHTTPRequest? I mean, stuff like, non-get requests are never cached? And does it respect the various competing ways of doing cache control from the server side, if so, which headers are supported?

To properly support multipart file uploads, the syntax for upload becomes a lot like the syntax for download. We might as well call download ‘request’ and if we have upload and download, make them shortcuts to the fuller functionality of a ‘request’… or do we call it a web_request? or a http_request? or maybe we just call it http? hrmm. Also, there’s another crazy possibility that Elliot Cable once showed me.

If http is a method, which accepts a symbol called ://something.blah as an argument, and symbols implement the / operator… well, it suddenly becomes rather possible to write simple http url’s inline in your code and have that be meaningful.

Semaphore Horse

said on August 15th 18:08

This sort of crazy thing: http://gist.github.com/5673

Kinda crazy, but you get the idea.

Birthday Pony

said on August 15th 23:40

I suppose it would work fine to have the syntax of upload be: upload “ URL ”, {:param_name => File.open(“whatever.mp4”)}

MenTaLguY

said on August 27th 16:19

Thank heavens. Threads really aren’t the tool for this stuff.

MenTaLguY

said on August 27th 16:20

Well, raw threads as an API anyway.

Comments are closed for this entry.