hoodwink.d enhanced


Nearing Greasemonkey for Browser-of-Choice with the Hoodlum Proxy! #

by why in inspect

Here it is. Day five of the Hoodwink’d docu-drama. Can you believe this? Here, look. We are left with a smoldering meteor from an alien planet in our hands:


No more greasemonkey.

No more HOSTS entry.

Run hoodlum.rb.

Firefox > Preferences > General > Connection Settings… > Manual Proxy > + 37004

Not yet for Safari and IE.

There’s so much to say about this proxy. I’ll start by saying that I barely wrote any of the code. MenTaLguY wrote the good stuff. And Goto Kentaro wrote the really good stuff because he wrote WEBrick::HTTPProxy, which we’re just clawing inside and operating from within.

The overhead view is this:

  1. The prewink method rewrites the request. We serve cached stuff from within the proxy (so the JS file doesn’t get retrieved everytime you go anywhere) and mimick HOSTS.
  2. The upwink method injects a Javascript into the page.
  3. The fixup_script method replaces GM_API methods with HoOdLuM_API methods. This is where browser compatibility stuff will go. We could end up implementing all of Firefox inside IE and Safari, though. So this is still up in the air.

I still want to see what MrCode’s got. Wonderland could be a lot cleaner and could use Ruby alot more effectively. For now, Hoodlum sits on the Javascript side.

said on 26 Aug 2005 at 19:16

why’s being modest. All the prewink goodness is his work, and I consider it the hard part.

said on 26 Aug 2005 at 19:29

One advantage to doing it from the Javascript side is that (at least on a fast connection) it makes browsing feel a lot faster since webrick doesn’t stream.

Also because of that I wouldn’t really want to consider attempting a large file download through the webrick proxy. You’d end up loading the whole blob into memory—not cool.

said on 26 Aug 2005 at 19:45

Hmmm, those may well be valid issues MenTaL. For now I shall ignore them, muwhahahahaha.

Plus I still consider Wonderland to be a bit of a prototype, even if it ends up being cleaner (or at least more flexible) than Hoodlum.

In the end, Greasemonkey may be the best solution, but we won’t know until we try.

said on 26 Aug 2005 at 19:46

why, mightn’t stopping at just a webproxy be a little premature? I’ll amuse myself with the delusion that you all have read Kottke’s thoughts on what an Internet Operating System would be like, basically, a flexible web application server integrated onto each computer allowing for the development of one version of a unified Internet Application serving both online and offline purposes.

With StarMonkey (or my favorite alternative, GreasyMonkey) as our “web hacks layer” on top of another WEBrick server handling the guts of the so-called iOS, Ruby could be the lingua franca of the Next Next Big Thing.

Are we not madmen powered sheerly by Delusions of Ultimate Grandeur? Of making Ruby the ONE TRUE Language?! Dare we not dream of Programs that dream of US?!



Sorry, I got a little carried away.

Did that make any sense?

said on 26 Aug 2005 at 19:49

MrCode: Oh, it should be possible to make the proxy stream, given some work, it just isn’t done by default.

said on 26 Aug 2005 at 20:19

MenTaL: I’m not worried about making the proxy stream, I’m worried about the fact that most Greasemonkeyish hacks require (or would work best with) a full document. In other words, we need to fully load the HTML in the proxy, pass it off to scripts for their hackery, then send the result to the browser (that is the current design of the in-progress Wonderland, StarMonkey, [insert favorite name here].)

That becomes much trickier when we are streaming and dealing with partial DOMs.

Danno: I’ll have to read that Kottke stuff…is that found at http://www.kottke.org?

said on 26 Aug 2005 at 20:31

MrCode: well, if you only stream the things that you aren’t starmonkeying, it would still make things a lot better.

Not streaming the things that ARE getting starmonkeyed is, I think, an acceptable tradeoff.

said on 26 Aug 2005 at 20:42

MenTaL: OK, I agree. I’ll add that to the as-yet-nonexistant TODO list.

Of course all it takes is one script with an @include = [/.*/] to make everything non-streaming. I think several such Greasemonkey scripts exist.

said on 26 Aug 2005 at 21:09

It’s good to see that my minions are hard at work! Mwaaaahaaahaaha!!!

Danno: Now you’re talking. Your dream of an iOS powered by Ruby is just about the right size. Yes, it’s amazing the things that are happening in this little corner of the web. Who would have guessed that something so historic would be born right here in the RedHanded comments section? What’s that rumbling I hear off in the distance? Could it be…. the end of the world as we know it? (oh, and I really like that bit about programs that dream of us – can we incorporate that into 2.0?)

MenTaLguY: any ideas for fixing that downloading problem?

said on 26 Aug 2005 at 21:31

Oh, and MonkeyBacon! Very Poignant.

said on 26 Aug 2005 at 21:37

I’m running that hoodlum.rb script and I set the proxy as indicated, but when I go to the hoodlum URL I get ‘service not available’. Am I doing something wrong, or is the hoodlum thingy down?

said on 27 Aug 2005 at 10:38

(I am the Greasemonkey guy)

Wow, awesome.

I started something like this in python, but couldn’t figure out how to make pages that use http compression work.

I don’t think you really need to change the GM_ references to HoOdLuM_API references since GM_ is not defined in the first place.

Just define it differently, so that it has cross-browser stuff and sends the munged URL to hoodwink.d instead of directly.

Or am I missing something?

said on 27 Aug 2005 at 10:44

Minor Correction to my original post: My favorite alternative name to StarMonkey is GreasyBacon

said on 27 Aug 2005 at 10:47

MrCode: Well, sure, but that’s still only affecting HTML files. You’re never going to be starmonkeying e.g. image files or tarballs, are you?

said on 27 Aug 2005 at 10:48

boogs: That’s precisely what I’ve done in the most recent version of hoodlum.rb, though why hasn’t posted my changes as of yet.

said on 27 Aug 2005 at 11:28

It’s not working for me! I have hoodlum.rb running.. proxy settings… and i see hoodlum spitting out everything that I ma doing… yet see no winkage links!

said on 27 Aug 2005 at 11:36

Robby: try going to http://hoodwink.d immediately after starting the proxy, and log in. You’ll get a 404 after doing so, but that will still prime it.

said on 27 Aug 2005 at 14:52

boogs: Hey, welcome. Thankyou for Greasemonkey. All the winkers thank you, positively.

I think the proxy’s biggest weakness is that our GM_API functions will be readily available to the page. We’re using a security token embedded in the Javascript source (which the proxy honors for cross-site requests), but I’m not sure how to keep it from scripts on the page. So.. the same old stuff you hit with the 0.3 series.

Hoodlum doesn’t do http compression, it just disables it by stripping the outgoing Accept-Encoding header.

said on 27 Aug 2005 at 15:58

Update: Wonderland will support HTTP compression (at least for gzip.) I just got it working. It isn’t that hard.

MenTaL: I was considering making images and other files besides HTML also hackable by Wonderland, but since it might be a rare thing I probably won’t do it for the initial version. I’ll look into the streaming as well.

P.S. As you may notice from my posts, I think I’m going to stick with Wonderland as the name for this project. Mostly because _why posted about it above :)

said on 27 Aug 2005 at 22:37

why: Yes, the keeping of the functions from the page content will be a hard one.

I posted about that in part IV, but I’ll restate here … I think the whitelisting of xmlhttp-able URLs is the best idea which has been proposed. It keeps the list of possible targets pretty small. If you’re running bookburro, then amazon.com can access powells.com, but why would they?

Also, you will have the same problems as Greasemonkey scripts where the page content can hijack DOM methods to counteract scripts. But that’s a smaller problem.

said on 28 Aug 2005 at 18:10

I think we can automatically whitelist the domain the script came from.

said on 28 Aug 2005 at 18:53

The question is, if we still can’t actually get to http://hoodwink.d/, is there any way to use this? Shouldn’t read-only access work even if the user isn’t logged in? :-/

Comments are closed for this entry.