Headless browsers and proxy authentication

For my most recent gig, I’ve been trying to drive Firefox headlessly using Selenium and a pretty excellent Elixir library called Hound.

One of the tasks I needed to accomplish was to connect to sites through a proxy server that required a username and password.

This is relatively trivial to do in Firefox when you have an interface, and you are able to type in the credentials when the box pops up. But driving it headlessly is a different animal – I spent the better part of the day trying to figure out where this username and password should live and how to even pass it through my app to begin with.

Save yourself the trouble and set up a separately-running proxy running on your local system (using a package such as Squid). This should be able to effortlessly connect to the proxy and authenticate, and all you would need to do is connect to your own local proxy without a user/pass combination. This is a great way to separate the connection logic from the application logic, and if you ever want to not use the proxy, just point to a different Firefox profile (or however your browser of choice accepts proxy configurations).

Posted in Uncategorized

Leave a Reply

Your email address will not be published. Required fields are marked *