2019-01-25 - Progress - Tony Finch
Earlier this month I wrote notes on some initial experiments in browser automation with WebDriver. The aim is to fix my superglue DNS delegation update scripts to work with currently-supported tools.
In the end I decided to rewrite the superglue-janet script in Perl,
since most of superglue is already Perl and I would like to avoid
rewriting all of it. This is still work in progress; superglue is
currently an unusable mess, so I don't recommend looking at it right
now :-)
My WebDriver library
Rather than using an off-the-shelf library, I have a very thin layer (300 lines of code, 200 lines of docs) that wraps WebDriver HTTP+JSON calls in Perl subroutines. It's designed for script-style usage, so I can write things like this (quoted verbatim):
# Find the domain's details page.
click '#commonActionsMenuLogin_ListDomains';
fill '#MainContent_tbDomainNames' => $domain,
'#MainContent_ShowReverseDelegatedDomains' => 'selected';
click '#MainContent_btnFilter';
This has considerably less clutter than the old PhantomJS / CasperJS code!
Asyncrony
I don't really understand the concurrency model between the WebDriver server and the activity in the browser. It appears to be remarkably similar to the way CasperJS behaved, so I guess it is related to the way JavaScript's event loop works (and I don't really understand that either).
The upshot is that in most cases I can click on a link, and the
WebDriver response comes back after the new page has loaded. I can
immediately interact with the new page, as in the code above.
However there are some exceptions.
On the JISC domain registry web site there are a few cases where selecting from a drop-down list triggers some JavaScript that causes a page reload. The WebDriver request returns immediately, so I have to manually poll for the page load to complete. (This also happened with CasperJS.) I don't know if there's a better way to deal with this than polling...
The WebDriver spec
I am not a fan of the WebDriver protocol specification. It is written as a description of how the code in the WebDriver server / browser behaves, written in spaghetti pseudocode.
It does not have any abstract syntax for JSON requests and responses - no JSON schema or anything like that. Instead, the details of parsing requests and constructing responses are interleaved with details of implementing the semantics of the request. It is a very unsafe style.
And why does the WebDriver spec include details of how to HTTP?
Next steps
This work is part of two ongoing projects:
I need to update all our domain delegations to complete the server renaming.
I need automated delegation updates to support automated DNSSEC key rollovers.
So I'm aiming to get superglue into a usable state, and hook it up
to BIND's dnssec-keymgr.