3 ms·

The Web Extension APi doesn't seem too suitable for browser automation as it doesn't have support for simulating key presses or mouse movements (other than HTML

The Web Extension APi doesn't seem too suitable for browser automation as it doesn't have support for simulating key presses or mouse movements (other than HTMLElement's click method). Yeah, you can simulate events, but that can be a lot of work (e.g. typing an 'a' key might require you to simulate all of keydown, keypress, and keyup and set various non-standardized properties on them). And that won't even work in a standard text input as isTrusted is set to False on events you generate. Simulating something like a Tab key press will require you writing code to try to replicate your browser's logic in determining what the next element should be. Why would this be preferable over something like Selenium?

timwis 9y ago
But we can just create/use a JS library that simulates typing. It would work for an app that needs it, or on your automation environment because it's just using the DOM API. Is selenium doing something special beyond triggering those events?
- scottfr 9y ago
  The point is you can't accurately simulate typing without something like Selenium. Neither the standard DOM API nor the additional features provided by the Web Extension API allow you to do so. You can simulate some parts of typing by mimicking browser behavior on a case by case basis, but there are places where this will be strictly impossible. For example, if you are working with anything that checks the isTrusted event bit, you're out of luck as you there is no mechanism for you to set that to true. Selenium on the other hand is actually triggering events as if a person physically triggered them. So, for instance, the isTrusted bit will be set to true when you use the send_keys method in Selenium.
  - timwis 9y ago
    Interesting. Do you know how selenium does that?
    - jarvuschris 8y ago
      Looks like it's a feature of the WebDriver API: https://www.w3.org/TR/webdriver/#element-interaction https://www.w3.org/TR/webdriver/#element-interaction
foob 9y ago
The Remote Browser framework is designed to be an API where interactions occur at a much lower level than they do in a library like Selenium or Puppeteer. In many ways, it's more similar to the Chrome DevTools Protocol than it is to Puppeteer. It's more a tool for building libraries with user-friendly interfaces than it is a user-friendly interface in and of itself. Like you mention, simulating user interactions with the DOM is relatively easy. The idea here isn't that you would implement that sort of thing yourself, it's that higher level libraries will be built which incorporate this sort of functionality for you. We have several projects in the works here at Intoli that fall into this category, and we're looking forward to rolling them out in the near future.

timwis 9y ago

But we can just create/use a JS library that simulates typing. It would work for an app that needs it, or on your automation environment because it's just using the DOM API. Is selenium doing something special beyond triggering those events?

scottfr 9y ago
The point is you can't accurately simulate typing without something like Selenium. Neither the standard DOM API nor the additional features provided by the Web Extension API allow you to do so. You can simulate some parts of typing by mimicking browser behavior on a case by case basis, but there are places where this will be strictly impossible. For example, if you are working with anything that checks the isTrusted event bit, you're out of luck as you there is no mechanism for you to set that to true. Selenium on the other hand is actually triggering events as if a person physically triggered them. So, for instance, the isTrusted bit will be set to true when you use the send_keys method in Selenium.
- timwis 9y ago
  Interesting. Do you know how selenium does that?
  - jarvuschris 8y ago
    Looks like it's a feature of the WebDriver API: https://www.w3.org/TR/webdriver/#element-interaction https://www.w3.org/TR/webdriver/#element-interaction

foob 9y ago

The Remote Browser framework is designed to be an API where interactions occur at a much lower level than they do in a library like Selenium or Puppeteer. In many ways, it's more similar to the Chrome DevTools Protocol than it is to Puppeteer. It's more a tool for building libraries with user-friendly interfaces than it is a user-friendly interface in and of itself. Like you mention, simulating user interactions with the DOM is relatively easy. The idea here isn't that you would implement that sort of thing yourself, it's that higher level libraries will be built which incorporate this sort of functionality for you. We have several projects in the works here at Intoli that fall into this category, and we're looking forward to rolling them out in the near future.