3 ms·

> Doh, yes Selenium Grid uses Selenium. And the argument about no major JS using Selenium is not an argument to not use it. Sure it is. If a developer will nev

by jeresig 18y ago

> Doh, yes Selenium Grid uses Selenium. And the argument about no major JS using Selenium is not an argument to not use it. Sure it is. If a developer will never switch to Selenium then Selenium Grid (in its current state) is useless. TestSwarm is completely test suite agnostic - which is huge. I don't know of a single Open Source JavaScript projects which uses Selenium. > Unless you right the appropriate daemon for those devices and integrate it into the existing Selenium Grid set up. Surely that's easier than starting afresh? It absolutely is. There are no daemons in TestSwarm. It's a single URL that you can open in any browser. No 3rd party software is required to be installed. > Huh? That's just bizarre. If you've got some new browser then reach Selenium how to start it. It's not that hard. So the only solution for an unknown browser is to wait for the Selenium team to add it? That's unacceptable. New browsers must be added instantly and on-demand (which TestSwarm handles). > This is solved by virtualization. No it's not, not at all. A virtualized environment requires a few things: A machine on which to run it and enough resources to run as many virtualizations as required. As an Open Source project we don't have that sort of money. Second, virtualization is insufficient for testing. Within the Mozilla project we've discussed problems with virtualization and how they are not always precise. Nothing is better than running the tests directly on an actual user's machine. Third, virtualization does not solve the problem of manual testing. We want to get tests in front of users wherever possible - throwing them to some dumb server does nothing to help us.

jgrahamc 18y ago
> So the only solution for an unknown browser is to wait > for the Selenium team to add it? That's unacceptable. New > browsers must be added instantly and on-demand No, the solution, as with any FLOSS project, would be for you (or someone else) to add the browser to Selenium and not wait for the Selenium team to do it. I don't understand the requirement for adding new browsers 'instantly and on-demand'. > Second, virtualization is insufficient for testing. Within > the Mozilla project we've discussed problems with > virtualization and how they are not always precise. Nothing > is better than running the tests directly on an actual > user's machine. I'd be curious to hear those arguments and to have examples of when virtualization is not sufficient and you need to go to the actual user's machine.
- jeresig 18y ago
  > I don't understand the requirement for adding new browsers 'instantly and on-demand'. Because the major JavaScript libraries' timetables should not be bound by how willing they are to twiddle config files/write new browser loaders in Java. It shouldn't be anymore complicated than hitting a button that says "Add this browser as a new test base." > I'd be curious to hear those arguments and to have examples of when virtualization is not sufficient and you need to go to the actual user's machine. Timing, performance analysis, and CPU/memory availability are huge ones. Being resource constrained in a VM is not an acceptable approximation of what a user's desktop actually is. Nor is running a mobile browser in a simulator. Nothing beats running an actual browser on an actual desktop. If you were to argue that you could have no resource constraints in your VM, that brings it back to the argument of cost and resources.
  - jgrahamc 18y ago
    > Timing, performance analysis, and CPU/memory availability are huge ones. Being resource constrained in a VM is not an acceptable approximation of what a user's desktop actually is. Fair enough. Don't you run into the same problem with volunteer machines though? You can't actually control their configuration in TestSwarm so you can't control the resources (e.g. you can't control what else the machine is doing). If you are going to do do any serious analysis of timing or performance then you are going to have to control the machines.
    - jeresig 18y ago
      > If you are going to do do any serious analysis of timing or performance then you are going to have to control the machines. Absolutely - the timing and performance won't be more accurate but it will be more realistic. For accuracy everything has to be done on the same machine in a 'clean' environment (and definitely not in a VM).

by jeresig 18y ago

jgrahamc 18y ago

> So the only solution for an unknown browser is to wait > for the Selenium team to add it? That's unacceptable. New > browsers must be added instantly and on-demand No, the solution, as with any FLOSS project, would be for you (or someone else) to add the browser to Selenium and not wait for the Selenium team to do it. I don't understand the requirement for adding new browsers 'instantly and on-demand'. > Second, virtualization is insufficient for testing. Within > the Mozilla project we've discussed problems with > virtualization and how they are not always precise. Nothing > is better than running the tests directly on an actual > user's machine. I'd be curious to hear those arguments and to have examples of when virtualization is not sufficient and you need to go to the actual user's machine.

jeresig 18y ago
> I don't understand the requirement for adding new browsers 'instantly and on-demand'. Because the major JavaScript libraries' timetables should not be bound by how willing they are to twiddle config files/write new browser loaders in Java. It shouldn't be anymore complicated than hitting a button that says "Add this browser as a new test base." > I'd be curious to hear those arguments and to have examples of when virtualization is not sufficient and you need to go to the actual user's machine. Timing, performance analysis, and CPU/memory availability are huge ones. Being resource constrained in a VM is not an acceptable approximation of what a user's desktop actually is. Nor is running a mobile browser in a simulator. Nothing beats running an actual browser on an actual desktop. If you were to argue that you could have no resource constraints in your VM, that brings it back to the argument of cost and resources.
- jgrahamc 18y ago
  > Timing, performance analysis, and CPU/memory availability are huge ones. Being resource constrained in a VM is not an acceptable approximation of what a user's desktop actually is. Fair enough. Don't you run into the same problem with volunteer machines though? You can't actually control their configuration in TestSwarm so you can't control the resources (e.g. you can't control what else the machine is doing). If you are going to do do any serious analysis of timing or performance then you are going to have to control the machines.
  - jeresig 18y ago
    > If you are going to do do any serious analysis of timing or performance then you are going to have to control the machines. Absolutely - the timing and performance won't be more accurate but it will be more realistic. For accuracy everything has to be done on the same machine in a 'clean' environment (and definitely not in a VM).