4 ms·

There are also open source tide and wave models, which is what researchers use and are fairly accurate. Not sure how much this commercial product differs, but I

There are also open source tide and wave models, which is what researchers use and are fairly accurate. Not sure how much this commercial product differs, but I know the Navy etc. use the open source ones with in house adjustments.

mkbosmans 4y ago
Sure, but tide and wave models generally cover a much larger area, such as whole coastal area's and over larger time spans. An example of such a model would be: https://www.youtube.com/watch?v=eN6CDaoMZ7U https://www.youtube.com/watch?v=eN6CDaoMZ7U In the Netherlands this is used e.g. to determine when to close the storm surge barriers to protect the river delta area from flooding due to a tide+storm combination. In contrast, the Next Ocean product is meant to model the direct area around a single ship, predicting minutes in advance with a second by second granularity. They are able to reuse the raw data coming from the navigation radar that is already installed on every sea-going ship. It uses the backscatter from water ripples to determine the wave field around the ship. Interestingly, for navigational purposes exactly this part of the data is considered noise and filtered out by the on board navigation system. (https://nextocean.nl/technology.php https://nextocean.nl/technology.php) [I did some work on the software implementation for Next Ocean a couple of years back] Anyway, neither type of model has anything to do with rogue waves.
- amelius 4y ago
  I wonder to what extent all the mathematical modeling can be replaced by modern run-of-the-mill data science techniques.
  - mkbosmans 4y ago
    My guess would be: not that much. But I work in the field of numerical mathematics and computational physics, so I could have some bias. :-D The more nuanced answer would be that taking the raw radar data as input to e.g. a neural network and train that to output the predicted timeseries of future ship motion is not feasible. It would take a giant network and too much compute to train for very unreliable results. This problem consists of a lot of subproblems, most of which are pretty well understood. For example how to translate the 6-dof motion of a ship to the vertical displacement of a heli platform on that ship is just some simple coordinate transforms. You don't gain anything by including that in the neural net. Potentially some data science techniques could be useful to handle some of the less understood submodels. Sort of like it is done in CFD with NN as a turbulence model within an existing PDE solver.
    - ShamelessC 4y ago
      Am an ML engineer with no experience on this subject. Do you think research like deep mind has done with now-casting could be useful here?
      - mkbosmans 4y ago
        Yes, I think ML could be useful at places where the current physics modelling falls short. The nowcasting rain example from DeepMind is in some respects pretty comparable with the Next Ocean wave prediction. In the wave prediction case, wave propagation and dispersion is pretty well understood. But one could add ML-based nonlinear terms to the equations to capture everything we don't know. That has the possibility of giving better predictions. In contrast with the rogue wave example, there is a lot of relevant input data (the radar backscatter) and the model output can be verified after the fact (the ship's 6-dof acceleration). What I was objecting to was the idea of: slap a ML model on the whole problem and call it a day.
  - AlotOfReading 4y ago
    Other than the computational cost, why would you want to use stats when you have an analytic model right there?
    - amelius 4y ago
      Flexibility: the problem might change in ways that are difficult to model. Also, the model might capture "unknown unknowns". And before you say that you need massive amounts of data: so does an analytical model, assuming you want to verify it.
    - mkbosmans 4y ago
      To be fair, most analytical models working with data already incorporate a lot of 'data science' techniques. That does not have to be ML or NN, but could be as trivial as a least squares regression to fit you model to a set of observations that overdetermine the system of equations. A more advanced example of a technique that was used before it was called data science is data assimilation (DA)[1]. Here you assume that you have observations (e.g. sensor data) that you want to use to inform the model, but they are noisy in some sense. With DA you take a set of observations at t=0 and fit a numerical model to that. Then you time-step the model to t=1 where you have new observations. The model and observations don't necessarily agree, but there is value in incorporating information from both. Based on e.g. your statistical description of the sensor noise, DA techniques give you the tools to combine data and models. A good example of DA is 4DCOOL[2], combining temperature sensors in a datacenter with a CFD model. Because the model is physics-based, after some time you get a good idea of the temperature distribution in the whole room, even if you only have pretty sparse sensor data. (disclosure: I work for the company) [1] https://en.wikipedia.org/wiki/Data_assimilation https://en.wikipedia.org/wiki/Data_assimilation [2] https://www.vortech.nl/en/projects/a-digital-twin-for-the-indoor-climate-in-data-centers/ https://www.vortech.nl/en/projects/a-digital-twin-for-the-in...
dzhiurgis 4y ago
> can we land skippy? > one moment, just downloading this project off github > just downloading few gigs of dependencies “Cannot use import statement outside module”

mkbosmans 4y ago

Sure, but tide and wave models generally cover a much larger area, such as whole coastal area's and over larger time spans. An example of such a model would be: https://www.youtube.com/watch?v=eN6CDaoMZ7U https://www.youtube.com/watch?v=eN6CDaoMZ7U In the Netherlands this is used e.g. to determine when to close the storm surge barriers to protect the river delta area from flooding due to a tide+storm combination. In contrast, the Next Ocean product is meant to model the direct area around a single ship, predicting minutes in advance with a second by second granularity. They are able to reuse the raw data coming from the navigation radar that is already installed on every sea-going ship. It uses the backscatter from water ripples to determine the wave field around the ship. Interestingly, for navigational purposes exactly this part of the data is considered noise and filtered out by the on board navigation system. (https://nextocean.nl/technology.php https://nextocean.nl/technology.php) [I did some work on the software implementation for Next Ocean a couple of years back] Anyway, neither type of model has anything to do with rogue waves.

amelius 4y ago
I wonder to what extent all the mathematical modeling can be replaced by modern run-of-the-mill data science techniques.
- mkbosmans 4y ago
  My guess would be: not that much. But I work in the field of numerical mathematics and computational physics, so I could have some bias. :-D The more nuanced answer would be that taking the raw radar data as input to e.g. a neural network and train that to output the predicted timeseries of future ship motion is not feasible. It would take a giant network and too much compute to train for very unreliable results. This problem consists of a lot of subproblems, most of which are pretty well understood. For example how to translate the 6-dof motion of a ship to the vertical displacement of a heli platform on that ship is just some simple coordinate transforms. You don't gain anything by including that in the neural net. Potentially some data science techniques could be useful to handle some of the less understood submodels. Sort of like it is done in CFD with NN as a turbulence model within an existing PDE solver.
  - ShamelessC 4y ago
    Am an ML engineer with no experience on this subject. Do you think research like deep mind has done with now-casting could be useful here?
    - mkbosmans 4y ago
      Yes, I think ML could be useful at places where the current physics modelling falls short. The nowcasting rain example from DeepMind is in some respects pretty comparable with the Next Ocean wave prediction. In the wave prediction case, wave propagation and dispersion is pretty well understood. But one could add ML-based nonlinear terms to the equations to capture everything we don't know. That has the possibility of giving better predictions. In contrast with the rogue wave example, there is a lot of relevant input data (the radar backscatter) and the model output can be verified after the fact (the ship's 6-dof acceleration). What I was objecting to was the idea of: slap a ML model on the whole problem and call it a day.
- AlotOfReading 4y ago
  Other than the computational cost, why would you want to use stats when you have an analytic model right there?
  - amelius 4y ago
    Flexibility: the problem might change in ways that are difficult to model. Also, the model might capture "unknown unknowns". And before you say that you need massive amounts of data: so does an analytical model, assuming you want to verify it.
  - mkbosmans 4y ago
    To be fair, most analytical models working with data already incorporate a lot of 'data science' techniques. That does not have to be ML or NN, but could be as trivial as a least squares regression to fit you model to a set of observations that overdetermine the system of equations. A more advanced example of a technique that was used before it was called data science is data assimilation (DA)[1]. Here you assume that you have observations (e.g. sensor data) that you want to use to inform the model, but they are noisy in some sense. With DA you take a set of observations at t=0 and fit a numerical model to that. Then you time-step the model to t=1 where you have new observations. The model and observations don't necessarily agree, but there is value in incorporating information from both. Based on e.g. your statistical description of the sensor noise, DA techniques give you the tools to combine data and models. A good example of DA is 4DCOOL[2], combining temperature sensors in a datacenter with a CFD model. Because the model is physics-based, after some time you get a good idea of the temperature distribution in the whole room, even if you only have pretty sparse sensor data. (disclosure: I work for the company) [1] https://en.wikipedia.org/wiki/Data_assimilation https://en.wikipedia.org/wiki/Data_assimilation [2] https://www.vortech.nl/en/projects/a-digital-twin-for-the-indoor-climate-in-data-centers/ https://www.vortech.nl/en/projects/a-digital-twin-for-the-in...

dzhiurgis 4y ago

> can we land skippy? > one moment, just downloading this project off github > just downloading few gigs of dependencies “Cannot use import statement outside module”