5 ms·
I honestly don't remember any more. At the time, we were working with NOAA, and I remember a problem that was solved by talking to an admin at NOAA (our IP need
by peterlk 7y ago
I honestly don't remember any more. At the time, we were working with NOAA, and I remember a problem that was solved by talking to an admin at NOAA (our IP needed to be on some official whitelist or something), but that may have been for a restricted data set. We didn't end up using it for long because the client said so.
But I dug around for some information to maybe get you started.
Installation: https://madis.ncep.noaa.gov/doc/INSTALL.unix https://madis.ncep.noaa.gov/doc/INSTALL.unix
API: https://madis.ncep.noaa.gov/madis_api.shtml https://madis.ncep.noaa.gov/madis_api.shtml
Data restrictions: https://madis.ncep.noaa.gov/madis_restrictions.shtml https://madis.ncep.noaa.gov/madis_restrictions.shtml
Another resource that may help: https://press3.mcs.anl.gov/forest/regional-models/global-data-acquisition/ https://press3.mcs.anl.gov/forest/regional-models/global-dat...
When I was working on this stuff, I found that a DFS on various government subdomains (like MADIS) was the best way to find information. It was tedious, but it worked.
It's also helpful to put on your fortran hat. For example, I once attended a Haskell meetup where someone wrote a parser to deal with parsing binary files from NOAA. I also was in a meeting (with some NOAA folks) once where I was asked if I "would prefer an ASCII file, or a binary one". This is not a world that operates on JSON or XML. Expect binary blobs with flags (bits) that change the meaning of other flags in fun and exotic ways. The binary nature of the data can help with data throughput limits, but boy is it a pain to deal with.
- jcadam 7y ago> This is not a world that operates on JSON or XML. Expect binary blobs with flags (bits) that change the meaning of other flags in fun and exotic ways. That bring back memories... As a government contractor I've had to work with sensor data (seismic, radar, etc) in various formats that were developed well before the rise of XML and JSON :( My favorite was a mixed ASCII and binary format, where each data record in a file had an ASCII header that described the format of the following block of binary, and pretty much anything could be different between records, even within the same data file (Time units? Integers? Floats? 16 bit integers? 64 bit? Big/Little Endian?). I had to write a parser for that :'(
- moftz 7y agoThe most "fun" I've ever had was decoding command and telemetry from piece of equipment for a ground station. The box would spit out this massive frame of data. It was a very long ASCII string that you would turn into binary to break into 6bit BCD values (no clue why they didn't use 4bit...). There were random flags of odd bit lengths (sometimes just a single bit, sometimes 5bits) thrown in between numbers for arbitrary reasons rather than just having all the binary flags up front. My python script was this ugly mess of slicing up the frame to turn it all into a very nice struct I could pass to the rest of the system. The manual with this piece of hardware was some old scan that must have been xeroxed a million times over so some portions of the document were just unreadable and you had to guess what those bits did. Other parts of the frame were just undocumented. Commands were send one by one as single letter with the actual ASCII representation of the numerical command parameter. When I started the project, I looked online to see if anyone had done any previous work on this thing. A vendor was selling a GUI for the thing for $2000, I scoffed at the price and started working it myself. By the time I was done, it had probably cost my employer more than that but at least we had our own code that could connect to whatever you wanted rather than a GUI with a no API.