6 ms·

Hello everyone. I quickly built this little tool based on public records provided by the government. I did the same a couple weeks ago with H-1Bs and I just ad

by negrit 11y ago

Hello everyone. I quickly built this little tool based on public records provided by the government. I did the same a couple weeks ago with H-1Bs and I just added over 1.1million green cards records. The data provided is not perfect so i'm still working on cleaning it but it should give you an idea of what is going on. If you have any questions, feel free to ask.

Jake232 11y ago
I can't seem to find the H1B statistics page, maybe I'm just looking in the wrong place though. Got a link?
roddit 11y ago
Kudos my friend.
Oras 11y ago
Great work mate
netheril96 11y ago
Any way for us to run `select` queries ourselves?
- negrit 11y ago
  I haven't planned it but why not. I don't really know how to do it though.
the_economist 11y ago
This is awesome Theo. When can I have data about Americans in France?
SuperKlaus 11y ago
Great tool! There seems to be a bug though when I drill down to year plus country of citizenship and then try to filter by job description and state or city. The job description search term ends up in the state input field. Also, filter by company would be great!
- negrit 11y ago
  My bad. Fixing it ASAP
jlees 11y ago
Neat tool, though it's a little disconcerting to be able to easily find my application - not your fault, though, the data was already there!
dalek2point3 11y ago
kinda weird that all the timeseries charts have time going backwards. going lower to higher is certainly the standard way to do it. for eg: http://data.jobsintech.io/companies/google-inc http://data.jobsintech.io/companies/google-inc
anorark 11y ago
This looks great! Can the green card data be shown by categories? Like how many green cards applications were filed in each category like EB1, EB2, etc.
bootload 11y ago
"If you have any questions, feel free to ask." What tools did you use/craft to clean the source data?
- negrit 11y ago
  PostgreSQL :)
  - bootload 11y ago
    so just dump the data in and process via sql. What about pre-processing ie: different data sources (pdf, cvs), incomplete or overlapping data? I imagine some code had to be written to do this?
    - negrit 11y ago
      All my datasource where csv and mdb files. So, I directly imported them to PostgreSQL. Very little code was written to clean the data. 99% of the cleaning was done with SQL queries. The code is mostly used to display the data. Cleaning the data with SQL queries is much faster than writing code.

by negrit 11y ago

Jake232 11y ago

I can't seem to find the H1B statistics page, maybe I'm just looking in the wrong place though. Got a link?

roddit 11y ago

Kudos my friend.

Oras 11y ago

Great work mate

netheril96 11y ago

Any way for us to run `select` queries ourselves?

negrit 11y ago
I haven't planned it but why not. I don't really know how to do it though.

the_economist 11y ago

This is awesome Theo. When can I have data about Americans in France?

SuperKlaus 11y ago

Great tool! There seems to be a bug though when I drill down to year plus country of citizenship and then try to filter by job description and state or city. The job description search term ends up in the state input field. Also, filter by company would be great!

negrit 11y ago
My bad. Fixing it ASAP

jlees 11y ago

Neat tool, though it's a little disconcerting to be able to easily find my application - not your fault, though, the data was already there!

dalek2point3 11y ago

kinda weird that all the timeseries charts have time going backwards. going lower to higher is certainly the standard way to do it. for eg: http://data.jobsintech.io/companies/google-inc http://data.jobsintech.io/companies/google-inc

anorark 11y ago

This looks great! Can the green card data be shown by categories? Like how many green cards applications were filed in each category like EB1, EB2, etc.

bootload 11y ago

"If you have any questions, feel free to ask." What tools did you use/craft to clean the source data?

negrit 11y ago
PostgreSQL :)
- bootload 11y ago
  so just dump the data in and process via sql. What about pre-processing ie: different data sources (pdf, cvs), incomplete or overlapping data? I imagine some code had to be written to do this?
  - negrit 11y ago
    All my datasource where csv and mdb files. So, I directly imported them to PostgreSQL. Very little code was written to clean the data. 99% of the cleaning was done with SQL queries. The code is mostly used to display the data. Cleaning the data with SQL queries is much faster than writing code.