4 ms·

It seems interesting, but what is the advantage over just \copy in Postgres from a CSV file - or better yet, import into SQLite without even providing the table

by sgt 5y ago

It seems interesting, but what is the advantage over just \copy in Postgres from a CSV file - or better yet, import into SQLite without even providing the table definition?

simlan 5y ago
Which is likely how this works under the hood.
- tanin 5y ago
  OP here. Yes! we use Sqlite under the hood with modified C extensions, so it can be as fast as it can be. (Just try not to touch JS code for this) However, I have recently learned that SIMD is even faster than C, but I haven't explored it yet.
SKILNER 5y ago
The advantage would be that you do not have your data in two places and can avoid all the problems and confusion that can result.
- jonnycomputer 5y ago
  How does this avoid having data in two places? You still have to have a CSV file to import the data...
  - SKILNER 5y ago
    You're saying that viewing your csv data via a SQL query is having it in two places. I don't think that's the normal concept of having multiple copies of data.
    - jonnycomputer 5y ago
      Not me. Are you responding to parent poster?
tanin 5y ago
The difference is between GUI and command-line. I work with a lot of CSV files, and for some reason I rarely load them into Postgres or use Python script. I still try very hard to use Excel. There is some degree of convenience that GUI offers over command-line, though some people still prefer command line.
- qorrect 5y ago
  I use the command line all day, it definitely has its place. But trying to manipulate large CSV files has never felt like a good fit for the CLI. I'm already using and enjoying this tool alot . I use to do this in DBeaver ( also a great tool ) , but I have to manually specify the types in a JDBC string to get all the goodies of SQL this so far has been much nicer. I have already made this a daily tool thanks!
  - tanin 5y ago
    Any feedback or suggestion is very welcome! Please don't hesitate to email me at tanin@superintendent.app

by sgt 5y ago

It seems interesting, but what is the advantage over just \copy in Postgres from a CSV file - or better yet, import into SQLite without even providing the table definition?

simlan 5y ago

Which is likely how this works under the hood.

tanin 5y ago
OP here. Yes! we use Sqlite under the hood with modified C extensions, so it can be as fast as it can be. (Just try not to touch JS code for this) However, I have recently learned that SIMD is even faster than C, but I haven't explored it yet.

SKILNER 5y ago

The advantage would be that you do not have your data in two places and can avoid all the problems and confusion that can result.

jonnycomputer 5y ago
How does this avoid having data in two places? You still have to have a CSV file to import the data...
- SKILNER 5y ago
  You're saying that viewing your csv data via a SQL query is having it in two places. I don't think that's the normal concept of having multiple copies of data.
  - jonnycomputer 5y ago
    Not me. Are you responding to parent poster?

tanin 5y ago

The difference is between GUI and command-line. I work with a lot of CSV files, and for some reason I rarely load them into Postgres or use Python script. I still try very hard to use Excel. There is some degree of convenience that GUI offers over command-line, though some people still prefer command line.

qorrect 5y ago
I use the command line all day, it definitely has its place. But trying to manipulate large CSV files has never felt like a good fit for the CLI. I'm already using and enjoying this tool alot . I use to do this in DBeaver ( also a great tool ) , but I have to manually specify the types in a JDBC string to get all the goodies of SQL this so far has been much nicer. I have already made this a daily tool thanks!
- tanin 5y ago
  Any feedback or suggestion is very welcome! Please don't hesitate to email me at tanin@superintendent.app