davinchia

6 ms·

1.
by davinchia 5mo ago
Haha indeed! On a more serious note, just as swyx mentioned in a comment further up, we do believe a lot of the challenges of reliably operationalising agents boil down to data. All of which is non-obvious to AI engineers (besides Frontier
2.
by davinchia 5mo ago
Sorry for that experience. We had a bad billing support routing issue and it’s since been fixed. Thank you for calling it out. We'll aim to do better!
3.
by davinchia 2y ago
Yeah, lots of blood sweat and tears here!
4.
by davinchia 2y ago
Let's gooooo!
5.
by davinchia 5y ago
Since this is executed from Java code, we did not want to have to bundle kubectl (I'm assuming that's what you are referring to) with the Jar. We also did not want to run an exec command. Are you asking why not GRPC between the va
6.
Scaling Data Pipelines on Kubernetes(airbyte.com)
3 points by davinchia 5y ago|2 comments
7.
by davinchia 5y ago
Hi, I'd encourage you to join our slack and ask around! The community is friendly and there are plenty of users that can give you feedback based on your specific situation. https://airbytehq.slack.com/join/shared_i
8.
by davinchia 5y ago
Airbyte engineer here. We use Dbt to normalise the delivered data at the final destination warehouse. If it's not a destination warehouse, how transformations are handled depends on the connector implementation.
9.
by davinchia 5y ago
Great points! "If you know you're going to spend 15 hours on it, then you might as well use a system you know will cost less to extend or scale over time." I wish younger me realised that earlier :) "And maybe even a can
10.
by davinchia 5y ago
Airflow is a general orchestration tool that fit into the Python stack extremely well. It wasn't build to scale though, so once you want to run something more than once a second, you are going to be jumping through hoops. My experience
11.
by davinchia 5y ago
We are working on it. This should be out next quarter!
12.
by davinchia 5y ago
Airbyte Engineer here. I think some of the points made here about ETL scripts being just 'ETL scripts' are very relevant. Definitely been on the other side of the table arguing for a quick 3-hour script. Having written plenty of E