jpau

7 ms·

31.
by jpau 6y ago
You sure can :) I see it as why the article supports Databricks as an RDBMS; it offers something others do not. You can't currently* do the same extensive UDFs in Snowflake or BQ and, sometimes, they are important. But with SnowPark co
32.
by jpau 6y ago
Databricks is good as a managed Spark platform. They have thought about how they can improve the DS experience. Inconsistent storage? DeltaLake. Slow Spark queries? Databricks Delta. Model management? MLFlow (I haven't adopted this, bu
33.
by jpau 6y ago
They use DeltaLake + Spark 3.0, and are mostly careful to partition well. Their datasets are small. Most tables are ~50GB, the odd table up to ~2TB. The clusters typically are nothing shabby for this size, defaults to ~[4-12]x32GB. The quer
34.
by jpau 6y ago
I'm deeply disappointed in Databricks as an RDBMS. As a DS/DE, there's a lot to love (not all, but a lot). The easy provision of Spark clusters. The jobs API. DeltaLake (mostly). Easy notebooks (please don't create a pro
35.
by jpau 6y ago
Well no, unfortunately. Remember that "data is a team sport". Together, we try and make better decisions (in manual or automated ways). A DE can produce great data but it's only useful if it helps the DA/DS. There's
36.
by jpau 6y ago
Snowflake's `Snowpark` product that they recently announced, which is to bring Spark-like APIs to Snowflake. Having a DS background, I love what SQL-orchestration tool dbt (and peers) have enabled: data consumers to rapidly create ou
37.
by jpau 6y ago
Similarly, there seems to be partial overlap with MLFlow for tracking iterations. I would find a comparison table vs. existing tools useful, to help me consider Orchest by placing it in my existing workflow.
38.
by jpau 6y ago
I personally agree that this does not seem too useful for any DS team that needs to deploy a model to production. But there are a whole horde of DS teams whose outputs are basically PPT presentations. Think like pricing forecasts, decision
39.
by jpau 6y ago
I sailed around Tahiti for a week with family on a 45’ catamaran. No phone, no laptop. So peaceful. I returned home refreshed, and it showed in my day-to-day focus. Though I find this in all adventures, so I’m not sure if it’s related to sc
40.
by jpau 6y ago
Success here is possible. I didn't believe it until I saw it :) I was in a regional strategy-esque team for Uber Eats. Eats have a nicely curated data warehouse of core operational data. They also have excellent, free, self-paced SQL t
41.
Ask HN: What problems are you having with your data (or lack of data)?
2 points by jpau 6y ago|0 comments
42.
by jpau 7y ago
Heyo! Data guy here. Airflow and its DAG-managing peers are important for us. Data transformations are one thing. For us, it’s the most important thing. Our data warehouse runs as a massive DAG of nightly batched transformations over app-ge
43.
by jpau 7y ago
My guess is it will, on average, increase demand. ML is finicky; the model training pipeline itself isn’t the hard part, compared to setting up for the right question and examples used to train the model. For small-to-mid firms, data scient
44.
by jpau 9y ago
Might [1] be "Linear Algebra Done Right" by Axler? http://www.springer.com/gp/book/9783319110790
45.
by jpau 10y ago
From what starting position are you asking from; what is your current math background?
46.
by jpau 10y ago
Excel for Mac is excellent, compared to the wider market. Unfortunately it is both missing many features and runs much slower than its Windows equivalent. For the work I do (as a financial analyst), Excel for Mac is unworkable. See Microsof
47.
by jpau 10y ago
Sydney's is great . I don't mind paying the $34 return like I do in Melbourne.
48.
by jpau 11y ago
This struck a little too close to home. I'm quite educated, but formally "only" to a bachelor degree from a not-very-prestigious (but good!) university. This lack of signal is causing me problems - not only in career progress
49.
by jpau 11y ago
You're right - they will probably make more by listing in the USA. Tech valuations on US exchanges are higher than on the ASX - even with recent falls, the NASDAQ is sitting at an average price-to-earnings of 22 vs the ASX200 IT index
50.
by jpau 11y ago
Hi Anakic, this looks great! I've been having to use SQL in Excel a lot lately, through ADO (VBA). I've been using it to create prepared operations; users click a button and, for example, Excel will generate a list of clients that
51.
by jpau 12y ago
I did some quick analysis comparing the proposed project against Li-ion, adjusting for storage and lifespan, and the result is very hopeful. I used values for $/KWh and lifespan from abdullahkhalids' comment. The analysis shows th
52.
by jpau 12y ago
I have the paperback. It's just fine. If you're happy with an ebook, it's available for free.
53.
by jpau 12y ago
Beat has hit the nail on the head. My day job is consumer sales for a big Australian telecom. Many of my customers visit to buy their first smartphone (!). I'll train them in how to call, text and browse before they leave and drill it
54.
by jpau 12y ago
A list of investor-investee relationships, each given an approximate establishment date. The website "watches the portfolio pages of some 150+ venture capital sites. Every night I look at each page, note any additions, and report them
55.
by jpau 12y ago
Customer support is expensive , as a proportion of fees. That said, I'd be happy to pay per support ticket (not prompted by a fault).
56.
by jpau 12y ago
Looks awesome, thanks!
57.
Ask HN: What apps did/do you use for cold calling?
6 points by jpau 12y ago|2 comments
58.
by jpau 13y ago
They may not necessarily have been checking for drugs - Customs may have been checking for bio risks. For example, here in Australia, only treated wood may be imported to prevent the introduction of pests and diseases. Unfortunately, treatm
59.
by jpau 13y ago
Nay. They accept mine (CBA Mastercard) just fine :)
60.
by jpau 13y ago
Brisbane, Australia -- relaxed. Oh dear!

More ›