alexott

13 ms·

91.
by alexott 4y ago
Regarding performance- try version 1.2.x or higher - it includes data skipping, etc. from what was Databricks-only implementation before. Regarding “that’s all” - can you give an estimate how much time it will take to reimplement it from sc
92.
by alexott 4y ago
Databricks includes pure SQL implementations - Databricks SQL, for example. Or for example, Delta Live Tables allow to write you pipelines using only SQL…
93.
by alexott 4y ago
There are a lot of optimizations in query engines as well - results cache, local caches of files stored in cloud, data layout optimization and skipping (to avoid reading not necessary files), etc.
94.
by alexott 4y ago
No, it’s not just API layer, but significant addon on top of existing tech… It brings transactions, faster queries, time travel, etc.
95.
by alexott 4y ago
Maybe you can look again - things are changing very fast, few years there were mo DBSQL, Photon, Delta Sharing, etc.
96.
by alexott 4y ago
For blocking checks - I personally use notion of errors and warnings, with errors definitely going to quarantine and propagated to good data, and warnings going to both good data and quarantine. It’s a trade off between not blocking all da
97.
by alexott 4y ago
I’m using 13 old iPod for one simple reason - its very handy for music and podcasts. Just set the play list “didn’t listen for N weeks” and music is automatically synced, listened podcasts are removed and new are uploaded, etc.
98.
by alexott 4y ago
I would suggest to look onto Delta Lake ( https://delta.io/ ) - it's built on top of the Parquet, but has advantages over the plain parquet: - transactions - you don't get a garbage in your table if your write faile
99.
by alexott 5y ago
For corporate customers it usually costs significantly less. Few years ago I worked in a very big company, and as I remember, O'Reilly subscription cost per person was < $50
100.
by alexott 5y ago
I wish that it would happen in Germany :-( Even in current situation, government is reluctant to open existing stations
101.
by alexott 5y ago
I like PlantUML a lot
102.
by alexott 5y ago
Very often they go into product management roles
103.
by alexott 5y ago
Yeah, language detection on short texts is quite complex. In my practice, N-grams doesn’t work well for them.
104.
by alexott 5y ago
You can try with dataset that I’ve used to evaluate fasttext against different language detectors - it’s linked to this blog post: https://alexott.blogspot.com/2017/10/evaluating-fasttexts-mo... I’ll try to find t
105.
by alexott 5y ago
Thank you
106.
by alexott 5y ago
it would be really useful to add release dates to this page...
107.
by alexott 5y ago
Just a suggestion for site - separate language from currency. For example, for me it’s much faster to read in English, but I’m paying in euro…
108.
by alexott 5y ago
Just in case, there is a Russian translation of this book: http://lisper.ru/pcl/
109.
by alexott 5y ago
Manning is very different from other publishers - combination of all these roles allow to produce a high quality content… that’s make it different from other publishers - O’Reilly, APress, etc.
110.
by alexott 5y ago
Primarily because it’s very hard to express complex conditions
111.
by alexott 5y ago
magic database has its own problems based on the language itself
112.
by alexott 5y ago
File was slower by at least 10 times slower (it was ~10 years ago). Second thing, for example it didn’t work with compound objects. Especially for things like word embedded in excel, etc. Similar for XML-based types, zip-based, … Plus rules
113.
by alexott 5y ago
yeah, detection of file types is a fun. I worked on several implementation of very precise type detection (for use in email & web proxies), and there are so many low-level details that affect that detection. The biggest fun is detection
114.
by alexott 5y ago
There is also Apache Tika ( https://tika.apache.org/ ) - file format detection & content extraction library
115.
by alexott 5y ago
I also prefer to use Maven to compile Scala code - it's much faster, better integrated with IDEs, etc.
116.
by alexott 5y ago
re2 has limited functionality. It’s works wel for specific patterns, like crawling, but not general purpose
117.
by alexott 5y ago
MobilePay in Denmark: https://en.wikipedia.org/wiki/MobilePay
118.
by alexott 5y ago
it's different - rxin talks about this: https://databricks.com/product/databricks-sql when you run Python, it's on Spark, although you now can use Photon engine that is used for DB SQL by default
119.
by alexott 5y ago
for DB that's old number - recent valuation is $38B
120.
by alexott 5y ago
As traveling a lot on trains in Germany - it’s a joke. Trains are are late so you miss connections, cancelled on short notice, etc. I never know if get to airport on time when I use train. You can easily half hour late on 1 hour trip…

More ›