kthielen

9 ms·

61.
by kthielen 6y ago
Re: no open source alternative to kdb I built this for Morgan Stanley (the same place where kdb originally came from) and put it to use in some major high-volume/low-latency trading systems where a significant fraction of US equity tra
62.
by kthielen 6y ago
Yep still used (pre and post trade, processing millions of orders and billions of market data events per day). I'd like to grow it into a successor to kdb, though kdb is very entrenched in finance and the company is extremely litigious
63.
by kthielen 6y ago
No it's used for a much more basic purpose than that, efficient order execution (which investors need as much as speculators). Anyway, it's generic and open source, so can be used for anything. I use it at home to make little vid
64.
by kthielen 6y ago
Shameless plug, I made hobbes and used it in high volume trading systems at Morgan Stanley: https://github.com/Morgan-Stanley/hobbes It’s kind of a structurally-typed variant of Haskell, integrates closely with C++, pr
65.
by kthielen 7y ago
I think that part of the problem is to define data structures in a way that can be produced and consumed in multiple languages, algebraic data types are a good "least common denominator" for this. Also w.r.t. reinventing the schem
66.
by kthielen 7y ago
Another voice in favor of logging. At Morgan Stanley we wind up recording about 1TB/day of structured log data, which also needs an efficient query engine to process. We carved a basic type system out for structured data, maps to C++
67.
by kthielen 7y ago
I agree especially with the proposition that we should exchange structured data (via files, network connections, ...). I started this project several years ago to support live code updates in production systems without sacrificing performan
68.
by kthielen 7y ago
> Anyway, I think the way out of this mess is to have a functional, Haskell-like language for processing tables I agree, and in a Haskell-like setting we can have any structured types (rather than just primitives). That’s why I built hob
69.
by kthielen 7y ago
I can't answer his "so what", I was just trying to clarify what was left implicit in this conversation (if not for you, then maybe for readers who don't know).
70.
by kthielen 7y ago
He means to say that Phil Wadler works for IOHK.
71.
by kthielen 7y ago
Well I've done some work to split small header-only libraries out of the project to make it useful in other places without having to link in everything. Low-latency structured logging, data files, a minimal x86_64 backend (LLVM altern
72.
by kthielen 7y ago
Well one way we deal with typical unstructured data is to prevent the unstructuring in the first place. The low-latency logging method (for example) uses something like a printf-style interface but stores all of the source data with exactl
73.
by kthielen 7y ago
Oh, well there are two stories actually. The first story is that it follows the tradition of naming programming languages after logicians/mathematicians (e.g. Haskell, Church, Russell, Pascal, ...). Because the Curry-Howard isomorphis
74.
by kthielen 7y ago
Nice to see this here again. :) I made this project several years back, it's been very fun and interesting. I've been meaning to write about some parts of it that haven't had much attention.
75.
by kthielen 7y ago
The paper is from 1991, when was your textbook written?
76.
by kthielen 7y ago
Yes I do.
77.
by kthielen 7y ago
Yes there are persisted data structures and databases built on top of them. The whole type system applies to persisted data structures so you can make your own data structures and fill out a few type class instances to query them with plai
78.
by kthielen 7y ago
Agreed, from the title I was expecting Swift language extensions with compiler plugins.
79.
by kthielen 7y ago
Except at Morgan Stanley (where the same could be said for kdb 20 years ago).
80.
by kthielen 7y ago
Morgan Stanley in recent years has used this where they’d previously used kdb: https://github.com/Morgan-Stanley/hobbes From the article, it sounds like the Kx folks are working on the issues that led to the developmen
81.
by kthielen 7y ago
That way of thinking has definitely proven useful in the domain where it started (at Morgan Stanley). We’ve also been looking at how to map it more closely to C/C++ and native hardware, kind of a structurally typed Haskell variant wit
82.
by kthielen 8y ago
Pretty cool, must be something in the air. This is a useful technique, and having it in a self-contained header-only lib is handy too. I made a similar library for writing data into memory mapped files, also a self-contained header-only li
83.
by kthielen 8y ago
Google has a lot of good people, but in my experience the interviews are specifically looking for a particular bias -- not just knowledgeable and personable people but specifically people who think a certain way. I had an interview there re
84.
by kthielen 8y ago
Yes, there is only Curry-Howard and all PLs are just projections (to greater and lesser degrees). Referential transparency, mutation, non-termination, coinduction, memory pointers, linear types, coroutines, continuations, blah blah blah. A
85.
by kthielen 8y ago
> Instead of just taking Haskell features and porting them > into already adopted languages, I'd really like to see > more companies just go directly with Haskell, [...] Unfortunately there are some places where aspects of the
86.
by kthielen 8y ago
I don't mean to be pedantic but IIRC Python's comprehensions actually came from the SETL language. Obviously it's ultimately the same thing either way, but that's the lineage (SETL -> ABC -> Python). https:/&
87.
by kthielen 8y ago
> Writing software is a trade-skill subject to market forces, literacy is a critical thinking skill more akin to logic itself. False dichotomy, these things you distinguish are just two sides of the same coin: https://en.wikip
88.
by kthielen 8y ago
Once you discard type structure, it's a fruitless task to try to reconstruct it. It's much easier to make sense of logs when we don't discard that type structure. https://github.com/Morgan-Stanley/hobbes#
89.
by kthielen 8y ago
> 2. you actually don't care about schema that much (e.g. logs) I would argue that this is one of the most overlooked cases _for_ schemas, because it's where schemas are the most complex and also where traditionally we've
90.
by kthielen 8y ago
> That said, I really wish proponents of other languages would get their > stuff together about creating libraries that can be used from other languages. > A library written in C can be used by anyone else. Many other languages are

More ›