9 ms·

As you implied, I don't think the answer is to make the perfect abstractions that can handle any scenario. But where software does fail in my humble opinion is

As you implied, I don't think the answer is to make the perfect abstractions that can handle any scenario. But where software does fail in my humble opinion is making it easy to pull in tried and true tested solutions to the problems that we do face, even if they are not as common. Because even though they may not seem common, I'm absolutely certain many face the same scenario. The amount of duplication solving the same problems is insane. But this is not an easy problem to solve and I don't intend to trivialize it. I think we need to come up with better tools. code sharing through libraries/repositories (like npm) is great, but it can't be the final solution. Back in the day in Haskell I dreamed of a system where you could type out a type signature and a fully tested rated implementation would be imported from an "open source" service. You could import modules, functions, data structures, anything. But that vision is still a long ways off.

feoren 4y ago
The problem is "pulling in a tried and tested solution" means pulling in a slice of the whole stack: CSS, Javascript, client/server communication, API endpoints, data-flow logic, and database schema. People are very very bad at this because they believe that software engineering principles apply only to little tiny portions of this stack. If you actually apply your software engineering principles to the whole stack, you end up with something that looks very different than what everyone else is doing (so there's not a lot of tooling to help with). I have a better way to do this than anything else I've ever seen, but even with that I struggly very strongly with the JS/API side.
- 7thaccount 4y ago
  Kinda reminds me of what they do in some areas of the finance world with Kx System's Kdb+. That entire install is tiny tiny and you get a high performance array language for querying your SSD database with either the main language "k" or a syntactical sugar dialect sitting on top called "q" that is more like SQL. I think of that as optimizing the entire stack for what the customer wants of rapid/high-speed analytics.
  - retcon 4y ago
    KDB is unique for translating "in house" tech to marketable CV entry status. Having Haskell and CS(FB) experience requires educating the recruitment agencies, for comparable recognition. Personally I consider every ill in computing as symptoms of the intellectual property system, from making reading potential prior art (aka learning) verboten to non disclosures making the Peter Principle look like the common cold to covid.
    - 7thaccount 4y ago
      I had trouble following your message. You've stated that Kdb+ looks good on a resume, Haskell requires explanation, and IP is bad? I'm not a big fan of closed source myself (and Kx Systems sounds like it would have a lot of red tape), but the technology is supposedly very fast and I find the language to be very simple and charming. I'd love for our analytics database to migrate , but I think the technology would really confuse our IT folks and it would be difficult to describe why it would make our jobs easier.
      - timeserious 4y ago
        Get in touch! I work within KX as a Solutions Architect and would love to see how we could help you out. [redacted]
        7thaccount 4y ago
        Will do at some point. Thanks for the invite!
        timeserious 4y ago
        Anytime! FYI we've released a new product KX Insights which is a cloud native version of kdb+ that supports ANSI-SQL (PostgreSQL) and has a bunch of ease-of-use and interoperability improvements to open the technology to a much wider user base :)
notriddle 4y ago
> Back in the day in Haskell I dreamed of a system where you could type out a type signature and a fully tested rated implementation would be imported from an "open source" service. The type signature of GPT-3 is `string -> string`
- nightski 4y ago
  Well I'd argue it's more Model => String => String if you are talking about using a fully pre-trained model. Even then you could probably expand on the type signature of the model to make it more useful. But if you look at like co-pilot for example, if that was given the ability to have type signatures serve as input you might get a lot more powerful results than what it does with raw text (which is very impressive). But this comes down to type signature design. You can encode any function using simple types like int -> int which aren't very useful. Where Haskell shines is when using types to limit the scope of inputs & outputs. What I am getting at is that you can still write uninformative type signatures in Haskell, but it also gives you the power to write more informative ones. I don't think Haskell is the answer, so please don't take that as what I am saying. I do think however using richer type systems could be a stepping stone towards a solution to this problem.
  - astrange 4y ago
    Type signatures (or ASTs) are structured data, and Copilot mainly works because GPT accepts any text and doesn't have to deal with your logical fully recursive concepts like ASTs. I feel like it wouldn't work if you could actually mathematically constrain the outputs to be syntactically correct either. That's probably one of those Gödel things.
mcphage 4y ago
> Back in the day in Haskell I dreamed of a system where you could type out a type signature and a fully tested rated implementation would be imported from an "open source" service. Type signatures don’t tell you which should be the “then” clause vs the “else” clause in any conditional.

feoren 4y ago

The problem is "pulling in a tried and tested solution" means pulling in a slice of the whole stack: CSS, Javascript, client/server communication, API endpoints, data-flow logic, and database schema. People are very very bad at this because they believe that software engineering principles apply only to little tiny portions of this stack. If you actually apply your software engineering principles to the whole stack, you end up with something that looks very different than what everyone else is doing (so there's not a lot of tooling to help with). I have a better way to do this than anything else I've ever seen, but even with that I struggly very strongly with the JS/API side.

7thaccount 4y ago
Kinda reminds me of what they do in some areas of the finance world with Kx System's Kdb+. That entire install is tiny tiny and you get a high performance array language for querying your SSD database with either the main language "k" or a syntactical sugar dialect sitting on top called "q" that is more like SQL. I think of that as optimizing the entire stack for what the customer wants of rapid/high-speed analytics.
- retcon 4y ago
  KDB is unique for translating "in house" tech to marketable CV entry status. Having Haskell and CS(FB) experience requires educating the recruitment agencies, for comparable recognition. Personally I consider every ill in computing as symptoms of the intellectual property system, from making reading potential prior art (aka learning) verboten to non disclosures making the Peter Principle look like the common cold to covid.
  - 7thaccount 4y ago
    I had trouble following your message. You've stated that Kdb+ looks good on a resume, Haskell requires explanation, and IP is bad? I'm not a big fan of closed source myself (and Kx Systems sounds like it would have a lot of red tape), but the technology is supposedly very fast and I find the language to be very simple and charming. I'd love for our analytics database to migrate , but I think the technology would really confuse our IT folks and it would be difficult to describe why it would make our jobs easier.
    - timeserious 4y ago
      Get in touch! I work within KX as a Solutions Architect and would love to see how we could help you out. [redacted]
      - 7thaccount 4y ago
        Will do at some point. Thanks for the invite!
        timeserious 4y ago
        Anytime! FYI we've released a new product KX Insights which is a cloud native version of kdb+ that supports ANSI-SQL (PostgreSQL) and has a bunch of ease-of-use and interoperability improvements to open the technology to a much wider user base :)

notriddle 4y ago

> Back in the day in Haskell I dreamed of a system where you could type out a type signature and a fully tested rated implementation would be imported from an "open source" service. The type signature of GPT-3 is `string -> string`

nightski 4y ago
Well I'd argue it's more Model => String => String if you are talking about using a fully pre-trained model. Even then you could probably expand on the type signature of the model to make it more useful. But if you look at like co-pilot for example, if that was given the ability to have type signatures serve as input you might get a lot more powerful results than what it does with raw text (which is very impressive). But this comes down to type signature design. You can encode any function using simple types like int -> int which aren't very useful. Where Haskell shines is when using types to limit the scope of inputs & outputs. What I am getting at is that you can still write uninformative type signatures in Haskell, but it also gives you the power to write more informative ones. I don't think Haskell is the answer, so please don't take that as what I am saying. I do think however using richer type systems could be a stepping stone towards a solution to this problem.
- astrange 4y ago
  Type signatures (or ASTs) are structured data, and Copilot mainly works because GPT accepts any text and doesn't have to deal with your logical fully recursive concepts like ASTs. I feel like it wouldn't work if you could actually mathematically constrain the outputs to be syntactically correct either. That's probably one of those Gödel things.

mcphage 4y ago

> Back in the day in Haskell I dreamed of a system where you could type out a type signature and a fully tested rated implementation would be imported from an "open source" service. Type signatures don’t tell you which should be the “then” clause vs the “else” clause in any conditional.