14 ms·

Does anyone else begin to feel like their role as a software developer is to maintain a mental search index of available techniques, languages, libraries, and m

by jka 6y ago

Does anyone else begin to feel like their role as a software developer is to maintain a mental search index of available techniques, languages, libraries, and metadata properties about each of them? It's becoming so easy to compose software from available open source components, and migrate functionality (like full-text search) to different layers of the stack (and that's fantastic!). It's just tricky to keep all the requirements and constraints (and implications) in mind when selecting the appropriate libraries :)

amelius 6y ago
It's sad that we still don't have automatic interoperability between languages. Someone should define a common API, and every language should adhere to it (or risk not be taken seriously). This is not trivial, since some languages have garbage collection, but it should be possible.
- ImprobableTruth 6y ago
  I sadly think this is just not possible unless the common API is extremely restrictive, and for that case we already have C FFIs. Even something relatively simple like generics just works so differently depending on the language. C++'s templates are just fundamentally incompatible with how Java generics work.
  - fulafel 6y ago
    Forget about generics, even strings can't be agreed on. (Sometimes not even within a single language...)
- adwn 6y ago
  Like ImprobableTruth said, this isn't really possible without restricting the expressivity of the interop API or the set of supported languages. At least not on the function-call level. A more flexible – though less efficient – approach would be a service-oriented protocol. You'd send requests in the form of messages (binary or text) over a byte-oriented bidirectional channel and receive the replies on the same channel. Unfortunately this approach would require more code to set up than primitive [1] function calls, and fine-grained interaction with the library would be harder. [1] "primitive" as in lower-level, not as in dumb.
  - adwn 6y ago
    Dammit, now I can't think about anything else but how to design such a protocol, and how to generate adapters which translate between this protocol and the API of a library... Edit: From the perspective of the interop protocol, it wouldn't make much difference if the library runs in the same address space or in a different process. Large blobs of data, like an picture or a long string, could be passed via pointers (in the same process) or via shared memory (in different processes).
    - asdfman123 6y ago
      If you're trying to make an API for all programming languages, aren't you essentially just recreating something like the Java virtual machine but with your own biases and assumptions inserted?
      - adwn 6y ago
        You're misunderstanding my idea. Don't think "C ABI with higher-level types and objects", think "HTTP with more structure".
        asdfman123 6y ago
        But it seems that kind of protocol would just be a way of telling a computer what to do, not how to do it. How would that be better than any other messaging format that exists? Genuinely curious, because I don't fully understand this myself but the idea is interesting.
        adwn 6y ago
        To be honest, I don't know. It was just a quick idea, and I'm increasingly less sure, whether it makes sense at all. Sorry to disappoint you :-(
        twic 6y ago
        Right, a way to broker requests between objects. Just a common architecture for that.
- openfuture 6y ago
  You mean like protobuf or json or xml or...? Garbage collectors ruin embeddability which is why people have been using C/C++ for all this time. The alternative is to serialize and deserialize like you are doing network communication.
- asdfman123 6y ago
  There is a (relatively) common API: machine language. Is that too architecture-specific? You could force all programming languages compile down to Java byte code, but that would be too restrictive. Not everyone wants to be bound by byte code. You could create a language that aims to accomplish what you describe, but then you'd just have yet another programming language. (Insert relevant XKCD here.) And humans either aren't smart enough to create a perfectly flexible language without warts that would work for people doing dramatically different things, or there are too many tradeoffs for those people to share the same den. Someone who has thought more about the philosophy of programming languages probably could answer that better than I could.
kodablah 6y ago
Don't try to keep them all in mind. You have to approach each problem/project holistically and include existing environment, maturity, etc in your evaluation process. Having too many uncurated choices can seem overwhelming, but it's a valuable skill to be able to recognize the difference between your needs/challenges and the ones of people toying around and/or solving very specialized problems. Admire the edge of the ecosystem with interest and intrigue, but apply practically.
asdfman123 6y ago
> Does anyone else begin to feel like their role as a software developer is to maintain a mental search index of available techniques, languages, libraries, and metadata properties about each of them? Yup. Every highly paid professional is highly paid for their knowledge and expertise. They need to be aware of all the relevant solutions, and the best way to apply them. Top lawyers know about law, top doctors know about medicine, and they farm out the work to their less-highly paid nurses and paralegals. When you reach the top of your field your main value is knowing what's out there and how to implement it, making sure more junior employees don't make mistakes along the way. (That is, if you stay on the IC track and don't go into management.)
- jka 6y ago
  Thanks; that's a straightforward and insightful way of thinking about it. It makes me wonder whether some/all of this can be encoded into an actual search engine or constraint satisfier. Arguably it's the dream of the Expert System[1] all over again. [1] - https://en.wikipedia.org/wiki/Expert_system https://en.wikipedia.org/wiki/Expert_system
jungletime 6y ago
I'm not so sure about that. Just try to get CSS to display properly across browsers, or display content in an iFrame that resizes properly, on different devices. Edge case problems, galore.

by jka 6y ago

amelius 6y ago

It's sad that we still don't have automatic interoperability between languages. Someone should define a common API, and every language should adhere to it (or risk not be taken seriously). This is not trivial, since some languages have garbage collection, but it should be possible.

ImprobableTruth 6y ago
I sadly think this is just not possible unless the common API is extremely restrictive, and for that case we already have C FFIs. Even something relatively simple like generics just works so differently depending on the language. C++'s templates are just fundamentally incompatible with how Java generics work.
- fulafel 6y ago
  Forget about generics, even strings can't be agreed on. (Sometimes not even within a single language...)
adwn 6y ago
Like ImprobableTruth said, this isn't really possible without restricting the expressivity of the interop API or the set of supported languages. At least not on the function-call level. A more flexible – though less efficient – approach would be a service-oriented protocol. You'd send requests in the form of messages (binary or text) over a byte-oriented bidirectional channel and receive the replies on the same channel. Unfortunately this approach would require more code to set up than primitive [1] function calls, and fine-grained interaction with the library would be harder. [1] "primitive" as in lower-level, not as in dumb.
- adwn 6y ago
  Dammit, now I can't think about anything else but how to design such a protocol, and how to generate adapters which translate between this protocol and the API of a library... Edit: From the perspective of the interop protocol, it wouldn't make much difference if the library runs in the same address space or in a different process. Large blobs of data, like an picture or a long string, could be passed via pointers (in the same process) or via shared memory (in different processes).
  - asdfman123 6y ago
    If you're trying to make an API for all programming languages, aren't you essentially just recreating something like the Java virtual machine but with your own biases and assumptions inserted?
    - adwn 6y ago
      You're misunderstanding my idea. Don't think "C ABI with higher-level types and objects", think "HTTP with more structure".
      - asdfman123 6y ago
        But it seems that kind of protocol would just be a way of telling a computer what to do, not how to do it. How would that be better than any other messaging format that exists? Genuinely curious, because I don't fully understand this myself but the idea is interesting.
        adwn 6y ago
        To be honest, I don't know. It was just a quick idea, and I'm increasingly less sure, whether it makes sense at all. Sorry to disappoint you :-(
      - twic 6y ago
        Right, a way to broker requests between objects. Just a common architecture for that.
openfuture 6y ago
You mean like protobuf or json or xml or...? Garbage collectors ruin embeddability which is why people have been using C/C++ for all this time. The alternative is to serialize and deserialize like you are doing network communication.
asdfman123 6y ago
There is a (relatively) common API: machine language. Is that too architecture-specific? You could force all programming languages compile down to Java byte code, but that would be too restrictive. Not everyone wants to be bound by byte code. You could create a language that aims to accomplish what you describe, but then you'd just have yet another programming language. (Insert relevant XKCD here.) And humans either aren't smart enough to create a perfectly flexible language without warts that would work for people doing dramatically different things, or there are too many tradeoffs for those people to share the same den. Someone who has thought more about the philosophy of programming languages probably could answer that better than I could.

kodablah 6y ago

Don't try to keep them all in mind. You have to approach each problem/project holistically and include existing environment, maturity, etc in your evaluation process. Having too many uncurated choices can seem overwhelming, but it's a valuable skill to be able to recognize the difference between your needs/challenges and the ones of people toying around and/or solving very specialized problems. Admire the edge of the ecosystem with interest and intrigue, but apply practically.

asdfman123 6y ago

> Does anyone else begin to feel like their role as a software developer is to maintain a mental search index of available techniques, languages, libraries, and metadata properties about each of them? Yup. Every highly paid professional is highly paid for their knowledge and expertise. They need to be aware of all the relevant solutions, and the best way to apply them. Top lawyers know about law, top doctors know about medicine, and they farm out the work to their less-highly paid nurses and paralegals. When you reach the top of your field your main value is knowing what's out there and how to implement it, making sure more junior employees don't make mistakes along the way. (That is, if you stay on the IC track and don't go into management.)

jka 6y ago
Thanks; that's a straightforward and insightful way of thinking about it. It makes me wonder whether some/all of this can be encoded into an actual search engine or constraint satisfier. Arguably it's the dream of the Expert System[1] all over again. [1] - https://en.wikipedia.org/wiki/Expert_system https://en.wikipedia.org/wiki/Expert_system

jungletime 6y ago

I'm not so sure about that. Just try to get CSS to display properly across browsers, or display content in an iFrame that resizes properly, on different devices. Edge case problems, galore.