5 ms·

This is a good point, there is not much good/accessible material on databases - because a lot of that material is considered academic. Honestly, I learned most

This is a good point, there is not much good/accessible material on databases - because a lot of that material is considered academic. Honestly, I learned mostly by experience of what frustrated me with current databases and then trying to come up with workaround solutions. Then I practiced by playing around with new ideas and making small prototypes in code. And then I would whiteboard out all the possible combinations of those ideas. Once I had all combinations listed, I reduced them all into their generic rules and which ones depended upon others. Finally, based off those constraints I worked out the mathematics of the system, the constraints, and tradeoffs. Maybe I should write a more intensive article on this to help other people. In the meantime, I think if anybody can read/watch classes on data structures, algorithms, etc. that will benefit them the most. Recently, we have worked on a some interactive animated explainers. Not quite database stuff, more like security (cryptography) and algorithms / data structures. Let me know if they are helpful: Cartoon Cryptography - http://gun.js.org/explainers/data/security.html http://gun.js.org/explainers/data/security.html Cartoon Conflict - http://gun.js.org/explainers/conflict/lexical.html http://gun.js.org/explainers/conflict/lexical.html Cartoon Sorting Algorithms - http://gun.js.org/explainers/basketball/basketball.html http://gun.js.org/explainers/basketball/basketball.html Cartoon Concurrency - http://gun.js.org/distributed/matters.html http://gun.js.org/distributed/matters.html And hopefully more to come. :) I know those are probably not immediately useful for your kdb+ like language. But, I'd love to learn more about what you are trying to do. :D

mamcx 10y ago
>Maybe I should write a more intensive article on this to help other people. This will be great. Look for example how this make "easy" to understand how a (basic) GC can be implemented: http://journal.stuffwithstuff.com/2013/12/08/babys-first-garbage-collector/ http://journal.stuffwithstuff.com/2013/12/08/babys-first-gar... Exist a lot of material in how understand at conceptual level things (that is what papers are for) but the actual implementation is "left to the reader". The trouble with that is that if I'm first learning about a subject, and I only see the final result but not the steps to get there, I get lost in why the final result is like that! So, I imagine something along the lines on how a nano-database could work (like, that only is a couple of tables (arrays or hash tables), and build from first principles the relational operators, then maybe later this is how you build the ACID, the query planner, etc). Not even necessary how build the sql parser because that part is covered elsewhere (and also, I think SQL distract from the understanding on this matter, because is a half-good and limited API to interact with a relational database). > But, I'd love to learn more about what you are trying to do. Most people have a pet theory that try push (like "make programming pure"). I just wish to recapture the experience I have building apps with FoxPro -that was more pleasant for database-based development than most- but with a modern twist and without the limitations of the past. I arrive to the conclusion that a relational language (in CONTRAST with a relational database!) could be in fact great. For example, in Fox you can write this: https://msdn.microsoft.com/en-us/library/aa978284(v=vs.71).aspx https://msdn.microsoft.com/en-us/library/aa978284(v=vs.71).a... SCAN FOR UPPER(country) = 'SWEDEN' AND isActive ? contact, company, city ENDSCAN (This is a FOR specialized to walk a table with WHERE and other stuff. A imperative ... WHERE sql + map + project) Some example along this is: http://www.try-alf.org/blog/2013-10-21-relations-as-first-class-citizen http://www.try-alf.org/blog/2013-10-21-relations-as-first-cl... My bet is that because the relational model is actually simple and provide "universal" query abilities is possible to build a language similar in scope to python+pandas, to build full apps, not just as part of the data manipulation side of the equation in a rdbms in the server. Eventually I found the APL/J/KDB+ family and have some similar ideas (only is based on arrays, than in relations). This is something I know can work. Fox was fairly popular in my time and it die not for lack of people but because MS kill it (along classic VB) to focus on .NET
- marknadal 10y ago
  Very much agreed, good thoughts/ideas - I'll keep these in mind! Thanks!

mamcx 10y ago

>Maybe I should write a more intensive article on this to help other people. This will be great. Look for example how this make "easy" to understand how a (basic) GC can be implemented: http://journal.stuffwithstuff.com/2013/12/08/babys-first-garbage-collector/ http://journal.stuffwithstuff.com/2013/12/08/babys-first-gar... Exist a lot of material in how understand at conceptual level things (that is what papers are for) but the actual implementation is "left to the reader". The trouble with that is that if I'm first learning about a subject, and I only see the final result but not the steps to get there, I get lost in why the final result is like that! So, I imagine something along the lines on how a nano-database could work (like, that only is a couple of tables (arrays or hash tables), and build from first principles the relational operators, then maybe later this is how you build the ACID, the query planner, etc). Not even necessary how build the sql parser because that part is covered elsewhere (and also, I think SQL distract from the understanding on this matter, because is a half-good and limited API to interact with a relational database). > But, I'd love to learn more about what you are trying to do. Most people have a pet theory that try push (like "make programming pure"). I just wish to recapture the experience I have building apps with FoxPro -that was more pleasant for database-based development than most- but with a modern twist and without the limitations of the past. I arrive to the conclusion that a relational language (in CONTRAST with a relational database!) could be in fact great. For example, in Fox you can write this: https://msdn.microsoft.com/en-us/library/aa978284(v=vs.71).aspx https://msdn.microsoft.com/en-us/library/aa978284(v=vs.71).a... SCAN FOR UPPER(country) = 'SWEDEN' AND isActive ? contact, company, city ENDSCAN (This is a FOR specialized to walk a table with WHERE and other stuff. A imperative ... WHERE sql + map + project) Some example along this is: http://www.try-alf.org/blog/2013-10-21-relations-as-first-class-citizen http://www.try-alf.org/blog/2013-10-21-relations-as-first-cl... My bet is that because the relational model is actually simple and provide "universal" query abilities is possible to build a language similar in scope to python+pandas, to build full apps, not just as part of the data manipulation side of the equation in a rdbms in the server. Eventually I found the APL/J/KDB+ family and have some similar ideas (only is based on arrays, than in relations). This is something I know can work. Fox was fairly popular in my time and it die not for lack of people but because MS kill it (along classic VB) to focus on .NET

marknadal 10y ago
Very much agreed, good thoughts/ideas - I'll keep these in mind! Thanks!