2 ms·

You just translated SQL to an ugly C-ish syntax, which says nothing to me. I believe if we forget the SQL cliche, a different approach, I mean syntactically, is

by mojuba 19y ago

You just translated SQL to an ugly C-ish syntax, which says nothing to me. I believe if we forget the SQL cliche, a different approach, I mean syntactically, is quite possible.

nostrademons 19y ago
I didn't. SQLAlchemy did. ;-) (http://www.sqlalchemy.org/ http://www.sqlalchemy.org/) You mean simply using normal CS data structures and keeping application data in memory? That approach certainly works - PG uses it for this site, and I believe it's used for Mailinator and Bloglines. But you should think of the services that a database gives you and consider whether you're going to use them. That includes: 1.) Indexing by arbitrary column 2.) Subranges & slices, again by arbitrary column 3.) Joins 4.) Sorting 5.) Quick summarizing, over any combination of columns 6.) Transactions 7.) Persistence 8.) Concurrency control 9.) Access from multiple languages 10.) Distribution If you aren't going to use any of these, then by all means Keep It Simple. If you're only going to use one or two - say you need persistence, indexing, and slicing - you can probably roll something up with your language's normal libraries. But if it looks like you'll need a good chunk of them, you'll drive yourself nuts trying to implement them yourself. Even maintaining multiple indexes via your hashtable approach can get very complicated when you have multiple requests inserting, deleting, and changing rows.
- mojuba 19y ago
  keeping application data in memory No, I leave that to optimization. Could be in memory, on disk or wherever else my compiler and the run-time system decide them to be. When I see performance isn't good I may start fine-tuning my run-time system. Indexing by arbitrary column That's a matter of optimization. Thinking in indexes is clearly premature optimization that can be done automatically. Subranges & slices, again by arbitrary column That's map/reduce with some clever infrastructure that can optimize things for me. Joins Pointers. Sorting Sorting is sorting :) Quick summarizing, over any combination of columns Map/reduce. Transactions, Persistence and Concurrency control Possible without the DBMS. Access from multiple languages Honestly, I give up here. If we are going to have a programming language that itself handles large data structures, then other languages are probably out. Distribution What you mean by distribution?
  - anamax 19y ago
    I don't know what the poster meant by distribution, but one of the nice properties of a DBMS is that it's available when your application is down and can be accessed by other applications. Everyone is a client, so no one else has to be a server (for that data).
  - nostrademons 19y ago
    "What you mean by distribution?" Ability to spread your data across multiple physical machines.
  - fauigerzigerk 19y ago
    Hard-coding access paths is what is premature optimisation. RDBMS defer decisions about what loops to process until runtime. So the optimiser can make good decisions based on statistics it has about the data. Decisions that programmers do not have to make prematurely. I think it's pretty clear that if you know exactly how your data is going to be used, then performance is better if you hard code everything and store it in a way that is optimised for retrieval. However, if data is used for multiple purposes, then hard-coding access paths is horrible in terms of performance and maintainance.

by mojuba 19y ago

You just translated SQL to an ugly C-ish syntax, which says nothing to me. I believe if we forget the SQL cliche, a different approach, I mean syntactically, is quite possible.

nostrademons 19y ago

I didn't. SQLAlchemy did. ;-) (http://www.sqlalchemy.org/ http://www.sqlalchemy.org/) You mean simply using normal CS data structures and keeping application data in memory? That approach certainly works - PG uses it for this site, and I believe it's used for Mailinator and Bloglines. But you should think of the services that a database gives you and consider whether you're going to use them. That includes: 1.) Indexing by arbitrary column 2.) Subranges & slices, again by arbitrary column 3.) Joins 4.) Sorting 5.) Quick summarizing, over any combination of columns 6.) Transactions 7.) Persistence 8.) Concurrency control 9.) Access from multiple languages 10.) Distribution If you aren't going to use any of these, then by all means Keep It Simple. If you're only going to use one or two - say you need persistence, indexing, and slicing - you can probably roll something up with your language's normal libraries. But if it looks like you'll need a good chunk of them, you'll drive yourself nuts trying to implement them yourself. Even maintaining multiple indexes via your hashtable approach can get very complicated when you have multiple requests inserting, deleting, and changing rows.

mojuba 19y ago
keeping application data in memory No, I leave that to optimization. Could be in memory, on disk or wherever else my compiler and the run-time system decide them to be. When I see performance isn't good I may start fine-tuning my run-time system. Indexing by arbitrary column That's a matter of optimization. Thinking in indexes is clearly premature optimization that can be done automatically. Subranges & slices, again by arbitrary column That's map/reduce with some clever infrastructure that can optimize things for me. Joins Pointers. Sorting Sorting is sorting :) Quick summarizing, over any combination of columns Map/reduce. Transactions, Persistence and Concurrency control Possible without the DBMS. Access from multiple languages Honestly, I give up here. If we are going to have a programming language that itself handles large data structures, then other languages are probably out. Distribution What you mean by distribution?
- anamax 19y ago
  I don't know what the poster meant by distribution, but one of the nice properties of a DBMS is that it's available when your application is down and can be accessed by other applications. Everyone is a client, so no one else has to be a server (for that data).
- nostrademons 19y ago
  "What you mean by distribution?" Ability to spread your data across multiple physical machines.
- fauigerzigerk 19y ago
  Hard-coding access paths is what is premature optimisation. RDBMS defer decisions about what loops to process until runtime. So the optimiser can make good decisions based on statistics it has about the data. Decisions that programmers do not have to make prematurely. I think it's pretty clear that if you know exactly how your data is going to be used, then performance is better if you hard code everything and store it in a way that is optimised for retrieval. However, if data is used for multiple purposes, then hard-coding access paths is horrible in terms of performance and maintainance.