4 ms·

In a talk once I heard web applications described as "database skins." Maybe it's hyperbole, but the general point seems sound - most web apps are CRUD, and the

In a talk once I heard web applications described as "database skins." Maybe it's hyperbole, but the general point seems sound - most web apps are CRUD, and the slow point in most applications is IO. I suspect not doing dumb IO is 9 times out of 10 the solution to performance problems. That said, working against oracle databases with Django is slow as all get out. Something about the driver requires numbers to be converted from number to string back to number, while calculating whether it's an integer/decimal/whatever. Simple queries returning 1000 rows could take ten seconds. I spent several days running that down. The analysts at my shop blamed django, which was pretty unfortunate.

combatentropy 10y ago
> I heard web applications described as "database skins." Not a bad description. > the slow point in most applications is IO The slowest part of web apps in my decade of experience is the database, yes, if that's what you mean by IO. Reading and writing files, also IO, is very fast though. A database can be very fast too (I've mostly used Postgres). Or it can be dog slow. It's just like how some programmers are 100 times better than other programmers. One SQL query is 100 times faster than another, and both get the same result. And it's not usually something simple like adding a column index. Usually it's a matter of a sub-select or some other squirrelly way of combining tables or aggregating a result. I wish more programmers learned SQL --- I mean really learned it through and through --- than concentrated on the latest fads.
- iofj 10y ago
  > I wish more programmers learned SQL --- I mean really learned it through and through --- than concentrated on the latest fads. And miss out on all the coffee breaks afforded by programmers just looping over entire databases in a horribly inefficient ORM ? Because all this condition abstraction in the ORM is really hard to use ... Don't worry though : they're optimizing it. I hear they're upgrading to run the loop in 100 threads instead of just one. Surely that will accelerate things a lot ! And if that doesn't work, we always have memcache, don't we ? Also we can now use cloud databases to spread the load, surely that'll help (I'm looking forward for the first double-digit page refresh, measured in dollars) (Having worked years for hosting providers where 20-30% of my job was making PHP/Python/Ruby/sometimes even Java applications run faster, and achieving 50x improvements on a regular basis, this had to get off my chest. And yes I've seen people implement mail address lookups by iterating over the ORM-provided collection ... and then show 50 mail addresses, wondering why the page load time was bordering on 3 digits when measured in minutes. Not even just once)

combatentropy 10y ago

> I heard web applications described as "database skins." Not a bad description. > the slow point in most applications is IO The slowest part of web apps in my decade of experience is the database, yes, if that's what you mean by IO. Reading and writing files, also IO, is very fast though. A database can be very fast too (I've mostly used Postgres). Or it can be dog slow. It's just like how some programmers are 100 times better than other programmers. One SQL query is 100 times faster than another, and both get the same result. And it's not usually something simple like adding a column index. Usually it's a matter of a sub-select or some other squirrelly way of combining tables or aggregating a result. I wish more programmers learned SQL --- I mean really learned it through and through --- than concentrated on the latest fads.

iofj 10y ago
> I wish more programmers learned SQL --- I mean really learned it through and through --- than concentrated on the latest fads. And miss out on all the coffee breaks afforded by programmers just looping over entire databases in a horribly inefficient ORM ? Because all this condition abstraction in the ORM is really hard to use ... Don't worry though : they're optimizing it. I hear they're upgrading to run the loop in 100 threads instead of just one. Surely that will accelerate things a lot ! And if that doesn't work, we always have memcache, don't we ? Also we can now use cloud databases to spread the load, surely that'll help (I'm looking forward for the first double-digit page refresh, measured in dollars) (Having worked years for hosting providers where 20-30% of my job was making PHP/Python/Ruby/sometimes even Java applications run faster, and achieving 50x improvements on a regular basis, this had to get off my chest. And yes I've seen people implement mail address lookups by iterating over the ORM-provided collection ... and then show 50 mail addresses, wondering why the page load time was bordering on 3 digits when measured in minutes. Not even just once)