6 ms·

PHP5 Database Iterators

14 points by leftnode 17y ago

leftnode 17y ago
I'm the author and submitter of the article, and I'd honestly love to hear what people think about this. I know there's a lot of (understandable) disdain for PHP, but I think you can do some neat things with it. This is one application that I've found particularly useful and wanted to share it.
- thaumaturgy 17y ago
  I don't generally comment on anything related to individual programmer's styles or programming methodologies; I think those are targeted more at inexperienced programmers that are still developing their technique. That said, my personal take on things like this is that it defeats PHP's strengths in an attempt to make it behave like something it isn't. The primary reasons for coding web applications in PHP instead of Java, Ruby, or Python or what-have-you is speed and portability. Your code isn't compatible with PHP4 -- which a lot of servers are still running -- and you're spending an awful lot of code to duplicate functionality that's already built into PHP. You can already perform a query and get the results back and if you want to iterate over them, you have an array to step through. That would be maybe around 5 lines of code, and you wouldn't need comments explaining what it does because it would be clear and obvious. That's a lot less code for the parser to evaluate, and a lot less code for it to run, which will probably translate to being able to handle a lot more hits. It would be interesting to load your code onto an app, and then load a simpler, more direct version that does the same thing, and see how their performance compares. Nicely written article though.
  - leftnode 17y ago
    Thanks for the thoughts. To be honest, I'm not really concerned about PHP4 anymore, it's been dead a while. Thanks giving me an idea for a followup article too, I'll do exactly that, write a small direct application and compare performance with large datasets. Also, doing this in a purely procedural method would be an interesting article too.
  - wizard_2 17y ago
    I don't think its a strength of php to have code portable to older versions of php. New versions mean new features and he's taken full advantage of some of the most powerful features in php5. I'll quote the article "The code in this article was adopted from Artisan System, Leftnode’s PHP5 framework." I'm curious how many situations where new code is stuck in a php4 environment. PHP4 is getting quite old, and has been end of life'd. Most hosts run php5 or can be asked to upgrade without much fuss. After all it has been years since it came out. As for iteration of a model vs arrays, having your data in a model has many more advantages then a collection of strings. Chances are you don't need the slight performance you'd gain out of sticking with strings and you'd loose the advantages of working with a framework. It's like saying ROR is inefficient or slow. The point is frameworks are designed to make development easier. If you need the performance you might gain out of not using the framework, then you're probably at a point where you're past needing a framework to help build your app.
    - thaumaturgy 17y ago
      Yeah, PHP4 was EOL'd quite a while back. Unfortunately, the transition from PHP4 to PHP5 wasn't easy for a lot of people, and application developers didn't make this any easier for end-users by having convoluted and often broken upgrade paths. Of the two hosts I've dealt with regularly, both of them still support PHP4 on their older accounts, and both of them have handled the transition by setting up new servers and having all new accounts installed on their PHP5 servers. I'm curious about the advantages you say that a model has over an array of strings. I can't honestly think of any. I don't disagree that frameworks are designed to make development easier. Quite the contrary: that's why I have a strong preference for frameworks that are very clear, very concise, and very lightweight. I'm way beyond busy right now, but I'm considering writing a competing procedural approach to leftnode's code, just for fun.
      - leftnode 17y ago
        Cool, I'd love to see what you come up with!
  - jrockway 17y ago
    The primary reason[...] for coding web applications in PHP instead of Java, Ruby, or Python or what-have-you is speed This seems like a bad reason: http://shootout.alioth.debian.org/u64q/benchmark.php?test=all&lang=php&lang2=java&box=1 http://shootout.alioth.debian.org/u64q/benchmark.php?test=al... PHP even uses more memory than Java on some benchmarks, which is pretty damn impressive. You have to work quite hard to use more RAM than Java. (You'll also notice that source size of PHP is not much smaller than the equivalent Java program. PHP is a very verbose programming language compared to its other "dynamic language" brethren, and even to non-"dynamic" langauges like Haskell.)
    - thaumaturgy 17y ago
      That's a really interesting link, thanks. Something about it kinda smells though: it doesn't match my experiences with PHP at all. (I can't speak for Java in this case.) I've got half of a CMS under development that's just recently -- sadly -- topped a thousand lines of code. It has a built-in parser for its own human-readable configuration file, which it has to parse every time; it has built-in cache management; it merges templates; it handles recursion for nested templates; and it does on-the-fly high quality image resizing. All of my out-and-back server response times are well under 1 second for all this work. I'll look around and see if there are other trustworthy benchmarks that corroborate this. EDIT: Yeah, IEEE disagrees: http://www2.computer.org/portal/web/csdl/doi/10.1109/ICWS.2008.71 http://www2.computer.org/portal/web/csdl/doi/10.1109/ICWS.20... I'm not surprised by PHP's memory usage, actually; I long ago figured it must be enormous just based on the way it handles data types. I don't think the shootout link is deliberately deceptive; rather, all the tests that I looked at were massively recursive, and this is not a task for which PHP was designed -- I think I would expect Java to perform better in those cases.
      - igouy 17y ago
        > Something about it kinda smells though: it doesn't match my experiences with PHP at all. Did you look at what the programs do before announcing that "it kinda smells"? Did you look at the "really interesting link" enough to realize the benchmarks game website is written in PHP? Did you look at that particular web page enough to see "PHP is rarely the bottleneck (HTML slides)" ?
- Maascamp 17y ago
  You're not the first to try this. Sapphire (the framework powering the SilverStripe CMS) implements this technique and has all of it's DataObject select calls return a 'DataObjectSet' which is basically the equivalent of your DB_Iterator.
  - leftnode 17y ago
    Cool! I'll check it out.
daok 17y ago
Well written article, very clear. Nice job. The only thing that I do not like is that you would have to put some database field name in the controller to set data to the persistence instead of having a DataAccessLayer to handle it. The main problem is if in the future you refactor the persistence of change it, you will have to go to multiple place to be able to make everything work. I might be wrong... but at the first look, this is what I see. Oh by the way, do you always but your email address on each method of all your code :P ?
- leftnode 17y ago
  Thanks for the comments. Most of that code was copied from my framework, which is full of Doxygen comments, and I use my username in our subversion repository and my email address for the author of each method since several methods can have multiple authors.
  - jrockway 17y ago
    What value do you derive from documenting the author of each method? That seems like a lot of overhead for very little benefit, especially when your VCS can do an annotation and give you more detail than one comment can.
zackattack 17y ago
I don't have much formal CS training, and I code primarily in PHP. That being said, I don't see how this is useful at all. I've started coding more in PHP5 recently: my last two projects have been using classes. I started creating a DBHandler class. Then, whenever I need a DB query, I have a method like this: function getSnsWithoutIds($customerid) { $query = sprintf("SELECT aim_sn FROM add_queue WHERE aim_id=0 AND cid=%d", $customerid); $result = mysql_query($query); $sns = array(); while($row = mysql_fetch_assoc($result)) { array_push($sns, $row['aim_sn']); } return $sns; } And then I simply iterate on the array back in the code. $dbh = new DBHandler(); $arr = $dbh->getSnsWithoutIds(); foreach($arr as $ele) { doSomething(); } What is the advantage to doing things your way?
- leftnode 17y ago
  Well, imagine that the query returns 1000 (or a million or what not) results. You've looped through it once already in the getSnsWithoutIds() to build the $sns array (which you should use $sns[] = $row['aim_sn'], its faster). So, you've looped through everything, which takes some time, and now you have a huge data structure in memory that you may not need. Then, you loop through it again with the foreach. What if you wanted to paginate those results quickly? What if you only wanted a subset of the whole pie? This class, as I'll show in a future article, can easily be extended to allow for easy pagination and filtering of your data.
  - zackattack 17y ago
    Well, it takes a trivial amount of time to loop through an array with a million results. So can you please clarify the benefit of the class as-is, besides its purported future extensiblity? I don't get the time to unset() the array before I return it, but you do? BTW I did a test, and it seems that $array[] is a faster way to append.
    - jrockway 17y ago
      BTW I did a test, and it seems that $array[] is a faster way to append. Classic PHP-style micro-optimization. If you can't use the right algorithm, you can at least make the wrong one run 0.34% faster.

14 points by leftnode 17y ago

I'm the author and submitter of the article, and I'd honestly love to hear what people think about this. I know there's a lot of (understandable) disdain for PHP, but I think you can do some neat things with it. This is one application that I've found particularly useful and wanted to share it.

thaumaturgy 17y ago
I don't generally comment on anything related to individual programmer's styles or programming methodologies; I think those are targeted more at inexperienced programmers that are still developing their technique. That said, my personal take on things like this is that it defeats PHP's strengths in an attempt to make it behave like something it isn't. The primary reasons for coding web applications in PHP instead of Java, Ruby, or Python or what-have-you is speed and portability. Your code isn't compatible with PHP4 -- which a lot of servers are still running -- and you're spending an awful lot of code to duplicate functionality that's already built into PHP. You can already perform a query and get the results back and if you want to iterate over them, you have an array to step through. That would be maybe around 5 lines of code, and you wouldn't need comments explaining what it does because it would be clear and obvious. That's a lot less code for the parser to evaluate, and a lot less code for it to run, which will probably translate to being able to handle a lot more hits. It would be interesting to load your code onto an app, and then load a simpler, more direct version that does the same thing, and see how their performance compares. Nicely written article though.
- leftnode 17y ago
  Thanks for the thoughts. To be honest, I'm not really concerned about PHP4 anymore, it's been dead a while. Thanks giving me an idea for a followup article too, I'll do exactly that, write a small direct application and compare performance with large datasets. Also, doing this in a purely procedural method would be an interesting article too.
- wizard_2 17y ago
  I don't think its a strength of php to have code portable to older versions of php. New versions mean new features and he's taken full advantage of some of the most powerful features in php5. I'll quote the article "The code in this article was adopted from Artisan System, Leftnode’s PHP5 framework." I'm curious how many situations where new code is stuck in a php4 environment. PHP4 is getting quite old, and has been end of life'd. Most hosts run php5 or can be asked to upgrade without much fuss. After all it has been years since it came out. As for iteration of a model vs arrays, having your data in a model has many more advantages then a collection of strings. Chances are you don't need the slight performance you'd gain out of sticking with strings and you'd loose the advantages of working with a framework. It's like saying ROR is inefficient or slow. The point is frameworks are designed to make development easier. If you need the performance you might gain out of not using the framework, then you're probably at a point where you're past needing a framework to help build your app.
  - thaumaturgy 17y ago
    Yeah, PHP4 was EOL'd quite a while back. Unfortunately, the transition from PHP4 to PHP5 wasn't easy for a lot of people, and application developers didn't make this any easier for end-users by having convoluted and often broken upgrade paths. Of the two hosts I've dealt with regularly, both of them still support PHP4 on their older accounts, and both of them have handled the transition by setting up new servers and having all new accounts installed on their PHP5 servers. I'm curious about the advantages you say that a model has over an array of strings. I can't honestly think of any. I don't disagree that frameworks are designed to make development easier. Quite the contrary: that's why I have a strong preference for frameworks that are very clear, very concise, and very lightweight. I'm way beyond busy right now, but I'm considering writing a competing procedural approach to leftnode's code, just for fun.
    - leftnode 17y ago
      Cool, I'd love to see what you come up with!
- jrockway 17y ago
  The primary reason[...] for coding web applications in PHP instead of Java, Ruby, or Python or what-have-you is speed This seems like a bad reason: http://shootout.alioth.debian.org/u64q/benchmark.php?test=all&lang=php&lang2=java&box=1 http://shootout.alioth.debian.org/u64q/benchmark.php?test=al... PHP even uses more memory than Java on some benchmarks, which is pretty damn impressive. You have to work quite hard to use more RAM than Java. (You'll also notice that source size of PHP is not much smaller than the equivalent Java program. PHP is a very verbose programming language compared to its other "dynamic language" brethren, and even to non-"dynamic" langauges like Haskell.)
  - thaumaturgy 17y ago
    That's a really interesting link, thanks. Something about it kinda smells though: it doesn't match my experiences with PHP at all. (I can't speak for Java in this case.) I've got half of a CMS under development that's just recently -- sadly -- topped a thousand lines of code. It has a built-in parser for its own human-readable configuration file, which it has to parse every time; it has built-in cache management; it merges templates; it handles recursion for nested templates; and it does on-the-fly high quality image resizing. All of my out-and-back server response times are well under 1 second for all this work. I'll look around and see if there are other trustworthy benchmarks that corroborate this. EDIT: Yeah, IEEE disagrees: http://www2.computer.org/portal/web/csdl/doi/10.1109/ICWS.2008.71 http://www2.computer.org/portal/web/csdl/doi/10.1109/ICWS.20... I'm not surprised by PHP's memory usage, actually; I long ago figured it must be enormous just based on the way it handles data types. I don't think the shootout link is deliberately deceptive; rather, all the tests that I looked at were massively recursive, and this is not a task for which PHP was designed -- I think I would expect Java to perform better in those cases.
    - igouy 17y ago
      > Something about it kinda smells though: it doesn't match my experiences with PHP at all. Did you look at what the programs do before announcing that "it kinda smells"? Did you look at the "really interesting link" enough to realize the benchmarks game website is written in PHP? Did you look at that particular web page enough to see "PHP is rarely the bottleneck (HTML slides)" ?
Maascamp 17y ago
You're not the first to try this. Sapphire (the framework powering the SilverStripe CMS) implements this technique and has all of it's DataObject select calls return a 'DataObjectSet' which is basically the equivalent of your DB_Iterator.
- leftnode 17y ago
  Cool! I'll check it out.

daok 17y ago

Well written article, very clear. Nice job. The only thing that I do not like is that you would have to put some database field name in the controller to set data to the persistence instead of having a DataAccessLayer to handle it. The main problem is if in the future you refactor the persistence of change it, you will have to go to multiple place to be able to make everything work. I might be wrong... but at the first look, this is what I see. Oh by the way, do you always but your email address on each method of all your code :P ?

leftnode 17y ago
Thanks for the comments. Most of that code was copied from my framework, which is full of Doxygen comments, and I use my username in our subversion repository and my email address for the author of each method since several methods can have multiple authors.
- jrockway 17y ago
  What value do you derive from documenting the author of each method? That seems like a lot of overhead for very little benefit, especially when your VCS can do an annotation and give you more detail than one comment can.

zackattack 17y ago

I don't have much formal CS training, and I code primarily in PHP. That being said, I don't see how this is useful at all. I've started coding more in PHP5 recently: my last two projects have been using classes. I started creating a DBHandler class. Then, whenever I need a DB query, I have a method like this: function getSnsWithoutIds($customerid) { $query = sprintf("SELECT aim_sn FROM add_queue WHERE aim_id=0 AND cid=%d", $customerid); $result = mysql_query($query); $sns = array(); while($row = mysql_fetch_assoc($result)) { array_push($sns, $row['aim_sn']); } return $sns; } And then I simply iterate on the array back in the code. $dbh = new DBHandler(); $arr = $dbh->getSnsWithoutIds(); foreach($arr as $ele) { doSomething(); } What is the advantage to doing things your way?

leftnode 17y ago
Well, imagine that the query returns 1000 (or a million or what not) results. You've looped through it once already in the getSnsWithoutIds() to build the $sns array (which you should use $sns[] = $row['aim_sn'], its faster). So, you've looped through everything, which takes some time, and now you have a huge data structure in memory that you may not need. Then, you loop through it again with the foreach. What if you wanted to paginate those results quickly? What if you only wanted a subset of the whole pie? This class, as I'll show in a future article, can easily be extended to allow for easy pagination and filtering of your data.
- zackattack 17y ago
  Well, it takes a trivial amount of time to loop through an array with a million results. So can you please clarify the benefit of the class as-is, besides its purported future extensiblity? I don't get the time to unset() the array before I return it, but you do? BTW I did a test, and it seems that $array[] is a faster way to append.
  - jrockway 17y ago
    BTW I did a test, and it seems that $array[] is a faster way to append. Classic PHP-style micro-optimization. If you can't use the right algorithm, you can at least make the wrong one run 0.34% faster.