4 ms·

Uh, hello!? Blekko is a web scale search engine that is built using Perl. It's not the language that makes handling big data slow. It's the algorithms and how

Uh, hello!? Blekko is a web scale search engine that is built using Perl. It's not the language that makes handling big data slow. It's the algorithms and how you move the bits around.

rieter 14y ago
Uhm, no. Google can't be implemented in Perl.
afsina 14y ago
Well, sure if you want to use 1000 servers instead of 100, go ahead and use it. No wonder Facebook noticed this and desperately trying to compile Php to C. Perhaps if Blekko ever gets popular we will see if their choice will bite them. Also an anecdotal example. Two months ago a friend of mine wrote a rather complex algorithm in Perl (He is very fluent with it). It was a novel sentence alignment algorithm using Gibbs sampling. Algorithm was hard to make parallel and not cache friendly. He needed to wait more than a day for training the system in a server. Well, long story short, with his help another developer converted it to Java in short time it worked around 200 times faster. So there. Moving bits around did not help Perl here at all.
- kamaal 14y ago
  >>Blekko ever gets popular we will see if their choice will bite them. What choice? The only reason why Blekko or Facebook is even able to launch and turn around things quickly to survive in competition is because they opt to use dynamic languages like Php and Perl. If they started doing their projects in C, with the current growth in rate of complexity they will never finish their projects ever. >>No wonder Facebook noticed this and desperately trying to compile Php to C. Dynamic languages aren't slower because they are not C. They are slow because they do a lot of magic. C is fast because the magic is left for you to perform. I can't see how any one pull out pace out of compiling Php to C directly, unless they sacrifice things along the way. Which really defeats the purpose of using Php at the first place. >>Well, long story short, with his help another developer converted it to Java in short time it worked around 200 times faster. So there. Moving bits around did not help Perl here at all. Number of programmers who even need to here the word 'bits' in their day to day activities(Talking of application programmers) are rare enough to make their case totally exceptional. Besides there are places where C makes perfect sense. There is hardly any other language heard of in the embedded programming world.
runarb 14y ago
Is the backend/search kernel in Perl to? I wrote the first prototype of Boitho (Norwegian internet search engine, now defunct) in Perl in 2000. That did not work at all because memory access and sorting was so slow. We had to write the next version in C. Of course a lot has happen with Perl in the last 13 years, but search kernel in Perl still sounds odd in my ears.
- randomstring 14y ago
  Granted, sometimes flipping bits is important. Then you write that part in C. There is no reason to pay the development tax of C for everything else. Not when you can build it in Perl, get it working, and then find the hotspots and convert those to C.
  - badgar 14y ago
    > There is no reason to pay the development tax of C for everything else. Not when you can build it in Perl, get it working, and then find the hotspots and convert those to C. Certainly not with fewer than 20 million req/day.

Uh, hello!? Blekko is a web scale search engine that is built using Perl. It's not the language that makes handling big data slow. It's the algorithms and how you move the bits around.

rieter 14y ago

Uhm, no. Google can't be implemented in Perl.

afsina 14y ago

Well, sure if you want to use 1000 servers instead of 100, go ahead and use it. No wonder Facebook noticed this and desperately trying to compile Php to C. Perhaps if Blekko ever gets popular we will see if their choice will bite them. Also an anecdotal example. Two months ago a friend of mine wrote a rather complex algorithm in Perl (He is very fluent with it). It was a novel sentence alignment algorithm using Gibbs sampling. Algorithm was hard to make parallel and not cache friendly. He needed to wait more than a day for training the system in a server. Well, long story short, with his help another developer converted it to Java in short time it worked around 200 times faster. So there. Moving bits around did not help Perl here at all.

kamaal 14y ago
>>Blekko ever gets popular we will see if their choice will bite them. What choice? The only reason why Blekko or Facebook is even able to launch and turn around things quickly to survive in competition is because they opt to use dynamic languages like Php and Perl. If they started doing their projects in C, with the current growth in rate of complexity they will never finish their projects ever. >>No wonder Facebook noticed this and desperately trying to compile Php to C. Dynamic languages aren't slower because they are not C. They are slow because they do a lot of magic. C is fast because the magic is left for you to perform. I can't see how any one pull out pace out of compiling Php to C directly, unless they sacrifice things along the way. Which really defeats the purpose of using Php at the first place. >>Well, long story short, with his help another developer converted it to Java in short time it worked around 200 times faster. So there. Moving bits around did not help Perl here at all. Number of programmers who even need to here the word 'bits' in their day to day activities(Talking of application programmers) are rare enough to make their case totally exceptional. Besides there are places where C makes perfect sense. There is hardly any other language heard of in the embedded programming world.

runarb 14y ago

Is the backend/search kernel in Perl to? I wrote the first prototype of Boitho (Norwegian internet search engine, now defunct) in Perl in 2000. That did not work at all because memory access and sorting was so slow. We had to write the next version in C. Of course a lot has happen with Perl in the last 13 years, but search kernel in Perl still sounds odd in my ears.

randomstring 14y ago
Granted, sometimes flipping bits is important. Then you write that part in C. There is no reason to pay the development tax of C for everything else. Not when you can build it in Perl, get it working, and then find the hotspots and convert those to C.
- badgar 14y ago
  > There is no reason to pay the development tax of C for everything else. Not when you can build it in Perl, get it working, and then find the hotspots and convert those to C. Certainly not with fewer than 20 million req/day.