4 ms·

I love Python, it gives me the best jobs of my life. Usually is like this. Customer happens to stumble some easy 3 lines program that solves the problem in an

I love Python, it gives me the best jobs of my life. Usually is like this. Customer happens to stumble some easy 3 lines program that solves the problem in an easy subset and after that it will create a full program around that one with all the blings. Then goes into production after a month and 15k USD lighter and its users are using it for full scale and the solving time now is in hours. Then it will search for an expert to solve the impossible problem of speeding up Python and curses at all the Indians that are scamming him and all experts who ask for another 15k USD to actually implement the program correctly. I shit you not, last one was a data scientist who paid like $3/h some Indian dude that got the job done in 2 days, took the money and contract was closed under those terms. Then this data scientist was all over his job posting crying that, while the initial matrix was in dozens of columns/rows now the same Python program would take 3 days when throwing at it a matrix that was having millions of columns/rows in size. I mean, it's several orders of magnitude higher, I was amazed that the program actually finished the job after 3 days instead of just crashing. So I had to drill into his thick skull the idea that while he initially went to war against Lichtenstein and won, definitely cannot win against US army using the same weapon. Only after that he agreed to scrap the Python program altogether and finally go with C++, because speed does matter in the end. Like I said, I love Python, especially for anything math related.

alokrai 6y ago
I wonder why it is necessary to mention "Indians that are scamming" him as if scamming is something only Indians do? Should I be careful only when dealing with Indian consultants and presume others won't lie or cheat? edit: grammar
- unnouinceput 6y ago
  Probably because Indians are the ones that are leaders in scamming? Also, if you paid attention, I've mentioned the initial Python dev was an honest Indian, he did his job within terms of contract. The data scientist, which in this case was a Canadian, is solely at fault for not disclosing full scale to the Indian one.
  - alokrai 6y ago
    >Probably because Indians are the ones that are leaders in scamming? I could respond to it but there is not much to be said. I think you have made my point.
  - rramadass 6y ago
    You really seem to have a way with words where, while the point of your original premise (i.e. lack of requirements, technical competence) is valid you have managed to bury that under completely unrelated negative and racist stereotyping.
- mlN90 6y ago
  >as if [bad thing] is something only [group] do? This is such a nonsense statement and it comes up everytime someone describes someones nationality in correlation with something awful. Honest question, when you read the line "[...]Chinese that are good at math [...]" do you read that all Chinese people are math-wizards? Because that is not what it says. What about "[...] black people that are fast runners [...]"? all 3 examples, including the one that turned on your torch of virtue, describes a sub-set of a group, the primary attribute of said group and nothing more. If anything the implication that the guy you were replied to is somehow biased and "racist" against the absolute-plague-tier of disproportional scammers coming out of India is based on nothing but your inability to differentiate between "broadspectrum-racism" and "critism of a subset of a group"
  - alokrai 6y ago
    This is a very charitable reading of the comment, and the examples stated seem somewhat unrelated. A closer analogy will be: "He was lost in New York City. Later, he cursed at all the Blacks who robbed him." Or "He had an intense negotiation with the financiers. He later cursed at all the Jews who were scamming him." As you may note, the term "jews" or "blacks" or "Indians" (in the original comment) is not merely stated as an adjective to describe the individuals, rather it is used in pejorative sense to denote a cultural trait within the group that makes them act in a particular manner. A child comment by the original poster makes his prejudice quite clear: "Probably because Indians are the ones that are leaders in scamming? " I get your whole point about talking about individual, subset, and group, but it looks like just a defence for calling Indians "world leaders in scamming.", rather than some data based, dispassionate description of the situation. Edit: grammar
    - mlN90 6y ago
      You have to resort to using analogies when the actual sentence in question transfers very well in my examples? I'm making extreme examples out of the sentence, but putting something 'awesome' with it. Being good at math / Fast runners etc - to make the point very concise and on point. Had i run with the theme and went "White people who shoots up schools [...]" or "Black people who sell crack cocaine" you would have likely missed the point entirely because I'm using negative-stereotypes. That the child-comment elaborates his thoughts into racist ramblings is frankly irrelevant to me. The guy is clearly both illiterate, insensitive and likely in the silly end of the bell curve.
alejoar 6y ago
Libraries like numpy are implemented in C under the hood, and you can also easily extend your Python code with C.
- enriquto 6y ago
  > you can also easily extend your Python code with C Or, more precisely, you can easily wrap your algorithm written in C by a bit of boilerplate Python code (if that is your fancy).
  - fauigerzigerk 6y ago
    Or you could use Cython to do a bit of both at the same time.
    - enriquto 6y ago
      I don't see the advantage. If you use cython then your code becomes quite difficult to use in the rest of the C ecosystem (where your algorithm belongs).
      - counters 6y ago
        Usually if you use Cython, f2py, numba, or something similar, the goal isn't to create code that fits into the ecosystem of the compiled language - it's to optimize a kernel that fits into the Python ecosystem with minimal baggage.
shepardrtc 6y ago
I'm sure that Python program could have been rewritten to complete in an acceptable amount of time. Yes, the C++ program will be faster, but a good Python dev could probably have fixed that scientist's code with numpy, proper BLAS libraries, and maybe a quick dash of Cython.
- dataflow 6y ago
  Don't forget Numba.
- galangalalgol 6y ago
  I post this whenever it comes up, but lots of small matricies or any operation gets hit too hard by the ffi overhead into numpy.
  - shepardrtc 6y ago
    Yes, for smaller stuff, numpy really does add a lot of overhead. It would make life much simpler if you could copy things into a numpy array more quickly, but oh well. In any case, you can find pretty tight pure Python code for most things. For instance, I needed to drop a relatively small Linear Regression calculation from like 500 microseconds using numpy or scipy (I don't remember) to double-digit microseconds somehow. I googled it for a little bit and after adapting some pure python code using regular lists, I got it into double digits. And then after converting the function rather easily into Cython (and just the function, not the entire program), its single-digit microseconds now.
    - deleted 6y ago
      [deleted]
objektif 6y ago
This is a horribly written post you may want to edit it.
analognoise 6y ago
Where can I hire somebody for $3/hour? Surely this is an exaggeration?

alokrai 6y ago

I wonder why it is necessary to mention "Indians that are scamming" him as if scamming is something only Indians do? Should I be careful only when dealing with Indian consultants and presume others won't lie or cheat? edit: grammar

unnouinceput 6y ago
Probably because Indians are the ones that are leaders in scamming? Also, if you paid attention, I've mentioned the initial Python dev was an honest Indian, he did his job within terms of contract. The data scientist, which in this case was a Canadian, is solely at fault for not disclosing full scale to the Indian one.
- alokrai 6y ago
  >Probably because Indians are the ones that are leaders in scamming? I could respond to it but there is not much to be said. I think you have made my point.
- rramadass 6y ago
  You really seem to have a way with words where, while the point of your original premise (i.e. lack of requirements, technical competence) is valid you have managed to bury that under completely unrelated negative and racist stereotyping.
mlN90 6y ago
>as if [bad thing] is something only [group] do? This is such a nonsense statement and it comes up everytime someone describes someones nationality in correlation with something awful. Honest question, when you read the line "[...]Chinese that are good at math [...]" do you read that all Chinese people are math-wizards? Because that is not what it says. What about "[...] black people that are fast runners [...]"? all 3 examples, including the one that turned on your torch of virtue, describes a sub-set of a group, the primary attribute of said group and nothing more. If anything the implication that the guy you were replied to is somehow biased and "racist" against the absolute-plague-tier of disproportional scammers coming out of India is based on nothing but your inability to differentiate between "broadspectrum-racism" and "critism of a subset of a group"
- alokrai 6y ago
  This is a very charitable reading of the comment, and the examples stated seem somewhat unrelated. A closer analogy will be: "He was lost in New York City. Later, he cursed at all the Blacks who robbed him." Or "He had an intense negotiation with the financiers. He later cursed at all the Jews who were scamming him." As you may note, the term "jews" or "blacks" or "Indians" (in the original comment) is not merely stated as an adjective to describe the individuals, rather it is used in pejorative sense to denote a cultural trait within the group that makes them act in a particular manner. A child comment by the original poster makes his prejudice quite clear: "Probably because Indians are the ones that are leaders in scamming? " I get your whole point about talking about individual, subset, and group, but it looks like just a defence for calling Indians "world leaders in scamming.", rather than some data based, dispassionate description of the situation. Edit: grammar
  - mlN90 6y ago
    You have to resort to using analogies when the actual sentence in question transfers very well in my examples? I'm making extreme examples out of the sentence, but putting something 'awesome' with it. Being good at math / Fast runners etc - to make the point very concise and on point. Had i run with the theme and went "White people who shoots up schools [...]" or "Black people who sell crack cocaine" you would have likely missed the point entirely because I'm using negative-stereotypes. That the child-comment elaborates his thoughts into racist ramblings is frankly irrelevant to me. The guy is clearly both illiterate, insensitive and likely in the silly end of the bell curve.

alejoar 6y ago

Libraries like numpy are implemented in C under the hood, and you can also easily extend your Python code with C.

enriquto 6y ago
> you can also easily extend your Python code with C Or, more precisely, you can easily wrap your algorithm written in C by a bit of boilerplate Python code (if that is your fancy).
- fauigerzigerk 6y ago
  Or you could use Cython to do a bit of both at the same time.
  - enriquto 6y ago
    I don't see the advantage. If you use cython then your code becomes quite difficult to use in the rest of the C ecosystem (where your algorithm belongs).
    - counters 6y ago
      Usually if you use Cython, f2py, numba, or something similar, the goal isn't to create code that fits into the ecosystem of the compiled language - it's to optimize a kernel that fits into the Python ecosystem with minimal baggage.

shepardrtc 6y ago

I'm sure that Python program could have been rewritten to complete in an acceptable amount of time. Yes, the C++ program will be faster, but a good Python dev could probably have fixed that scientist's code with numpy, proper BLAS libraries, and maybe a quick dash of Cython.

dataflow 6y ago
Don't forget Numba.
galangalalgol 6y ago
I post this whenever it comes up, but lots of small matricies or any operation gets hit too hard by the ffi overhead into numpy.
- shepardrtc 6y ago
  Yes, for smaller stuff, numpy really does add a lot of overhead. It would make life much simpler if you could copy things into a numpy array more quickly, but oh well. In any case, you can find pretty tight pure Python code for most things. For instance, I needed to drop a relatively small Linear Regression calculation from like 500 microseconds using numpy or scipy (I don't remember) to double-digit microseconds somehow. I googled it for a little bit and after adapting some pure python code using regular lists, I got it into double digits. And then after converting the function rather easily into Cython (and just the function, not the entire program), its single-digit microseconds now.
  - deleted 6y ago
    [deleted]

objektif 6y ago

This is a horribly written post you may want to edit it.

analognoise 6y ago

Where can I hire somebody for $3/hour? Surely this is an exaggeration?