5 ms·

I'm curious if you know much about jobs doing this kind of thing. I've worked a ton with Python, and know a lot about using Cython to write extension modules, w

by p4wnc6 10y ago

I'm curious if you know much about jobs doing this kind of thing. I've worked a ton with Python, and know a lot about using Cython to write extension modules, wrap C++ code, etc. I really want a job where I find performance bottlenecks in research or business analytics or financial analytics code, migrate parts to robust Cython implementations, and then build and maintain the broader Python libraries and APIs surrounding that. But what I'm unfortunately finding is that basically nobody uses this stuff anywhere in practice. I've interviewed at a ton of places doing everything in C++ entirely, and I just see their code and I think, my god, why would anyone choose this? You can write the critical stuff in C++ just as you are, and then so easily wrap it for Python, then write the other 99% of the code in super easy, dynamic Python, even with various kinds of validating systems, or using typing and type annotations to avoid certain kinds of bugs, and just everyone's life is easier. But they are just so entrenched in the old way of doing it that nobody is willing to try. I've been consistently amazed at how few jobs in the monthly Who Is Hiring thread are tagged even just with NumPy. I think I've seen maybe one job in 6 months that mentions Cython, and then I found two others outside of HN and the interviews with those places didn't work out. How to find a job doing this stuff is like my main vexing work problem at the moment. It's so useful, but it's sort of locked outside of some kind of energy barrier that quant teams seem unwilling to cross.

partycoder 10y ago
Well, people have found success embedding Lua or JavaScript, since they're easier to sandbox than Python. Now, Python extensions can be a bit hard to get right. Especially since Python has a specific threading model and if you are trying to wrap an existing C++ code base that is multithreaded in Python... it can be challenging. Anyways, best of luck with your quest.
- bogomipz 10y ago
  Can you elaborate on why those are easier to sandbox than Python?
  - partycoder 10y ago
    Python offers a standard library with many, many functionalities. You can start processes, do networking, manipulate files, etc. You might not necessarily want to give your scripting environment those facilities.
S4M 10y ago
Why not write a blog post explaining your approach with some examples and mention on the page that you're available for a job/consulting job?
osullivj 10y ago
Suggest you look for an opening on JP Morgan's Athena project, or Bank of America Merrill Lynch Quartz. Both those platforms are C++ on the inside with Python APIs. The originators have gone independent here: http://www.wsq.io/about-us/ http://www.wsq.io/about-us/ The site says they're hiring.
- brianwawok 10y ago
  A few trading companies do this kind of work also. Since you really want to write C and Python... Do you care if you have to write trading code to help billionaires make more money?
  - p4wnc6 10y ago
    I don't think I would be a good match for bank culture or HFT, but if the work you describe is in a small-to-medium sized asset manager or hedge fund, then I would not mind, and would find it enjoyable. I worked at such a place a number of years ago and it was one of the better jobs I've had. The trouble I find is that a lot of small-to-medium sized asset managers and hedge funds don't have very good technology practices in general, and are pretty far from sophisticated use of low-level Python. There are often a lot of political reasons why they cling to Excel/VBA, R, or MATLAB, and there's almost always some political battle happening between the people that want to rationalize the system to a proper software design, and people who just want to keep cranking on the hodge podge of existing tools. I'd probably be fine with some trade-off regarding all of that, if the pay was acceptable and there was a strong commitment to a healthy work/life balance. But among finance firms this is extremely hard to find. So even though I am not at all opposed to doing this work in finance, I still find the volume of acceptable job openings in that field to be too small to make it realistic.
- throwaway160728 10y ago
  Obvious throwaway. Do NOT work on Quartz or Athena or WSQ. I have worked on two of these. I have seen the code for all three. These systems have too many WTFs to count. They have lead to technical crisis at both JP Morgan and Bank of America Merrill Lynch. At Bank of America Merrill Lynch, the crisis lead to resignations of the chief architect and the CTO. Stay away.
infinite8s 10y ago
Continuum.io (the guys behind Anaconda and a lot of new python data libraries like blaze, numba, dask) do this kind of work as consultants.
- p4wnc6 10y ago
  I actually worked at Continuum already. It's a great place with some of the best Python engineers you can possibly find. I found that the particular combination of start-up environment and consulting was not a good fit for me, and I suspect I would feel the same about Enthought or other kinds of analytics consulting companies, so alas this is not an option for me.
pm90 10y ago
https://www.enthought.com/company/careers/ https://www.enthought.com/company/careers/ - I worked at Enthought and can vet for the team that works there. https://www.continuum.io/join-us-empower-people-solve-worlds-greatest-challenges-using-open-source https://www.continuum.io/join-us-empower-people-solve-worlds... - I haven't worked here personally but have heard good things. Both are in Austin, Texas.
- p4wnc6 10y ago
  I worked remotely for Continuum before. As I mentioned in another comment, I think Continuum is a great place with amazing engineers, but my particular working style was not well-suited for the mixture of start-up environment and ad hoc consulting. My projects also did not happen to feature the use of any C stuff, only database stuff, pure Python, and pandas. I do think some other projects involved the Cython stuff, but nothing I was assigned to work on. So unfortunately these types of options aren't likely a good match for the sort of work I am looking for.
aldanor 10y ago
I do all of the stuff you mentioned on a daily basis (quant in a high-frequency trading firm) -- NumPy (plus its C API), Cython, numba, a good measure of C++, and even some occasional LLVM. The typical projects range from ingesting market data in high volumes to doing complex machine learning mumbo jumbo :)
- p4wnc6 10y ago
  I've tried applying at SIG in Philly in the past; I have all the things you mention on my resume (I worked previously at Continuum and also at a quant asset manager), but have not ever heard any kind of reply to my resume (not even a rejection) unfortunately.
  - aldanor 10y ago
    It's funny because I do work at SIG (Dublin, Ireland), and we do know a few folks from Continuum personally. For a quant role, even if in the end it is likely to involve a lot of hardcore low level coding, you're (generally) expected to have a PhD in math/phys/stat/eeng though, that's the rules. For developers roles I don't know all the details. But one thing for sure, we're hiring for quant roles pretty aggressively right now :)
    - p4wnc6 10y ago
      When I worked at a large quant asset manager as a quant analyst in Boston, I did not have a PhD and neither did any of the other analysts except maybe 1 or 2. It did seem you needed a PhD for portfolio management or higher, but not for quant analyst. This was also true at a hedge fund in Boston ... I ultimately turned down their offer, but they seemd to mostly hire master's degree folks (I have a master's in stats/machine learning) not full PhDs. Are there any openings (that would sponsor a visa) in the Ireland office? I'd love to apply and think SIG could be a good fit, but I've just had no luck responding to the Stack Overflow ads from the Philly office.
      - aldanor 10y ago

by p4wnc6 10y ago

partycoder 10y ago

Well, people have found success embedding Lua or JavaScript, since they're easier to sandbox than Python. Now, Python extensions can be a bit hard to get right. Especially since Python has a specific threading model and if you are trying to wrap an existing C++ code base that is multithreaded in Python... it can be challenging. Anyways, best of luck with your quest.

bogomipz 10y ago
Can you elaborate on why those are easier to sandbox than Python?
- partycoder 10y ago
  Python offers a standard library with many, many functionalities. You can start processes, do networking, manipulate files, etc. You might not necessarily want to give your scripting environment those facilities.

S4M 10y ago

Why not write a blog post explaining your approach with some examples and mention on the page that you're available for a job/consulting job?

osullivj 10y ago

Suggest you look for an opening on JP Morgan's Athena project, or Bank of America Merrill Lynch Quartz. Both those platforms are C++ on the inside with Python APIs. The originators have gone independent here: http://www.wsq.io/about-us/ http://www.wsq.io/about-us/ The site says they're hiring.

brianwawok 10y ago
A few trading companies do this kind of work also. Since you really want to write C and Python... Do you care if you have to write trading code to help billionaires make more money?
- p4wnc6 10y ago
  I don't think I would be a good match for bank culture or HFT, but if the work you describe is in a small-to-medium sized asset manager or hedge fund, then I would not mind, and would find it enjoyable. I worked at such a place a number of years ago and it was one of the better jobs I've had. The trouble I find is that a lot of small-to-medium sized asset managers and hedge funds don't have very good technology practices in general, and are pretty far from sophisticated use of low-level Python. There are often a lot of political reasons why they cling to Excel/VBA, R, or MATLAB, and there's almost always some political battle happening between the people that want to rationalize the system to a proper software design, and people who just want to keep cranking on the hodge podge of existing tools. I'd probably be fine with some trade-off regarding all of that, if the pay was acceptable and there was a strong commitment to a healthy work/life balance. But among finance firms this is extremely hard to find. So even though I am not at all opposed to doing this work in finance, I still find the volume of acceptable job openings in that field to be too small to make it realistic.
throwaway160728 10y ago
Obvious throwaway. Do NOT work on Quartz or Athena or WSQ. I have worked on two of these. I have seen the code for all three. These systems have too many WTFs to count. They have lead to technical crisis at both JP Morgan and Bank of America Merrill Lynch. At Bank of America Merrill Lynch, the crisis lead to resignations of the chief architect and the CTO. Stay away.

infinite8s 10y ago

Continuum.io (the guys behind Anaconda and a lot of new python data libraries like blaze, numba, dask) do this kind of work as consultants.

p4wnc6 10y ago
I actually worked at Continuum already. It's a great place with some of the best Python engineers you can possibly find. I found that the particular combination of start-up environment and consulting was not a good fit for me, and I suspect I would feel the same about Enthought or other kinds of analytics consulting companies, so alas this is not an option for me.

pm90 10y ago

https://www.enthought.com/company/careers/ https://www.enthought.com/company/careers/ - I worked at Enthought and can vet for the team that works there. https://www.continuum.io/join-us-empower-people-solve-worlds-greatest-challenges-using-open-source https://www.continuum.io/join-us-empower-people-solve-worlds... - I haven't worked here personally but have heard good things. Both are in Austin, Texas.

p4wnc6 10y ago
I worked remotely for Continuum before. As I mentioned in another comment, I think Continuum is a great place with amazing engineers, but my particular working style was not well-suited for the mixture of start-up environment and ad hoc consulting. My projects also did not happen to feature the use of any C stuff, only database stuff, pure Python, and pandas. I do think some other projects involved the Cython stuff, but nothing I was assigned to work on. So unfortunately these types of options aren't likely a good match for the sort of work I am looking for.

aldanor 10y ago

I do all of the stuff you mentioned on a daily basis (quant in a high-frequency trading firm) -- NumPy (plus its C API), Cython, numba, a good measure of C++, and even some occasional LLVM. The typical projects range from ingesting market data in high volumes to doing complex machine learning mumbo jumbo :)

p4wnc6 10y ago
I've tried applying at SIG in Philly in the past; I have all the things you mention on my resume (I worked previously at Continuum and also at a quant asset manager), but have not ever heard any kind of reply to my resume (not even a rejection) unfortunately.
- aldanor 10y ago
  It's funny because I do work at SIG (Dublin, Ireland), and we do know a few folks from Continuum personally. For a quant role, even if in the end it is likely to involve a lot of hardcore low level coding, you're (generally) expected to have a PhD in math/phys/stat/eeng though, that's the rules. For developers roles I don't know all the details. But one thing for sure, we're hiring for quant roles pretty aggressively right now :)
  - p4wnc6 10y ago
    When I worked at a large quant asset manager as a quant analyst in Boston, I did not have a PhD and neither did any of the other analysts except maybe 1 or 2. It did seem you needed a PhD for portfolio management or higher, but not for quant analyst. This was also true at a hedge fund in Boston ... I ultimately turned down their offer, but they seemd to mostly hire master's degree folks (I have a master's in stats/machine learning) not full PhDs. Are there any openings (that would sponsor a visa) in the Ireland office? I'd love to apply and think SIG could be a good fit, but I've just had no luck responding to the Stack Overflow ads from the Philly office.
    - aldanor 10y ago