3 ms·

Having worked with flask+Python eco-system and golang do you think if you continued with Python for v2 it might have been better (since you were already familia

Having worked with flask+Python eco-system and golang do you think if you continued with Python for v2 it might have been better (since you were already familiar with Python even though golang is statically typed)? Also do you think Python 3.6 static checking with mypy would have eliminated need for go? Another question is for algorithmic scheduling why you didn't use Python libraries but Julia, given your v1 was written in Python?

philip1209 10y ago
No, I was having issues with too much "magic" in Python libraries. I'm continuing to use Go for my new company. In particular, running SQLAlchemy was a nightmare (see a reddit comment I made about it here [1]). I also wanted a more parallel language - I was tired of having to use queues to do parallel tasks. Go's static typing made everything easier, from auto-generating Swagger definitions to having usable code documentation via Godoc. Most of our runtime errors while using Python were due to type mismatches that could have been caught by a compiler. I'm unfamiliar with Python 3.6 static checking. I'll have to look into it. For the algorithmic scheduling - our algorithms in Julia predate the existence of a web app. We had customers using the Julia-based algorithm using the protocol "spreadsheets over email". The website came later, and we ended up switching the algorithms from Julia to Python after the web-app launch [2]. [1] https://www.reddit.com/r/Python/comments/5w3o6t/staffjoy_v1_has_been_open_sourced_if_you_want_to/de96ybk/?context=3 https://www.reddit.com/r/Python/comments/5w3o6t/staffjoy_v1_... [2] https://blog.staffjoy.com/retro-on-the-julia-programming-language-7655121ea341 https://blog.staffjoy.com/retro-on-the-julia-programming-lan...
- dragonsh 10y ago
  Thanks for the reply and insight. Why you had issues with magic in Python libraries, as I understand one of the main design goals of Python and its libraries is "Explicit is better then implicit" PEP-20 [1]. Indeed for our own project we chose python instead of ruby and ruby on rails just because of this PEP-20[1]. Your blog and moving to go alarmed us if there is anything wrong with python eco-system. We tried go and due to too much boilerplate code and quality of the libraries we dropped it and when it comes to interacting with database especially postgreSQL all the speed advantage of go is not very useful as most tasks were IO intensive. Also the database driver in golang for postgresql is not as mature as psycopg. For CPU intensive tasks we relied on scientific python which is written in C and it worked well with multiprocessing and asyncio in python. Also for the specific error you encountered in race conditions on update with multiple request. We solved it by using SQLAlchemy with_for_update()[2] which actually takes care at database level to make sure transactions are properly handled. As database we were using is postgreSQL, we wanted the database to handle ACID compliance, not implement in application code. [1]https://www.python.org/dev/peps/pep-0020/ https://www.python.org/dev/peps/pep-0020/ [2]http://docs.sqlalchemy.org/en/latest/core/selectable.html#sqlalchemy.sql.expression.Select.with_for_update http://docs.sqlalchemy.org/en/latest/core/selectable.html#sq...

philip1209 10y ago

No, I was having issues with too much "magic" in Python libraries. I'm continuing to use Go for my new company. In particular, running SQLAlchemy was a nightmare (see a reddit comment I made about it here [1]). I also wanted a more parallel language - I was tired of having to use queues to do parallel tasks. Go's static typing made everything easier, from auto-generating Swagger definitions to having usable code documentation via Godoc. Most of our runtime errors while using Python were due to type mismatches that could have been caught by a compiler. I'm unfamiliar with Python 3.6 static checking. I'll have to look into it. For the algorithmic scheduling - our algorithms in Julia predate the existence of a web app. We had customers using the Julia-based algorithm using the protocol "spreadsheets over email". The website came later, and we ended up switching the algorithms from Julia to Python after the web-app launch [2]. [1] https://www.reddit.com/r/Python/comments/5w3o6t/staffjoy_v1_has_been_open_sourced_if_you_want_to/de96ybk/?context=3 https://www.reddit.com/r/Python/comments/5w3o6t/staffjoy_v1_... [2] https://blog.staffjoy.com/retro-on-the-julia-programming-language-7655121ea341 https://blog.staffjoy.com/retro-on-the-julia-programming-lan...

dragonsh 10y ago
Thanks for the reply and insight. Why you had issues with magic in Python libraries, as I understand one of the main design goals of Python and its libraries is "Explicit is better then implicit" PEP-20 [1]. Indeed for our own project we chose python instead of ruby and ruby on rails just because of this PEP-20[1]. Your blog and moving to go alarmed us if there is anything wrong with python eco-system. We tried go and due to too much boilerplate code and quality of the libraries we dropped it and when it comes to interacting with database especially postgreSQL all the speed advantage of go is not very useful as most tasks were IO intensive. Also the database driver in golang for postgresql is not as mature as psycopg. For CPU intensive tasks we relied on scientific python which is written in C and it worked well with multiprocessing and asyncio in python. Also for the specific error you encountered in race conditions on update with multiple request. We solved it by using SQLAlchemy with_for_update()[2] which actually takes care at database level to make sure transactions are properly handled. As database we were using is postgreSQL, we wanted the database to handle ACID compliance, not implement in application code. [1]https://www.python.org/dev/peps/pep-0020/ https://www.python.org/dev/peps/pep-0020/ [2]http://docs.sqlalchemy.org/en/latest/core/selectable.html#sqlalchemy.sql.expression.Select.with_for_update http://docs.sqlalchemy.org/en/latest/core/selectable.html#sq...