4 ms·

When you have a relatively tiny budget, and an absolutely massive difficult system to build (requiring input from a wide range of specialists), there is no poss

When you have a relatively tiny budget, and an absolutely massive difficult system to build (requiring input from a wide range of specialists), there is no possible way to make significant progress without fostering a strong environment of community and collaboration. Integrating heterogenous components from a wide range of high-quality specialist communities (GAP, Pari, Maxima, etc.) as I did with Sage, brought many of these communities together, with them work toward a common goal. We had over 60 coding spring workshops in the last 10 years (http://wiki.sagemath.org/#Past_and_future_workshops http://wiki.sagemath.org/#Past_and_future_workshops). Also, we wrote an enormous amount of new code in Sage itself, likely more lines of code than all other math components put together.

fdej 12y ago
I somehow doubt the amount of code in Sage itself is close to the sum of its mathematical components, though Sage does contain a huge amount of new code (most importantly a lot of high level mathematical code not found anywhere else). According to openhub.net [https://www.openhub.net/p/sage/analyses/latest/languages_summary https://www.openhub.net/p/sage/analyses/latest/languages_sum...], Sage has 1.7 million lines of code. Maxima alone has 700K lines, Singular has 640K lines, SciPy has 650K lines, SymPy has 480K lines, etc.
- williamstein 12y ago
  We haven't done "lines of code" estimates for Sage in a while, so I'm not sure. Of course, lines of code being large isn't necessarily something to be proud of (though there's some meaning in lines of code that have come and gone overall). I'm very dubious about your lines-of-code estimates above. For example, you claim "Maxima alone has 700K lines", but I just downloaded Maxima from http://sourceforge.net/projects/maxima/files/ http://sourceforge.net/projects/maxima/files/, then ran sloccount on the src directory and I get 129,584 lines of code: /tmp/maxima-5.34.1$ sloccount src ... Total Physical Source Lines of Code (SLOC) = 129,584
  - fdej 12y ago
    Yeah, I don't know what's up with that. I got the number from openhub. But sloccount tells me that sage-6.2 only has 570K lines of code. And sloccount says that Singular has 280K lines of code.
danbruc 12y ago
I did obviously not consider other constraints like budget and availability of developers. What if these were no (major) issues? Do you think it would still be (by far) the best approach (in the case of Sage) or is the integration complex enough and forces enough compromises onto you that you would at least consider doing everything from scratch?
- williamstein 12y ago
  If I had resources similar to commercial math software companies (so I could hire hundreds of professional engineers), I would still do a mix of writing new code from scratch and using existing libraries. What would change is the mix. Also, I would identify components of Sage of questionable quality/design, and put substantial resources into improving them (thus giving back to their communities). For example, Jakob Kroeker has been frequently posting on the Sage development mailing list about how he was -- out of interest -- systematically auditing code in Singular, and he found some serious bugs. Unfortunately, he is about to leave mathematics (since he's not going to get supported doing auditing?), but if I had more resources, I could hire him fulltime to systematically auditing open source mathematical software. It's a waste that such people systematically slip through the cracks, people who are passionate about auditing math software source code. The minute they start doing some of the things we really, really need, they vanish due to lack of money. After watching this for over 10 years, I'm starting to get really annoyed.
  - asgard1024 12y ago
    I just read your blogpost about Sage mentioned above, and it made me a little sad (I actually expected it to be optimistic). I think it smells of burnout, and you should perhaps take a break. You certainly deserve it! It's not your fault that economic resources in human societies are hugely misallocated, and that social progress is always painfully slow.

fdej 12y ago

I somehow doubt the amount of code in Sage itself is close to the sum of its mathematical components, though Sage does contain a huge amount of new code (most importantly a lot of high level mathematical code not found anywhere else). According to openhub.net [https://www.openhub.net/p/sage/analyses/latest/languages_summary https://www.openhub.net/p/sage/analyses/latest/languages_sum...], Sage has 1.7 million lines of code. Maxima alone has 700K lines, Singular has 640K lines, SciPy has 650K lines, SymPy has 480K lines, etc.

williamstein 12y ago
We haven't done "lines of code" estimates for Sage in a while, so I'm not sure. Of course, lines of code being large isn't necessarily something to be proud of (though there's some meaning in lines of code that have come and gone overall). I'm very dubious about your lines-of-code estimates above. For example, you claim "Maxima alone has 700K lines", but I just downloaded Maxima from http://sourceforge.net/projects/maxima/files/ http://sourceforge.net/projects/maxima/files/, then ran sloccount on the src directory and I get 129,584 lines of code: /tmp/maxima-5.34.1$ sloccount src ... Total Physical Source Lines of Code (SLOC) = 129,584
- fdej 12y ago
  Yeah, I don't know what's up with that. I got the number from openhub. But sloccount tells me that sage-6.2 only has 570K lines of code. And sloccount says that Singular has 280K lines of code.

danbruc 12y ago

I did obviously not consider other constraints like budget and availability of developers. What if these were no (major) issues? Do you think it would still be (by far) the best approach (in the case of Sage) or is the integration complex enough and forces enough compromises onto you that you would at least consider doing everything from scratch?

williamstein 12y ago
If I had resources similar to commercial math software companies (so I could hire hundreds of professional engineers), I would still do a mix of writing new code from scratch and using existing libraries. What would change is the mix. Also, I would identify components of Sage of questionable quality/design, and put substantial resources into improving them (thus giving back to their communities). For example, Jakob Kroeker has been frequently posting on the Sage development mailing list about how he was -- out of interest -- systematically auditing code in Singular, and he found some serious bugs. Unfortunately, he is about to leave mathematics (since he's not going to get supported doing auditing?), but if I had more resources, I could hire him fulltime to systematically auditing open source mathematical software. It's a waste that such people systematically slip through the cracks, people who are passionate about auditing math software source code. The minute they start doing some of the things we really, really need, they vanish due to lack of money. After watching this for over 10 years, I'm starting to get really annoyed.
- asgard1024 12y ago
  I just read your blogpost about Sage mentioned above, and it made me a little sad (I actually expected it to be optimistic). I think it smells of burnout, and you should perhaps take a break. You certainly deserve it! It's not your fault that economic resources in human societies are hugely misallocated, and that social progress is always painfully slow.