5 ms·

Math is totally unnecessary for everyday software development. It is necessary for interesting software development though. Someone has to build the core that

by 10098 13y ago

Math is totally unnecessary for everyday software development. It is necessary for interesting software development though. Someone has to build the core that all the other applications rely on, and these engineers actually need to know all that nerd stuff.

upquark 13y ago
>Math is totally unnecessary for everyday software development. You are writing the simplest possible CRUD app for storing / retrieving some client records. You have a database of these records on the back-end, and users can add/edit/remove these records. Now, your boss asks you to add a feature of bulk insertion of new client records from some other legacy data source, with proper de-duplication. There are no unique ids across the legacy and new data stores. I hope this qualifies as "everyday software development". Is math totally unnecessary for solving this problem?
- mamcx 13y ago
  Of curse, is unnecesary (ie: the lack of have a math solution is not make this task hard/impossible to do). If a person know database/sql, then "math" is not necessary here. Or what exactly are you talking about?
  - m0nastic 13y ago
    > If a person know database/sql, then "math" is not necessary here SQL? Like in relational databases? Like in relational algebra? I think this demonstrates the point someone made upthread that people don't consider a lot of math to be math.
  - deleted 13y ago
    [deleted]
  - deleted 13y ago
    [deleted]
  - fbomb 13y ago
    The irony here is that you mention SQL (the language of Relational Calculus) and in the same sentence claim that math is unnecessary.
    - 10098 13y ago
      but you see, my point is that the actual hard problem of executing a SQL query has been solved by those who designed the DBMS. our task boils down to writing SQL, which is way easier than designing the machinery that implements relational algebra. EDIT: not to mention that you don't even have to understand relational algebra to write SQL. There are HORDES of programmers who don't know what it is yet they write SQL.
      - fbomb 13y ago
        Relational Algebra is about executing the query. Relational Calculus is about formulating the query in the first place. Unless you copy-paste all your SQL code, you wind up using Relational Calculus. I would agree that most developers don't have a formal understanding of it (I sure don't) or even an awareness of it. They may not think of it as doing math. They may be thinking "how do I use a select with a join to get this particular set of data". But it's still math.
    - mamcx 13y ago
      Yeah, I know from where come the relational databases. Still, claim that using SQL is using math is like say using a car is using math. Like using python is using assembler.. and at the same time is not. This is kinda zend: If a developer don't think about math....
  - upquark 13y ago
    Nope, there's plenty of math and stats involved in solving this problem correctly. The amount of math you would apply here depends on your familiarity with the problem domain (this is where CS education comes into play), and your willingness and ability to work with higher level math and stats (this is where math proves necessary even in mundane software engineering tasks). For the record, even basic relational algebra can be seen as part of applied math, but the rabbit hole goes way deeper. The best solutions for de-duplication of records without direct clues like unique IDs involve pretty cool applications of probability theory and machine learning, you'd find things like the Expectation-Maximization algorithm as prerequisites for understanding the current research. The foundational solution in this space is Fellegi-Sunter record linkage [1]. More advanced solutions include things like hierarchical Bayesian models [2]. If you dig deeper, especially if you are dealing with large datasets and basic techniques turn out to be too slow, you'd look into locality sensitive hashing and approximate nearest neighbors, which is a very mathematical subject, see for example [3]. In other words, math is encountered in even the most mundane software engineering problems, and you have to know where to look and what to look for. [1] http://www.purdue.edu/discoverypark/vaccine/assets/pdfs/publications/pdf/Duplicate%20Record.pdf http://www.purdue.edu/discoverypark/vaccine/assets/pdfs/publ... [2] http://arxiv.org/pdf/1207.4180.pdf http://arxiv.org/pdf/1207.4180.pdf [3] http://www.cs.princeton.edu/courses/archive/spring05/cos598E/bib/p253-datar.pdf http://www.cs.princeton.edu/courses/archive/spring05/cos598E...
    - mamcx 13y ago
      I do this sort of task routinely. Can you point to a specific dataset where the regular DBAs skills falls apart? Because just see a lot of formulas without the context is part of the problem with math education (ie: show the solution and the claims it solve a contrived problem.. but where is the actual problem?)
      - upquark 13y ago
        The actual problem I described above (bulk de-duplication of records without any direct clues, such as unique ids, to decide whether two records are duplicates) falls under this category. The first paper I provided is a survey of commonly used methods in this space, it's the exact opposite of a lot of formulas without the context.

by 10098 13y ago

upquark 13y ago

>Math is totally unnecessary for everyday software development. You are writing the simplest possible CRUD app for storing / retrieving some client records. You have a database of these records on the back-end, and users can add/edit/remove these records. Now, your boss asks you to add a feature of bulk insertion of new client records from some other legacy data source, with proper de-duplication. There are no unique ids across the legacy and new data stores. I hope this qualifies as "everyday software development". Is math totally unnecessary for solving this problem?

mamcx 13y ago
Of curse, is unnecesary (ie: the lack of have a math solution is not make this task hard/impossible to do). If a person know database/sql, then "math" is not necessary here. Or what exactly are you talking about?
- m0nastic 13y ago
  > If a person know database/sql, then "math" is not necessary here SQL? Like in relational databases? Like in relational algebra? I think this demonstrates the point someone made upthread that people don't consider a lot of math to be math.
- deleted 13y ago
  [deleted]
- deleted 13y ago
  [deleted]
- fbomb 13y ago
  The irony here is that you mention SQL (the language of Relational Calculus) and in the same sentence claim that math is unnecessary.
  - 10098 13y ago
    but you see, my point is that the actual hard problem of executing a SQL query has been solved by those who designed the DBMS. our task boils down to writing SQL, which is way easier than designing the machinery that implements relational algebra. EDIT: not to mention that you don't even have to understand relational algebra to write SQL. There are HORDES of programmers who don't know what it is yet they write SQL.
    - fbomb 13y ago
      Relational Algebra is about executing the query. Relational Calculus is about formulating the query in the first place. Unless you copy-paste all your SQL code, you wind up using Relational Calculus. I would agree that most developers don't have a formal understanding of it (I sure don't) or even an awareness of it. They may not think of it as doing math. They may be thinking "how do I use a select with a join to get this particular set of data". But it's still math.
  - mamcx 13y ago
    Yeah, I know from where come the relational databases. Still, claim that using SQL is using math is like say using a car is using math. Like using python is using assembler.. and at the same time is not. This is kinda zend: If a developer don't think about math....
- upquark 13y ago
  Nope, there's plenty of math and stats involved in solving this problem correctly. The amount of math you would apply here depends on your familiarity with the problem domain (this is where CS education comes into play), and your willingness and ability to work with higher level math and stats (this is where math proves necessary even in mundane software engineering tasks). For the record, even basic relational algebra can be seen as part of applied math, but the rabbit hole goes way deeper. The best solutions for de-duplication of records without direct clues like unique IDs involve pretty cool applications of probability theory and machine learning, you'd find things like the Expectation-Maximization algorithm as prerequisites for understanding the current research. The foundational solution in this space is Fellegi-Sunter record linkage [1]. More advanced solutions include things like hierarchical Bayesian models [2]. If you dig deeper, especially if you are dealing with large datasets and basic techniques turn out to be too slow, you'd look into locality sensitive hashing and approximate nearest neighbors, which is a very mathematical subject, see for example [3]. In other words, math is encountered in even the most mundane software engineering problems, and you have to know where to look and what to look for. [1] http://www.purdue.edu/discoverypark/vaccine/assets/pdfs/publications/pdf/Duplicate%20Record.pdf http://www.purdue.edu/discoverypark/vaccine/assets/pdfs/publ... [2] http://arxiv.org/pdf/1207.4180.pdf http://arxiv.org/pdf/1207.4180.pdf [3] http://www.cs.princeton.edu/courses/archive/spring05/cos598E/bib/p253-datar.pdf http://www.cs.princeton.edu/courses/archive/spring05/cos598E...
  - mamcx 13y ago
    I do this sort of task routinely. Can you point to a specific dataset where the regular DBAs skills falls apart? Because just see a lot of formulas without the context is part of the problem with math education (ie: show the solution and the claims it solve a contrived problem.. but where is the actual problem?)
    - upquark 13y ago
      The actual problem I described above (bulk de-duplication of records without any direct clues, such as unique ids, to decide whether two records are duplicates) falls under this category. The first paper I provided is a survey of commonly used methods in this space, it's the exact opposite of a lot of formulas without the context.