10 ms·

Tackling bioinformatics as a webdev: why is it so hard? "I am a professional in Field X but I am doing work in Field Y despite having minimal skills or experie

Tackling bioinformatics as a webdev: why is it so hard? "I am a professional in Field X but I am doing work in Field Y despite having minimal skills or experience and I ran into loads of problems and issues that I don't know how to fix! Field Y is total bullshit and people need to make it easier so people with no skills or experience can do it! Wah wah!" ... or to be less facetious, modern web/frontend development is a genuine skill set that benefits from experience and knowledge. Should we expect anyone to just jump in cold with minimal experience and not find it difficult? E.g. in the same way would we expect people to start bioinformatics cold and be productive immediately? I'd doubt it. On the past web dev has been looked down upon as somehow "lesser" than "real" engineering, with sneering criticism from people more used to traditional development approaches (particularly criticism of JavaScript). It is just some web pages right? HTML and CSS? How hard can it be? Well guess what: it is a genuine skillset and people should not expect to just be able to jump into something without knowledge training or experience and be proficient at it, just like we'd not for brain surgery, car mechanics, concert piano, structural engineering, or indeed bioinformatics. </rant>

hmwhy 7y ago
I second this. I have some experience in both web development and scientific computing in various fields (not bioinformatics). Things felt just as hard at the beginning in all cases—I don't recall things being any better in scientific computing compared to web development. I agree that documentation is terrible most of the time, but that's not something limited to web development in my own experience. (On the contrary, check out the documentation of the website in question in this thread: https://news.ycombinator.com/item?id=23031408 https://news.ycombinator.com/item?id=23031408, it's absolutely fantastic). Some things become second nature when you have developed expertise in that field, and it's easy to criticise something that you have comparatively less knowledge in and pass it off as "too difficult", particularly when the web development part seems to be a means to an end (which seems to be the case here). Edit: missing conjunction, tidied up the last sentence for clarity.
- LifeIsBio 7y ago
  Oh, interesting. I just wrote a reply saying the exact opposite. I remember learning scientific computing as being much less frustrating. Sometimes you'd have some error spawning out of some ancient C/Fortran library. But that was rare for me. Even after a couple of years on the web, I still feel like I encounter issues where I just don't even know how to start.
- zhdc1 7y ago
  Same, and they're not. If anything, programming for academia is more of a mess than web development. Case in point - I spent the better part of a day rewriting what's essentially a parallelized for loop because the library I'm using in Julia to open HDF5 files has a well known memory leak. I have the impression that something similar in any popular web dev framework or library would have been ruthlessly purged a long time ago.
  - mattkrause 7y ago
    Bugs in libraries and code that overpromises are maddening (and too common), but I think it's different. You can get the data out of an HDF5 file with using HDF5 c = h5open(filename, "r") do file read(file, "data") end And you're done. You're not somehow locked into plotting data with the Plotly backend, but not GR. You know exactly where the data came and can trivially point it somewhere else with some 'magic' breaking.
    - zhdc1 7y ago
      HDF5.jl suffers from a memory leak when HDF5 files are opened in succession. Bug report: https://github.com/JuliaIO/HDF5.jl/issues/349 https://github.com/JuliaIO/HDF5.jl/issues/349
      - mattkrause 7y ago
        Right, but that's a bug. It's annoying it isn't fixed and working around it is undoubtedly annoying. This seems different from a system where there's lots of complexity in just choosing what frameworks to start with and if/how they play well together, and all of that rapidly changing over time. (I'd bet, for example, that many of those examples used to work).
throwaway713 7y ago
> It is a genuine skillset and people should not expect to just be able to jump into something qithout knowledge training or experience and be proficient at it Sure, but this also applies to understanding the U.S. tax code, which I think is a needlessly overcomplicated, poorly designed system that requires layers of inefficiencies to extract value from, much like web development.
- fastball 7y ago
  You can still make a webpage that is just pure, simple, structured HTML. In fact, the HTML5 standard arguably made it easier to write a simple HTML webpage. The complexity in webdev exists for good reasons much (most?) of the time. If you want interactivity, you need an actual programming language. If you're using an actual programming language, then there is a lot of tooling that can be used to make your life easier in the long run. Not intially, but definitely from a maintenance / bug-fixing / etc perspective. If you want to work on a large, complex project with a large team, this "complexity" is invaluable. That being said, JS as a language kinda sucks still, but "complexity" has been added to attempt to ameliorate that too, such as with TypeScript / CoffeeScript / etc.
  - iainmerrick 7y ago
    As I read the original post, Javascript the language isn’t the major problem. It’s things like configuration, certificates and (lack of) inter-operability of frameworks. I agree that this stuff is difficult in web development and I think it’s largely accidental complexity. (That is, the web is a distributed system and distributed programming is hard, sure. But the configuration hell you often run into is separate from that.)
    - luckylion 7y ago
      It's also a decentral system. Even for the points of interaction we have protocols that not everybody implements the same (often by mistake, sometimes by choice), but for internals of frameworks and tools, there's very little standardization even within one language, much less intra-language.
WilliamEdward 7y ago
Webdev is more like a trade than a science; you benefit more from remembering a large body of facts and details that comes with experience. There's no theory to learn, you just have to get practise in things like databases, browsers, HTTP, frameworks, etc. This is probably why the author is struggling. This is a real world industry tackling real world problems. It is not the neat and cut world of science research.
- LifeIsBio 7y ago
  > It is not the neat and cut world of science research. You had me up until this last sentence. Everything I've experienced in science research is the opposite of neat or clean cut. Actually, I think I do my best programming and debugging with a treat it like a science experiment.
  - WilliamEdward 7y ago
    I don't have any experience in scientific research, so that's a fair justification.
- SaxonRobber 7y ago
  Spoken like a bootcamp luddite
- seer 7y ago
  On the contrary, learning the fundamentals of CS has made me a much better dev, more so than any fact or tidbit. Its a testament to the awesomeness of the web however that so so many people even without any formal training can go about creating value. You _can_ be a webdev tradesman, fixing things till it works, kinda, or you can delve into the fundamentals and be much more productive. But you don’t have to and I think thats awesome. For example I still remember that conference that introduced Ruby on Rails to the world so many years ago. Most webdevs did not know there was an underlying theory and standard (REST) that can be used to make web apis bearable! After rails, we kinda figured huh, so this RFC for the web was sitting there all those years gathering dust and only weirdos and graybeards talked about it. But it was actually very very useful.
LifeIsBio 7y ago
You're absolutely right. Webdev is certainly its own fully fleshed out fields. Oddly, I think I can play off your facetious quote and turn it into a more analytical way of framing what the post was about. "I am a biochem major but I decided to work in bioinformatics/scientific-programming despite having minimal skills or experience and I ran into loads of problems and issues that I was able to steadily solve because the ecosystem felt sane, the tracebacks were clear, and the debugging usually had a clear path forward. Then I was a bioinformatics professional doing work in webdev [...] I ran into loads of problems [...] that felt consistently more opaque and magical than when I was learning scientific programming" I think that's the heart of it. The transition to the web feels so much more chaotic than the transition to bioinformatics.
fxtentacle 7y ago
There used to be a time when web frameworks strived for simplicity. I fully agree with the article's opinion that things have become needlessly complicated. The new best practice seems to be to install NodeJS, then use the npm repository manager to install bower, then use that to download your packages. And 5000 dependencies and 500 MB in downloads later, you have bootstrapped a Hello World template. Or name me one person who can, from memory, correctly configure the rails sprockets asset pipeline :p There's no denying that most of the tutorials and contemporary web development advice boils down to "follow these instructions carefully + magic happens"
- dalke 7y ago
  Fun bit of trivia: the first two major web libraries - cgi-lib.pl and CGI.pm - were written by bioinformaticians. Steven E. Brenner and Lincoln Stein, respectively.
  - davidjnelson 7y ago
    Cgi.pm was great, really nice little abstraction. Create react app is also a really nice abstraction, although I wouldn’t call it little :-)
- pixelbash 7y ago
  Is anyone still using bower? Also webpacker for rails is somehow both insanely complex and alright to use.
  - TeMPOraL 7y ago
    A perfect proof for GP's/TFA's point. To this I'll only add that "follow these instructions carefully + magic happens" works only for ~6 months after a tutorial is published, longer than that and some dependency or toolchain update somewhere is bound to break the instructions in a way incomprehensible to a novice.
    - fxtentacle 7y ago
simion314 7y ago
IMO because of the poor building blocks of web dev (we are trying to build applications with document markup) things that should be simple are too complicated. As an example say I give you an url that will respond with a json of an array of persons and you want to view this json in a table( just view some raw data, I do not want fancy themes or features). So if we ask web developers how to solve the above problem we get at least 25 different ways to sovle with with JS , and some other 25 ways if we use some backend PHP,Python, Java. If you ask some Qt/Flex developers you will get the same response, Drag and drop the DataGrid and set the data source to your data (and you get a ton of fancy features like adjustable columns, sorting,,,) and you don't have to install 300 dependencies.
- mattlondon 7y ago
  But Qt/Flex is just one way of building desktop UIs :) Comparing the entire gamut of web development against a single library seems unfair. What about asking the same question of .net developers, VB classic developers, golang developers, electron developers, python developers, Java developers, c++ developers, swift developers? They'd all do it differently than QT developers would because there are different GUI frameworks that need to be used. This problem is not unique to web dev.
  - simion314 7y ago
    I am not sure is true, For example the .Net Windows Forms have a DataGrid widget that works exactly the same , the only differences between competent GUI frameworks is that Widget names and functions might differ. so in summary in any GUI framerwork the reaponse is the same drag the DataGrid and connect it to the data(the names can differ), maybe some Delphi,GTK,JavaFX,WPF devs can confirm or disprove me.
    - mattlondon 7y ago
      I think you are ignoring a lot of the complexity and assuming it is trivial in desktop GUI development, but complex for web development. I do not think this is the case - e.g. just look at the two different options just for Java (Swing vs JavaFx): https://docs.oracle.com/javase/tutorial/uiswing/components/table.html https://docs.oracle.com/javase/tutorial/uiswing/components/t... https://docs.oracle.com/javafx/2/ui_controls/table-view.htm https://docs.oracle.com/javafx/2/ui_controls/table-view.htm This is not just dragging and dropping some common component - each thing is implemented differently, and so has a different API to control it (and require huge tutorials to work out how to use them) so everyone has to do it differently. They're not just different APIs to the same underlying OS-provided control. A Java Swing developer will not just be able to do the same thing they always do just in Qt, TK, Wx, WPF, WinForms, Gtk etc etc etc - it is still totally different. Sure the .net languages are all the same (i.e. same API ... but then do you use WinForms? WPF? Gtk#? others?), but there is more to GUI development than .net.
emsy 7y ago
This shouldn’t be the top comment. Yes, switching fields and expecting to just work professionally is naive and insulting towards the other profession. Has web dev a lot of legacy cruft and suffers from over-engineering? Absolutely. Even for seasoned programmers starting web dev is harder than it needs to be. Front end development has JS frameworks, CSS or compile-to-CSS languages, templating, web packing, JS transpilers, tree pruning, multiple package managers and what not. But in the end it’s just a bunch of text, images and buttons.
- goto11 7y ago
  But you don't need any of that. A static HTML page and a bit of CSS for styling and you have a web site. If you don't think package management or transpilers makes your particular project simpler, then don't use them. They are tools.
  - emsy 7y ago
    Yes, that’s correct. And also missing the point.
    - goto11 7y ago
      No, that is quite relevant to the point. Web development have become easier over time. For example styling is much easier with better and more consistent CSS support in browsers. But at the same time people are attempting much more complex stuff which requires complex tools. Perhaps there is a problem in the educational resources available if people think they need npm and typescript to build a simple website.
      - emsy 7y ago
        The point is that for someone starting out, web development looks complex. And it is. I know it because I teach this stuff. Ignoring 99% of the industry and use the bare minimum to write a web app is something an expert does. When you’re a beginner you lack the knowledge to recognize expertise. And when most tutorials suggest a heavy stack you‘ll likely stick with the majority. And even when you’re lucky and learn how to write simple web apps from scratch, CSS, HTML and JS are anything but sensible technologies, even with the latest additions that admittedly improved things a lot.
azangru 7y ago
100 percent this. And yet, even within the web development community, we hear a lot of complaints about how this or that isn't beginner-friendly; or how such-and-such is gatekeeping, and so on. This has been baffling to me. Is it the same in other disciplines? Are engineering, bioinformatics or medicine, for example, expected to be beginner-friendly?
mettamage 7y ago
IMO web dev isn't that hard, given one has a CS background. I came out of college, immediately started to teach a bootcamp (they had a lot of faith in me). What I learned about the difficulty of web dev: - Finding out the right curriculum by yourself is hard - Once you have the right curriculum, then you can learn it yourself provided one has the right background. In fact, they even can teach that curriculum to people that don't know how to program or do web dev yet When I was done teaching NodeJS/ReactJS, I went on to freelance for a company that had a microservice backend in Java. Since my CS degree was a Java school, I picked up Spring Boot quite quickly. I needed to build on top of code and not create it from scratch. That really helped, starting from scratch would've been more difficult and does require a lot more experience (so altering an open-source project might be a good way to practice). After 3 months the job was done and I left. Now, I know that I don't have a lot of experience, but I honestly haven't seen any hard challenges yet when it comes to web dev at the 3 year old startup level. There were no scalability or security challenges to take care of. The web dev rabbit hole can go deep (and that is hard), but in practice it mostly doesn't (unless you work at Google scale, I presume). Deployment with IBM and Kubernetes was just: read the docs and know how to debug and troubleshoot. It's not easy per se, but if you start with the right background (CS and a lot of programming courses in such a CS curriculum) and get a good coding school curriculum after that, then I would argue that web dev isn't that hard. There are enough startups where you'd be productive. Though, to be fair: if I couldn't use a debugger, I'd be totally lost. Using a debugger makes it not that hard. This is also why I had a harder time with learning Webpack. What I found way harder is wrapping your mind around concepts that seem completely alien. I have this still a bit with functional programming (not covered in my CS degree) or back in the day with databases or certain forms of math. Reading API documentation isn't part of it which is what I feel web dev is mostly nowadays. Even when you learn another programming language, it's all procedural/object oriented, so you just need to learn the quirks of that particular language. When I first started out I also had to deal with emotional fear of being inadequate. But after your fifth language of feeling that way, you know it's part of the process and you shouldn't take it to mean that you are inadequate.
JeremyNT 7y ago
I would suggest that it's really easy to envision some much less painful abstractions for web development. Sure, learning an entire field from first principals is going to be hard, and to do science you need to understand those principals. But the general concept of "make a dynamic web page" is not computer science! What most people (and the author of the blog post) need is a clean abstraction that allows them to glue a few APIs together with some custom forms.
- Taikonerd 7y ago
  That's the premise of low-code, right? "I don't need a fully general language with configurable everything; I just need a simple tool for this one common use case." I'm excited about what Dark (https://darklang.com/ https://darklang.com/) is doing with this idea. As you said, "a clean abstraction that allows them to glue a few APIs together with some custom forms."
rsynnott 7y ago
> ... or to be less facetious, modern web/frontend development is a genuine skill set that benefits from experience and knowledge. I mean, there's a strong argument that it has become too complex for no particularly good reason. It is _far_ more complex today than a decade or so ago, and the results in many cases are arguably worse for the user. If you compare the _average_ modern react-y website to a Ruby on Rails one with minimal or no Javascript from 2007 or so, the react-y one is far more difficult (and slower) to work on, but is it really delivering extra value to the end user? There are exceptions, of course, but I think for most CRUD web apps things have gone in a very bad direction.
- bitexploder 7y ago
  Funny thing is, those rails and simple Django apps are just as effective today as they were 10 years ago. In fact, not letting your app get bogged down in the mess that is modern front ends is likely a competitive advantage these days. Keep it simple. Django and Rails are the best they have ever been. Most software is about data, not front ends. It takes dedicated teams to tame “modern” front ends. Let the data show you the way and avoid front end stuff until you absolutely can’t. Maybe, after you need something to do with a lot of VC money or you can justify a certain UX in your app.
mattkrause 7y ago
> Should we expect anyone to just jump in cold with minimal experience and not find it difficult? My take on this is that the author had some (relevant) programming experience and thought it would transfer better, hence the frustration. That was my experience, as someone in the same boat (computational scientist trying to put together a webapp). I have a CS degree and my job involves a lot of diverse coding, from low-level stuff to interface with hardware to graphics/ML/data stuff, not to mention gluing together other people's code. I nevertheless found it very frustrating to get started with web dev. There seem to be a million slightly different ways to do things and people have strong opinions about which ones are best, so it's hard to settle on authoritative source, especially one that's up to date. Many tutorials bounce between basic web stuff (which I wanted) and basic programming (which makes me zone out). Plus, there's a lurking fear that I'm gonna screw something up in a way that exposes a bunch of data or costs me a fortune in hosting (etc) while trying to make something that I sketched out on three pieces of paper. This seems a little different from bioinformatics, where a lot of the complexity is baked into the subject matter. For background, textbooks certainly vary in how they present information, but they all pretty much agree: Molecular Biology of the Cell isn't going to tell you that DNA is where it's at, with RNA being a fad vs. The Cell claiming that DNA is outmoded and it's all RNA, baby!
- hysan 7y ago
  OP’s comment still seems pretty on point though even with your example. Taking the exampling to another field in programming that’s also diverse like gamedev and you run into a similar “too many choices” problem. The solution of course is to just pick a popular tutorial and start. You find and fill in the gaps along the way. Chasing for the elusive best practice is just setting yourself up for failure as a beginner.
downerending 7y ago
If you have any doubt that software takes real skill, wade into the muck that is software written by scientists. You will discover classes of errors at all levels that you didn't even think possible. Testing? What's that? Exponential run time? Does that matter? And why does my program give different answers now than it did last year? Those numbers worked for my paper--make the program give them again!
doteka 7y ago
I am a software engineer who works closely with a team of bioinformaticians. Your reply suggests you only have one side of the story. A bioinformatician is someone who has quite serious programming skills on top of their biology knowledge. They are frequently bottlenecked by compute and so tend to have a good understanding of data structures and algorithms. They usually know 3 or more languages and are no strangers to databases, version control, containers, et cetera. And yet I recognize the feeling from the article: they get stuck in the heap of complexity we (as an industry) have created around a bog standard web frontend. The majority of us are not doing rocket science - the applications I’m working on today do similar things to the ones I worked on five years ago. Yet somehow, the ‘how’ of it, the whole Rube Goldberg machine, tends to get more convoluted every passing year. I often get the feeling a nonzero percentage of it is introduced purely so web developers can feel they’re “real engineers” whatever that may be. There’s nothing to prove, now stop piling on junk and reinventing wheels.

hmwhy 7y ago

I second this. I have some experience in both web development and scientific computing in various fields (not bioinformatics). Things felt just as hard at the beginning in all cases—I don't recall things being any better in scientific computing compared to web development. I agree that documentation is terrible most of the time, but that's not something limited to web development in my own experience. (On the contrary, check out the documentation of the website in question in this thread: https://news.ycombinator.com/item?id=23031408 https://news.ycombinator.com/item?id=23031408, it's absolutely fantastic). Some things become second nature when you have developed expertise in that field, and it's easy to criticise something that you have comparatively less knowledge in and pass it off as "too difficult", particularly when the web development part seems to be a means to an end (which seems to be the case here). Edit: missing conjunction, tidied up the last sentence for clarity.

LifeIsBio 7y ago
Oh, interesting. I just wrote a reply saying the exact opposite. I remember learning scientific computing as being much less frustrating. Sometimes you'd have some error spawning out of some ancient C/Fortran library. But that was rare for me. Even after a couple of years on the web, I still feel like I encounter issues where I just don't even know how to start.
zhdc1 7y ago
Same, and they're not. If anything, programming for academia is more of a mess than web development. Case in point - I spent the better part of a day rewriting what's essentially a parallelized for loop because the library I'm using in Julia to open HDF5 files has a well known memory leak. I have the impression that something similar in any popular web dev framework or library would have been ruthlessly purged a long time ago.
- mattkrause 7y ago
  Bugs in libraries and code that overpromises are maddening (and too common), but I think it's different. You can get the data out of an HDF5 file with using HDF5 c = h5open(filename, "r") do file read(file, "data") end And you're done. You're not somehow locked into plotting data with the Plotly backend, but not GR. You know exactly where the data came and can trivially point it somewhere else with some 'magic' breaking.
  - zhdc1 7y ago
    HDF5.jl suffers from a memory leak when HDF5 files are opened in succession. Bug report: https://github.com/JuliaIO/HDF5.jl/issues/349 https://github.com/JuliaIO/HDF5.jl/issues/349
    - mattkrause 7y ago
      Right, but that's a bug. It's annoying it isn't fixed and working around it is undoubtedly annoying. This seems different from a system where there's lots of complexity in just choosing what frameworks to start with and if/how they play well together, and all of that rapidly changing over time. (I'd bet, for example, that many of those examples used to work).

throwaway713 7y ago

> It is a genuine skillset and people should not expect to just be able to jump into something qithout knowledge training or experience and be proficient at it Sure, but this also applies to understanding the U.S. tax code, which I think is a needlessly overcomplicated, poorly designed system that requires layers of inefficiencies to extract value from, much like web development.

fastball 7y ago
You can still make a webpage that is just pure, simple, structured HTML. In fact, the HTML5 standard arguably made it easier to write a simple HTML webpage. The complexity in webdev exists for good reasons much (most?) of the time. If you want interactivity, you need an actual programming language. If you're using an actual programming language, then there is a lot of tooling that can be used to make your life easier in the long run. Not intially, but definitely from a maintenance / bug-fixing / etc perspective. If you want to work on a large, complex project with a large team, this "complexity" is invaluable. That being said, JS as a language kinda sucks still, but "complexity" has been added to attempt to ameliorate that too, such as with TypeScript / CoffeeScript / etc.
- iainmerrick 7y ago
  As I read the original post, Javascript the language isn’t the major problem. It’s things like configuration, certificates and (lack of) inter-operability of frameworks. I agree that this stuff is difficult in web development and I think it’s largely accidental complexity. (That is, the web is a distributed system and distributed programming is hard, sure. But the configuration hell you often run into is separate from that.)
  - luckylion 7y ago
    It's also a decentral system. Even for the points of interaction we have protocols that not everybody implements the same (often by mistake, sometimes by choice), but for internals of frameworks and tools, there's very little standardization even within one language, much less intra-language.

WilliamEdward 7y ago

Webdev is more like a trade than a science; you benefit more from remembering a large body of facts and details that comes with experience. There's no theory to learn, you just have to get practise in things like databases, browsers, HTTP, frameworks, etc. This is probably why the author is struggling. This is a real world industry tackling real world problems. It is not the neat and cut world of science research.

LifeIsBio 7y ago
> It is not the neat and cut world of science research. You had me up until this last sentence. Everything I've experienced in science research is the opposite of neat or clean cut. Actually, I think I do my best programming and debugging with a treat it like a science experiment.
- WilliamEdward 7y ago
  I don't have any experience in scientific research, so that's a fair justification.
SaxonRobber 7y ago
Spoken like a bootcamp luddite
seer 7y ago
On the contrary, learning the fundamentals of CS has made me a much better dev, more so than any fact or tidbit. Its a testament to the awesomeness of the web however that so so many people even without any formal training can go about creating value. You _can_ be a webdev tradesman, fixing things till it works, kinda, or you can delve into the fundamentals and be much more productive. But you don’t have to and I think thats awesome. For example I still remember that conference that introduced Ruby on Rails to the world so many years ago. Most webdevs did not know there was an underlying theory and standard (REST) that can be used to make web apis bearable! After rails, we kinda figured huh, so this RFC for the web was sitting there all those years gathering dust and only weirdos and graybeards talked about it. But it was actually very very useful.

LifeIsBio 7y ago

You're absolutely right. Webdev is certainly its own fully fleshed out fields. Oddly, I think I can play off your facetious quote and turn it into a more analytical way of framing what the post was about. "I am a biochem major but I decided to work in bioinformatics/scientific-programming despite having minimal skills or experience and I ran into loads of problems and issues that I was able to steadily solve because the ecosystem felt sane, the tracebacks were clear, and the debugging usually had a clear path forward. Then I was a bioinformatics professional doing work in webdev [...] I ran into loads of problems [...] that felt consistently more opaque and magical than when I was learning scientific programming" I think that's the heart of it. The transition to the web feels so much more chaotic than the transition to bioinformatics.

fxtentacle 7y ago

There used to be a time when web frameworks strived for simplicity. I fully agree with the article's opinion that things have become needlessly complicated. The new best practice seems to be to install NodeJS, then use the npm repository manager to install bower, then use that to download your packages. And 5000 dependencies and 500 MB in downloads later, you have bootstrapped a Hello World template. Or name me one person who can, from memory, correctly configure the rails sprockets asset pipeline :p There's no denying that most of the tutorials and contemporary web development advice boils down to "follow these instructions carefully + magic happens"

dalke 7y ago
Fun bit of trivia: the first two major web libraries - cgi-lib.pl and CGI.pm - were written by bioinformaticians. Steven E. Brenner and Lincoln Stein, respectively.
- davidjnelson 7y ago
  Cgi.pm was great, really nice little abstraction. Create react app is also a really nice abstraction, although I wouldn’t call it little :-)
pixelbash 7y ago
Is anyone still using bower? Also webpacker for rails is somehow both insanely complex and alright to use.
- TeMPOraL 7y ago
  A perfect proof for GP's/TFA's point. To this I'll only add that "follow these instructions carefully + magic happens" works only for ~6 months after a tutorial is published, longer than that and some dependency or toolchain update somewhere is bound to break the instructions in a way incomprehensible to a novice.
  - fxtentacle 7y ago

simion314 7y ago

IMO because of the poor building blocks of web dev (we are trying to build applications with document markup) things that should be simple are too complicated. As an example say I give you an url that will respond with a json of an array of persons and you want to view this json in a table( just view some raw data, I do not want fancy themes or features). So if we ask web developers how to solve the above problem we get at least 25 different ways to sovle with with JS , and some other 25 ways if we use some backend PHP,Python, Java. If you ask some Qt/Flex developers you will get the same response, Drag and drop the DataGrid and set the data source to your data (and you get a ton of fancy features like adjustable columns, sorting,,,) and you don't have to install 300 dependencies.

mattlondon 7y ago
But Qt/Flex is just one way of building desktop UIs :) Comparing the entire gamut of web development against a single library seems unfair. What about asking the same question of .net developers, VB classic developers, golang developers, electron developers, python developers, Java developers, c++ developers, swift developers? They'd all do it differently than QT developers would because there are different GUI frameworks that need to be used. This problem is not unique to web dev.
- simion314 7y ago
  I am not sure is true, For example the .Net Windows Forms have a DataGrid widget that works exactly the same , the only differences between competent GUI frameworks is that Widget names and functions might differ. so in summary in any GUI framerwork the reaponse is the same drag the DataGrid and connect it to the data(the names can differ), maybe some Delphi,GTK,JavaFX,WPF devs can confirm or disprove me.
  - mattlondon 7y ago
    I think you are ignoring a lot of the complexity and assuming it is trivial in desktop GUI development, but complex for web development. I do not think this is the case - e.g. just look at the two different options just for Java (Swing vs JavaFx): https://docs.oracle.com/javase/tutorial/uiswing/components/table.html https://docs.oracle.com/javase/tutorial/uiswing/components/t... https://docs.oracle.com/javafx/2/ui_controls/table-view.htm https://docs.oracle.com/javafx/2/ui_controls/table-view.htm This is not just dragging and dropping some common component - each thing is implemented differently, and so has a different API to control it (and require huge tutorials to work out how to use them) so everyone has to do it differently. They're not just different APIs to the same underlying OS-provided control. A Java Swing developer will not just be able to do the same thing they always do just in Qt, TK, Wx, WPF, WinForms, Gtk etc etc etc - it is still totally different. Sure the .net languages are all the same (i.e. same API ... but then do you use WinForms? WPF? Gtk#? others?), but there is more to GUI development than .net.

emsy 7y ago

This shouldn’t be the top comment. Yes, switching fields and expecting to just work professionally is naive and insulting towards the other profession. Has web dev a lot of legacy cruft and suffers from over-engineering? Absolutely. Even for seasoned programmers starting web dev is harder than it needs to be. Front end development has JS frameworks, CSS or compile-to-CSS languages, templating, web packing, JS transpilers, tree pruning, multiple package managers and what not. But in the end it’s just a bunch of text, images and buttons.

goto11 7y ago
But you don't need any of that. A static HTML page and a bit of CSS for styling and you have a web site. If you don't think package management or transpilers makes your particular project simpler, then don't use them. They are tools.
- emsy 7y ago
  Yes, that’s correct. And also missing the point.
  - goto11 7y ago
    No, that is quite relevant to the point. Web development have become easier over time. For example styling is much easier with better and more consistent CSS support in browsers. But at the same time people are attempting much more complex stuff which requires complex tools. Perhaps there is a problem in the educational resources available if people think they need npm and typescript to build a simple website.
    - emsy 7y ago
      The point is that for someone starting out, web development looks complex. And it is. I know it because I teach this stuff. Ignoring 99% of the industry and use the bare minimum to write a web app is something an expert does. When you’re a beginner you lack the knowledge to recognize expertise. And when most tutorials suggest a heavy stack you‘ll likely stick with the majority. And even when you’re lucky and learn how to write simple web apps from scratch, CSS, HTML and JS are anything but sensible technologies, even with the latest additions that admittedly improved things a lot.

azangru 7y ago

100 percent this. And yet, even within the web development community, we hear a lot of complaints about how this or that isn't beginner-friendly; or how such-and-such is gatekeeping, and so on. This has been baffling to me. Is it the same in other disciplines? Are engineering, bioinformatics or medicine, for example, expected to be beginner-friendly?

mettamage 7y ago

IMO web dev isn't that hard, given one has a CS background. I came out of college, immediately started to teach a bootcamp (they had a lot of faith in me). What I learned about the difficulty of web dev: - Finding out the right curriculum by yourself is hard - Once you have the right curriculum, then you can learn it yourself provided one has the right background. In fact, they even can teach that curriculum to people that don't know how to program or do web dev yet When I was done teaching NodeJS/ReactJS, I went on to freelance for a company that had a microservice backend in Java. Since my CS degree was a Java school, I picked up Spring Boot quite quickly. I needed to build on top of code and not create it from scratch. That really helped, starting from scratch would've been more difficult and does require a lot more experience (so altering an open-source project might be a good way to practice). After 3 months the job was done and I left. Now, I know that I don't have a lot of experience, but I honestly haven't seen any hard challenges yet when it comes to web dev at the 3 year old startup level. There were no scalability or security challenges to take care of. The web dev rabbit hole can go deep (and that is hard), but in practice it mostly doesn't (unless you work at Google scale, I presume). Deployment with IBM and Kubernetes was just: read the docs and know how to debug and troubleshoot. It's not easy per se, but if you start with the right background (CS and a lot of programming courses in such a CS curriculum) and get a good coding school curriculum after that, then I would argue that web dev isn't that hard. There are enough startups where you'd be productive. Though, to be fair: if I couldn't use a debugger, I'd be totally lost. Using a debugger makes it not that hard. This is also why I had a harder time with learning Webpack. What I found way harder is wrapping your mind around concepts that seem completely alien. I have this still a bit with functional programming (not covered in my CS degree) or back in the day with databases or certain forms of math. Reading API documentation isn't part of it which is what I feel web dev is mostly nowadays. Even when you learn another programming language, it's all procedural/object oriented, so you just need to learn the quirks of that particular language. When I first started out I also had to deal with emotional fear of being inadequate. But after your fifth language of feeling that way, you know it's part of the process and you shouldn't take it to mean that you are inadequate.

JeremyNT 7y ago

I would suggest that it's really easy to envision some much less painful abstractions for web development. Sure, learning an entire field from first principals is going to be hard, and to do science you need to understand those principals. But the general concept of "make a dynamic web page" is not computer science! What most people (and the author of the blog post) need is a clean abstraction that allows them to glue a few APIs together with some custom forms.

Taikonerd 7y ago
That's the premise of low-code, right? "I don't need a fully general language with configurable everything; I just need a simple tool for this one common use case." I'm excited about what Dark (https://darklang.com/ https://darklang.com/) is doing with this idea. As you said, "a clean abstraction that allows them to glue a few APIs together with some custom forms."

rsynnott 7y ago

> ... or to be less facetious, modern web/frontend development is a genuine skill set that benefits from experience and knowledge. I mean, there's a strong argument that it has become too complex for no particularly good reason. It is _far_ more complex today than a decade or so ago, and the results in many cases are arguably worse for the user. If you compare the _average_ modern react-y website to a Ruby on Rails one with minimal or no Javascript from 2007 or so, the react-y one is far more difficult (and slower) to work on, but is it really delivering extra value to the end user? There are exceptions, of course, but I think for most CRUD web apps things have gone in a very bad direction.

bitexploder 7y ago
Funny thing is, those rails and simple Django apps are just as effective today as they were 10 years ago. In fact, not letting your app get bogged down in the mess that is modern front ends is likely a competitive advantage these days. Keep it simple. Django and Rails are the best they have ever been. Most software is about data, not front ends. It takes dedicated teams to tame “modern” front ends. Let the data show you the way and avoid front end stuff until you absolutely can’t. Maybe, after you need something to do with a lot of VC money or you can justify a certain UX in your app.

mattkrause 7y ago

> Should we expect anyone to just jump in cold with minimal experience and not find it difficult? My take on this is that the author had some (relevant) programming experience and thought it would transfer better, hence the frustration. That was my experience, as someone in the same boat (computational scientist trying to put together a webapp). I have a CS degree and my job involves a lot of diverse coding, from low-level stuff to interface with hardware to graphics/ML/data stuff, not to mention gluing together other people's code. I nevertheless found it very frustrating to get started with web dev. There seem to be a million slightly different ways to do things and people have strong opinions about which ones are best, so it's hard to settle on authoritative source, especially one that's up to date. Many tutorials bounce between basic web stuff (which I wanted) and basic programming (which makes me zone out). Plus, there's a lurking fear that I'm gonna screw something up in a way that exposes a bunch of data or costs me a fortune in hosting (etc) while trying to make something that I sketched out on three pieces of paper. This seems a little different from bioinformatics, where a lot of the complexity is baked into the subject matter. For background, textbooks certainly vary in how they present information, but they all pretty much agree: Molecular Biology of the Cell isn't going to tell you that DNA is where it's at, with RNA being a fad vs. The Cell claiming that DNA is outmoded and it's all RNA, baby!

hysan 7y ago
OP’s comment still seems pretty on point though even with your example. Taking the exampling to another field in programming that’s also diverse like gamedev and you run into a similar “too many choices” problem. The solution of course is to just pick a popular tutorial and start. You find and fill in the gaps along the way. Chasing for the elusive best practice is just setting yourself up for failure as a beginner.

downerending 7y ago

If you have any doubt that software takes real skill, wade into the muck that is software written by scientists. You will discover classes of errors at all levels that you didn't even think possible. Testing? What's that? Exponential run time? Does that matter? And why does my program give different answers now than it did last year? Those numbers worked for my paper--make the program give them again!

doteka 7y ago

I am a software engineer who works closely with a team of bioinformaticians. Your reply suggests you only have one side of the story. A bioinformatician is someone who has quite serious programming skills on top of their biology knowledge. They are frequently bottlenecked by compute and so tend to have a good understanding of data structures and algorithms. They usually know 3 or more languages and are no strangers to databases, version control, containers, et cetera. And yet I recognize the feeling from the article: they get stuck in the heap of complexity we (as an industry) have created around a bog standard web frontend. The majority of us are not doing rocket science - the applications I’m working on today do similar things to the ones I worked on five years ago. Yet somehow, the ‘how’ of it, the whole Rube Goldberg machine, tends to get more convoluted every passing year. I often get the feeling a nonzero percentage of it is introduced purely so web developers can feel they’re “real engineers” whatever that may be. There’s nothing to prove, now stop piling on junk and reinventing wheels.