4 ms·

This is 2015, 2MB files shouldn't be news worthy for a text editor. We seem to be bloating software faster than hardware improvements can offset it.

mseebach 11y ago
This has nothing to do with bloat. While unimpressive not to be able to open files larger than 2MB, it's easy to see how and why such a limit came about, and as Atom seems to be styled as a text editor for programming, not really unreasonable, as very few human-created text documents get anywhere near 2MB.
- nightski 11y ago
  Sorry, it's not obvious at all how the limit came about. That is pretty terrible.
  - manigandham 11y ago
    Atom is a desktop/native wrapper around browser-like internals. The stuff inside is all just HTML/javascript/css running on node. There are some neat advantages, as well as just seeing how far a web interface for a desktop app can go. The limit came about because trying to open a 100MB file as an html page (even without all the extra styling and interactive options) will break any browser. It was just a hard problem to solve, having an HTML-based editor deal with so much text at once.
    - rplnt 11y ago
      > trying to open a 100MB file as an html page (even without all the extra styling and interactive options) will break any browser. I'd love to test that. Anyone willing to privide an example file? HTML with css perhaps, no js. Should be pretty small when zipped.
      - icebraining 11y ago
        Assuming you don't need anything complex, it's pretty easy to generate it. Here's a 100MB HTML file (compressed to 300KB): https://curl.io/get/yt7zbmfq/0eb1564f570236acf1d9f304855a7c79d4346fa9 https://curl.io/get/yt7zbmfq/0eb1564f570236acf1d9f304855a7c7... As a side note, VIM opens it in an instant /troll
    - userbinator 11y ago
      The limit came about because trying to open a 100MB file as an html page (even without all the extra styling and interactive options) will break any browser. I remember opening HTML files in that size range (they were logs) in IE6 around a decade or more ago, on a machine with <1GB of RAM, and it handled them just fine - there was a noticeable delay, but it wasn't unusable. Even searching for strings worked.
  - mseebach 11y ago
    It is obvious to me: I think they chose to load, parse, prettify, colour etc. and keep the entire file into an in-memory data structure, which is easy to reason about, and totally reasonable for "human sized" data. However, for arbitrary sized files, you need a mechanism for leaving most of the file in a raw state - that's harder (but not impossible, obviously). So, it makes sense to me that they would assume only "human sized" data (and expect users to use another tool for looking at large files, "do one thing well" and all that) and opt for a slightly simpler programming model. Apparently, they decided that the assumption was wrong, and so changed the model.
- dagw 11y ago
  very few human-created text documents get anywhere near 2MB. True, but as a programmer i very often generate and need to view files greater than 2MB.
  - errtnsd 11y ago
    This. Not to mention log files which commonly consume megabytes.
    - bradleyland 11y ago
      Nitpick inbound. You shouldn't open log files with an editor unless your intent is to edit the file. There are a whole host of tools (grep, less, awk, sed, etc) useful for parsing and exploring log files.
  - mseebach 11y ago
    You also very often do X, but that doesn't mean your programming text editor is the correct tool for doing X. Insisting on all tools having an X because X is a useful thing is how you get the very bloat the GP is talking about.
    - lolhackernews 11y ago
      On the other hand, virtually every other editor is the correct tool to do that task, yet this one is not. Are we supposed to accept that?
- andrewchambers 11y ago
  2MB is 0.025 percent of my ram, I shudder to think how much ram is actually being used by atom... At least the html looks pretty.
- userbinator 11y ago
  The biggest problem I see is that this is an artificial, arbitrary hard-coded limit: https://github.com/atom/atom/commit/ec65a71d6dc8c3c44cb1b77a6cff09e9da39ccac https://github.com/atom/atom/commit/ec65a71d6dc8c3c44cb1b77a... If I have 64GB of RAM, I expect my applications to be able to use all of it when asked to, and not just give up completely. It's fine to slow down, and giving a warning "Opening large files may be slow. Do you want to continue?" would be nice, but working with a 2097151-byte file and giving up if it's 2097152 bytes is really unpleasant.
zamalek 11y ago
Interestingly, Visual Studio disables all editor services (cold folding, intellisense, etc.) except syntax highlighting on XML files bigger than 10MB. Code editors are surprisingly complex, it's not about simply putting lines on the screen. A good exercise in understanding the complexity is writing a simple syntax highlighting code viewer with native controls (not HTML) and see how much memory/CPU cycles your implementation uses for a 2MB file. Tons of optimization needs to happen - for example, as far as I know very little information about lines that aren't visible is kept in memory in Visual Studio - so called "virtual lines." I'm sure other more mature code editors have this type of optimization as well. From a native perspective this type of optimization isn't a complete nightmare to implement (although it's hard if you consider code folding), however, imagine trying to do it in HTML. The browser is still going too do all the reflow etc. for the lines that aren't even visible, eating up CPU cycles and memory. Even merely syntax highlighting is a problem. If you make one edit the entire file might be re-lexed: once again you need a unique solution for editors ("progressive lexing").
- pavlov 11y ago
  imagine trying to do it in HTML The question on my mind is, why do it in HTML? Especially if you're going to wrap that browser in something that approximates a desktop app anyway. Imagine if designers tried to do everything in PowerPoint for some unknown reason. "3D animation? Yeah, we can do it in PowerPoint with this incredible hack that makes slides sort of like faces on a mesh..." That's how it is with HTML today.
  - linker3000 11y ago
    "Imagine if designers tried to do everything in PowerPoint for some unknown reason." A fair percentage of the employees where I work seem to have reached that point already - reports, training material, data for general discussion, GANTT charts with individual symbols that I guess must be manually moved every week. Gah!
  - egeozcan 11y ago
    Because of extensibility concerns and the sheer number of JavaScript developers? I'd agree if you say it's not a good compromise, but it is one nevertheless.
deleted 11y ago
[deleted]

mseebach 11y ago

This has nothing to do with bloat. While unimpressive not to be able to open files larger than 2MB, it's easy to see how and why such a limit came about, and as Atom seems to be styled as a text editor for programming, not really unreasonable, as very few human-created text documents get anywhere near 2MB.

nightski 11y ago
Sorry, it's not obvious at all how the limit came about. That is pretty terrible.
- manigandham 11y ago
  Atom is a desktop/native wrapper around browser-like internals. The stuff inside is all just HTML/javascript/css running on node. There are some neat advantages, as well as just seeing how far a web interface for a desktop app can go. The limit came about because trying to open a 100MB file as an html page (even without all the extra styling and interactive options) will break any browser. It was just a hard problem to solve, having an HTML-based editor deal with so much text at once.
  - rplnt 11y ago
    > trying to open a 100MB file as an html page (even without all the extra styling and interactive options) will break any browser. I'd love to test that. Anyone willing to privide an example file? HTML with css perhaps, no js. Should be pretty small when zipped.
    - icebraining 11y ago
      Assuming you don't need anything complex, it's pretty easy to generate it. Here's a 100MB HTML file (compressed to 300KB): https://curl.io/get/yt7zbmfq/0eb1564f570236acf1d9f304855a7c79d4346fa9 https://curl.io/get/yt7zbmfq/0eb1564f570236acf1d9f304855a7c7... As a side note, VIM opens it in an instant /troll
  - userbinator 11y ago
    The limit came about because trying to open a 100MB file as an html page (even without all the extra styling and interactive options) will break any browser. I remember opening HTML files in that size range (they were logs) in IE6 around a decade or more ago, on a machine with <1GB of RAM, and it handled them just fine - there was a noticeable delay, but it wasn't unusable. Even searching for strings worked.
- mseebach 11y ago
  It is obvious to me: I think they chose to load, parse, prettify, colour etc. and keep the entire file into an in-memory data structure, which is easy to reason about, and totally reasonable for "human sized" data. However, for arbitrary sized files, you need a mechanism for leaving most of the file in a raw state - that's harder (but not impossible, obviously). So, it makes sense to me that they would assume only "human sized" data (and expect users to use another tool for looking at large files, "do one thing well" and all that) and opt for a slightly simpler programming model. Apparently, they decided that the assumption was wrong, and so changed the model.
dagw 11y ago
very few human-created text documents get anywhere near 2MB. True, but as a programmer i very often generate and need to view files greater than 2MB.
- errtnsd 11y ago
  This. Not to mention log files which commonly consume megabytes.
  - bradleyland 11y ago
    Nitpick inbound. You shouldn't open log files with an editor unless your intent is to edit the file. There are a whole host of tools (grep, less, awk, sed, etc) useful for parsing and exploring log files.
- mseebach 11y ago
  You also very often do X, but that doesn't mean your programming text editor is the correct tool for doing X. Insisting on all tools having an X because X is a useful thing is how you get the very bloat the GP is talking about.
  - lolhackernews 11y ago
    On the other hand, virtually every other editor is the correct tool to do that task, yet this one is not. Are we supposed to accept that?
andrewchambers 11y ago
2MB is 0.025 percent of my ram, I shudder to think how much ram is actually being used by atom... At least the html looks pretty.
userbinator 11y ago
The biggest problem I see is that this is an artificial, arbitrary hard-coded limit: https://github.com/atom/atom/commit/ec65a71d6dc8c3c44cb1b77a6cff09e9da39ccac https://github.com/atom/atom/commit/ec65a71d6dc8c3c44cb1b77a... If I have 64GB of RAM, I expect my applications to be able to use all of it when asked to, and not just give up completely. It's fine to slow down, and giving a warning "Opening large files may be slow. Do you want to continue?" would be nice, but working with a 2097151-byte file and giving up if it's 2097152 bytes is really unpleasant.

zamalek 11y ago

Interestingly, Visual Studio disables all editor services (cold folding, intellisense, etc.) except syntax highlighting on XML files bigger than 10MB. Code editors are surprisingly complex, it's not about simply putting lines on the screen. A good exercise in understanding the complexity is writing a simple syntax highlighting code viewer with native controls (not HTML) and see how much memory/CPU cycles your implementation uses for a 2MB file. Tons of optimization needs to happen - for example, as far as I know very little information about lines that aren't visible is kept in memory in Visual Studio - so called "virtual lines." I'm sure other more mature code editors have this type of optimization as well. From a native perspective this type of optimization isn't a complete nightmare to implement (although it's hard if you consider code folding), however, imagine trying to do it in HTML. The browser is still going too do all the reflow etc. for the lines that aren't even visible, eating up CPU cycles and memory. Even merely syntax highlighting is a problem. If you make one edit the entire file might be re-lexed: once again you need a unique solution for editors ("progressive lexing").

pavlov 11y ago
imagine trying to do it in HTML The question on my mind is, why do it in HTML? Especially if you're going to wrap that browser in something that approximates a desktop app anyway. Imagine if designers tried to do everything in PowerPoint for some unknown reason. "3D animation? Yeah, we can do it in PowerPoint with this incredible hack that makes slides sort of like faces on a mesh..." That's how it is with HTML today.
- linker3000 11y ago
  "Imagine if designers tried to do everything in PowerPoint for some unknown reason." A fair percentage of the employees where I work seem to have reached that point already - reports, training material, data for general discussion, GANTT charts with individual symbols that I guess must be manually moved every week. Gah!
- egeozcan 11y ago
  Because of extensibility concerns and the sheer number of JavaScript developers? I'd agree if you say it's not a good compromise, but it is one nevertheless.

deleted 11y ago

[deleted]