11 ms·
This is a great example that shows how poor most non-web debugging and logging. The ability to mix and match data types and presentation format is extremely use
by azinman2 8y ago
This is a great example that shows how poor most non-web debugging and logging. The ability to mix and match data types and presentation format is extremely useful. Almost everywhere else we just have printf. And that’s printf going to stdout — there isn’t a universally deployed format to even allow it to travel easily over the network and be viewed on one of many useful and advanced viewers.
I think part of the problem is that programmers rarely want to get into these supportive details; they just printf and get on with it.
- a_lieb 8y agoThat, in turn, shows how far the web has come. I remember a time when we didn't even have console.log()!
- hoorayimhelping 8y agoGood point! The first time I used firebug, probably 3 years before I used Chrome, it blew my mind and immediately made clear how much faster it was going to make styling pages, debugging errors, and just testing JavaScript expressions.
- deathanatos 8y agoSo much this! I've long wanted something where by I can chain, into a sort of tree, to more easily follow or hide large subcomponents and subservices. For example, say I make two calls in my service A to two other services B and C, and all report logs to a centralized location; that could get rendered as, ─ info: GET /foo/endpoint ├─ info: user <user ID> valid auth ├─ info: request to service B │ ├─ debug: opening new connection │ ├─ debug: success, result = ... │ └─ info: request took 1 seconds ├─ info: request to service B │ ├─ debug: opening new connection │ ├─ debug: success, result = ... │ └─ info: request took 1 seconds ├─ info: preparing result took 1 seconds └─ info: http request took 3 seconds with the ability to hide sections, perhaps grep for only certain messages (particularly if you keep the formatting and the message separate, this should be doable, I think), attach metadata to messages… As it is, we have a fairly standard shove everything into syslog, and then pipe to a downstream logging system and a local file. But the downstream system is not very good at search (this is probably mostly our fault) and requires the message to be in JSON, so the stuff in the log file is _also_ JSON, b/c that's what syslog got. There are definitely better ways with our existing tools, but it sure makes one dream up what the perfect logging solution could look like.
- azinman2 8y agoTracing calls across threads/servers is exactly the kind of stuff that should be easy and out of the box. If we had this built into a kind of universal logging protocol, you can imagine some viewers getting extremely sophisticated with network-level analysis, parsing & plotting times for distributions, anomaly detection, etc. Other viewers might be more geared towards mobile development, which might emphasize where network requests were generated in the code and its effect on the UI. Both should be possible from the same input data stream.
- tonyarkles 8y agoThis is what I did my MSc thesis on. I should consider open sourcing the framework, even though it’s relatively “homework grade” A couple of places to look though: - opentracing.io - Fonseca et al’s X-Trace work I don’t have a link handy, and the visualizations haven’t aged well, but you can probably find a copy of my thesis or conference paper under Anthony Arkles in google scholar. I extended the X-Trace protocol a bit to make it easier to reassemble function calls that potentially had parallelism.
- joshribakoff 8y agoHtml5 timing API + chrome devtools performance tab "user timing" section is a great workflow in my opinion for frontend.
- chrisweekly 8y agoTake a look at lnav (https://lnav.org https://lnav.org), you might be pleasantly surprised.
- masklinn 8y ago> As it is, we have a fairly standard shove everything Assuming "everything" is "static log statements" which is the first issue I have with this, a developer must have thought before deployment that this could be useful, which usually results in a lot of useless garbage and a lack of actually useful information. And little by little you build up actually technically useful data collection, and rather than being spread throughout the codebase all of the probes are centralised and readable. I've been thinking about using dynamic instrumentation tools (bcc/dtrace) for that purpose instead, you know you need something when you actually do need it, at that point you can add it to the probes/instrumentation (which is external to the program and deployable separately), and all information would be collected in a structured form in a database you can interact with (probably not something relational).
- hoosieree 8y agoMy absolute favorite web-console feature (in Chrome and Firefox at least) is the ability to go back in the REPL history to edit a previous function definition. A standard REPL just gives you back one line at a time, so if you want to make a change to line 3 of a 4 line function, then you have to type: UP UP UP UP ENTER UP UP UP ENTER UP UP before you can edit that line. But the web consoles allow you to type UP just once, and then move your cursor around the entire function definition. I wish all REPLs had this feature.
- pdkl95 8y agoYou should try binding UP/DOWN to history-search-backward and history-search-forward[1]. Adding this to your ~/.inputrc "\e[A": history-search-backward "\e[B": history-search-forward enables searching by the current line's prefix in all programs that use readline (a lot of languages use readline for their REPL). That is, if you type "cp<UP>" you immediately get the previous command that started with "cp", regardless of how far back it is in your history. [1] https://codeinthehole.com/tips/the-most-important-command-line-tip-incremental-history-searching-with-inputrc/ https://codeinthehole.com/tips/the-most-important-command-li...
- kbp 8y agoThis is a useful trick, but I don't see how it addresses the issue they talked about. As I understand it, they were saying that Firefox and Chrome's Javascript REPL's scroll up through expressions rather than lines. If you say "const f = () => {" and press enter, it doesn't submit it until you close that brace, and when you use the up arrow to scroll back to it later, you get the whole thing (newlines preserved) instead of single lines of it.
- Arnavion 8y ago> UP UP UP UP ENTER UP UP UP ENTER UP UP If the REPL uses readline (or mimics it), then UP UP UP UP ^O ^O will do the same thing as that sequence.
- taeric 8y agoThis is trivial in the right environments. Not shockingly, in emacs I get that. It actually takes a little getting used to at first, that up literally just moves the cursor up into the output of the previous command. Nowdays, it is severely restricting to have the same commands move the cursor that change the current command. (That is, left/right move cursor, up/down do a ton more in most terminals. That is not the case in emacs shells.) Really, though, the repl is a fun stepping stone to just sending function definitions from the file you are working on into an environment. https://github.com/skeeto/skewer-mode https://github.com/skeeto/skewer-mode is a fun video showing the idea for javascript, amusingly. That said, there was a fun video a while back of the guy that did Minecraft live coding a video game. Basically, the game would be running, and he could make a change and it would live update, just like the javascript video above. But it was in java. This is harder in non-managed runtimes, I believe. But not impossible.
- pdkl95 8y ago> shows how poor most non-web debugging and logging Nonsense - web programming is inherently tied to the browser, which severely limits debugging and output options. > The ability to mix and match data types and presentation format is extremely useful. That's a common feature of dynamic type languages. > Almost everywhere else we just have printf. Most languages have logging libraries, and many have them built into the stdlib or runtime. Popular languages usually have many logging libraries to choose from. > that’s printf going to stdout Of course - printf(3) by definition "shall place output on the standard output stream stdout"[1]. If you want stderr or any other output stream, use fprintf(3). #include <stdio.h> int main() { fprintf(stderr, "Error: %s\n", "LP0 on fire"); return 0; } Other languages usually handle that for you as part of the logging library. > there isn’t a universally deployed format Sure there is: "text"[2]. > to even allow it to travel easily over the network Copying files to a remote host is easy with scp. There are numerous ways to send data over a network. Web software is often more work as you have to move your debugging data/logs out of the browser first or use a specialized tool that does that for you. Sending program output realtime to a remote host only requires appending "| ssh ${host} \"${remote_cmd}\"" to your command. > advanced viewers That's great if you have specific needs and a viewer for your complex data structures, but the only universal solution is text. Fortunately, there are many tools available to handle structured data as text. > I think part of the problem is that programmers rarely want to get into these supportive details; they just printf and get on with it. I think you might want to explore the work programmers have done in other languages. [1] http://pubs.opengroup.org/onlinepubs/9699919799/functions/printf.html http://pubs.opengroup.org/onlinepubs/9699919799/functions/pr... [2] http://www.catb.org/esr/writings/taoup/html/ch05s01.html http://www.catb.org/esr/writings/taoup/html/ch05s01.html
- azinman2 8y agoYou're totally missing my point(s). Of course web programming is tied to the browser, but why does that mean debugging via logs has to be tied to whatever particular console is attached to where a program is running? Why can't I have a console.table() equivalent in Python, and Ruby, and C, and Go, and Java that all can render via a common protocol to any viewer that can handle it? By printf I didn't _literally_ mean printf -- I meant any logging type equivalent whether that IS printf or a built-in logging system. The point is there's no inter-op between them with any rich detail, which prohibits universal log viewers. When I say travel over the network, I don't mean scp. I mean allow my iOS/Raspberry Pi/local Erlang server device to stream in real-time it's logs to any viewer that I want, whether that's on the same device or somewhere else in the world. That's the kind of portability required when you can't guarantee that complex display can be part of the same system that generated it -- that's a unique quality of web browsers. IT'S ABOUT THE PROTOCOL, NOT THE MECHANISM. Text is great, and most logs will be in text. But I also sometimes want to inspect data, or expand JSON, or view an image, etc etc. But let's even stick with text -- a good viewer would let me set up filters on a host of structured qualities -- filenames, lines, log levels, keywords, etc. It would output in a way that allows me to easily view such info in a UI that supports infinite scroll back, easy copy/paste, elegantly formats so different log instances are clearly separate from each other, adds color to enhance readability, allows me to hide/collapse sections or duplicate entries, provides columns for the structured logging fields like timestamps, allow interactive sorting on these columns like Excel, etc etc all without losing my data. Apple's Console.app is an elementary example of what I'm suggesting I'd like to see. Except I don't want it tied to the syslogs of an Apple device. I want every language/platform to be able to output to it in real-time. And for it's capabilities to be as rich or richer than what we have in the browser. We already have universal text editors like VIM/Emacs -- why can't we have universal log viewers?
- greghendershott 8y agoAWS Cloud Watch Logs can take JSON format log events. This displays better in the AWS console -- and also it enables better queries than raw text, e.g. `{$.response.code >= 400}`.
- azinman2 8y agoWhile I agree it's nice they can take it, I don't think their viewer is particularly readable or advanced. Apple's very basic Console.app is a much stronger example, and it's super simple in capabilities.
- aaron-santos 8y agoClojure's not often mentioned clojure.inspector namespace contains two functions: inspect-table and inspect-tree. They roughly approximate Console.table() and Console.dir(). I tend to get more mileage out of inspect-tree, but even so I find it limiting. I dream for a graphical object diffs and a version that when passed an atom or ref, auto-updates when it changes.
- lpghatguy 8y agoOne of the under-shared features of Rust that I adore is #[derive(Debug)] -- having object output like a dynamic language on low-level, statically typed struct definitions is AWESOME. Huge boon to productivity, especially when leveraged in tests.
- creichenbach 8y agoA couple years back at university, I created DoodleDebug[1,2], tackling some of your mentioned issues. It can be thought of as a graphical version of System.out.println(). There is a similar output format customization mechanism as toString(), but you can also customize the rendering of arbitrary types you don't control the source code of. DoodleDebug's output is HTML-based and interactive with semantic zoom. That means you can click parts of printed objects to inspect them. However, I didn't work on it in years due to lack of time, and even the linked description page is a bit outdated (it's not an Eclipse plugin anymore). [1]: http://scg.unibe.ch/wiki/projects/DoodleDebug http://scg.unibe.ch/wiki/projects/DoodleDebug [2]: https://github.com/CedricReichenbach/DoodleDebug https://github.com/CedricReichenbach/DoodleDebug Edit: Maybe this PDF of my Bachelor's thesis gives a better overview of what DD is and does: http://scg.unibe.ch/archive/projects/Reic13a.pdf http://scg.unibe.ch/archive/projects/Reic13a.pdf
- westoncb 8y agoDamn, glad to see this, but they're also kinda encroaching on my territory! :P I've been working on something like this that's language agnostic and works with not only tables but also trees, graphs, lists, hashmaps, etc., and animates the visuals as the data structures are modified in real time (and you can pause, playback, step, etc.): http://symbolflux.com/projects/avd http://symbolflux.com/projects/avd The api I was conceiving earlier was more complex, but if you read the copy there, you'll notice it's essentially the same now: `MObject.monitor(...)` or MTable.monitor() MStateMachine.monitor() etc. Edit: really am happy to see some other projects like this in total, partly because I want to use 'em now if possible—but also because I've had a surprisingly hard time communicating to other developers why it might be desirable to do something like this in the first place, so I'll be very happy to have general awareness raised. I read the first line of their docs as rather dry humor, "Displays tabular data as a table." —like yeah, of course you would want to see tabular data as a table, and hierarchical data as a tree, etc.!
- nojvek 8y agoThat video was impressive. Are you using OpenGL/gpu for the animations?
- westoncb 8y agoYes, sadly I rolled my own engine directly on top of Java/OpenGL, rather than using something pre-existing. (There were some decent reasons for it considering how specialized the visuals I'm working with are—but it was a big mistake over all. Should've done it in a more prototype-y fashion). To code the animations I use something in the engine which resembles CSS transitions and can be attached to arbitrary Java objects, which I call 'property modulators'. So most of the animation you see comes from setting up one of those on one property or another.
- aldanor 8y agoWould this possibly work on top of WebGL?
- noway421 8y agoWe are too hell-bent on that unix philosophy that everything is a text, stdin, stderr and pipes are enough. It is not. We should have ways to log objects. We should have ways to pass those objects around and serialise, deserialise and inspect them. It's not just text. Grep is not enough for investigation.
- hvidgaard 8y agoA proper debugger like the one in VS and general structured logging would like to challenge that statement. I'll take those over what you have in the web space any day.
- rounce 8y agoWhat are the criteria that define whether a debugger is 'proper' or not?
- hvidgaard 8y agoThere are many, but for my line of work: * Conditional Breakpoints * Function breakpoints to add ad-hoc logging * Show type information * Ability to custom render state, a graph for instance. * GUI to browse the state * CLI to query the state * Ability to alter state * Multithread debugging support * Record execution history and state so you can debug after the fact for hard non reproducible bugs. * Remote debugging * Support for network traffic debugging
- rounce 8y agoThanks for this list and I'm full agreement. Chrome developer tools will do all of these except > * Record execution history and state so you can debug after the fact for hard non reproducible bugs. ^ Which would frankly be awesome. Various libraries provide their own userland support for this but it's all fragmented with hugely varying levels of quality and flexibility, but out of the box support would make me so happy (unless this already exists in which case, someone please enlighten me!).
- sedatk 8y agoPowerShell uses objects as output. They get automatically formatted and save you from custom parsing code.
- frankus 8y agoI wish PowerShell and/or some of its ideas had a bit more mindshare outside of the MS ecosystem. Other folks have talked about this at length (e.g. https://mkremins.github.io/blog/unix-not-acceptable-unix/ https://mkremins.github.io/blog/unix-not-acceptable-unix/), but the fact that there is no standard way to pass anything other than strings of plain text between UNIX shell commands is incredibly limiting. Heck even if we could pass a flag (say, -j) to something like ls and have it spit out JSON instead of plain text would be an upgrade.
- deleted 8y ago[deleted]
- wruza 8y ago>Almost everywhere else we just have printf Because you see the black console almost everywhere. We debug tabular or structured data in visual debugger, console or specialized snapshot-view that is not hard to create. All depends on programmer’s will to make their life easier. Console.table() is just one of the simplest things that you could write for yourself in dynamic language, but the fact that most web “devs” cared to read the docs only after it hit top of HN makes it look amazing. >how poor most non-web debugging and logging Praise the web.
- pjmlp 8y agoOnly for those developers that insist in not using IDEs. Common Lisp and Smalltalk environments have something like console.table since its early days. Xerox even went to the trouble of adding similar capabilities to the Mesa/Cedar enviroment, because they thought they should cater to their Interlisp-D and Smalltalk developers. Java and .NET even allow to provide metadata so that debuggers know what is the best way to render the data in debug view. Visual Studio allows similar capabilities for C++ via data visualizers. Now if you insist in vim and Emacs, there is printf.
- mort96 8y agoIt has little to do with IDEs and everything to do with debuggers. When writing code in emacs or vim, as long as you compile with -g and run the binary with gen, you get the features you're talking about.
- pjmlp 8y agoUnless you are speaking about something like DDD, not really. Also it has to do with IDE, because in an IDE there is two way communication between both tools, like the ability to do edit-and-continue.
- mort96 8y agoLook at this: https://news.ycombinator.com/item?id=17310418 https://news.ycombinator.com/item?id=17310418 It's not quite as fancy as an IDE, but it's a lot more than printf.
- deleted 8y ago[deleted]
- oblio 8y agoCould you show a screenshot?
- 8y ago
- megous 8y ago> there isn’t a universally deployed format to even allow it to travel easily over the network and be viewed on one of many useful and advanced viewers. printf("<tabe><tr><td>%s</td></tr></table>", htmlescape(blabla), ...); Copy paste to the browser and you're done.
- baud147258 8y ago> how poor most non-web debugging and logging Well, since that time when chrome was unable to find the javascript file of the page I was looking at, forcing me to go back to internet explorer, I disagree. Also is it possible to edit the javascript code without reloading the page? I know how to do it with java+eclipse, but not in the browser.
- alerighi 8y agoIf you are debugging a program inserting printf or other debug prints in the code you are doing it wrong, for debugging you should use a debugger. It's slow and inefficient, you add the debug print, then you need to recompile the program, maybe you need another debug print, recompile the program, and so on, and then you must remember to delete all the debug prints that you added! And let's hope that your debug print didn't add some strange side effect (they might have!). With a modern debugger you could print everything you want, at every memory location, you could set watchpoints, isn't it better than printf ? Also where you printf on an embedded system...
- megaman22 8y agoSometimes you just can't do that, without completely breaking everything while the program is halted in the debugger. Particularly real-time, multithreaded applications.
- organsnyder 8y agoIn those situations, even adding printf statements can be enough to change the behavior of the system.
- TeMPOraL 8y agoNot necessarily. It's about timescales and other environmental factors. Pausing in a debugger is measured in seconds. Your network-using application won't notice extra half ms for a print statement, but might change its behaviour if you pause it for 5 seconds, as requests will timeout. A video game running in fullscreen won't mind printing to console/file in the background, but may not handle well being interrupted in the middle of rendering and having you switch focus to the debugger (less of a problem these days, but I remember this being a pain ~10 years ago).
- gkya 8y agoSometimes it's just easier. Especially if you have a nice incremental build system or an interpreter to which you can just send expressions and definitions and see the results. Edit: also, you don't need to remember to weed out debug prints, you can, in almost all languages, define a function or a macro to do that which can be undefined or an ignore function if not built in debug mode. With cpp or Lisp, that'd be a compile time decision, and I suppose many languages could optimise out a function that is defined to just ignore it's arguments and do nothing.
- hawski 8y agoIs there a reversible AKA time travelling debugger for JavaScript?
- timrichard 8y agoI think the closest thing for this would be something like SpyJS, which is available for both NodeJS and browser JavaScript in JetBrains products such as Idea and Webstorm. You can see previous executions of the code you're interested in, with a highlight of the paths followed through the logic and snapshots of the variables as they were at the time.
- TeMPOraL 8y agoOn the one hand, I like it. It's good this function is there. On the other hand, I'm disappointed. With all the power the web browser tools have - basically, arbitrary HTML rendering in console - this is the best we get? A simple, dumb table? There are so many to be explored here! > I think part of the problem is that programmers rarely want to get into these supportive details; they just printf and get on with it. A lot of programmers aren't aware that yes, they can and should modify their environment to suit their needs. Yes, they are allowed and should write their own debugging tools. Those tools don't have to be complicated. You can start with an ASCII-based console.table() equivalent. Or a script that parses the output and feeds it to some half-baked D3-based visualization. Or even gnuplot. Or graphviz. Plenty of possibilities there, and yet most devs I know never even think of them.
- tripzilch 8y agoI want a thing that can plot a numerical array as a line graph of data points. A histogram and a scatter plot would be nice, too. And I don't mean for computational/data science exploring, I've got numpy/matplotlib for that. But really as a debugging tool in the browser's dev console. It's not uncommon that I have an array of numbers (or something that looks like one), a few hundred or more. Finding the bug or unexpected behaviour in my program is often about finding the numbers in such a list that "stand out" in some way. Now I could specify and filter those elements that "stand out", but I could also plot a graph and grasp its behaviour in a single glance. Just dumping the numbers to the console and hunting for aberrant behaviour likely takes me out of my flow, where a graph would not.
- TeMPOraL 8y agoYup. This is something I'd love to have too - in particular, point/line plots and scatterplots at resolution greater than possible in text-only terminals. I already render text-based histograms when I need. Come to think of it, it shouldn't be hard to extend Emacs to eat specially prefixed output (e.g. #<scatterplot: (1 2) (3 4)...>), feed the data to external program and place a rendered chart inline. I might pick it up as a small project. Thanks!
- gowld 8y agoEclipse, VisualStudio, and all other IDEs have had tabular expression viewers for decades.