7 ms·
For RoR, see every method call, parameter and return value in production
- aantix 3y agoHere's the client code for those that would like to inspect. https://github.com/callstacking/callstacking-rails https://github.com/callstacking/callstacking-rails
- CPLX 3y agoThis seems ridiculously useful. What’s the catch?
- jtokoph 3y agoThese types of profiling gems usually kill performance
- aantix 3y agoThe instrumentation has a performance overhead. You enable the instrumentation with a prepend_before_action, e.g. prepend_around_action :callstacking_setup, if: -> { params[:debug] == '1' } When the request is completed, the instrumented methods are removed (thus removing the overhead). You have to enable it judiciously. But for a problematic request, it will give the entire team a holistic view as to what is really happening for a given request. What methods are called, their calling parameters, and return values, all are given visibility. You no longer have to reconstruct production scenarios piecemeal via the rails console.
- dev-ns8 3y agoI'm not familiar with rails, so sorry if your reply above inherently answered this.. So you're enabling the tracing with a call in your controller method, but how is the tool capturing function params and returned values for sub-calls in the respective controller method? Is it waiting for execution to return to the controller method and polling the stack trace from there?
- cschneid 3y agoas far as I can tell, it only executes the trace when asked. It's not an APM like newrelic. Most likely the trace meaningfully slows down the individual request. When I was at ScoutAPM, we built a version of this that was stochastic instead of 100% predictable. We sampled the call stack every 10-50ms. Much lower overhead, and it caught the slower methods, which is quite helpful on its own, especially since slow behavior often isn't uniform, it happens on only a small handful of your biggest customers. But it certainly missed many fast executed methods. Different approaches for sure, solve different issues.
- progne 3y agoThis is a product where the SaaS doesn't seem to add much that couldn't be done as easily locally, other than monetizing the process.
- aantix 3y agoDisagree. 1) In a large-scale production scenario, you typically do not have the data, nor the interaction flow, to reproduce the bug locally. The idea is that you enable Call Stacking on the fly, when needed. Turn it off when not needed. 2) Having multiple runtime captures of the same endpoint across two different deployments or time periods allows you to quickly compare for logic or data changes (argument values and return values are visible). 3) Commenting on individual lines of execution allows for the team to have a specific discussion surrounding logic changes.
- VeejayRampay 3y agolocal is never really a production environment though
- deleted 3y ago[deleted]
- mortallywounded 3y ago[flagged]
- aantix 3y agoHaha - I can't abandon my first programming love. :D Ruby is fantastic.
- x0x0 3y agoBut it's so much more pleasant -- esp the testing and mocking support -- than all the other languages I've tried.
- allknowingfrog 3y agoWell, if you never get around to releasing the feature, you'll never have production issues. Ruby embodies "perfect is the enemy of done" in a way that I tend to appreciate.
- GGO 3y agoI have not come across this sentiment before. Is there something specific about Ruby that makes you think this way or is this your general view of dynamic languages without a strong type system?
- kayodelycaon 3y agoSome of this already exists to some degree: https://github.com/MiniProfiler/rack-mini-profiler https://github.com/MiniProfiler/rack-mini-profiler
- aantix 3y agoDifferent emphasis. The goal is to quickly be able to see just the important, executed methods for a given request. E.g. you may have a 2,000-line User model, but Call Stacking allows you to pinpoint, "Oh, only these three methods are actually being called during authentication. And here are the subsequent calls that those methods make. And here's where the logic change occurred."
- stevepike 3y agoHow does this work on the backend? Does it only trace method calls when an exception is thrown, or does it profile the call stack of every request? Something I've been interested in is the performance impact of using https://docs.ruby-lang.org/en/3.2/Coverage.html https://docs.ruby-lang.org/en/3.2/Coverage.html to find unused code by profiling production. Particularly using that to figure out any gems that are never called in production. Seems like it could be made fast.
- aantix 3y agoThe idea is to turn it on for a given request when needed - via a parameter, feature flag, etc. prepend_around_action :callstacking_setup, if: -> { params[:debug] == '1' } Once the request completes, the instrumented methods are removed to remove the performance overhead.
- ysavir 3y agoWould this mean that any data I happened to have in memory during the flow now permanently lives in callstacking's data stores? How does it handle all the data flowing through from a security perspective?
- jxf 3y agoIt respects the normal RoR toolchain parameter filtering, so anything that you say is sensitive (or everything by default, if you'd like) also doesn't get sent to CallStacking.
- aantix 3y agoThe same filtering mechanism you have in place for your application logs is applied to the argument hash before being sent to the server. https://github.com/callstacking/callstacking-rails/blob/599d4ceeed944a48e69081957d12a653f7d5c669/lib/callstacking/rails/instrument/base.rb#L16 https://github.com/callstacking/callstacking-rails/blob/599d...
- jmholla 3y agoThe calculator at the bottom of the page is doing some weird calculations. If you have no incidents a year, it still costs you money. So do incidents that take zero minutes to resolve. I took apart the code, and this seems to be the equation in use: (revenueTarget / 8760) * resolutionTimeTarget * numIncidentsTarget * resolutionTimeTarget + numEmployeesTarget * avgEmployeeTargeRate This means revenue lost is correlated to the square of the lost time and the cost from employees is a static yearly cost. There are a couple things wrong with it. The third * should be switched with a + and the last term need to be multiplied by the number of incidents. (revenueTarget / 8760) * resolutionTimeTarget * numIncidentsTarget + resolutionTimeTarget * numEmployeesTarget * avgEmployeeTargeRate * numIncidentsTarget Which if anyone at Call Stacking is here, just means changing o = (n / 8760) * e * t * e + i * r; to o = (n / 8760) * e * t + e * i * r * t; or more succinctly o = t * e * (n / 8760 + i * r) I'm assuming that's minified, so numIncidentsTarget * resolutionTimeTarget * (revenueTarget / 8760 + numEmployeesTarget * avgEmployeeRateTarget) Edit: With the correct math, the example is wildly different. It should be $37,277.81, not $87,991.23.
- xmcqdpt2 3y agoMaybe there is a tool that can help them trace where this bug is coming from in their cost calculator?
- berkes 3y agoThe irony is that the issues this helps with could be solved far before production. Compile time, or some local runtime even. Just not in Ruby. Nearly all the issues this shows you quickly are issues that static typing would prevent compile time, or type-hints would show you in dev-time. I've been doing fulltime Rails for 12+ years now, PHP before that, C before that. But always I developed side-gigs in Java, C# and other typed languages and now, finally fulltime over to Rust. They solve this. Before production. You want this solved before production. Really.
- aantix 3y agoOf the bugs that I've experienced in large-scale, Rails production systems, typing is a small subset. Manually reconstructing logistical errors based on a combination of user input and system data, are the most time-consuming issues to diagnose. When your codebase is 500,000+ lines of code, which code paths are relevant for a given endpoint? What methods were called and under what context? How do we begin to reconstruct this bug? These are the scenarios for which Call Stacking gives instant visibility to.
- IshKebab 3y agoDepends on the type system. I would say for Java / Python level static types they catch a small but significant fraction of bugs (10-20% according to the only objective measurement I've seen, which is easily worth it). However some languages like Rust, Haskell and OCaml let you express much more in the type system. Subjectively it feels like that catches more like 30-60% of bugs. So this thing is still useful but Berkes is right that you need it a lot less if you use better static types. > which code paths are relevant for a given endpoint? This is exactly the question that static types can answer... statically. You don't need a runtime log to find out. You do need a runtime log to see the actual values though. So it's not like a debugger is completely useless in Rust. But I definitely reach for it much less than in other languages.
- theonething 3y ago> This is exactly the question that static types can answer... statically. You don't need a runtime log to find out. This tool seems to be able to display the relevant code paths. That sounds super convenient and useful. Do statically typed languages have tools to do similar?
- theonething 3y agoThis is advertised as a tool for production. Seems like it would be useful in development too.
- aantix 3y agoAnd new engineer onboarding. You have a new engineer. Point them to the Call Stacking dashboard. "Here's a list of all of our endpoints for our application. Click on a trace, and you can see the relevant methods for that endpoint and the context in which they are called." This will get your new engineers up to speed, much quicker.
- theonething 3y agowow!
- mrinterweb 3y agoHow does this product handle sensitive data? I'm guessing this is not a HIPAA compliant service.
- aantix 3y agoCorrect. The SaaS version is not HIPPAA compliant. On-premise is an option.
- saaspirant 3y agoDoes similar tool exist for Django / Python?
- antod 3y agohttps://werkzeug.palletsprojects.com/en/3.0.x/debug/#using-the-debugger https://werkzeug.palletsprojects.com/en/3.0.x/debug/#using-t... ? Not sure if Django could use it, but used an earlier version in Flask and Pyramid.
- antod 3y agoAnd by similar to the Rails tool, I mean the functionality, not the intended usecase. Werkzeug is intended for local dev mode only.
- crystaln 3y agoThis is very cool and useful. Also it is only necessary because of the lack of type enforcement which means no code can be relied and on and all code has to be constantly inspected for new bugs. Ugh.
- yen223 3y agoI can see this being useful in strongly-typed languages too. There is a massive class of logical bugs that will type-check correctly, that will still result in wrong results being returned.
- jaggederest 3y agoI built a toy raytracer in Haskell for fun. I found all of these bugs. It turns out that when you implement dot product slightly wrong, the image output is very confusing.
- aantix 3y agoTypes do not prove logistical correctness. Imagine a million-line codebase. There are half a dozen suspicious methods with complex sets of if/else if/else statements. And each of those statements make subsequent method calls. Determining that code path is a nightmare. Types won't save you.
- masklinn 3y agoDo you mean logical? Types absolutely do prove logical correctness. In fact that’s all they do. However they can only prove the correctness of logic that’s type encoded. If your program is a primitive soup, there’s not much logic for them to prove.
- 59nadir 3y agoEven just redundant call paths are useful to spot. I recently looked through the standard library of a strongly, statically typed language and found that in one pretty basic function a validation function was called over 8 times despite reliably returning exactly the same result every time and this tool would highlight that very easily. That's not even mentioning the logic bugs you can spot more easily as well.
- aantix 3y agoIf someone is up for writing a Call Stacking client for the language platform of their choice, please email me. This would be your reference implementation. https://github.com/callstacking/callstacking-rails https://github.com/callstacking/callstacking-rails I'm sure we could work out an arrangement. jim@callstacking.com
- maz1b 3y agoPretty cool. Not sure what pricing is, seems like it's focused for enterprise.
- theonething 3y agoDoes this handle multithreading?
- mediumsmart 3y agoAfter signup I see a typo in : The trace URL will also be outut via the Rails log. And the local usage section is hard to read white text on light blue background in this safari browser.
- aantix 3y agoGood catches. Should be fixed now.
- n3storm 3y agoI would love to see this for Laravel.
- block_dagger 3y agoTypo on front page: "What's does an hour of downtime cost you?"
- ilamparithi 3y agoRecently did something similar for a java project using AOP. Basically adding an annotation to each method and logging the parameters before the method call and return values after the method call. Whenever there is an exception, a mail will be sent with the stacktrace along with the entire request path(including method calls, parameters and return values). Extremely useful for debugging and to proactively fix the issues.
- saagarjha 3y agoCurious how you do this performantly for any non-trivial codebase. Like, consider a class whose logging representation is the data it contains, which can be arbitrarily large. Generally this is not an issue because it’s only logged rarely and on designated paths where people actually care about this and it was expected to be used in that fashion. How would this work for a large object that you pass to a function repeatedly, or in a deeply nested stack trace?
- ilamparithi 3y agoThe project I worked on has a non-trivial codebase. So far I haven't seen any performance issues though I was worried initially. The idea is to use it during development and beta testing and switch it off later once the application is stable enough. Might keep it on for some more time if there are no performance issues.
- deliriousferret 3y agoDoes it work for background jobs as well?