6 ms·
Heroku Gets Sweet Logging
- ai09 16y agoUntil the Advanced functionality is rolled out publicly, does anyone have suggestions for logging all requests to something like S3? I was going to write my own this afternoon since I couldn't find anything. I see the market currently providing two options, neither of which encompass my use case. 1) Exception Logging via existing Heroku addons like Exceptional 2) Tracking via javascript, e.g. Google Analytics The problem I'm experiencing is that I have successful ajax calls to a Sinatra app that I'd like to log. Running 'heroku logs' will give me the last couple hundred but I'm generating many thousands. The new new Heroku logging functionality will expand that tail to showing me a few thousand queries across my dynos but as I understand it, it won't write out to S3 (nor a file since Heroku is write-only). I planned on writing a custom logger that will output to S3. If anyone has suggestions in lieu of rolling my own, please advise. Otherwise I can post the code I create on github for others in my situation.
- knodi 16y agoYou can use MongoDB Logger which works pretty good https://github.com/peburrows/mongo_db_logger https://github.com/peburrows/mongo_db_logger Although I'm not sure if its still being maintained.
- sync 16y agocentral_logger is the new mongo_db_logger: https://github.com/customink/central_logger https://github.com/customink/central_logger We use it in production on heroku and it works great.
- blasdel 16y agoS3 is not the best AWS for the functionality you seek: http://aws.amazon.com/simpledb/usecases_logging/ http://aws.amazon.com/simpledb/usecases_logging/
- dholowiski 16y agoThat's great. My first attempt at using heroku was frustrated by their not-so-sweet logging. I'll have to try it out again for my next project
- yuxt 16y agomy favorite: 2010-10-21T10:11:16-07:00 app[web.2]: Company acquired /_party (0.1ms)
- steveklabnik 16y agoOnly a partial party? I guess they would be re-using it multiple times, hopefully...
- fizx 16y agoand it didn't last for very long ;)
- idlewords 16y agohttp://en.wikipedia.org/wiki/Inner-platform_effect http://en.wikipedia.org/wiki/Inner-platform_effect
- sgrove 16y agoI understand it seems witty article to link to, but it's not giving the Heroku guys much credit. It seems that they ran into some fundamentally difficult problems while trying to implement a pleasant logging system - I'm inclined to give them the benefit of the doubt for a number of reasons, but I know that anyone can fall into NIH trap. What, then, would you propose as a solution, given the nature of Heroku's architecture and customer base?
- idlewords 16y agoUnix
- d_r 16y agoDoesn't the second paragraph under examples more or less describe the motivation behind NoSQL? In the database world, developers are sometimes tempted to bypass the RDBMS, for example by storing everything in one big table with two columns labelled key and value If so, maybe this is not always an anti-pattern.
- avk 16y ago$ heroku logs --tail ! Realtime tail and filter are not available with basic logging, please upgrade to expanded logging :(
- marcamillion 16y agoProlly not the place to ask this...but I will go ahead anyway...I recently heard that if you have a Rails app that will be accepting files from users (e.g. say images/videos being uploaded) Heroku is not the best host to use. Anyone have any thoughts/experiences with that? Is that complete rubbish or is there truth to it and why?
- grahamr 16y agoI use the paperclip gem and store files on S3 with my Heroku apps and it works great. I believe this approach is very common.
- marcamillion 16y agoI too am implementing paperclip...but I am wondering about backing up the uploads. How do you backup those uploads? I was thinking that I would store the first copy on the host - then push them to s3 for backup (and maybe 1 other CDN, just to be ultra paranoid). But, it seems to me, that if you are just passing through Heroku altogether - that removes one point of fault tolerance and redundancy - right?
- nborgo 16y agoI'm guessing they said that because Heroku doesn't let you save files to their system. You have to use something like s3. Other than that, I see no reason why someone would say that. You can save things to the tmp directory for a little while. The files should last long enough to do some work on them. For example, I've had an app parse an uploaded XML file that gets saved in tmp. This might not be a good practice, but I don't feel like setting up anything special for that app.
- tpz 16y agoAFAIK, if you have a Rails app that will be accepting files from users, Rails is probably not the best thing to use. Ruby likewise, short of using something evented (and I say this as someone thoroughly enjoying working with both Ruby and Rails.) Seeing as you'll be tying up processes rather quickly that way, no? That aside for the moment, and based on your S3-related comments nested below, I thought I might pop by to mention that I (using Rails on Heroku) have been handling uploads by using direct pre-authorized posting to S3 and it has been very nice to work with.