5 ms·
Debugging a Long-Running Apache Spark Application: A War Story
- mbb70 9y agoExcellent write up! Running a single long running Spark Context that accepts and runs jars compiled from user input is a curse I thought we were suffering alone at my company. We too have fought the ContextCleaner, though for us it was G1GC and `-XX:+ExplicitGCInvokesConcurrent` on the driver that saved us.
- zng00 9y agoIs this dynamic class execution a common use case? How frequently are you broadcasting new bytecode to your workers?