3 ms·

I've worked a lot with this pattern in Python (Flask), Java and Scala. I have found it a little inconvenient beyond the simple hello world app when dealing with

by jonaf 10y ago

I've worked a lot with this pattern in Python (Flask), Java and Scala. I have found it a little inconvenient beyond the simple hello world app when dealing with Flask, but I haven't encountered this problem with Java/Scala. To date, I've worked on probably 4-5 separate applications using this pattern and it's worked quite well. I have also used Spring 4 on the JVM, which ... feels more flexible, but also has a lot of cognitive overhead, and so I'm not crazy about it. I believe Spring Boot follows closely after Jersey, most likely as a result of the pattern's popularity. In my experience, it is pretty straightforward to build your resources (as Jersey calls them) into various classes without any issue, even for applications with many API endpoints. As for metrics, security and routing, most of these are canonically managed using annotations in Java. In Spring, application context XML configurations are used, but they are very big and not close to their actual implementations, which makes them get out of control quickly (and inconvenient when they aren't out of control). It feels like losing either way. I'm a big fan of Coda Hale's work on Dropwizard, which uses metrics, security and routing through annotations, and it works very well for large scale production applications. I use these techniques at scale at Bazaarvoice with great success. To be fair, I won't go so far as to claim the pattern is imperfect. It's just the best pattern I've used for web services to date, even at scale. I would love for you to expound more on your own experiences in more detail! edit: clarification on where I've used the pattern

strictfp 10y ago
I've worked on everything you mention and more, and I find the simplicity of say node.js or SparkJava very refreshing when coming from metaprogramming frameworks. Once you go metaprogramming everything is suddenly metaprogramming. Data serialization is metaprogramming. Middleware is metaprogramming. Adding your own plugins to the framework becomes metaprogramming. Also, because the framework wants to create your classes you cannot simply send in the objects dependencies into the constructor anymore. You find yourself in need of a DI framework instantiating your classes. And suddenly the data dependencies of your app become very vague. Adapters get registered based on the presence of JARs and/or classes in directories which are scanned in runtime. You have reinvented the work of the compiler and the language, and what do you gain? A big mess where you no longer have programmatic control. I cannot easily start parts of my application for testing, I have to rely on Spring or Jersey having some sort of metaprogramming magic triggered by some magic annotation supplied by a JUnit<->Spring integration module. Suddenly modules like JUnit and Spring aren't combinable without metaprogramming. You see this trend spreading like wildfire: "Is your lib Spring compatible?". We should instead be asking ourself why the heck unrelated libs have to be compatible with one another. They are supposed to be orthogonal, that's the point! To see an example of how ridicilous this can get, have a look here: http://stackoverflow.com/questions/35957287/cucumber-testng-springboot http://stackoverflow.com/questions/35957287/cucumber-testng-... I wouldn't go as far as saying that frameworks like Spring or Jersey are unworkable. But I believe you could acheive the same level of service a LOT easier if you wouldn't go metaprogramming crazy. Just consider the amount of time wasted trying to figure out why something isn't wired correctly, why some plugin is loaded, why your component isn't picked up, why some property isn't resolved etc. With normal programming, you fire up the debugger and step through the code. Solved in 5 min. With metaprogramming, it's a struggle on a whole different level.
- jonaf 10y ago
  Thanks for clarifying. I agree with you, mostly. This is exactly the reason why I refuse to use Lombok. It literally mutates your bytecode! That's a disaster waiting to happen, and a potential debugging nightmare. I think that there is a fair balance where metaprogramming has great utility. For example, I'm a huge proponent of Guice. And yes, sometimes it makes debugging issues a little more complicated. OTOH, without it, there's a lot of boilerplate. Admittedly, DI in Java is a workaround for an inherent Java problem (boilerplate). Maybe it wouldn't be so useful in other languages/ecosystems. I have written services both with and without DI, and prefer it, and haven't had significant issues debugging or managing it. But it must remain scoped. As for library compatibility issues, that is more of a byproduct of dynamically linked libraries than metaprogramming. If you're writing in a JVM language, you're kind of stuck with this (OK, you could shade your library and all its dependencies to get away from it, if you want to deal with the bloat / impact to the JIT'er). Having said all that, your statements are arguing against metaprogramming, and I'm not really arguing for or against metaprogramming in my original comment. Simply, I think the pattern of defining HTTP resources in the aforementioned way is very intuitive from a software development perspective. If it can be done without metaprogramming, then great. But the presence of metaprogramming doesn't, in my mind, preclude the effectiveness of the pattern.
  - strictfp 10y ago
    Ok, I hear you. I agree somewhat, but not totally :) To me, the most important aspect of programming is to retain full control. I for instance want to be able to register my HTTP endpoints based on some data read from a file. I want to be able to start my app in server mode and run requests against it in any way I want during testing. What many of these frameworks do is to remove this power. Bytecode modification is not too bad IMO, as long as it's within reason. I use a notnull annotation weaver to add notnull assertions. Not Lombok, that's too much for me as well. Should be another language really, like Kotlin. Metaprogramming does have utility. But it has a tendency to go overboard. I actually find Jersey to be a good example of a nice and clean API for defining resources. I recently tried to integrate it into a small app where we do all the wiring manually for clarity. The problem is that it's hard to stay in programmatic control. You define a resource. You realize you need a Service in your resource. You realize that the only way of getting it into your object is to inject it with the JAX-RS annotations. And so suddenly your service has to be a JAX-RS @Singleton service. You have to make your whole app JAX-RS compatible. The framework takes control. This is really bad IMO. A library which requires control over your program. This is not an intrinsic problem of metaprogramming, but IMO it's a big problem of Java as it stands today. We need to give up on frameworks, make sure that all libraries are combinable and that they don't take control of your app. Doing so would make everything a lot better. I wish I could just do: new Jersey(config).addRoutesIn(new HelloWorldResource(new MyBackendService(dbDriver))).addSerializer(new JsonSerializer()).start(); This would then utilize some reflection when routing requests, but not take over my entire app. Regarding the libraries, my point was not that you get conflicting classes (DLL hell sort of thing), but that we're building multiple products which want to take over the world. JUnit wants to be in control, and so does Spring. So when you try to combine them, you end up with a problem. Can you start Spring programmatically from a JUnit test? Well, not really. You can, but it's very involved. So they want to provide you with a Spring-JUnit integration. This is a really bad idea, since every linear combination of libs has to get an integration, like the link I included in my post. If all these libraries just tried to stay out of the way of the programmer and just provide a programmatic API, there would be no need for such integrations.

by jonaf 10y ago

strictfp 10y ago

I've worked on everything you mention and more, and I find the simplicity of say node.js or SparkJava very refreshing when coming from metaprogramming frameworks. Once you go metaprogramming everything is suddenly metaprogramming. Data serialization is metaprogramming. Middleware is metaprogramming. Adding your own plugins to the framework becomes metaprogramming. Also, because the framework wants to create your classes you cannot simply send in the objects dependencies into the constructor anymore. You find yourself in need of a DI framework instantiating your classes. And suddenly the data dependencies of your app become very vague. Adapters get registered based on the presence of JARs and/or classes in directories which are scanned in runtime. You have reinvented the work of the compiler and the language, and what do you gain? A big mess where you no longer have programmatic control. I cannot easily start parts of my application for testing, I have to rely on Spring or Jersey having some sort of metaprogramming magic triggered by some magic annotation supplied by a JUnit<->Spring integration module. Suddenly modules like JUnit and Spring aren't combinable without metaprogramming. You see this trend spreading like wildfire: "Is your lib Spring compatible?". We should instead be asking ourself why the heck unrelated libs have to be compatible with one another. They are supposed to be orthogonal, that's the point! To see an example of how ridicilous this can get, have a look here: http://stackoverflow.com/questions/35957287/cucumber-testng-springboot http://stackoverflow.com/questions/35957287/cucumber-testng-... I wouldn't go as far as saying that frameworks like Spring or Jersey are unworkable. But I believe you could acheive the same level of service a LOT easier if you wouldn't go metaprogramming crazy. Just consider the amount of time wasted trying to figure out why something isn't wired correctly, why some plugin is loaded, why your component isn't picked up, why some property isn't resolved etc. With normal programming, you fire up the debugger and step through the code. Solved in 5 min. With metaprogramming, it's a struggle on a whole different level.

jonaf 10y ago
Thanks for clarifying. I agree with you, mostly. This is exactly the reason why I refuse to use Lombok. It literally mutates your bytecode! That's a disaster waiting to happen, and a potential debugging nightmare. I think that there is a fair balance where metaprogramming has great utility. For example, I'm a huge proponent of Guice. And yes, sometimes it makes debugging issues a little more complicated. OTOH, without it, there's a lot of boilerplate. Admittedly, DI in Java is a workaround for an inherent Java problem (boilerplate). Maybe it wouldn't be so useful in other languages/ecosystems. I have written services both with and without DI, and prefer it, and haven't had significant issues debugging or managing it. But it must remain scoped. As for library compatibility issues, that is more of a byproduct of dynamically linked libraries than metaprogramming. If you're writing in a JVM language, you're kind of stuck with this (OK, you could shade your library and all its dependencies to get away from it, if you want to deal with the bloat / impact to the JIT'er). Having said all that, your statements are arguing against metaprogramming, and I'm not really arguing for or against metaprogramming in my original comment. Simply, I think the pattern of defining HTTP resources in the aforementioned way is very intuitive from a software development perspective. If it can be done without metaprogramming, then great. But the presence of metaprogramming doesn't, in my mind, preclude the effectiveness of the pattern.
- strictfp 10y ago
  Ok, I hear you. I agree somewhat, but not totally :) To me, the most important aspect of programming is to retain full control. I for instance want to be able to register my HTTP endpoints based on some data read from a file. I want to be able to start my app in server mode and run requests against it in any way I want during testing. What many of these frameworks do is to remove this power. Bytecode modification is not too bad IMO, as long as it's within reason. I use a notnull annotation weaver to add notnull assertions. Not Lombok, that's too much for me as well. Should be another language really, like Kotlin. Metaprogramming does have utility. But it has a tendency to go overboard. I actually find Jersey to be a good example of a nice and clean API for defining resources. I recently tried to integrate it into a small app where we do all the wiring manually for clarity. The problem is that it's hard to stay in programmatic control. You define a resource. You realize you need a Service in your resource. You realize that the only way of getting it into your object is to inject it with the JAX-RS annotations. And so suddenly your service has to be a JAX-RS @Singleton service. You have to make your whole app JAX-RS compatible. The framework takes control. This is really bad IMO. A library which requires control over your program. This is not an intrinsic problem of metaprogramming, but IMO it's a big problem of Java as it stands today. We need to give up on frameworks, make sure that all libraries are combinable and that they don't take control of your app. Doing so would make everything a lot better. I wish I could just do: new Jersey(config).addRoutesIn(new HelloWorldResource(new MyBackendService(dbDriver))).addSerializer(new JsonSerializer()).start(); This would then utilize some reflection when routing requests, but not take over my entire app. Regarding the libraries, my point was not that you get conflicting classes (DLL hell sort of thing), but that we're building multiple products which want to take over the world. JUnit wants to be in control, and so does Spring. So when you try to combine them, you end up with a problem. Can you start Spring programmatically from a JUnit test? Well, not really. You can, but it's very involved. So they want to provide you with a Spring-JUnit integration. This is a really bad idea, since every linear combination of libs has to get an integration, like the link I included in my post. If all these libraries just tried to stay out of the way of the programmer and just provide a programmatic API, there would be no need for such integrations.