7 ms·

I think reflection is "slow" is most languages unless you use a kind of preprocessor. Is there any exception?

camus2 10y ago
That's why macros like the Crystal implementation are often the way to go : https://crystal-lang.org/docs/syntax_and_semantics/macros.html https://crystal-lang.org/docs/syntax_and_semantics/macros.ht... It's a meta language that works at compile time. It's not just text substitution like with GCC, so the syntax can be validated. And no need for "go gen" like silly tricks. Languages that have reflection should do everything so that using reflection is the exception, not the norm. Some languages which boast about simplicity are often the worst offenders in practice, when it comes to reflection, just look at Go code out there, full of reflection everywhere, because the language is way too simplistic for a lot of use cases. Go even allows adhoc type definition at run time, out of thin air. Is this really what gophers mean by "simplicity" ? because the maintainers refuse to add any new syntax they now basically cram everything they need into the reflect package. Then dare say its use is "not idiomatic" when they abuse reflection themselves ?
- eropple 10y ago
  Crystal's implementation wouldn't work for something like C#, though, because C# supports dynamic assembly loading at runtime and must provide reflective facilities for operating on types loaded from them. (I once wrote an extensibility system[1] and on-the-fly compiler for a game engine that took advantage of this, it was neat.) Macros don't help you here and limiting yourself to "I can compile the whole blob at once" cuts off a lot of avenues of extensibility. [1] - https://github.com/eropple/exor https://github.com/eropple/exor
- themihai 10y ago
  >> because the maintainers refuse to add any new syntax they now basically cram everything they need into the reflect package. I think they want to try different avenues before to decide what features to add. Generators("go generate") are used a lot too but they are a pain to develop. Most of them still use templates or print statements instead of specilised packages such go/ast , go/types, go/parser etc to actually generate/print code. Usually you should use reflection to prototype and switch to generators only where the performance is really a big issue or where reflection can't help(i.e define new named types, import/use arbitrary packages/functions)
  - eropple 10y ago
    Alternatively, you build a type system that can express what developers actually need it to express and solve the problem without reflection at all. =)
    - themihai 10y ago
      That's easier said than done. There are tradeoffs on expressive type systems too. That being said I wouldn't mind more expressivity in the Go type system. I'm actually looking to implement my own small features(i.e optionals, variant types) in a Go superset.
      - camus2 10y ago
        Where do you think struct tags fit in all this ? do you really think go designers took the time to consider the tradeoffs of such feature ? I don't think so.
    - DrJokepu 10y ago
      Any sufficiently advanced type system is indistinguishable from magic.
      - camus2 10y ago
        But putting struct tags in Go was a good idea that promote simplicity ... I prefer the magic of generics or pattern matching or covariance to insane features like this, thank you.
spullara 10y ago
The JVM has very fast reflective invocations. Lookup is reasonable.
- matthewwarren 10y ago
  From what I understand, the JVM uses 'inflation' to make reflection faster. See https://blogs.oracle.com/buck/entry/inflation_system_properties https://blogs.oracle.com/buck/entry/inflation_system_propert... and http://stackoverflow.com/questions/10082523/java-what-is-jitcs-reflection-inflation http://stackoverflow.com/questions/10082523/java-what-is-jit... for more info
chrisseaton 10y ago
> I think reflection is "slow" is most languages unless you use a kind of preprocessor. Is there any exception? Yes! I wrote a PhD on making reflection in Ruby fast. It is possible to make most reflection operations run with no peak time performance overhead at all compared to normal operations. http://chrisseaton.com/phd/ http://chrisseaton.com/phd/ The main techniques needed are dynamic code compilation with speculation and deoptimisation, polymorphic inline caching, splitting and method inlining. Unfortunately I think the .NET JIT isn't dynamic (might be mistaken, not an expert on .NET) so can't do speculative optimisations. Basically you see what method names have been used last time in a reflective call, and create a little call that's more like a conventional call using those names. Each time you do the reflective call you check the name against the list of names you have created conventional calls for, and use them if you can. You need to make string comparison fast, which can be done using a rope data structure. Then you need to be able to remove the check entirely if the string comes from somewhere you can control, like a constant. Then you need to inline the reflection method so you are just left with the synthetic conventional call. So it isn't easy! Which is why it isn't usually done. But it's possible. In Ruby you need to make it fast no matter how hard it is, as Ruby libraries tend to use reflection (they call it metaprogramming) in inner-loop operations.
- Locke1689 10y ago
  > It is possible to make most reflection operations run with no peak time performance overhead at all compared to normal operations. Hmm... but my discussions with people who have worked on Ruby execution engines say that Ruby's normal operations are slow (because Ruby is constrained to do many slow things for normal invocations). How does Ruby reflection compare to .NET reflection and how does Ruby invocation compare to .NET invocation?
  - chrisseaton 10y ago
    I mean that for Ruby it is possible to compile at runtime a + b And a.send('+', b) Into the same thing - machine code like add ra, rb, rx jo Except for the jo to detect overflow, this is as good as you will get from any compiler including .NET, Java, C and so on. Ruby really isn't constrained to do anything slow for normal method invocations - you can optimise I think literally all of them away entirely if you have a JIT. For normal method invocations this all all solved back in the early 1990s in the research on making Smalltalk fast, but wasn't fully applied to Ruby until my work (and Topaz, Ruby on PyPy achieved similar results around the same time actually). I extended it to work with reflection (metaprogramming).
__old_dude__ 10y ago
Java 7 has introduced a very specific API (java.lang.invoke) to be able to inline reflection calls. So lookups are quite slow, but invocations are fully inlined. java.lang.invoke API is fast to the point that for doing faster string concatenation in Java 9, Aleksey Shipilev has used this API instead of introducing a new opcode in the JVM [1]. [1] https://youtu.be/wIyeOaitmWM?list=PL2ekzZZrxVUmBXtrwS7pLabUu8v-FCE8O&t=2278 https://youtu.be/wIyeOaitmWM?list=PL2ekzZZrxVUmBXtrwS7pLabUu...

That's why macros like the Crystal implementation are often the way to go : https://crystal-lang.org/docs/syntax_and_semantics/macros.html https://crystal-lang.org/docs/syntax_and_semantics/macros.ht... It's a meta language that works at compile time. It's not just text substitution like with GCC, so the syntax can be validated. And no need for "go gen" like silly tricks. Languages that have reflection should do everything so that using reflection is the exception, not the norm. Some languages which boast about simplicity are often the worst offenders in practice, when it comes to reflection, just look at Go code out there, full of reflection everywhere, because the language is way too simplistic for a lot of use cases. Go even allows adhoc type definition at run time, out of thin air. Is this really what gophers mean by "simplicity" ? because the maintainers refuse to add any new syntax they now basically cram everything they need into the reflect package. Then dare say its use is "not idiomatic" when they abuse reflection themselves ?

eropple 10y ago
Crystal's implementation wouldn't work for something like C#, though, because C# supports dynamic assembly loading at runtime and must provide reflective facilities for operating on types loaded from them. (I once wrote an extensibility system[1] and on-the-fly compiler for a game engine that took advantage of this, it was neat.) Macros don't help you here and limiting yourself to "I can compile the whole blob at once" cuts off a lot of avenues of extensibility. [1] - https://github.com/eropple/exor https://github.com/eropple/exor
themihai 10y ago
>> because the maintainers refuse to add any new syntax they now basically cram everything they need into the reflect package. I think they want to try different avenues before to decide what features to add. Generators("go generate") are used a lot too but they are a pain to develop. Most of them still use templates or print statements instead of specilised packages such go/ast , go/types, go/parser etc to actually generate/print code. Usually you should use reflection to prototype and switch to generators only where the performance is really a big issue or where reflection can't help(i.e define new named types, import/use arbitrary packages/functions)
- eropple 10y ago
  Alternatively, you build a type system that can express what developers actually need it to express and solve the problem without reflection at all. =)
  - themihai 10y ago
    That's easier said than done. There are tradeoffs on expressive type systems too. That being said I wouldn't mind more expressivity in the Go type system. I'm actually looking to implement my own small features(i.e optionals, variant types) in a Go superset.
    - camus2 10y ago
      Where do you think struct tags fit in all this ? do you really think go designers took the time to consider the tradeoffs of such feature ? I don't think so.
  - DrJokepu 10y ago
    Any sufficiently advanced type system is indistinguishable from magic.
    - camus2 10y ago
      But putting struct tags in Go was a good idea that promote simplicity ... I prefer the magic of generics or pattern matching or covariance to insane features like this, thank you.

spullara 10y ago

The JVM has very fast reflective invocations. Lookup is reasonable.

matthewwarren 10y ago
From what I understand, the JVM uses 'inflation' to make reflection faster. See https://blogs.oracle.com/buck/entry/inflation_system_properties https://blogs.oracle.com/buck/entry/inflation_system_propert... and http://stackoverflow.com/questions/10082523/java-what-is-jitcs-reflection-inflation http://stackoverflow.com/questions/10082523/java-what-is-jit... for more info

chrisseaton 10y ago

> I think reflection is "slow" is most languages unless you use a kind of preprocessor. Is there any exception? Yes! I wrote a PhD on making reflection in Ruby fast. It is possible to make most reflection operations run with no peak time performance overhead at all compared to normal operations. http://chrisseaton.com/phd/ http://chrisseaton.com/phd/ The main techniques needed are dynamic code compilation with speculation and deoptimisation, polymorphic inline caching, splitting and method inlining. Unfortunately I think the .NET JIT isn't dynamic (might be mistaken, not an expert on .NET) so can't do speculative optimisations. Basically you see what method names have been used last time in a reflective call, and create a little call that's more like a conventional call using those names. Each time you do the reflective call you check the name against the list of names you have created conventional calls for, and use them if you can. You need to make string comparison fast, which can be done using a rope data structure. Then you need to be able to remove the check entirely if the string comes from somewhere you can control, like a constant. Then you need to inline the reflection method so you are just left with the synthetic conventional call. So it isn't easy! Which is why it isn't usually done. But it's possible. In Ruby you need to make it fast no matter how hard it is, as Ruby libraries tend to use reflection (they call it metaprogramming) in inner-loop operations.

Locke1689 10y ago
> It is possible to make most reflection operations run with no peak time performance overhead at all compared to normal operations. Hmm... but my discussions with people who have worked on Ruby execution engines say that Ruby's normal operations are slow (because Ruby is constrained to do many slow things for normal invocations). How does Ruby reflection compare to .NET reflection and how does Ruby invocation compare to .NET invocation?
- chrisseaton 10y ago
  I mean that for Ruby it is possible to compile at runtime a + b And a.send('+', b) Into the same thing - machine code like add ra, rb, rx jo Except for the jo to detect overflow, this is as good as you will get from any compiler including .NET, Java, C and so on. Ruby really isn't constrained to do anything slow for normal method invocations - you can optimise I think literally all of them away entirely if you have a JIT. For normal method invocations this all all solved back in the early 1990s in the research on making Smalltalk fast, but wasn't fully applied to Ruby until my work (and Topaz, Ruby on PyPy achieved similar results around the same time actually). I extended it to work with reflection (metaprogramming).

__old_dude__ 10y ago

Java 7 has introduced a very specific API (java.lang.invoke) to be able to inline reflection calls. So lookups are quite slow, but invocations are fully inlined. java.lang.invoke API is fast to the point that for doing faster string concatenation in Java 9, Aleksey Shipilev has used this API instead of introducing a new opcode in the JVM [1]. [1] https://youtu.be/wIyeOaitmWM?list=PL2ekzZZrxVUmBXtrwS7pLabUu8v-FCE8O&t=2278 https://youtu.be/wIyeOaitmWM?list=PL2ekzZZrxVUmBXtrwS7pLabUu...