6 ms·

> Inlining functions also has the benefit of not making it possible to call the function from other places. I’ve really gone to town with this in Python. de

> Inlining functions also has the benefit of not making it possible to call the function from other places. I’ve really gone to town with this in Python. def parse_news_email(…): def parse_link(…): … def parse_subjet(…): … … If you are careful, you can rely on the outer function’s variables being available inside the inner functions as well. Something like a logger or a db connection can be passed in once and then used without having to pass it as an argument all the time: # sad def f1(x, db, logger): … def f2(x, db, logger): … def f3(x, db, logger): … def g(xs, db, logger): for x0 in xs: x1 = f1(x0, db, logger) x2 = f2(x1, db, logger) x3 = f3(x2, db, logger) yikes x3 # happy def g(xs, db, logger): def f1(x): … def f2(x): … def f3(x): … for x in xs: yield f3(f2(f1(x))) Carmack commented his inline functions as if they were actual functions. Making actual functions enforces this :) Classes and “constants” can also quite happily live inside a function but those are a bit more jarring to see, and classes usually need to be visible so they can be referred to by the type annotations.

eru 2y ago
Funny enough, the equivalent of your Python example is how Haskell 'fakes' all functions with more than one argument (at least by default). Imperative blocks of code in Haskell (do-notation) also work like this.
InDubioProRubio 2y ago
So inlining is the private of functions without a object. Pop it all to stack, add arguments, set functionpointer to instructionstart of inline code, challenge accepted, lets go to..
bitwize 2y ago
It's possible to nest subprograms within subprograms in Ada. I take advantage of this ability to break a large operation into one or more smaller simpler "core" operations, and then in the main body of the procedure write some setup code followed by calls to the core operation(s).
raverbashing 2y ago
It might be a benefit in some cases, but I do feel that f1/f2/f3 are the prime candidates for actual unit testing
toenail 2y ago
> Inlining functions also has the benefit of not making it possible to call the function from other places. Congrats, you've got an untestable unit.
- xboxnolifes 2y ago
  The unit here is the email, not the email's link or subjects. Those are implementation details.
  - mgsouth 2y ago
    What do you use unit tests for, other than verifying implementation details? Perhaps we have a difference in definition. To me, a unit test for a function such as "parse_news_email" would explore variations in parameters and states. Because of combinatorial explosion, that often means at least some white-box testing. I'm not going to generate random subjects and senders, and received-froms, I'm going to target based on internal details. Are we doing smart things with the message ID hostname? Then what happens if two messages come in with the same message ID but from different relays? The objective is that the unit test wrings out the implementation details, and the caller's unit test doesn't need to exhaustively test them. This white-box texting may require directly poking at or mocking internal functions or at least abusing how they're called. For example, parsing the news item might entail pulling up and modifying conversation thread cache entries or state. For some of the tests you may need hand-crafted cache state, it's not feasible to create unique states for each parameter combination you're testing, and testing a combination will pollute the state for the following combinations. Or maybe the function depends upon an external resource you can't beat to death with a million identical requests. So the least-bad, simplest solution may be to freeze or back out part of the normal state update in the unit test. Which would usually involve directly invoking the internal routines. Can this lead to fragile, false-positve to the point of useless tests? You betcha. That's where entertaining two contrary viewpoints is needed :) Use experience and good judgement about pros and cons in the particular situation.
    - randomdata 2y ago
scbrg 2y ago
Not sure I believe the benefit of this approach outweighs the added difficulty wrt testing, but I certainly agree that Python needs a yikes keyword :-)
- nuancebydefault 2y ago
  What is the benefit of such a yikes? Or do you consider it a yikes language as a whole? Personally I like that functions can be inside functions, as a trade off between inlining and functional seperation in C++. The scope reduction makes it easier to track bugs while it has the benefits of separation of concern.
  - scbrg 2y ago
    > What is the benefit of such a yikes? Or do you consider it a yikes language as a whole? None, it was just a simple joke based on the typo in the post I replied to. I like Python, and have in fact been happily using it as my main language for over 20 years.
    - gorgoiler 2y ago
      Ahhh, now I (top level author) get it :)
wraptile 2y ago
The latter pattern is very popular in Python web scraping and data parsing niches as the code is quite verbose and specific and I'm very happy with this approach. Easy to read and debug and the maintenance is naturally organized.
grumbel 2y ago
That's not an improvement, as it screws up the code flow. The point of inline blocks is that you can read the code the same way as it is executed. No surprised that code might be called twice or that a function call could be missed or reordered. Adding real functions causes exactly the indirection that one wanted to avoid in the first place. If the block has no name you know that it will only be executed right where it is written.
- gorgoiler 2y ago
  Yeah that’s a valid point. I tend to have in mind that as soon as I pull any of the inner functions out to the publicly visible module level I can say goodbye to ever trying to stop people reusing the code when I don’t really want them to. For example, if your function has an implicit, undocumented contract such as assuming the DB is only a few milliseconds away, but they then reuse the code for logging to DBs over the internet, then they find it’s slow and speed it up with caching. Now your DB writing code has to suffer their cache logic bugs when it didn’t have to.
a_t48 2y ago
You can do this in C++, too, but the syntax is a little uglier.
- kllrnohj 2y ago
  Not that bad? int main() { int a = -1; [&] { a = 42; printf("I'm an uncallable inline block"); }(); printf(" "); [&] { printf("of code\n"); }(); [&] { printf("Passing state: %d\n", a); }(); return 0; }
  - a_t48 2y ago
    It’s not horrible, a little bit verbose though.
  - int_19h 2y ago
    At this point, why wouldn't you just use a nested block?
LoganDark 2y ago
Remember to `nonlocal xs, db, logger` inside those inner functions. I'm not sure if this is needed for variables that are only read, but I wouldn't ever leave it out.
- pansa2 2y ago
  > I'm not sure if this is needed for variables that are only read It’s not needed. In fact, you should leave it out for read-only variables. That’s standard practice - if you use `nonlocal` people reading the code will expect to see writes to the variables.
  - LoganDark 2y ago
    > That’s standard practice - if you use `nonlocal` people reading the code will expect to see writes to the variables. Since when? I was under the impression Python virtually doesn't have lexical scoping at all and that's why `nonlocal` exists. I mean hell, in CPython you can literally access and modify the local variables of your caller (and everything else up the call stack too). I never associated `nonlocal` at all with specifically writes. Just access in general.
    - pansa2 2y ago
      > I was under the impression Python virtually doesn't have lexical scoping at all and that's why `nonlocal` exists Python has had lexical scoping since version 2.2. PEP 227 [0] "describes the addition of statically nested scoping (lexical scoping)" - allowing access to (but not assignment to) names in outer scopes. `nonlocal` was introduced later, in Python 3.0 [1], specifically to allow assignment to names in outer scopes. [0] https://peps.python.org/pep-0227/ https://peps.python.org/pep-0227/ [1] https://peps.python.org/pep-3104/ https://peps.python.org/pep-3104/
orf 2y ago
That’s gonna be quite expensive, don’t do this in hot loops. You’re re-defining and re-creating the function object each time the outer function is called.
- gorgoiler 2y ago
  Good point. I measured it for 10^6 loops: (1) 40ms for inline code; (2) 150ms for an inner function with one expression; (3) 200ms for a slightly more complex inner function; and (4) 4000ms+ for an inner function and an inner class. def f1(n: int) -> int: return n * 2 def f2(n: int) -> int: def g(): return n * 2 return g() def f3(n: int) -> int: def g(): for _ in range(0): try: pass except Exception as exc: if isinstance(exc, 1): pass else: while True: pass raise Exception() return n * 2 return g() def f4(n: int) -> int: class X: def __init__(self, a, b, c): pass def _(self) -> float: return 1.23 def g(): for _ in range(0): try: pass except Exception as exc: if isinstance(exc, 1): pass else: while True: pass raise Exception() return n * 2 return g()
zelphirkalt 2y ago
Where is the part, where this is "careful"? This is just how scopes work. I don't see what is special about the inner functions using things in the scope of the outer functions.
- int_19h 2y ago
  Excessive use of external bindings in a closure can make it hard to reason about lifetimes in cases where that matters (e.g. when you find out that a huge object graph is alive solely because some callback somewhere is a lambda that closed over one of the objects in said graph).

eru 2y ago

Funny enough, the equivalent of your Python example is how Haskell 'fakes' all functions with more than one argument (at least by default). Imperative blocks of code in Haskell (do-notation) also work like this.

InDubioProRubio 2y ago

So inlining is the private of functions without a object. Pop it all to stack, add arguments, set functionpointer to instructionstart of inline code, challenge accepted, lets go to..

bitwize 2y ago

It's possible to nest subprograms within subprograms in Ada. I take advantage of this ability to break a large operation into one or more smaller simpler "core" operations, and then in the main body of the procedure write some setup code followed by calls to the core operation(s).

raverbashing 2y ago

It might be a benefit in some cases, but I do feel that f1/f2/f3 are the prime candidates for actual unit testing

toenail 2y ago

> Inlining functions also has the benefit of not making it possible to call the function from other places. Congrats, you've got an untestable unit.

xboxnolifes 2y ago
The unit here is the email, not the email's link or subjects. Those are implementation details.
- mgsouth 2y ago
  What do you use unit tests for, other than verifying implementation details? Perhaps we have a difference in definition. To me, a unit test for a function such as "parse_news_email" would explore variations in parameters and states. Because of combinatorial explosion, that often means at least some white-box testing. I'm not going to generate random subjects and senders, and received-froms, I'm going to target based on internal details. Are we doing smart things with the message ID hostname? Then what happens if two messages come in with the same message ID but from different relays? The objective is that the unit test wrings out the implementation details, and the caller's unit test doesn't need to exhaustively test them. This white-box texting may require directly poking at or mocking internal functions or at least abusing how they're called. For example, parsing the news item might entail pulling up and modifying conversation thread cache entries or state. For some of the tests you may need hand-crafted cache state, it's not feasible to create unique states for each parameter combination you're testing, and testing a combination will pollute the state for the following combinations. Or maybe the function depends upon an external resource you can't beat to death with a million identical requests. So the least-bad, simplest solution may be to freeze or back out part of the normal state update in the unit test. Which would usually involve directly invoking the internal routines. Can this lead to fragile, false-positve to the point of useless tests? You betcha. That's where entertaining two contrary viewpoints is needed :) Use experience and good judgement about pros and cons in the particular situation.
  - randomdata 2y ago

scbrg 2y ago

Not sure I believe the benefit of this approach outweighs the added difficulty wrt testing, but I certainly agree that Python needs a yikes keyword :-)

nuancebydefault 2y ago
What is the benefit of such a yikes? Or do you consider it a yikes language as a whole? Personally I like that functions can be inside functions, as a trade off between inlining and functional seperation in C++. The scope reduction makes it easier to track bugs while it has the benefits of separation of concern.
- scbrg 2y ago
  > What is the benefit of such a yikes? Or do you consider it a yikes language as a whole? None, it was just a simple joke based on the typo in the post I replied to. I like Python, and have in fact been happily using it as my main language for over 20 years.
  - gorgoiler 2y ago
    Ahhh, now I (top level author) get it :)

wraptile 2y ago

The latter pattern is very popular in Python web scraping and data parsing niches as the code is quite verbose and specific and I'm very happy with this approach. Easy to read and debug and the maintenance is naturally organized.

grumbel 2y ago

That's not an improvement, as it screws up the code flow. The point of inline blocks is that you can read the code the same way as it is executed. No surprised that code might be called twice or that a function call could be missed or reordered. Adding real functions causes exactly the indirection that one wanted to avoid in the first place. If the block has no name you know that it will only be executed right where it is written.

gorgoiler 2y ago
Yeah that’s a valid point. I tend to have in mind that as soon as I pull any of the inner functions out to the publicly visible module level I can say goodbye to ever trying to stop people reusing the code when I don’t really want them to. For example, if your function has an implicit, undocumented contract such as assuming the DB is only a few milliseconds away, but they then reuse the code for logging to DBs over the internet, then they find it’s slow and speed it up with caching. Now your DB writing code has to suffer their cache logic bugs when it didn’t have to.

a_t48 2y ago

You can do this in C++, too, but the syntax is a little uglier.

kllrnohj 2y ago
Not that bad? int main() { int a = -1; [&] { a = 42; printf("I'm an uncallable inline block"); }(); printf(" "); [&] { printf("of code\n"); }(); [&] { printf("Passing state: %d\n", a); }(); return 0; }
- a_t48 2y ago
  It’s not horrible, a little bit verbose though.
- int_19h 2y ago
  At this point, why wouldn't you just use a nested block?

LoganDark 2y ago

Remember to `nonlocal xs, db, logger` inside those inner functions. I'm not sure if this is needed for variables that are only read, but I wouldn't ever leave it out.

pansa2 2y ago
> I'm not sure if this is needed for variables that are only read It’s not needed. In fact, you should leave it out for read-only variables. That’s standard practice - if you use `nonlocal` people reading the code will expect to see writes to the variables.
- LoganDark 2y ago
  > That’s standard practice - if you use `nonlocal` people reading the code will expect to see writes to the variables. Since when? I was under the impression Python virtually doesn't have lexical scoping at all and that's why `nonlocal` exists. I mean hell, in CPython you can literally access and modify the local variables of your caller (and everything else up the call stack too). I never associated `nonlocal` at all with specifically writes. Just access in general.
  - pansa2 2y ago
    > I was under the impression Python virtually doesn't have lexical scoping at all and that's why `nonlocal` exists Python has had lexical scoping since version 2.2. PEP 227 [0] "describes the addition of statically nested scoping (lexical scoping)" - allowing access to (but not assignment to) names in outer scopes. `nonlocal` was introduced later, in Python 3.0 [1], specifically to allow assignment to names in outer scopes. [0] https://peps.python.org/pep-0227/ https://peps.python.org/pep-0227/ [1] https://peps.python.org/pep-3104/ https://peps.python.org/pep-3104/

orf 2y ago

That’s gonna be quite expensive, don’t do this in hot loops. You’re re-defining and re-creating the function object each time the outer function is called.

gorgoiler 2y ago
Good point. I measured it for 10^6 loops: (1) 40ms for inline code; (2) 150ms for an inner function with one expression; (3) 200ms for a slightly more complex inner function; and (4) 4000ms+ for an inner function and an inner class. def f1(n: int) -> int: return n * 2 def f2(n: int) -> int: def g(): return n * 2 return g() def f3(n: int) -> int: def g(): for _ in range(0): try: pass except Exception as exc: if isinstance(exc, 1): pass else: while True: pass raise Exception() return n * 2 return g() def f4(n: int) -> int: class X: def __init__(self, a, b, c): pass def _(self) -> float: return 1.23 def g(): for _ in range(0): try: pass except Exception as exc: if isinstance(exc, 1): pass else: while True: pass raise Exception() return n * 2 return g()

zelphirkalt 2y ago

Where is the part, where this is "careful"? This is just how scopes work. I don't see what is special about the inner functions using things in the scope of the outer functions.

int_19h 2y ago
Excessive use of external bindings in a closure can make it hard to reason about lifetimes in cases where that matters (e.g. when you find out that a huge object graph is alive solely because some callback somewhere is a lambda that closed over one of the objects in said graph).