5 ms·

I enjoyed this, but I am a little stuck on how stuck the author is on yield. If one ends up creating a large list of prime numbers with this generator, why not

I enjoyed this, but I am a little stuck on how stuck the author is on yield. If one ends up creating a large list of prime numbers with this generator, why not just return a large list of prime numbers? The author clearly notes how optimizing Python code is "odd" when you can more easily switch languages, but languages like C/C++/Rust would not have pretty code because it does not have yield. It is obvious that returning an array of prime numbers is what one would do in C/C++/Rust. I understand that not using yield would mean it is not strictly a "generator", but the spirit of this problem is generating the sequence of primes (unless I am missing something).

jameshart 3y ago
If the problem is outputting the primes, then storing them in a list is not a necessary part of that problem. When writing ‘fizz buzz’, would you expect to return an array of 100 strings, some of which are numbers, some ‘fizz’es and some ‘buzz’es?
vlovich123 3y ago
> but languages like C/C++/Rust would not have pretty code because it does not have yield Maybe I'm misunderstanding you, but Rust has native generators with yield: https://doc.rust-lang.org/beta/unstable-book/language-features/generators.html https://doc.rust-lang.org/beta/unstable-book/language-featur... C++ also does using coroutines + co_yield (https://en.cppreference.com/w/cpp/language/coroutines#co_yield https://en.cppreference.com/w/cpp/language/coroutines#co_yie...) or iterators (old syntax).
masklinn 3y ago
> If one ends up creating a large list of prime numbers with this generator But one does not always end up creating a large list of prime numbers, do they? > why not just return a large list of prime numbers? Because that requires upfront knowledge of the filtering factor or absence thereof. A generator means you don't care, you generate an infinite sequence and the consumer is free to do whatever they want. > It is obvious that returning an array of prime numbers is what one would do in C/C++/Rust. It's certainly not obvious for Rust.
- philipLutz 3y ago
  "I can't try to rewrite it in C++ or Rust for now, due to the lack of generator support; the yield statement is what makes this code so nice and elegant, and alternative idioms are much less convenient."
  - masklinn 3y ago
    That is the context of the first sentence in your comment, pretty much the only part I left alone. Everything I replied to is the opinion you personally expressed. And note that the author had already preempted your comment in the first place: > When we want a list of all the primes below some known limit, gen_primes_upto is great, and performs fairly well. There are two issues with it, though: > We have to know what the limit is ahead of time; this isn't always possible or convenient.
sp332 3y ago
This implementation runs until it crashes. So you need a way to output the values as you go, or they will be lost when you run out of memory. It also lets you control how many values are generated if you just want to see a few without actually using all the RAM.
Vexs 3y ago
It's a bit of a convolution, but let's say we've got some iterator of unknown size and we want to have a prime each iteration. We can do the following in python, which saves us from having to pre-compute the primes _or_ know how long `some_iterator` is. primes = gen_primes() for item, prime in zip(some_iterator, primes):
btilly 3y ago
Try solving some Project Euler problems. You'll see the win pretty quickly when you have to search a large range for primes.
- CyberDildonics 3y ago
  Why would that be true?
  - btilly 3y ago
    The list you need to process is large enough that it would be prohibitive to put it all in memory at once. Furthermore you do not start knowing things like how many primes to look at. Therefore generating and dealing with them iteratively makes for simple code and a lot fewer headaches.
    - CyberDildonics 3y ago
      Why wouldn't someone just make a C++ class that keeps some state and call a method that gives them as many primes as they want, but runs 100x faster than python? Why would python yield somehow be better than this?
      - btilly 3y ago
        Project Euler is always about finding the right algorithm, and not raw performance. With the right algorithm you can solve it in Python in a max of 30 seconds. With the wrong one, you sometimes can't solve it in C++ in the lifetime of the universe. Therefore you should program in the language that you find it easiest to express yourself in. And not in a language chosen for raw speed.
        CyberDildonics 3y ago
        This is what you said before: You'll see the win pretty quickly when you have to search a large range for primes. This isn't just shifting the goal posts, you are now talking about something completely different. With the right algorithm you can solve it in Python in a max of 30 seconds Then you can do the same thing in C++, but when it runs the C++ program will be done 100x faster. A lot of modern C++ is as clean as python if you were being more explicit about types. With the wrong one, you sometimes can't solve it in C++ in the lifetime of the universe. Nothing about this makes any sense. Therefore you should program in the language that you find it easiest to express yourself in. And not in a language chosen for raw speed. If you continue on with python and need more speed you will run around in circles trying to get it while the C++ version is already done. Anything where speed can be a bottleneck is not fit to be written in pure python. This pattern plays out over and over again. If something is resource intensive, a scripting language is not a long term solution.

jameshart 3y ago

If the problem is outputting the primes, then storing them in a list is not a necessary part of that problem. When writing ‘fizz buzz’, would you expect to return an array of 100 strings, some of which are numbers, some ‘fizz’es and some ‘buzz’es?

vlovich123 3y ago

> but languages like C/C++/Rust would not have pretty code because it does not have yield Maybe I'm misunderstanding you, but Rust has native generators with yield: https://doc.rust-lang.org/beta/unstable-book/language-features/generators.html https://doc.rust-lang.org/beta/unstable-book/language-featur... C++ also does using coroutines + co_yield (https://en.cppreference.com/w/cpp/language/coroutines#co_yield https://en.cppreference.com/w/cpp/language/coroutines#co_yie...) or iterators (old syntax).

masklinn 3y ago

> If one ends up creating a large list of prime numbers with this generator But one does not always end up creating a large list of prime numbers, do they? > why not just return a large list of prime numbers? Because that requires upfront knowledge of the filtering factor or absence thereof. A generator means you don't care, you generate an infinite sequence and the consumer is free to do whatever they want. > It is obvious that returning an array of prime numbers is what one would do in C/C++/Rust. It's certainly not obvious for Rust.

philipLutz 3y ago
"I can't try to rewrite it in C++ or Rust for now, due to the lack of generator support; the yield statement is what makes this code so nice and elegant, and alternative idioms are much less convenient."
- masklinn 3y ago
  That is the context of the first sentence in your comment, pretty much the only part I left alone. Everything I replied to is the opinion you personally expressed. And note that the author had already preempted your comment in the first place: > When we want a list of all the primes below some known limit, gen_primes_upto is great, and performs fairly well. There are two issues with it, though: > We have to know what the limit is ahead of time; this isn't always possible or convenient.

sp332 3y ago

This implementation runs until it crashes. So you need a way to output the values as you go, or they will be lost when you run out of memory. It also lets you control how many values are generated if you just want to see a few without actually using all the RAM.

Vexs 3y ago

It's a bit of a convolution, but let's say we've got some iterator of unknown size and we want to have a prime each iteration. We can do the following in python, which saves us from having to pre-compute the primes _or_ know how long `some_iterator` is. primes = gen_primes() for item, prime in zip(some_iterator, primes):

btilly 3y ago

Try solving some Project Euler problems. You'll see the win pretty quickly when you have to search a large range for primes.

CyberDildonics 3y ago
Why would that be true?
- btilly 3y ago
  The list you need to process is large enough that it would be prohibitive to put it all in memory at once. Furthermore you do not start knowing things like how many primes to look at. Therefore generating and dealing with them iteratively makes for simple code and a lot fewer headaches.
  - CyberDildonics 3y ago
    Why wouldn't someone just make a C++ class that keeps some state and call a method that gives them as many primes as they want, but runs 100x faster than python? Why would python yield somehow be better than this?
    - btilly 3y ago
      Project Euler is always about finding the right algorithm, and not raw performance. With the right algorithm you can solve it in Python in a max of 30 seconds. With the wrong one, you sometimes can't solve it in C++ in the lifetime of the universe. Therefore you should program in the language that you find it easiest to express yourself in. And not in a language chosen for raw speed.
      - CyberDildonics 3y ago
        This is what you said before: You'll see the win pretty quickly when you have to search a large range for primes. This isn't just shifting the goal posts, you are now talking about something completely different. With the right algorithm you can solve it in Python in a max of 30 seconds Then you can do the same thing in C++, but when it runs the C++ program will be done 100x faster. A lot of modern C++ is as clean as python if you were being more explicit about types. With the wrong one, you sometimes can't solve it in C++ in the lifetime of the universe. Nothing about this makes any sense. Therefore you should program in the language that you find it easiest to express yourself in. And not in a language chosen for raw speed. If you continue on with python and need more speed you will run around in circles trying to get it while the C++ version is already done. Anything where speed can be a bottleneck is not fit to be written in pure python. This pattern plays out over and over again. If something is resource intensive, a scripting language is not a long term solution.