4 ms·

I guess the point is that there are different levels of coders. For coders who can't write code that print a hundred random lines from a dictionary file, GPT-4

by jxy 3y ago

I guess the point is that there are different levels of coders. For coders who can't write code that print a hundred random lines from a dictionary file, GPT-4 is already a significant threat. On the other hand, even if you can write it, does it mean you should write it? How much time does it cost you to write it? How much money your company is paying you in that amount of time? Is it more economical for GPT-4 to do it? So I was curious and decided to ask the mistral-7b-instruct, and it gave me the following, > Here is an example of a Python script that reads a dictionary file and prints 100 random lines: ```python import random with open('dictionary.txt', 'r') as f: lines = f.readlines() random_lines = random.sample(lines, 100) for line in random_lines: print(line.strip()) ``` > This script uses the `random` module to select a random subset of the lines in the file. The `open` function is used to open the file and read its contents into a list of strings, which are then passed to the `random.sample` function to select 100 random lines. Finally, the selected lines are printed using a for loop and the `strip` method to remove any leading or trailing whitespace. > Note that this script assumes that the dictionary file is located in the same directory as the script and is named "dictionary.txt". You may need to modify the script to specify the correct path and filename for your dictionary file.

klipt 3y ago
Not bad, but suppose the dictionary is too big to load into memory - could AI suggest an online (reservoir sampling) algorithm as an alternative?
- coder543 3y ago
  Sure: https://chat.openai.com/share/0a3e52c6-1db8-422a-a98c-cb35006e5066 https://chat.openai.com/share/0a3e52c6-1db8-422a-a98c-cb3500... I laid out the constraints, but I did not mention reservoir sampling at all. The script seems to work as expected when I run it against a dictionary file.
  - klipt 3y ago
    Not bad, but suppose the dictionary has n lines and you only want to randomly sample k=100 of them, where n is so huge that you don't want to scan over the whole file at all. Can you use random access into the file to sample k lines in O(k) time instead of O(n) time?
    - coder543 3y ago
      That is a problematic request for multiple obvious reasons, and for those same reasons, ChatGPT resisted providing an implementation that didn't require indexing the file. By telling it "no indexing is allowed, provide a best effort solution" it relented and provided a best effort solution. Here is the provided solution and some discussion of the problems with the problem itself: https://chat.openai.com/share/54807663-17ca-4e7d-bc76-cd3cf3abc176 https://chat.openai.com/share/54807663-17ca-4e7d-bc76-cd3cf3...
      - klipt 3y ago
        > That is a problematic request for multiple obvious reasons I'd prefer to think it's more like a real engineering problem, and less like a simple interview question :-) And it definitely shows the limits of GPT here: it pointed out that the ends of the file might be tricky, but ignored the very conceptually simple solution of considering the file as circular (if you go past either end you simply wrap around). And it misses the real problem with its implementation: the probability of sampling each line is now directly proportional to the length of the line before it (because it seeks into that line first and then skips it!) So the word after "begins" is twice as likely to come up as the word after "and". PS in the case of dictionary words with a length limit of say 30 letters, there is still an O(k) general solution using rejection sampling.
        coder543 3y ago
        If you had actually read what it wrote: "Remember, this is a probabilistic approach and works well if the lines in your file are roughly the same length. If the line lengths vary significantly, some lines will have a higher or lower chance of being selected." It had already addressed "the real problem with its implementation" that you pointed out. > PS in the case of dictionary words with a length limit of say 30 letters, there is still an O(k) general solution using rejection sampling. Again, what ChatGPT wrote: "In a typical scenario where lines can have variable lengths, true O(k) random sampling isn't feasible without some prior knowledge about the file." Knowing that the limit is 30 characters without question counts as "some prior knowledge". As an interviewer, it sounds like you're not hearing what the candidate is saying. > And it definitely shows the limits of GPT here I don't think anyone here is claiming that ChatGPT is limitless. The topic is "a coder considers the waning days of the craft", not "a coder considers the bygone days of the craft." ChatGPT is capable of solving many real world problems already. If it continues improving, some people are concerned about what that could mean, especially for less experienced developers. How many people have you interviewed with that brainteaser that have actually provided the complete solution you're looking for? Vanishingly few, I would imagine, unless you were dropping some serious hints. It's not a real world problem. Most brainteasers have solutions that are "conceptually simple" once you already know the solution. > I'd prefer to think it's more like a real engineering problem, and less like a simple interview question It's absolutely not, though. It's exactly like the infamous trick questions that many tech interviews are known for, which have nothing to do with real engineering that you would encounter on the job. You might as well have someone invert a binary tree for all the value that it provides.
- louai 3y ago
  ChatGPT 4 > Use reservoir sampling to get 100 random lines from a dictionary that doesn’t fit into memory. Give me an implementation in Python. > Reservoir sampling is a great choice for randomly selecting a fixed number of items from a stream of unknown size, especially when the entire dataset cannot fit into memory. Here's a Python implementation for selecting 100 random lines from a large file, like a dictionary, using reservoir sampling: ```python import random def reservoir_sampling(file_name, k): reservoir = [] with open(file_name, 'r') as file: for i, line in enumerate(file): if i < k: # Fill the reservoir array reservoir.append(line.strip()) else: # Randomly replace elements in the reservoir # with a decreasing probability. # Choose an integer between 0 and i (inclusive) j = random.randint(0, i) if j < k: reservoir[j] = line.strip() return reservoir # Example usage file_path = 'path_to_your_large_file.txt' # Replace with your file path sampled_lines = reservoir_sampling(file_path, 100) print(sampled_lines) ``` > This script opens a file and reads through it line by line. For the first 100 lines, it adds them directly to the reservoir. For each subsequent line, it randomly decides whether to replace an existing line in the reservoir with the new line. The probability of replacing decreases as it progresses through the file, ensuring each line has an equal chance of being included in the final sample.
fiddlerwoaroof 3y ago
<dict shuf | head -n100
- qbrass 3y ago
  <dict shuf -n 100
  - coder543 3y ago
    Similarly: https://chat.openai.com/share/bc4a533e-c007-48ea-9232-0e548b72f882 https://chat.openai.com/share/bc4a533e-c007-48ea-9232-0e548b... The "<" is actually an unnecessary additional character to type.
    - fiddlerwoaroof 3y ago
      Except, I intentionally don’t use either form because they don’t extend nicely (and I dislike using flags when I could use another pipe segment or positional arguments when I could use standard input). I can iterate quickly by adding pipeline segments on top of the basic formula, because all of the segments have the same general shape and “calling convention”. Finally, because I’ve built up familiarity with the shell over my career, I can come up with this formula as fast as I can type it. At this point, ChatGPT would slow me down: (1) because this sort of thing is basically muscle memory and (2) I have to actually think about the code ChatGPT produces to verify it is correct, which is almost as difficult as producing it in the first place.

by jxy 3y ago

klipt 3y ago

Not bad, but suppose the dictionary is too big to load into memory - could AI suggest an online (reservoir sampling) algorithm as an alternative?

coder543 3y ago
Sure: https://chat.openai.com/share/0a3e52c6-1db8-422a-a98c-cb35006e5066 https://chat.openai.com/share/0a3e52c6-1db8-422a-a98c-cb3500... I laid out the constraints, but I did not mention reservoir sampling at all. The script seems to work as expected when I run it against a dictionary file.
- klipt 3y ago
  Not bad, but suppose the dictionary has n lines and you only want to randomly sample k=100 of them, where n is so huge that you don't want to scan over the whole file at all. Can you use random access into the file to sample k lines in O(k) time instead of O(n) time?
  - coder543 3y ago
    That is a problematic request for multiple obvious reasons, and for those same reasons, ChatGPT resisted providing an implementation that didn't require indexing the file. By telling it "no indexing is allowed, provide a best effort solution" it relented and provided a best effort solution. Here is the provided solution and some discussion of the problems with the problem itself: https://chat.openai.com/share/54807663-17ca-4e7d-bc76-cd3cf3abc176 https://chat.openai.com/share/54807663-17ca-4e7d-bc76-cd3cf3...
    - klipt 3y ago
      > That is a problematic request for multiple obvious reasons I'd prefer to think it's more like a real engineering problem, and less like a simple interview question :-) And it definitely shows the limits of GPT here: it pointed out that the ends of the file might be tricky, but ignored the very conceptually simple solution of considering the file as circular (if you go past either end you simply wrap around). And it misses the real problem with its implementation: the probability of sampling each line is now directly proportional to the length of the line before it (because it seeks into that line first and then skips it!) So the word after "begins" is twice as likely to come up as the word after "and". PS in the case of dictionary words with a length limit of say 30 letters, there is still an O(k) general solution using rejection sampling.
      - coder543 3y ago
        If you had actually read what it wrote: "Remember, this is a probabilistic approach and works well if the lines in your file are roughly the same length. If the line lengths vary significantly, some lines will have a higher or lower chance of being selected." It had already addressed "the real problem with its implementation" that you pointed out. > PS in the case of dictionary words with a length limit of say 30 letters, there is still an O(k) general solution using rejection sampling. Again, what ChatGPT wrote: "In a typical scenario where lines can have variable lengths, true O(k) random sampling isn't feasible without some prior knowledge about the file." Knowing that the limit is 30 characters without question counts as "some prior knowledge". As an interviewer, it sounds like you're not hearing what the candidate is saying. > And it definitely shows the limits of GPT here I don't think anyone here is claiming that ChatGPT is limitless. The topic is "a coder considers the waning days of the craft", not "a coder considers the bygone days of the craft." ChatGPT is capable of solving many real world problems already. If it continues improving, some people are concerned about what that could mean, especially for less experienced developers. How many people have you interviewed with that brainteaser that have actually provided the complete solution you're looking for? Vanishingly few, I would imagine, unless you were dropping some serious hints. It's not a real world problem. Most brainteasers have solutions that are "conceptually simple" once you already know the solution. > I'd prefer to think it's more like a real engineering problem, and less like a simple interview question It's absolutely not, though. It's exactly like the infamous trick questions that many tech interviews are known for, which have nothing to do with real engineering that you would encounter on the job. You might as well have someone invert a binary tree for all the value that it provides.
louai 3y ago
ChatGPT 4 > Use reservoir sampling to get 100 random lines from a dictionary that doesn’t fit into memory. Give me an implementation in Python. > Reservoir sampling is a great choice for randomly selecting a fixed number of items from a stream of unknown size, especially when the entire dataset cannot fit into memory. Here's a Python implementation for selecting 100 random lines from a large file, like a dictionary, using reservoir sampling: ```python import random def reservoir_sampling(file_name, k): reservoir = [] with open(file_name, 'r') as file: for i, line in enumerate(file): if i < k: # Fill the reservoir array reservoir.append(line.strip()) else: # Randomly replace elements in the reservoir # with a decreasing probability. # Choose an integer between 0 and i (inclusive) j = random.randint(0, i) if j < k: reservoir[j] = line.strip() return reservoir # Example usage file_path = 'path_to_your_large_file.txt' # Replace with your file path sampled_lines = reservoir_sampling(file_path, 100) print(sampled_lines) ``` > This script opens a file and reads through it line by line. For the first 100 lines, it adds them directly to the reservoir. For each subsequent line, it randomly decides whether to replace an existing line in the reservoir with the new line. The probability of replacing decreases as it progresses through the file, ensuring each line has an equal chance of being included in the final sample.

fiddlerwoaroof 3y ago

<dict shuf | head -n100

qbrass 3y ago
<dict shuf -n 100
- coder543 3y ago
  Similarly: https://chat.openai.com/share/bc4a533e-c007-48ea-9232-0e548b72f882 https://chat.openai.com/share/bc4a533e-c007-48ea-9232-0e548b... The "<" is actually an unnecessary additional character to type.
  - fiddlerwoaroof 3y ago
    Except, I intentionally don’t use either form because they don’t extend nicely (and I dislike using flags when I could use another pipe segment or positional arguments when I could use standard input). I can iterate quickly by adding pipeline segments on top of the basic formula, because all of the segments have the same general shape and “calling convention”. Finally, because I’ve built up familiarity with the shell over my career, I can come up with this formula as fast as I can type it. At this point, ChatGPT would slow me down: (1) because this sort of thing is basically muscle memory and (2) I have to actually think about the code ChatGPT produces to verify it is correct, which is almost as difficult as producing it in the first place.