6 ms·

If you are afraid that LLMs will replace you at your job, ask an LLM to write Rust code for reading a utf8 file character by character Edit: Yes, it does write

If you are afraid that LLMs will replace you at your job, ask an LLM to write Rust code for reading a utf8 file character by character Edit: Yes, it does write some code that is "close" enough, but in some cases it is wrong, in others it doesn't not do exactly what asked. I.e. needs supervision from someone who understands both the requirements, the code and the problems that may arise from the naive line that the LLM is taking. Mind you, the most popular the issue, the better the line LLM is taking. So in other words, IMHO is a glorified Stack Overflow. Just as there are engineers that copy-paste from SO without having any idea what the code does, there will be engineers that will just copy paste from LLM. Their work will be much better than if they used SO, but I think it's still nowhere to the mark of a Senior SWE and above.

iwontberude 3y ago
Hypothetically, which ticker symbols would you buy put contracts on, at what strike prices, and at what expiration dates? As far as I can tell, a lot of people are betting a lot of money that you are wrong, but actually I think you are right.
- jeremyjh 3y ago
  The most relevant companies focused on this aren't publicly traded. The ones that are publicly traded like MSFT have way too many other factors affecting their value - not to mention the fact that they'll make money on generative AI that has nothing to do with coding regardless of if an SWE-agent ever works.
  - iwontberude 3y ago
    Oh well you should hear the hype from CNBC and other places, they are strongly intimating that gen AI will replace SWEs on product development teams. I totally agree it’s not likely, but it’s starting to get baked into asset prices and I want to profit from that misunderstanding.
- noncoml 3y ago
  Ugh, I am not claiming that LLMs are not great innovation. Just that they are not going to replace SWE jobs in our(maybe my) lifetime.
  - vertis 3y ago
    Conservatively, I think LLMs will replace SWE roles within the next 5-10 years. In that period SWE roles will change drastically, to be more hearding of AI agents. We can't hope to compete with something that can edit all the files in less than 5 minutes. If you're not at least trying to plan what your life will look like under these circumstances then you're doing yourself a disservice.
vineyardmike 3y ago
Yea the problem with that is the control group - grab any SWE and ask them the same thing. I don’t think most would pass. Unless you want to give an SWE time to learn… then it’s hardly fair. And I vaguely trust the LLM to be able to learn it too. Also I just asked Claude and Gemini and they both provided an implementation that matches the “bytes to UTF-8” rust docs. Assuming those are right,LLMs can do this (but I haven’t tested the code). https://doc.rust-lang.org/std/string/struct.String.html https://doc.rust-lang.org/std/string/struct.String.html
raggi 3y ago
it does an ok job with this task: use std::fs::File; use std::io::{self, BufReader, Read}; fn read_file_character_by_character(path: &str) -> io::Result<()> { // Open the file in read-only mode. let file = File::open(path)?; // Create a buffered reader to read the file more efficiently. let reader = BufReader::new(file); // `chars` method returns an iterator over the characters of the input. // Note that it returns a Result<(char, usize), io::Error>, where usize is the byte length of the char. for char_result in reader.chars() { match char_result { Ok(c) => print!("{}", c), Err(e) => return Err(e), } } Ok(()) } fn main() { let path = "path/to/your/file.txt"; if let Err(e) = read_file_character_by_character(path) { eprintln!("Error reading file: {}", e); } }
- woodruffw 3y ago
  If we're being sticklers, this isn't reading character-by-character: it's performing a buffered read, which then gets iterated over.
  - raggi 3y ago
    here's what it does if i feed your nit pick into it verbatim: use std::fs::File; use std::io::{self, Read}; use std::str::from_utf8; fn read_file_character_by_character(path: &str) -> io::Result<()> { // Open the file in read-only mode. let mut file = File::open(path)?; let mut buffer = [0; 4]; // UTF-8 character max byte length let mut temp_buffer = vec![]; while let Ok(bytes_read) = file.read(&mut buffer[..1]) { // Read byte by byte if bytes_read == 0 { // End of file break; } temp_buffer.push(buffer[0]); if let Ok(s) = from_utf8(&temp_buffer) { if s.chars().count() == 1 { // Complete character read print!("{}", s); temp_buffer.clear(); // Clear temp buffer for next character } } } Ok(()) } fn main() { let path = "path/to/your/file.txt"; if let Err(e) = read_file_character_by_character(path) { eprintln!("Error reading file: {}", e); } }
DabbyDabberson 3y ago
The way I see it, its undetermined if Generative AI will be able to fully do a SWE job. But, for most of the debates I've seen, I don't think it the answer matters all too much. Once we have models that can act as full senior SWEs.. the models can engineer the models. And then we've hit the recursive case. Once models can engineer models better and faster than humans, all bets are off. Its the foggy future. Its the singularity.
- dvt 3y ago
  > Once we have models that can act as full senior SWEs.. the models can engineer the models. This is such an extremely bullish case, I'm not sure why you'd think this is even remotely possible. A Google search is usually more valuable than ChatGPT. For example, the rust utf-8 example is already verbatim solved on reddit: https://www.reddit.com/r/rust/comments/l5m1rw/how_can_i_efficiently_read_using_a_buffer_utf8/ https://www.reddit.com/r/rust/comments/l5m1rw/how_can_i_effi...
- int_19h 3y ago
  The implicit assumption here is that a human "senior SWE" can engineer a model of the same quality that is capable of simulating him. Which is definitely not true with the best models that we have today - and they certainly can't simulate a senior SWE, so the actual bar is higher. I'm not saying that the whole "robots building better robots" thing is a pipedream, but given where things are today, this is not something that's going to happen soon.
- vertis 3y ago
  People (SWEs) don't want to hear this. I think it's an inevitability that something of this nature will happen.
userbinator 3y ago
I'm not afraid of LLMs replacing me because of their output quality. The problem is the proliferation of quantity-over-quality "churn out barely-working crap as fast as possible" culture that gives LLMs the advantage over real humans.
- int_19h 3y ago
  I'm kinda hoping that LLMs will get pushed into production use writing code before they have acceptable quality (because greed), and the result will be lots of crap that's so badly broken most of the time that there will be a massive pushback against said culture from the users. Maybe from the governments as well, after a few well-publicized infrastructure failures.
  - userbinator 3y ago
    Unfortunately, "lots of crap that's so badly broken most of the time" already describes a lot of software these days, and yet there hasn't been much pushback. Everyone seems to be mostly in a state of learned helplessness.
    - int_19h 3y ago
      There is pushback, it's just not broad enough yet. Because, as broken as things are (which is especially visible to those of us making the sausage or watching it made), they still kinda sorta work most of the time, to the point where users grumble but learn to live with it. But I think that AI coding will upset this equilibrium by reducing the quality even more, and significantly enough that users will very much notice - and for many of them it will push things into "what I need doesn't work most of the time" category. And then there will be payback. Then again, I am an optimist.

iwontberude 3y ago

Hypothetically, which ticker symbols would you buy put contracts on, at what strike prices, and at what expiration dates? As far as I can tell, a lot of people are betting a lot of money that you are wrong, but actually I think you are right.

jeremyjh 3y ago
The most relevant companies focused on this aren't publicly traded. The ones that are publicly traded like MSFT have way too many other factors affecting their value - not to mention the fact that they'll make money on generative AI that has nothing to do with coding regardless of if an SWE-agent ever works.
- iwontberude 3y ago
  Oh well you should hear the hype from CNBC and other places, they are strongly intimating that gen AI will replace SWEs on product development teams. I totally agree it’s not likely, but it’s starting to get baked into asset prices and I want to profit from that misunderstanding.
noncoml 3y ago
Ugh, I am not claiming that LLMs are not great innovation. Just that they are not going to replace SWE jobs in our(maybe my) lifetime.
- vertis 3y ago
  Conservatively, I think LLMs will replace SWE roles within the next 5-10 years. In that period SWE roles will change drastically, to be more hearding of AI agents. We can't hope to compete with something that can edit all the files in less than 5 minutes. If you're not at least trying to plan what your life will look like under these circumstances then you're doing yourself a disservice.

vineyardmike 3y ago

Yea the problem with that is the control group - grab any SWE and ask them the same thing. I don’t think most would pass. Unless you want to give an SWE time to learn… then it’s hardly fair. And I vaguely trust the LLM to be able to learn it too. Also I just asked Claude and Gemini and they both provided an implementation that matches the “bytes to UTF-8” rust docs. Assuming those are right,LLMs can do this (but I haven’t tested the code). https://doc.rust-lang.org/std/string/struct.String.html https://doc.rust-lang.org/std/string/struct.String.html

raggi 3y ago

it does an ok job with this task: use std::fs::File; use std::io::{self, BufReader, Read}; fn read_file_character_by_character(path: &str) -> io::Result<()> { // Open the file in read-only mode. let file = File::open(path)?; // Create a buffered reader to read the file more efficiently. let reader = BufReader::new(file); // `chars` method returns an iterator over the characters of the input. // Note that it returns a Result<(char, usize), io::Error>, where usize is the byte length of the char. for char_result in reader.chars() { match char_result { Ok(c) => print!("{}", c), Err(e) => return Err(e), } } Ok(()) } fn main() { let path = "path/to/your/file.txt"; if let Err(e) = read_file_character_by_character(path) { eprintln!("Error reading file: {}", e); } }

woodruffw 3y ago
If we're being sticklers, this isn't reading character-by-character: it's performing a buffered read, which then gets iterated over.
- raggi 3y ago
  here's what it does if i feed your nit pick into it verbatim: use std::fs::File; use std::io::{self, Read}; use std::str::from_utf8; fn read_file_character_by_character(path: &str) -> io::Result<()> { // Open the file in read-only mode. let mut file = File::open(path)?; let mut buffer = [0; 4]; // UTF-8 character max byte length let mut temp_buffer = vec![]; while let Ok(bytes_read) = file.read(&mut buffer[..1]) { // Read byte by byte if bytes_read == 0 { // End of file break; } temp_buffer.push(buffer[0]); if let Ok(s) = from_utf8(&temp_buffer) { if s.chars().count() == 1 { // Complete character read print!("{}", s); temp_buffer.clear(); // Clear temp buffer for next character } } } Ok(()) } fn main() { let path = "path/to/your/file.txt"; if let Err(e) = read_file_character_by_character(path) { eprintln!("Error reading file: {}", e); } }

DabbyDabberson 3y ago

The way I see it, its undetermined if Generative AI will be able to fully do a SWE job. But, for most of the debates I've seen, I don't think it the answer matters all too much. Once we have models that can act as full senior SWEs.. the models can engineer the models. And then we've hit the recursive case. Once models can engineer models better and faster than humans, all bets are off. Its the foggy future. Its the singularity.

dvt 3y ago
> Once we have models that can act as full senior SWEs.. the models can engineer the models. This is such an extremely bullish case, I'm not sure why you'd think this is even remotely possible. A Google search is usually more valuable than ChatGPT. For example, the rust utf-8 example is already verbatim solved on reddit: https://www.reddit.com/r/rust/comments/l5m1rw/how_can_i_efficiently_read_using_a_buffer_utf8/ https://www.reddit.com/r/rust/comments/l5m1rw/how_can_i_effi...
int_19h 3y ago
The implicit assumption here is that a human "senior SWE" can engineer a model of the same quality that is capable of simulating him. Which is definitely not true with the best models that we have today - and they certainly can't simulate a senior SWE, so the actual bar is higher. I'm not saying that the whole "robots building better robots" thing is a pipedream, but given where things are today, this is not something that's going to happen soon.
vertis 3y ago
People (SWEs) don't want to hear this. I think it's an inevitability that something of this nature will happen.

userbinator 3y ago

I'm not afraid of LLMs replacing me because of their output quality. The problem is the proliferation of quantity-over-quality "churn out barely-working crap as fast as possible" culture that gives LLMs the advantage over real humans.

int_19h 3y ago
I'm kinda hoping that LLMs will get pushed into production use writing code before they have acceptable quality (because greed), and the result will be lots of crap that's so badly broken most of the time that there will be a massive pushback against said culture from the users. Maybe from the governments as well, after a few well-publicized infrastructure failures.
- userbinator 3y ago
  Unfortunately, "lots of crap that's so badly broken most of the time" already describes a lot of software these days, and yet there hasn't been much pushback. Everyone seems to be mostly in a state of learned helplessness.
  - int_19h 3y ago
    There is pushback, it's just not broad enough yet. Because, as broken as things are (which is especially visible to those of us making the sausage or watching it made), they still kinda sorta work most of the time, to the point where users grumble but learn to live with it. But I think that AI coding will upset this equilibrium by reducing the quality even more, and significantly enough that users will very much notice - and for many of them it will push things into "what I need doesn't work most of the time" category. And then there will be payback. Then again, I am an optimist.