5 ms·
Forgive the long reply, but I'm going to go ahead and "press X to doubt," because I have been trying to use both ChatGPT and Copilot, and the results are abysma
by dvt 3y ago
Forgive the long reply, but I'm going to go ahead and "press X to doubt," because I have been trying to use both ChatGPT and Copilot, and the results are abysmal apart from anything but "write me a linked list" or "give me some React boilerplate." If you have examples of it working for you, please share them. Here's a very salient example of it not working for me: I spent about 1-2 days figuring this out, so it's definitely non-trivial, but it's not something insane either. I'm working on a new project/startup idea and this is part of it (verbatim prompt I just used on chatGPT):
hi, can you write me a function in rust that, if running on windows:
1. figures out if the topmost window is a windows explorer window
2. if it's a windows explorer window, it returns the path (as a string) that the windows explorer window is actively viewing
3. if it's not a windows explorer window, it returns a blank string
This is a deceptively difficult task, as it requires all kinds of COM finagling, and ChatGPT gets it royally wrong:
#[cfg(windows)]
fn get_active_explorer_path() -> String {
use std::ptr;
use winapi::um::winuser::{EnumWindows, GetWindowTextW, IsWindowVisible};
use winapi::shared::windef::HWND;
use std::os::raw::c_int;
use std::ffi::OsString;
use std::os::windows::ffi::OsStringExt;
let mut path = String::new();
let mut active_explorer = None;
unsafe extern "system" fn enum_windows_callback(hwnd: HWND, lparam: LPARAM) -> c_int {
let active_explorer = &mut *(lparam as *mut Option<HWND>);
let mut buffer: [u16; 512] = [0; 512];
if IsWindowVisible(hwnd) != 0 && GetWindowTextW(hwnd, buffer.as_mut_ptr(), 512) != 0 {
let window_title = OsString::from_wide(&buffer[..]).to_string_lossy();
if window_title == "File Explorer" {
*active_explorer = Some(hwnd);
return 0; // Stop enumeration
}
}
1 // Continue enumeration
}
unsafe {
EnumWindows(Some(enum_windows_callback), &mut active_explorer as *mut _ as LPARAM);
}
if let Some(hwnd) = active_explorer {
let mut buffer: [u16; 512] = [0; 512];
if GetWindowTextW(hwnd, buffer.as_mut_ptr(), 512) != 0 {
let window_title = OsString::from_wide(&buffer[..]).to_string_lossy();
if window_title == "File Explorer" {
let shell32 = winapi::um::shellapi::GetShellWindow();
if shell32 != hwnd {
let mut path_buffer: [u16; 512] = [0; 512];
if winapi::um::shellapi::SHGetPathFromIDListW(ptr::null_mut(), path_buffer.as_mut_ptr()) != 0 {
let path_str = OsString::from_wide(&path_buffer[..]).to_string_lossy();
path = path_str.into_owned();
}
}
}
}
}
path
}
I'm actually on my Mac right now, and I don't even need to test it, because the first red flag is `"window_title == "File Explorer"`. File Explorer will change its window titlebar to the currently viewed folder (so the conditional will always fail). There's also other comical mistakes, including passing a null pointer to `SHGetPathFromIDListW`, using `GetShellWindow` which returns the desktop, and not taking into account that Explorer windows can have multiple tabs. Okay, let's see if we fare better on MacOS.
#[cfg(target_os = "macos")]
fn get_active_finder_path() -> String {
use std::ptr;
use objc::runtime::{Class, Object};
use objc::{class, msg_send, sel, sel_impl};
use objc_id::Id;
let mut path = String::new();
unsafe {
let workspace: Id<Object> = msg_send![class!(NSWorkspace), sharedWorkspace];
let front_window: Id<Object> = msg_send![workspace, frontmostApplication];
let front_window_pid: u32 = msg_send![front_window, processIdentifier];
let running_apps: Id<Object> = msg_send![workspace, runningApplications];
let app_count: usize = msg_send![running_apps, count];
for i in 0..app_count {
let app: Id<Object> = msg_send![running_apps, objectAtIndex: i];
let app_pid: u32 = msg_send![app, processIdentifier];
if app_pid == front_window_pid {
let bundle_url: Id<Object> = msg_send![app, bundleURL];
let bundle_path: *const Object = msg_send![bundle_url, path];
let path_str: *const i8 = msg_send![bundle_path, UTF8String];
if !path_str.is_null() {
let c_str = std::ffi::CStr::from_ptr(path_str);
path = c_str.to_string_lossy().to_string();
}
break;
}
}
}
path
}
#[cfg(not(target_os = "macos"))]
fn get_active_finder_path() -> String {
String::new()
}
fn main() {
let active_finder_path = get_active_finder_path();
println!("Active Finder Path: {}", active_finder_path);
}
Doesn't look like it: this is what I got, which doesn't even compile. I'm not as familiar with MacOS systems programming, so I'm not even sure what's wrong with it, frankly. My solution involved using osascript on MacOS (which is the first Google hit, though I'll have to port it, as some folks may not have it on their systems).
So what is ML code generation useful for? Apart from helping on the Nth CRUD app someone's building, I just don't see it.
- frankgrecojr 3y agoWhen I have use cases similar to this one (similar in complexity), I'll pair program with it. If the code fails, I'll tell it what error I got and ask it to refactor. If the explanation it gives with the code shows it didn't understand the question, I'll guide it along the correct track. It's not perfect at "zero-shot" answers but from my experience is very good when you work with it conversationally.
- johnfn 3y agoI get a lot of mileage out of GPT, so let me see if I can explain. I wouldn't ask GPT/Copilot to do what you asked. After using them for a while you start to get an intuitive sense of what's easy and what's hard for them to do, and your specific example is indeed too difficult to get GPT to do properly. They have the biggest utility for me for stuff that I would ask a junior programmer to do - like a new grad. The real value proposition of GPT isn't that it can solve really hard problems. The value proposition is that it's about as capable as a junior engineer, except it can write code much faster than any junior engineer, so it can speed you along on the boilerplate-y parts of coding that otherwise would be a lot of manual effort. It's especially useful for things that feel "easy" but which I don't have the relevant domain expertise. For instance, the other day I needed to write a fairly trivial shell script to parse some JSON files. I never write shell scripts and I always forget the syntax, but GPT wrote it correctly on the first try. That probably saved me 20-30 minutes of googling for how to do things like read files, etc in shell scripts.
- ElFitz 3y agoIt’s great at simple tedious stuff. It’s useful for average things and concepts we might be unfamiliar with. It’s of limited use, if not utterly useless, for most things a senior would have a hard time wrapping their head around. And where it really shines, is when adapting an answer you’d otherwise look for on Google to your code, context, and constraints.
- clarge1120 3y agoThanks for including this example. It helps me understand what you mean. ChatGPT is not really equipped to do this level of code design because there are too many steps for it to handle at once. It can handle your instructions if it could first write a detailed design spec, then write code and unit tests, then run them, read the compiler output, and iteratively make changes. IOW, this would require a purpose-built solution which uses ChatGPT's underlying engine to take your list of requirements and turn them into built and tested code.