4 ms·

It's refreshing to see someone like Terence Tao embrace GPT-4 as an assistant. A few friends of mine (who are ML practitioners!) still don't trust LLMs or find

It's refreshing to see someone like Terence Tao embrace GPT-4 as an assistant. A few friends of mine (who are ML practitioners!) still don't trust LLMs or find them useful in their day-to-day work. I've noticed a similar trend among folks who work on compilers, preferring to "stay in their lane" instead of embracing GPT-4. The opposite appears true among those who work on user-facing applications, adoption of LLMs is much higher in their day-to-day.

SOLAR_FIELDS 3y ago
It is rather annoying because anyone who spends 15 minutes with the tool given the right context can easily learn its limitations and conclude that it’s a useful albeit imperfect tool. At this point not using an LLM in your day to day is like not using Google. Sure you can do it, but are you going to be outputting work as efficiently as possible? Wouldn’t you feel like doing your job without Google is hamstringing you a bit?
- kweingar 3y ago
  I definitely leave a lot on the table in terms of outputting work as efficiently as possible. In the past I’ve used personal wikis, time tracking software, personal project management software, etc. and it was a boon for my productivity. These days I don’t bother. I’m sure some people view chatbots the same way.
- PhilipRoman 3y ago
  TBH my views on Google (and search engines) in general have also changed over time. The information available on the internet is surprisingly shallow. Anything related to proprietary tech, hardware, geographically local info, reverse engineering or just anything that relatively few people are working on is very hard to find.
moonchild 3y ago
> I've noticed a similar trend among folks who work on compilers, preferring to "stay in their lane" instead of embracing GPT-4 Research into ML-guided optimisation predates chatjippity.
- rowanG077 3y ago
  chatjippity what?
- visarga 3y ago
  http://chatjippity.com/ http://chatjippity.com/ is taken, btw
thatguysaguy 3y ago
Could you tell me some typical uses beyond just asking for code snippets? I'm an NLP researcher, and currently I still find myself using Google for other types of information seeking. I'm working on NLG decoding algorithms currently. I see a lot of people saying it's replaced some large fraction of their search usage, but they generally don't explain what type of queries they're making.
- yodsanklai 3y ago
  Asking for code snippets has replaced most of my Stack Overflow usage. I'd say 90% of the cases, it answers something usable. Maybe it's not relevant for an NLP researchers, but for a regular SWE like me, it's handy several times per day.
  - thatguysaguy 3y ago
    Completely agreed on this one, although it's maybe more like 70% for me. I did leave an exception for that specifically in my original comment.
- ttul 3y ago
  I’ll often ask GPT to explain a concept to me, but giving it a hint as to my existing level of knowledge in the subject area. With Google, you’re likely to get the most popular page on a topic; not the page that is appropriate for people with your expertise. For instance, if I want to know about extreme ultraviolet lithography and I tell GPT-4 that I have a degree in engineering and that I studied advanced optics, the explanation is much richer in useful detail, going way beyond what any of the pages I could find on Google would reveal.
  - kaba0 3y ago
    That’s an area where they genuinely suck, unless there is an exact wikipedia article on the topic, readily within its training data set. If you wander off of that knowledge sphere, and you are not sufficiently knowledgeable yourself about the topic, it can tell you some really stupid stuff. Nonetheless, I do use it quite regularly in everyday life, as it is basically the best reverse dictionary (that works for any language) there is. For work (programming), I didn’t find a better use case than sometimes passing it a list of stuff, giving it an example on the first element on what I want, and using it to generate it for the rest of the elements. But that is <1% of what I do each day.
jiggawatts 3y ago
An observation I've made is that people that were early adopters and effective users of web search such as AltaVista or these days Google are also good at using LLMs. Almost every criticism of generative chat AI that I've seen in forums is someone holding it wrong, the direct equivalent of troubleshooting by searching for the term "my PC crashed" instead of a specific error code or whatever. Similarly, the people complaining about ChatGPT not being very good don't realise they're using the free-tier v3.5 instead of the paid GPT4 that's much smarter. The equivalent is people who use the default browser (IE) and use the default search engine (Bing) and complain that "the Internet is not useful". Just recently I had to review a bunch of legacy C# code, and I've found GPT4 to be enormously useful. It can find bugs and security issues in seconds, will suggest fixes to them, explain why they're bad, etc... Note that I don't have to trust it to write security-bug-free code! I'm asking it to find bugs that I will then fix myself.
- DonHopkins 3y ago
  [flagged]
  - jiggawatts 3y ago
    There is always that kid in the class that Googles the question and then prints out the first page as their assignment submission.
    - DonHopkins 3y ago
      Which in no way supports or defends or justifies any of your ignorant fallacious attacks on science, tired false equivalencies about religion, and nasty ad-hominem attacks on scientists. Plus it totally contradicts you assertions above about the usefulness of Google and ChatGPT. Flip, flop, flip, flop! You could at least have the integrity and intellectual honesty to reply directly to the actual real live scientist, a professional quantum physicist, who took the time to correctly rebut your wild conspiracy theories and gross misunderstandings of science, but you chose not to, and the crickets are still chirping away, awaiting your response. But your lack of a response to him says so much more about you than your response possibly could (short of an admission that you are wrong and an apology for attacking scientists and a promise to do better next time). Do you see the irony? Nope. You just contradicted yourself and flip-flopped yet again, reflexively clinging to your anti-scientific conspiracy theories even tighter, as revenge for being called out with the facts by a professional scientist, just like any dime-a-dozen foaming-at-the-mouth Q-Anon freak would. That only makes you even more wrong and ridiculous looking, and doesn't hurt or fool anybody else.
PheonixPharts 3y ago
It's not about not trusting it or "staying in your lane", LLM results for domain specific work are terrible, and additionally useless if you're writing fairly novel code (which most ML practitioners/Compiler programmers are doing if they're doing interesting work). The usefulness of LLMs for writing code is strongly correlated with how "Stack Overflowable" your work is.

SOLAR_FIELDS 3y ago

It is rather annoying because anyone who spends 15 minutes with the tool given the right context can easily learn its limitations and conclude that it’s a useful albeit imperfect tool. At this point not using an LLM in your day to day is like not using Google. Sure you can do it, but are you going to be outputting work as efficiently as possible? Wouldn’t you feel like doing your job without Google is hamstringing you a bit?

kweingar 3y ago
I definitely leave a lot on the table in terms of outputting work as efficiently as possible. In the past I’ve used personal wikis, time tracking software, personal project management software, etc. and it was a boon for my productivity. These days I don’t bother. I’m sure some people view chatbots the same way.
PhilipRoman 3y ago
TBH my views on Google (and search engines) in general have also changed over time. The information available on the internet is surprisingly shallow. Anything related to proprietary tech, hardware, geographically local info, reverse engineering or just anything that relatively few people are working on is very hard to find.

moonchild 3y ago

> I've noticed a similar trend among folks who work on compilers, preferring to "stay in their lane" instead of embracing GPT-4 Research into ML-guided optimisation predates chatjippity.

rowanG077 3y ago
chatjippity what?
visarga 3y ago
http://chatjippity.com/ http://chatjippity.com/ is taken, btw

thatguysaguy 3y ago

Could you tell me some typical uses beyond just asking for code snippets? I'm an NLP researcher, and currently I still find myself using Google for other types of information seeking. I'm working on NLG decoding algorithms currently. I see a lot of people saying it's replaced some large fraction of their search usage, but they generally don't explain what type of queries they're making.

yodsanklai 3y ago
Asking for code snippets has replaced most of my Stack Overflow usage. I'd say 90% of the cases, it answers something usable. Maybe it's not relevant for an NLP researchers, but for a regular SWE like me, it's handy several times per day.
- thatguysaguy 3y ago
  Completely agreed on this one, although it's maybe more like 70% for me. I did leave an exception for that specifically in my original comment.
ttul 3y ago
I’ll often ask GPT to explain a concept to me, but giving it a hint as to my existing level of knowledge in the subject area. With Google, you’re likely to get the most popular page on a topic; not the page that is appropriate for people with your expertise. For instance, if I want to know about extreme ultraviolet lithography and I tell GPT-4 that I have a degree in engineering and that I studied advanced optics, the explanation is much richer in useful detail, going way beyond what any of the pages I could find on Google would reveal.
- kaba0 3y ago
  That’s an area where they genuinely suck, unless there is an exact wikipedia article on the topic, readily within its training data set. If you wander off of that knowledge sphere, and you are not sufficiently knowledgeable yourself about the topic, it can tell you some really stupid stuff. Nonetheless, I do use it quite regularly in everyday life, as it is basically the best reverse dictionary (that works for any language) there is. For work (programming), I didn’t find a better use case than sometimes passing it a list of stuff, giving it an example on the first element on what I want, and using it to generate it for the rest of the elements. But that is <1% of what I do each day.

jiggawatts 3y ago

An observation I've made is that people that were early adopters and effective users of web search such as AltaVista or these days Google are also good at using LLMs. Almost every criticism of generative chat AI that I've seen in forums is someone holding it wrong, the direct equivalent of troubleshooting by searching for the term "my PC crashed" instead of a specific error code or whatever. Similarly, the people complaining about ChatGPT not being very good don't realise they're using the free-tier v3.5 instead of the paid GPT4 that's much smarter. The equivalent is people who use the default browser (IE) and use the default search engine (Bing) and complain that "the Internet is not useful". Just recently I had to review a bunch of legacy C# code, and I've found GPT4 to be enormously useful. It can find bugs and security issues in seconds, will suggest fixes to them, explain why they're bad, etc... Note that I don't have to trust it to write security-bug-free code! I'm asking it to find bugs that I will then fix myself.

DonHopkins 3y ago
[flagged]
- jiggawatts 3y ago
  There is always that kid in the class that Googles the question and then prints out the first page as their assignment submission.
  - DonHopkins 3y ago
    Which in no way supports or defends or justifies any of your ignorant fallacious attacks on science, tired false equivalencies about religion, and nasty ad-hominem attacks on scientists. Plus it totally contradicts you assertions above about the usefulness of Google and ChatGPT. Flip, flop, flip, flop! You could at least have the integrity and intellectual honesty to reply directly to the actual real live scientist, a professional quantum physicist, who took the time to correctly rebut your wild conspiracy theories and gross misunderstandings of science, but you chose not to, and the crickets are still chirping away, awaiting your response. But your lack of a response to him says so much more about you than your response possibly could (short of an admission that you are wrong and an apology for attacking scientists and a promise to do better next time). Do you see the irony? Nope. You just contradicted yourself and flip-flopped yet again, reflexively clinging to your anti-scientific conspiracy theories even tighter, as revenge for being called out with the facts by a professional scientist, just like any dime-a-dozen foaming-at-the-mouth Q-Anon freak would. That only makes you even more wrong and ridiculous looking, and doesn't hurt or fool anybody else.

PheonixPharts 3y ago

It's not about not trusting it or "staying in your lane", LLM results for domain specific work are terrible, and additionally useless if you're writing fairly novel code (which most ML practitioners/Compiler programmers are doing if they're doing interesting work). The usefulness of LLMs for writing code is strongly correlated with how "Stack Overflowable" your work is.