4 ms·

Emil's Story as a Self-Taught AI Researcher (2020)

101 points by eamag 2y ago

croissants 2y ago
Previous discussion: https://news.ycombinator.com/item?id=22101066 https://news.ycombinator.com/item?id=22101066 IMO the confident Tweets about how to become a good researcher and hire good researchers look pretty weird next to the lack of any apparent research papers (or even visible research products) five years later.
- gwern 2y ago
  Isn't he running a company (Palette) which makes heavy use of DL?
  - emilwallner 2y ago
    Indeed, I realized it's hard to compete with PhD students for grants, and subsidising my work with content marketing does not fit my style, and I prefer owning my work and choosing my own research direction. I also want people to use my work, and create solutions that are cost-effective. So the most logical way was to bootstrap an AI start-up in the area I'm interested, so that's what I'm doing. Unfortunately, it's hard to publish or contribute to open-source, since it becomes too easy to copy, which cuts my margins and ability to fund my research and compute. Now I spend most of my days doing AI research, and outsource most other parts, really enjoying it :)
    - gopher_space 2y ago
      I think you'd attract a more sympathetic audience if you reframed Emil's Story as "How to get people to leave you alone and let you think." E.g. half of your AI autodidact degree boils down to 'make money with what you\'ve learned', and it's always interesting to see people unpack their approaches to this idea. Your thinking around target numbers would be valuable.
      - emilwallner 2y ago
        I agree, it was written some time ago, and the language is hyperbolic, but many of the key points still hold true. Building a skillset to reach $100/hour consulting gigs was key to work 2-3 days per week, have time for research, and saving money for an ML rig, while living in Paris. Around $5K MRR is sufficient to live off, $10K MRR is more comfortable and allows renting a few extra A6000 Ada and start outsourcing, and $20K MRR to afford full outsourcing to focus on research and renting 8xH100s. That's a good goal to balance research freedom with other responsibilities. After that, it's optional to trade research freedom for higher MRR and more responsibilities.
  - croissants 2y ago
    Sure, so it seems fair for him to offer advice on how to apply AI or start a company based on it, but doing AI research means generating new knowledge about AI itself, and I don't see any evidence of that.
    - numba888 2y ago
      > doing AI research means generating new knowledge about AI itself, and I don't see any evidence of that. Wouldn't colorization count as research? In vision domain there are a lot of papers like this. Just arranging and rearranging known blocks and getting SOTA result on some datasets. ;)
- king_magic 2y ago
  Guy seems & speaks like a charlatan.
  - Xmd5a 2y ago
    >I moved from Sweden to Ghana, West Africa. I started working as a teacher in the countryside, but after invoking the spirit of their dead chief, they later annotated me the king of their village.
    - RestartKernel 2y ago
      For those casually browsing the comments, this is an actual quote from the article.
      - emilwallner 2y ago
        True story, happened when I was around 18: https://imgur.com/fNd55hB https://imgur.com/fNd55hB
    - dkarl 2y ago
      I knew a guy like this. He was fairly transparent about the fact that he talked a lot of bullshit about himself, in the sense that he didn't keep it realistic and didn't respond to reasonable skepticism the way a normal person would, but a lot of people responded positively to it. The people it worked on, they recognized that he wasn't speaking literal truth and mentally discounted what he said by some large percentage, thinking that they were critically adjusting for his boastfulness. But he painted such a grandiose picture of himself that anyone who subconsciously assumed that 10% of it was true would still be impressed. He was a weird dude. I think he grew up in a household where outrageous boasting was common and had been doing it so long that it was his authentic self. He desperately wanted people to be impressed with him, and he had mastered a strategy that had a consistent yield, but the people it didn't work on looked down on him pretty severely, and he didn't have any way of earning their respect. He had to trust in the fact that over time he would tend to find himself surrounded by people he could manipulate.
    - 2y ago
emilwallner 2y ago
Hey, Emil from the article here. I end-up doing part-time work for Google at the interaction of Art/Culture and ML doing project like this (https://artsandculture.google.com/story/the-klimt-color-enigma/SQWxuZfE5ki3mQ?hl=en https://artsandculture.google.com/story/the-klimt-color-enig...), I saved up enough to build an ML rig (https://www.emilwallner.com/p/ml-rig https://www.emilwallner.com/p/ml-rig), since I worked 2-3 days a week, I could spend the rest of my time doing research. I spent 1-2 years working on reasoning, trying different adaptive compute mechanisms and RL on code and mathematics (similar to R1/o1), however, I realised it was hard to compete with the established labs, and if I published my work it was hard to monetize it to have enough time to stop doing consulting work and fund my compute needs. Instead, I started researching AI colorization, and launched it as a side-project (https://www.reddit.com/r/InternetIsBeautiful/comments/xe6avh/i_made_a_new_and_free_ai_colorizer_tool_colorize/ https://www.reddit.com/r/InternetIsBeautiful/comments/xe6avh...), I ended up having a few hundred thousand users in a few weeks and realized it had enough legs to bootstrap into a company. So I left my consulting gig at Google to go full-time on the colorization project (Palette: https://palette.fm/ https://palette.fm/). Fast forward to today, Palette is still running with a healthy margin, I’ve outsourced most of the things and I can spend most of my time doing AI research. I’d love to publish and open-source more, but since it becomes too easy to copy, it makes it hard to fund myself and my compute needs. Happy to answer any questions.
- tobr 2y ago
  I got stuck at ”after invoking the spirit of their dead chief, they later annotated me the king of their village”. What kind of interviewer doesn’t ask a follow-up question on that anecdote?
  - emilwallner 2y ago
    tldr, most of technology in rural Ghana was considered magic, like phones and tvs, and the only explanation was that it came from "the white man". The chief had recently died and for most of the time I was the only caucasian person in the region, and i met many that had never seen a caucasian person before. The locals started joking by greeting my as the king, the joke picked up and soon most were greeting me as the king. I always though it was a joke, and went a long with it. However, one day I was summed by the neighbouring regional kings and the elders of the village I lived in. After chatting for 30 minutes in the local language, they were convinced that the dead chief's spirit had entered me and they asked in english, "So, do you want to be our king?", I asked what it meant, one person said, I would be given 4 wives and they would slaughter a goat and pour the blood over me, another said I should ask the king of Sweden what it meant, every person said a different thing. I sat there utterly confused, but thought to myself, yolo. So I agreed. I still feel like it was a dream, a surreal experience. The anointing ceremony was a lot of fun, since many of the people in the village were christians, the dropped many of the accent rituals since they were considered unethical under Christianity. There were a hundred or so people, many of the regional kings attended, they offered me a new outfit, a tunic, carved a chair out of wood that only chiefs can sit on, lots of music, dancing, and regional ceremonial aspects. Here's a pic from the ceremony: https://imgur.com/fNd55hB https://imgur.com/fNd55hB

101 points by eamag 2y ago

Previous discussion: https://news.ycombinator.com/item?id=22101066 https://news.ycombinator.com/item?id=22101066 IMO the confident Tweets about how to become a good researcher and hire good researchers look pretty weird next to the lack of any apparent research papers (or even visible research products) five years later.

gwern 2y ago
Isn't he running a company (Palette) which makes heavy use of DL?
- emilwallner 2y ago
  Indeed, I realized it's hard to compete with PhD students for grants, and subsidising my work with content marketing does not fit my style, and I prefer owning my work and choosing my own research direction. I also want people to use my work, and create solutions that are cost-effective. So the most logical way was to bootstrap an AI start-up in the area I'm interested, so that's what I'm doing. Unfortunately, it's hard to publish or contribute to open-source, since it becomes too easy to copy, which cuts my margins and ability to fund my research and compute. Now I spend most of my days doing AI research, and outsource most other parts, really enjoying it :)
  - gopher_space 2y ago
    I think you'd attract a more sympathetic audience if you reframed Emil's Story as "How to get people to leave you alone and let you think." E.g. half of your AI autodidact degree boils down to 'make money with what you\'ve learned', and it's always interesting to see people unpack their approaches to this idea. Your thinking around target numbers would be valuable.
    - emilwallner 2y ago
      I agree, it was written some time ago, and the language is hyperbolic, but many of the key points still hold true. Building a skillset to reach $100/hour consulting gigs was key to work 2-3 days per week, have time for research, and saving money for an ML rig, while living in Paris. Around $5K MRR is sufficient to live off, $10K MRR is more comfortable and allows renting a few extra A6000 Ada and start outsourcing, and $20K MRR to afford full outsourcing to focus on research and renting 8xH100s. That's a good goal to balance research freedom with other responsibilities. After that, it's optional to trade research freedom for higher MRR and more responsibilities.
- croissants 2y ago
  Sure, so it seems fair for him to offer advice on how to apply AI or start a company based on it, but doing AI research means generating new knowledge about AI itself, and I don't see any evidence of that.
  - numba888 2y ago
    > doing AI research means generating new knowledge about AI itself, and I don't see any evidence of that. Wouldn't colorization count as research? In vision domain there are a lot of papers like this. Just arranging and rearranging known blocks and getting SOTA result on some datasets. ;)
king_magic 2y ago
Guy seems & speaks like a charlatan.
- Xmd5a 2y ago
  >I moved from Sweden to Ghana, West Africa. I started working as a teacher in the countryside, but after invoking the spirit of their dead chief, they later annotated me the king of their village.
  - RestartKernel 2y ago
    For those casually browsing the comments, this is an actual quote from the article.
    - emilwallner 2y ago
      True story, happened when I was around 18: https://imgur.com/fNd55hB https://imgur.com/fNd55hB
  - dkarl 2y ago
    I knew a guy like this. He was fairly transparent about the fact that he talked a lot of bullshit about himself, in the sense that he didn't keep it realistic and didn't respond to reasonable skepticism the way a normal person would, but a lot of people responded positively to it. The people it worked on, they recognized that he wasn't speaking literal truth and mentally discounted what he said by some large percentage, thinking that they were critically adjusting for his boastfulness. But he painted such a grandiose picture of himself that anyone who subconsciously assumed that 10% of it was true would still be impressed. He was a weird dude. I think he grew up in a household where outrageous boasting was common and had been doing it so long that it was his authentic self. He desperately wanted people to be impressed with him, and he had mastered a strategy that had a consistent yield, but the people it didn't work on looked down on him pretty severely, and he didn't have any way of earning their respect. He had to trust in the fact that over time he would tend to find himself surrounded by people he could manipulate.
  - 2y ago

emilwallner 2y ago

Hey, Emil from the article here. I end-up doing part-time work for Google at the interaction of Art/Culture and ML doing project like this (https://artsandculture.google.com/story/the-klimt-color-enigma/SQWxuZfE5ki3mQ?hl=en https://artsandculture.google.com/story/the-klimt-color-enig...), I saved up enough to build an ML rig (https://www.emilwallner.com/p/ml-rig https://www.emilwallner.com/p/ml-rig), since I worked 2-3 days a week, I could spend the rest of my time doing research. I spent 1-2 years working on reasoning, trying different adaptive compute mechanisms and RL on code and mathematics (similar to R1/o1), however, I realised it was hard to compete with the established labs, and if I published my work it was hard to monetize it to have enough time to stop doing consulting work and fund my compute needs. Instead, I started researching AI colorization, and launched it as a side-project (https://www.reddit.com/r/InternetIsBeautiful/comments/xe6avh/i_made_a_new_and_free_ai_colorizer_tool_colorize/ https://www.reddit.com/r/InternetIsBeautiful/comments/xe6avh...), I ended up having a few hundred thousand users in a few weeks and realized it had enough legs to bootstrap into a company. So I left my consulting gig at Google to go full-time on the colorization project (Palette: https://palette.fm/ https://palette.fm/). Fast forward to today, Palette is still running with a healthy margin, I’ve outsourced most of the things and I can spend most of my time doing AI research. I’d love to publish and open-source more, but since it becomes too easy to copy, it makes it hard to fund myself and my compute needs. Happy to answer any questions.

tobr 2y ago
I got stuck at ”after invoking the spirit of their dead chief, they later annotated me the king of their village”. What kind of interviewer doesn’t ask a follow-up question on that anecdote?
- emilwallner 2y ago
  tldr, most of technology in rural Ghana was considered magic, like phones and tvs, and the only explanation was that it came from "the white man". The chief had recently died and for most of the time I was the only caucasian person in the region, and i met many that had never seen a caucasian person before. The locals started joking by greeting my as the king, the joke picked up and soon most were greeting me as the king. I always though it was a joke, and went a long with it. However, one day I was summed by the neighbouring regional kings and the elders of the village I lived in. After chatting for 30 minutes in the local language, they were convinced that the dead chief's spirit had entered me and they asked in english, "So, do you want to be our king?", I asked what it meant, one person said, I would be given 4 wives and they would slaughter a goat and pour the blood over me, another said I should ask the king of Sweden what it meant, every person said a different thing. I sat there utterly confused, but thought to myself, yolo. So I agreed. I still feel like it was a dream, a surreal experience. The anointing ceremony was a lot of fun, since many of the people in the village were christians, the dropped many of the accent rituals since they were considered unethical under Christianity. There were a hundred or so people, many of the regional kings attended, they offered me a new outfit, a tunic, carved a chair out of wood that only chiefs can sit on, lots of music, dancing, and regional ceremonial aspects. Here's a pic from the ceremony: https://imgur.com/fNd55hB https://imgur.com/fNd55hB