9 ms·

Draft of the Fast.ai Book

509 points by AgharaShyam 7y ago

classified 7y ago
> If you make any pull requests to this repo, then you are assigning copyright of that work to Jeremy Howard and Sylvain Gugger. That's not how the GPL works.
montenegrohugo 7y ago
Love the work Jeremy Howard does. Would definitely recommend the fastai course for anyone considering getting into ML.
- alexcnwy 7y ago
  Totally. Jeremy changed my life.
  - stazz1 7y ago
    Are you using ML in your business now? Just wondering Thanks =)
- Kunix 7y ago
  Same, long-time developer, I've followed the course and learned a lot (as long as you invest the effort ^^). Great content!
pritovido 7y ago
It always amazes me how bad some technical people is at basic promotion: What is Fastai? Why do I need it? Something as basic as an elevator speech that introduces your product in your github page and book intro can mean 10x or 100x more sales. If you force people into having to search it for you, you have already lost most of them. For this author it is as you already know everything about Fastai, but if you did, you would not be needing this book in the first place. It happens a lot to technical writers because they have spent years thinking about a topic, so they could not put themselves in the shoes of someone who does not.
- Arrrlex 7y ago
  This is a draft. I expect that, when the first finished version is ready, the authors will promote it effectively (IMO they are very good at promoting their courses at fast.ai).
- mkl 7y ago
  Fast.ai courses, software, and articles are frequently posted on HN: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=fast.ai&sort=byDate&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu... Few people here need to look it up; the basic promotion has already been done very effectively for the target audience. Besides, the intro chapter explains very clearly what the book is about and who it's for.
- sytelus 7y ago
  You are downvoted by fanboys but you are exactly right. I am surrounded by researchers working in DL and I have say at least 40% of them have never heard of FastAI or Jeremy Howard. However folks who are active on Twitter, listening to popular podcasts, popular media, HN etc would be very familiar with name Jeremy Howard and what FastAI is and need no introduction. In research world, an astonishing number of good researchers have little to none online presence. They have little to no time other than keeping track of research papers in their sub-field. It also surprises me when authors sweat for months to churn out 100s of polished pages but couldn’t spend 15 minutes to write a paragraph of introduction in readme.MD.
  - DrNuke 7y ago
    Fast.ai really democratizes the bleeding edge research for the masses, though, that’s why it’s popular among outcasts and outsiders. In general, I would be more wary of people working within closed environments and organizations than people making all they do public and open to review.
    - sytelus 7y ago
      fastai is popular among practitioners as well as many researchers rather than just outsiders or outcasts. I've personally learned from it a lot and is amazing contribution. However, there is still a large population that is still unaware and it would be great to have a quick intro paragraph in readme so they know what all the fuzz is about.
mci 7y ago
> There are also "agglutinative languages", like Polish, which can add many morphemes together to create very long "words" which include a lot of separate pieces of information. [1] Polish does not work this way. Source: I am Polish. Perhaps jph00 meant Turkish. Issue filed. [1] https://github.com/fastai/fastbook/blob/master/10_nlp.ipynb https://github.com/fastai/fastbook/blob/master/10_nlp.ipynb
- machiaweliczny 7y ago
  Doesn't German work this way?
  - mkl 7y ago
    Not really. "German grammar allows for the construction of long compounded noun phrases which are expressed as one word in written language. Compounding is not really the same as agglutination.": https://www.quora.com/Is-German-considered-a-true-agglutinative-language?share=1 https://www.quora.com/Is-German-considered-a-true-agglutinat... There are quite a lot of languages that do though: https://en.wikipedia.org/wiki/Agglutinative_language https://en.wikipedia.org/wiki/Agglutinative_language
    - gumby 7y ago
      Not just in written language, although the difference between a “word” and “noun phrase” in spoken language is in the ear of the beholder. But in a linguistic sense indeed, German is not at all an agglutinative language.
  - pault 7y ago
    English too. Policeman, bathwater, catwalk, headstone, toothbrush, etc.
- jph00 7y ago
  Yes you're right, in our NLP course we used Turkish as our example. But for the book I mentioned Polish due to this paper: https://arxiv.org/abs/1810.10222 https://arxiv.org/abs/1810.10222 . But as you say, now the word "agglutinative" isn't technically correct. I'm actually not sure what the right word is to describe languages that have lots of big compounds with no spaces. (Which is the key issue here, as to why we need subword tokenization techniques).
ratsimihah 7y ago
It looks promising! Minor point, a requirements.txt file or something would be convenient to get started quickly.
- jph00 7y ago
  Once the book is released there will be a whole website and prebuilt environments and lots more to get started quickly. We didn't expect the draft to get this much attention, frankly!
  - honzzz 7y ago
    Any estimate when the book will be released? Thanks :-)
    - jph00 7y ago
      July.
  - ratsimihah 7y ago
    That's exciting! If pull requests are enabled I can always send one once I'm up and running! Looking forward to checking out the good stuff in there!
    - jph00 7y ago
      Actually someone already did a PR for it!
      - ratsimihah 7y ago
        Oh brilliant! That's what happens when I slack off checking out face masks haha!
cube2222 7y ago
I recommend everybody who didn’t check those out to do so. Not really being interested in ML, but doing all the available most popular courses to keep up, I really liked how fastai doesn’t just teach you ready and known models, but also how to compose differentiable building blocks to design NN’s yourself.
DrNuke 7y ago
Another smart move from fast.ai, this book is going to be the state-of-the-art reference for 2020 and a classic anthology of algo techniques for the mid term.
chrisa 7y ago
It's a neat idea to write the entire book as notebooks - so you can have run-able code right in line. Definitely excited to check this out; thanks Jeremy and Sylvain!
- smohare 7y ago
  These sorts of books have existed since the dawn of notebooks. While I think them useful for demonstrations, I don’t think there is much pedantic value (beyond a more static medium) in all honesty. Well-worked examples a student can consult while attempting to solve problems on their own is still of paramount importance.
  - localhost 7y ago
    Why can’t we have both? Great examples AND an interactive, reproducible medium?
  - timClicks 7y ago
    When you say "pedantic value", do you mean "pedagogic value"?
- weego 7y ago
  It's an awful idea, notebook stability ages like milk
  - SkyMarshal 7y ago
    What do you mean by stability in this context?
    - khazhoux 7y ago
      I think he means they bit-rot. Stop working.
  - fredmonroe 7y ago
    i think that's true of almost all tech books in paper form personally i think these work great because i can add cells to inspect data or try experiments easily as i'm reading to help me understand whats going on
sytelus 7y ago
Wow, Orielly lawyers are determined to screw this up. The thing is GPL v3 licensed which means I can’t copy any of the book code in my closed-source product or competitions or even MIT licensed code. The readme says I cannot make copies of this material but it’s ok to fork. Huh?
- jph00 7y ago
  No, the readme says you can make copies for personal use. If you want to use code in the book under a non GPL license, then you could just buy the book when it comes out. That doesn't seem like an unreasonable burden. PS: none of this is anything to do with O'Reilly or their lawyers.
  - sytelus 7y ago
    Wait... so if you buy book then it seizes to be GPLed? This is quite confusing. For DL research, most code is MIT licenced and legal folks at many industrial labs would be quite hesitent to permit use of code from this repo with feels like legal minefield with different restrictions spread over multiple places including LICENSE, README, fastai website and perhaps printed book. I would highly recommand converting to one simple MIT license and call it a day (except for markdown cells).
    - bonoboTP 7y ago
      I don't get this perspective about the GPL. Look, they are giving you something for free, including the source code and the right to build upon it and publish modified versions. You can do basically whatever you want with it, as long as you pass on the freedoms that were granted to you. Is that unfair? Enjoying getting freedoms but not passing them on is not nice.
      - sytelus 7y ago
        I get GPL and fully appreciate its philosophy. The problem happens when you actually use it in practice. Because of its viral nature, anyone with different licensing must convert to GPL if they use your code. For many scenarios, this is actually not possible not just because of commercial secrets but the potential for opening up for security vulnerabilities when you don’t have resources or competitions where you should keep code secret until some time or simply because you have dependencies on other code which is very expensive to get rid off. Due to this reason, many companies forbid the use of GPL licensed software as well as release anything under it (because then you can’t use your own code!). Many other companies simply don't want the headache of checking all of their mess of legacy codebases with a myriad of dependencies that would be hard to untangle into GPL compatible open-source release. The legal and economic overhead when you use or release GPLed code is non-trivial. For this reason, the vast majority of open-source code released by big tech companies on GitHub is MIT/BSD licensed, which ironically is more "freeier" than GPL.
coderunner 7y ago
How does this compare to the video course? e.g. what are the differences?
- jph00 7y ago
  It's all new material. It'll be the basis of the next course coming in July. Or you can join the in person course from March in SF https://www.usfca.edu/data-institute/certificates/deep-learning-part-one https://www.usfca.edu/data-institute/certificates/deep-learn...
he11ow 7y ago
This is not intending to minimize in the slightest the amazing work that Jeremy does - I am a huge fan. But Fast.ai has TWO co-founders, and somehow, Rachel doesn't seem to get any credit in these discussions (not the book specifically, I'm talking about the overall enterprise). Not quite sure why; A lot of the content on the website is written by her, and it's clear she adds a lot of value to the endeavor as a whole.
- FranzFerdiNaN 7y ago
  I can think of a reason she isn’t getting the credit she deserves.
- jph00 7y ago
  Thank you for mentioning Rachel! :) She is working as the Founding Director of the Center for Applied Data Ethics nowadays, which is a very full-time job. So she hasn't been involved much in fastai v2 or the book (other than chapter 3, of which she's a co-author). She created and taught the NLP and Computational Linear Algebra courses, and has written most of the material on the fast.ai blog, and of course (as noted) co-founded fast.ai. Overall, I'd agree that she doesn't get as much credit as she deserves. That's perhaps partly due to her increasing focus on ethics issues, which aren't generally discussed much on HN (sadly). I also would say that Sylvain Gugger doesn't get as much credit as he should -- he has been an equal partner with me in creating the book and fastai library. (I discussed this response with Rachel prior to posting it.)

509 points by AgharaShyam 7y ago

> If you make any pull requests to this repo, then you are assigning copyright of that work to Jeremy Howard and Sylvain Gugger. That's not how the GPL works.

montenegrohugo 7y ago

Love the work Jeremy Howard does. Would definitely recommend the fastai course for anyone considering getting into ML.

alexcnwy 7y ago
Totally. Jeremy changed my life.
- stazz1 7y ago
  Are you using ML in your business now? Just wondering Thanks =)
Kunix 7y ago
Same, long-time developer, I've followed the course and learned a lot (as long as you invest the effort ^^). Great content!

pritovido 7y ago

It always amazes me how bad some technical people is at basic promotion: What is Fastai? Why do I need it? Something as basic as an elevator speech that introduces your product in your github page and book intro can mean 10x or 100x more sales. If you force people into having to search it for you, you have already lost most of them. For this author it is as you already know everything about Fastai, but if you did, you would not be needing this book in the first place. It happens a lot to technical writers because they have spent years thinking about a topic, so they could not put themselves in the shoes of someone who does not.

Arrrlex 7y ago
This is a draft. I expect that, when the first finished version is ready, the authors will promote it effectively (IMO they are very good at promoting their courses at fast.ai).
mkl 7y ago
Fast.ai courses, software, and articles are frequently posted on HN: https://hn.algolia.com/?dateRange=all&page=0&prefix=false&query=fast.ai&sort=byDate&type=story https://hn.algolia.com/?dateRange=all&page=0&prefix=false&qu... Few people here need to look it up; the basic promotion has already been done very effectively for the target audience. Besides, the intro chapter explains very clearly what the book is about and who it's for.
sytelus 7y ago
You are downvoted by fanboys but you are exactly right. I am surrounded by researchers working in DL and I have say at least 40% of them have never heard of FastAI or Jeremy Howard. However folks who are active on Twitter, listening to popular podcasts, popular media, HN etc would be very familiar with name Jeremy Howard and what FastAI is and need no introduction. In research world, an astonishing number of good researchers have little to none online presence. They have little to no time other than keeping track of research papers in their sub-field. It also surprises me when authors sweat for months to churn out 100s of polished pages but couldn’t spend 15 minutes to write a paragraph of introduction in readme.MD.
- DrNuke 7y ago
  Fast.ai really democratizes the bleeding edge research for the masses, though, that’s why it’s popular among outcasts and outsiders. In general, I would be more wary of people working within closed environments and organizations than people making all they do public and open to review.
  - sytelus 7y ago
    fastai is popular among practitioners as well as many researchers rather than just outsiders or outcasts. I've personally learned from it a lot and is amazing contribution. However, there is still a large population that is still unaware and it would be great to have a quick intro paragraph in readme so they know what all the fuzz is about.

mci 7y ago

> There are also "agglutinative languages", like Polish, which can add many morphemes together to create very long "words" which include a lot of separate pieces of information. [1] Polish does not work this way. Source: I am Polish. Perhaps jph00 meant Turkish. Issue filed. [1] https://github.com/fastai/fastbook/blob/master/10_nlp.ipynb https://github.com/fastai/fastbook/blob/master/10_nlp.ipynb

machiaweliczny 7y ago
Doesn't German work this way?
- mkl 7y ago
  Not really. "German grammar allows for the construction of long compounded noun phrases which are expressed as one word in written language. Compounding is not really the same as agglutination.": https://www.quora.com/Is-German-considered-a-true-agglutinative-language?share=1 https://www.quora.com/Is-German-considered-a-true-agglutinat... There are quite a lot of languages that do though: https://en.wikipedia.org/wiki/Agglutinative_language https://en.wikipedia.org/wiki/Agglutinative_language
  - gumby 7y ago
    Not just in written language, although the difference between a “word” and “noun phrase” in spoken language is in the ear of the beholder. But in a linguistic sense indeed, German is not at all an agglutinative language.
- pault 7y ago
  English too. Policeman, bathwater, catwalk, headstone, toothbrush, etc.
jph00 7y ago
Yes you're right, in our NLP course we used Turkish as our example. But for the book I mentioned Polish due to this paper: https://arxiv.org/abs/1810.10222 https://arxiv.org/abs/1810.10222 . But as you say, now the word "agglutinative" isn't technically correct. I'm actually not sure what the right word is to describe languages that have lots of big compounds with no spaces. (Which is the key issue here, as to why we need subword tokenization techniques).

ratsimihah 7y ago

It looks promising! Minor point, a requirements.txt file or something would be convenient to get started quickly.

jph00 7y ago
Once the book is released there will be a whole website and prebuilt environments and lots more to get started quickly. We didn't expect the draft to get this much attention, frankly!
- honzzz 7y ago
  Any estimate when the book will be released? Thanks :-)
  - jph00 7y ago
    July.
- ratsimihah 7y ago
  That's exciting! If pull requests are enabled I can always send one once I'm up and running! Looking forward to checking out the good stuff in there!
  - jph00 7y ago
    Actually someone already did a PR for it!
    - ratsimihah 7y ago
      Oh brilliant! That's what happens when I slack off checking out face masks haha!

cube2222 7y ago

I recommend everybody who didn’t check those out to do so. Not really being interested in ML, but doing all the available most popular courses to keep up, I really liked how fastai doesn’t just teach you ready and known models, but also how to compose differentiable building blocks to design NN’s yourself.

DrNuke 7y ago

Another smart move from fast.ai, this book is going to be the state-of-the-art reference for 2020 and a classic anthology of algo techniques for the mid term.

chrisa 7y ago

It's a neat idea to write the entire book as notebooks - so you can have run-able code right in line. Definitely excited to check this out; thanks Jeremy and Sylvain!

smohare 7y ago
These sorts of books have existed since the dawn of notebooks. While I think them useful for demonstrations, I don’t think there is much pedantic value (beyond a more static medium) in all honesty. Well-worked examples a student can consult while attempting to solve problems on their own is still of paramount importance.
- localhost 7y ago
  Why can’t we have both? Great examples AND an interactive, reproducible medium?
- timClicks 7y ago
  When you say "pedantic value", do you mean "pedagogic value"?
weego 7y ago
It's an awful idea, notebook stability ages like milk
- SkyMarshal 7y ago
  What do you mean by stability in this context?
  - khazhoux 7y ago
    I think he means they bit-rot. Stop working.
- fredmonroe 7y ago
  i think that's true of almost all tech books in paper form personally i think these work great because i can add cells to inspect data or try experiments easily as i'm reading to help me understand whats going on

sytelus 7y ago

Wow, Orielly lawyers are determined to screw this up. The thing is GPL v3 licensed which means I can’t copy any of the book code in my closed-source product or competitions or even MIT licensed code. The readme says I cannot make copies of this material but it’s ok to fork. Huh?

jph00 7y ago
No, the readme says you can make copies for personal use. If you want to use code in the book under a non GPL license, then you could just buy the book when it comes out. That doesn't seem like an unreasonable burden. PS: none of this is anything to do with O'Reilly or their lawyers.
- sytelus 7y ago
  Wait... so if you buy book then it seizes to be GPLed? This is quite confusing. For DL research, most code is MIT licenced and legal folks at many industrial labs would be quite hesitent to permit use of code from this repo with feels like legal minefield with different restrictions spread over multiple places including LICENSE, README, fastai website and perhaps printed book. I would highly recommand converting to one simple MIT license and call it a day (except for markdown cells).
  - bonoboTP 7y ago
    I don't get this perspective about the GPL. Look, they are giving you something for free, including the source code and the right to build upon it and publish modified versions. You can do basically whatever you want with it, as long as you pass on the freedoms that were granted to you. Is that unfair? Enjoying getting freedoms but not passing them on is not nice.
    - sytelus 7y ago
      I get GPL and fully appreciate its philosophy. The problem happens when you actually use it in practice. Because of its viral nature, anyone with different licensing must convert to GPL if they use your code. For many scenarios, this is actually not possible not just because of commercial secrets but the potential for opening up for security vulnerabilities when you don’t have resources or competitions where you should keep code secret until some time or simply because you have dependencies on other code which is very expensive to get rid off. Due to this reason, many companies forbid the use of GPL licensed software as well as release anything under it (because then you can’t use your own code!). Many other companies simply don't want the headache of checking all of their mess of legacy codebases with a myriad of dependencies that would be hard to untangle into GPL compatible open-source release. The legal and economic overhead when you use or release GPLed code is non-trivial. For this reason, the vast majority of open-source code released by big tech companies on GitHub is MIT/BSD licensed, which ironically is more "freeier" than GPL.

coderunner 7y ago

How does this compare to the video course? e.g. what are the differences?

jph00 7y ago
It's all new material. It'll be the basis of the next course coming in July. Or you can join the in person course from March in SF https://www.usfca.edu/data-institute/certificates/deep-learning-part-one https://www.usfca.edu/data-institute/certificates/deep-learn...

he11ow 7y ago

This is not intending to minimize in the slightest the amazing work that Jeremy does - I am a huge fan. But Fast.ai has TWO co-founders, and somehow, Rachel doesn't seem to get any credit in these discussions (not the book specifically, I'm talking about the overall enterprise). Not quite sure why; A lot of the content on the website is written by her, and it's clear she adds a lot of value to the endeavor as a whole.

FranzFerdiNaN 7y ago
I can think of a reason she isn’t getting the credit she deserves.
jph00 7y ago
Thank you for mentioning Rachel! :) She is working as the Founding Director of the Center for Applied Data Ethics nowadays, which is a very full-time job. So she hasn't been involved much in fastai v2 or the book (other than chapter 3, of which she's a co-author). She created and taught the NLP and Computational Linear Algebra courses, and has written most of the material on the fast.ai blog, and of course (as noted) co-founded fast.ai. Overall, I'd agree that she doesn't get as much credit as she deserves. That's perhaps partly due to her increasing focus on ethics issues, which aren't generally discussed much on HN (sadly). I also would say that Sylvain Gugger doesn't get as much credit as he should -- he has been an equal partner with me in creating the book and fastai library. (I discussed this response with Rachel prior to posting it.)