6 ms·

For more detail plus working code, lesson 4 of the fast.ai course uses this technique to obtain (what was at time of writing) a state of the art result on the i

• dantheman 8y ago
  For anyone just getting started on this I can't recommend fast.ai enough. It's extremely well done and I found it very intuitive. You are able to quickly apply some very advanced techniques.
  • sweezyjeezy 8y ago
    I want to like fast.ai more, but in my opinion their code quality is just not good enough. Everything is badly named, badly explained / not explained at all, and if you need to adapt any of their code to work in different domains, you are going to have a bad time.
    • jph00 8y ago
      The naming convention is fully documented and based on decades of research : https://github.com/fastai/fastai/blob/master/docs/style.md https://github.com/fastai/fastai/blob/master/docs/style.md It's fine if you don't like it (although it may be you're just not quite used to it yet), but I'm not sure it's fair to call it "bad". Everything is fully customisable in pytorch and this is explained with many examples in part 2 of the course. Written documentation is being worked on as we move towards a first release later this year (currently the library is still pre-release - it works fine and is used at many big and small companies, but there's still much to do to get it to a v1.0).
      • meesterdude 8y ago
        I agree with the poster you're replying to - the naming is not intuitive or approachable without that previous body of knowledge. It's enough that I would call it bad - I would flag the crap out of it in a code review. I'm glad you linked to the style guide - i've often wondered where certain names come from. It's not all bad. K for key, V for value, i for index... no disagreement. But the seeming aversion to any variable name longer than 2-3 characters might be great in mathematics, but makes for a nightmare of code which is focused more on writing. It doesn't need to go the extreme of UIMutableUserNotificationAction; but at least use words. AI is already clever enough; it doesn't need to be made more cryptic with poor naming conventions. I get that for people in the industry it may be fine - but for it to break out into general use it'll need to be more social; which means simpler & clearer verbage.
      • elyase 8y ago
        fast.ai is great and I also want to love it but I find the coding style extremely unconventional and uninviting to the point that I only use the library when there is really no better alternative (ex for the lr finding features). Variable naming and wildcard imports would be my main complains. But thanks a lot Jeremy for the course and for posting the style guide.
      • sweezyjeezy 8y ago
        > It's fine if you don't like it (although it may be you're just not quite used to it yet), but I'm not sure it's fair to call it "bad". It is totally fair to call a naming convention where all variables are 2 or 3 letter acronyms bad, at the very least un-pythonic, and certainly not suitable for an educational tool. If you think that it's good practice to make your code shorter at the expense your users needing an abbreviation guide, you're going to make adoption a much harder sell, especially if you're pitching at python devs. Unlike the last few decades, we now live in an age of autocomplete, there is absolutely no need for this. Here is a few lines from the imdb ipython notebook (https://github.com/fastai/fastai/blob/master/courses/dl2/imdb.ipynb https://github.com/fastai/fastai/blob/master/courses/dl2/imd...), that gave me a headache just to look at - trn_dl = LanguageModelLoader(np.concatenate(trn_lm), bs, bptt) val_dl = LanguageModelLoader(np.concatenate(val_lm), bs, bptt) md = LanguageModelData(PATH, 1, vs, trn_dl, val_dl, bs=bs, bptt=bptt) The cognitive load put on someone unfamiliar with your code is not acceptable: LanguageModelData takes two LanguageModelLoaders as arguments, and then later in the code it produces a model? Surely these two classes are named the wrong way round? You haven't explained what 'bs', or '1' are supposed to be. To figure out/ remember what the other variables are, you need to chase up and down the notebook, which would easily be avoided if you just gave them descriptive names. You only use these variables once or twice, so I don't even understand what is gained by making their names so terse.
• jph00 8y ago
  Well spotted - this is where I first created the algorithm that became ULMFiT! I wanted to show an example of transfer learning outside of computer vision for the course but couldn't find anything compelling. I was pretty sure a language model would work really well in NLP so tried it out, and was totally shocked when the very first model best the previous state of the art! Sebastian (author of this article) saw the lesson, and was kind enough to complete lots of experiments to test out the approach more carefully, and did a great job of writing up the results in a paper, which was then accepted by the ACL.
  • deleted 8y ago
    [deleted]
• mikert5671 8y ago
  Why is this in the comments section of every AI article?