9 ms·
Flashlight: Fast and flexible machine learning in C++
- angry_octet 6y agoThe name has unfortunate verbal similarities with a well known sex toy, especially when different accents are considered. I am actually astounded they went ahead with this name.
- andrew_v4 6y agoReally? In North American english, a flashlight is a handheld light, there is no thought of the toy you're referring to when it is mentioned. That "light" got its name as a play on flashlight. It's funny to see that now become the primary connotation in some peoples minds.
- CharlesW 6y ago> In North American english, a flashlight is a handheld light, there is no thought of the toy you're referring to when it is mentioned. Sample size of 1, but this North American immediately thought "oh no, how unfortunate" on seeing the headline. (Admittedly, I do listen to more podcasts than the average person.)
- whoopdedo 6y agoAnd yet millions of electric torches are sold in the United States each year without confusion. (Although not, let's be honest, without amusement.)
- angry_octet 6y agoI'd just choose a name with a greater Hamming distance from sexual harassment.
- aspaviento 6y agoUntil your super chaste name is used for something related with sex.
- whimsicalism 6y agoNow that I think about it... Docker does sound a bit like Dick-er.
- cultofmetatron 6y agotheres also "docking." a sex act between two men of which the details I'll let you refer to google at your own discretion.
- whimsicalism 6y agoWe really are an innovative species!
- eptcyka 6y agoThe sex toy came after flashlight the portable light source.
- igorkraw 6y agoIs this not a thing in the US? https://en.wikipedia.org/wiki/Flashlight https://en.wikipedia.org/wiki/Flashlight I assume they are going for "batteries included" connections?
- Narew 6y agoFlashlight vs torch (pytorch). ^^
- angry_octet 6y agoYes, I could see the construction, I just can't think why they wouldn't anticipate the problems it could cause down the track.
- whimsicalism 6y agoWhat? Nobody in their right mind thinks "fleshlight" immediately upon hearing the word "flashlight" Is this some sort of performance art comment?
- throwaway5849 6y agoReviving my old throwaway account to confirm that I indeed thought of "fleshlight" immediately after reading the announcement. Keep in mind that the average coder is far more likely to read Reddit daily (where this reference comes up surprisingly often) than to do things that require a flashlight (own a house somewhere remote, etc)
- jcelerier 6y agoas a non-native english speaker I'll humbly admit that fleshlight was indeed the first word that popped into my mind when reading the title, because I never say flashlight but lampe-torche (while a "fleshlight" is an occasional fun present / recurrent meme for friends birthday or stuff like that)
- kyawzazaw 6y ago
- randshift 6y agoI'm going to venture a guess and say it's more likely that people will associate this with millions upon millions of flashlights that have been sold in the United States, rather than whatever it is you're talking about.
- a-dub 6y agoyou do realize that the recently introduced sex toy was both named and designed to resemble flashlights, right? that said, interesting decision to ditch the scripting language. when i think of these sorts of things, immediately think that embedding or supporting a scripting language makes a ton of sense. i'm curious what the thinking was to just go full c++? i suppose they just decided that modern c++ was easy enough and this was better for simplifying production?
- angry_octet 6y agoWell this observation has produced a furious negative reaction, which I suspect is of both puritanical and chauvinist origin. Perhaps because I can't see very well I had uncertainty when reading the article title. But unless you're quite innocent (i.e. not a user of the internet) I don't see how you could avoid knowing about such devices. Knowing many programmers, I also know that they span the full spectrum from Bible-studies to BDSM (though that might be a horseshoe), and many are prone to juvenile humour which can become sexually toned. So I know that someone will use it in this context. When you become a manager you choose to avoid creating such problems. Some of you probably see this tone policing -- you are correct, suck it up. For those who don't understand what Hamming distance is: https://en.m.wikipedia.org/wiki/Hamming_distance https://en.m.wikipedia.org/wiki/Hamming_distance
- secondcoming 6y agoYou accidentally the whole thing
- machineko 6y agoBuilding flashlights was so frustrating on WSL for me take ~3/4h and still cuda wasn't working.
- whimsicalism 6y agoI wish Flashlight had a way to easily export models to other frameworks.
- suyash 6y agothat can be done is they support ONNX
- kyawzazaw 6y agoflashlight is such a common word used for an appliance. I would never think of anything else upon hearing this. Millions of people I assume as well.
- dwrodri 6y agoThis seems really cool, but I don't get why they would pour work into this while simultaneously building a C++ front-end for PyTorch[1]. Per the blog post, both frameworks have the goal of empowering ML researchers to iterate on ML models in such a way that it becomes easier to reason about performance than it would be talking to a bunch of dynamically linked object files behind an interpreter. Facebook is a huge company with lots of money, but production-ready ML frameworks are a HUGE undertaking. I don't get how tech companies can be simultaneously recruiting and doing interviews year-round, paying huge salaries and then putting them under a layer of management that thinks two different C++ ML frameworks with the same goals is a good idea. Facebook's work on Tensor Comprehensions, Halide (not originally FB but they have contributed heavily), Glow, and PyTorch all contributed to the ML space by offering alternatives (with innovative UX/technical differences) to the Tensorflow ecosystem. Not all of these contributions had novelty, but I respect FB's choice to work on something for the sole purpose of not having it's direction beholden to the whims of Alphabet (see: Swift for Tensorflow). I just don't see what this adds which FB isn't already working on within a different project. What am I missing? 1 = https://pytorch.org/cppdocs/ https://pytorch.org/cppdocs/
- whimsicalism 6y agoFlashlight is much lower level and gives more fine-grained performance control. For instance, I don't think there is really any way to do real-time speech recognition that is fast with PyTorch because of how it is architected. From my understanding, Tensor Comprehensions and Halide are both very tentative research projects. > not having it's direction beholden to the whims of Alphabet (see: Swift for Tensorflow). I don't think this is an accurate recreation of the history that led to FB working on pytorch.
- dwrodri 6y agoThis is a very interesting claim. I find it credible because it stands to reason that projects like DeepSpeed[1] and TorchScript[2] wouldn't need to exist if inference performance of research PyTorch models was satisfactory for production, but often case it isn't. It appears as though Flashlight is built on ArrayFire. I haven't seen how gradients are managed in arrayfire-ml, but perhaps it is the case that the autograd implementation in PyTorch was a bottleneck and this is a ground up approach. Editing as I didn't address your second point. I can neither confirm or deny the motivations for creating Torch being related to FB's desire to depend on an Alphabet-managed codebase. I know there are lots of reasons why programmers prefer the UX of the PyTorch Python API (I do as well), and there are probably other reasons I can't recall off the top of my head. I am only saying that PyTorch is contributing to the ML ecosystem already by the sole virtue that it isn't a Google product. [1] = https://github.com/microsoft/DeepSpeed https://github.com/microsoft/DeepSpeed [2] = https://pytorch.org/docs/stable/jit.html https://pytorch.org/docs/stable/jit.html
- hnachraf 6y agoGreat work by the team, I like that they re-used ArrayFire instead of Yet Another Tensor Library feels refreshing to use native code again to build ML Pipelines for these kind of tasks.
- melonakos 6y agoThank you!
- drtournier 6y agoFlashlight vs. Scikitlearn ...
- dynamite-ready 6y agoDoes anyone else think that C++ makes more sense for ML work than Python? I'd been thinking so for years. Both for deployment/performance and data wrangling purposes.
- whimsicalism 6y agoIn general, I think languages with static typing are preferable. C++ seems ideal for me right now because it is the only other language with a somewhat mature stack (perhaps Julia as well, but I haven't played too much around with that).
- eigenspace 6y agoJulia is dynamically typed FYI. However, I find that usually when people say they want a statically typed language, they don't actually want static typing (which is just a restriction of language semantics), but instead they want a language with a powerful type system that does work for them. In this case, julia absolutely is worth checking out. It does static analysis on it's intermediate representation to automatically identify and isolate statically inferrable regions of programs and then stitches them together if it ever encounters any dynamism. Julia's type system is extremely powerful and expressive and can do a lot of things that are incredibly difficult in fully static languages precisely because it does allow dynamism.
- codebolt 6y agoAbsolutely. In 2010 I used FANN for my Msc thesis research, and found it pretty easy to make my own little training sim for stock price data on top of it. Haven't done any ML work since, but I always scratched my head over how Python became the most popular language for this domain.
- person_of_color 6y agostock price ml? why aren’t you rich?
- 6y ago
- suyash 6y agoWonder why they didn't leverage TorchScript ? However this is a welcome news in ML Community. Would love to see this library break away from NVIDIA CUDA dependency and optimize for Apple M1 chips
- pilooch 6y agoTorchscript is very complicated. Its internal API was not truely documented for external use last time I checked it. You can't even export to torchscript from libtorch C++, it only works from Python AFAIK.
- wdpk 6y agoAnyone has experience with the JIT in arrayfire? https://arrayfire.com/performance-improvements-to-jit-in-arrayfire-v3-4/ https://arrayfire.com/performance-improvements-to-jit-in-arr... Many interesting features in this framework, autograd looks neat too. I guess the deployment in real world C++ apps will be easier than PyTorch or Tensorflow, especially at the edge in scenarios with little or no network access.
- melonakos 6y agoHappy to speak to JIT in ArrayFire if you have any specific questions. I am ArrayFire's CEO/Co-Founder.
- nbeleski 6y agoCan this statically compiled? Why would anyone opt for this instead of Torch or DLIB?
- stephc_int13 6y agoGiven the very high computing requirements of Machine Learning, I've always be perplexed by the seemingly widespread and unquestionable preference of Python over native code (typically C/C++). I guess performance was considered less critical than clarity/flexibility. But it seems that people are discovering that complex code tend to be hard to read/modify no matter the language...
- andrew_v4 6y agoPython (in common ML frameworks) is really just wrapping well optimized native code. The overhead is very minimal. The advantage here is about easy access to the internals. I don't know that any material speedup is expected just because its written in C++
- stephc_int13 6y agoI've worked at some point on a commercial game engine written in C++ at the core but with many Lua components and API, for convenience. The reasoning wat the same, all the heavy lifting done with fast native code, and everything written in Lua was mostly glue code without real performance impact. Turns out the engine was slow and difficult to maintain because of the many interfaces. They ditched it a few years later... I think you could say the same thing about javascript on the web, this is mostly glue code, all the heavy lifting is done by optimized C++ or Rust code in the browser. And yet it is difficult to run a web mail client if your computer is a bit old... This is probably related in some ways to Amdahl's law https://en.wikipedia.org/wiki/Amdahl%27s_law https://en.wikipedia.org/wiki/Amdahl%27s_law
- pjmlp 6y agoThat was the same reasoning why Unreal dropped UnrealScript. https://www.gamasutra.com/view/news/213647/Epics_Tim_Sweeney_lays_out_the_case_for_Unreal_Engine_4.php https://www.gamasutra.com/view/news/213647/Epics_Tim_Sweeney... However a couple of years later and Epic is bringing scripting back with Verse https://twitter.com/saji8k/status/1339709691564179464?s=20 https://twitter.com/saji8k/status/1339709691564179464?s=20 Because the problem is not the scripting, but how it is done. I imagine that the engine you mention did not use any kind of compiler for Lua, e.g. LuaJIT, nor batch calls across the marshalling layer. Same happens on the Web, JavaScript JIT can outperform WebAssembly if one keeps switching all the time between layers, instead of doing batch requests.
- flemishgun 6y agoVery cool to see more C++-based machine learning efforts. The language still needs a good dataframe abstraction (maybe XFrame?), but with matrix algebra provided by Armadillo/Eigen and other long-time machine learning libraries like mlpack, Shogun, Shark (and if you want to include C, Darknet), personally I think the future is bright for machine learning in C++---especially for production and deployment applications.
- deleted 6y ago[deleted]
- maeil 6y agoI sometimes use Talon, a voice control app mostly used by developers. Iirc the developer incorporated Flashlight but encountered sigificant bugs and slow response times to issues and ended up switching to a different framework. At the very least it didn't feel much tested for real-world usage yet.
- brobdingnagians 6y agoThis makes a lot of sense when done well. Researchers need to have full control and quickly change things with the least amount of cognitive overhead for comprehending the engine. For people who know C++ well, it is a minimalistic C++ library with exactly what you need and nothing else. No cross language concerns or bindings, just plain C++. Using modern c++: > Modern C++ also obviates the need for tasks like memory management while providing powerful tools for functional programming. > Flashlight supports doing research in C++ with no need to adjust external fixtures or bindings and no need for adapters to do things like threading, memory mapping, or interoperating with low-level hardware. Easy integration with other C++ libraries: > Flashlight makes it trivial to build new low-level computational abstractions. You can cleanly integrate CUDA or OpenCL kernels, Halide AOT pipelines, or other custom C/C++ code with minimal effort. Customizable with fast build times: > And when you change Flashlight’s core components, it takes just seconds to rebuild the entire library and its training pipelines, thanks to its minimalist design and freedom from language bindings.
- melonakos 6y agoFlashlight is a great project, and we're proud to be part of it. I'm with ArrayFire. If anyone has questions or feedback for ArrayFire, I'm here to be useful however I can!