liuliu

57 ms·

751.
by liuliu 6y ago
Good article! But the comment on NVLink / PCIe4 (and PCIe3 x4 is good enough) doesn’t fit my experience. PCIe3 x4 can really impact your all-reduce performance for models such as Transformers. For ResNet, it can impact a bit too, but t
752.
by liuliu 6y ago
Yeah. Even if it is not a Titan, it is at a price that I am glad to pay for the performance (if the 1.5x faster than Titan claim holds true). Bert / Transformer models are incredibly memory hungry, and a sub-2k graphics card with 24GB
753.
by liuliu 6y ago
These use-cases are different. They mainly designed to workaround the issue where a differentiable loss cannot be designed. Thinking it as a e2e system for autonomous driving. Your end-goal is miles-driven without accident. This metrics is
754.
by liuliu 6y ago
It is from my own experience. But it looks like inference is a bit faster on CPU: https://medium.com/huggingface/benchmarking-transformers-pyt...
755.
by liuliu 6y ago
It absolutely can. PCIe 3x16 is at around 16GB/s speed, hence, uploading parameters at full speed would be around 50 seconds. That probably is the bottleneck for this case though. A minor issue is that the NVIDIA consumer cards (or any
756.
by liuliu 6y ago
I don't know. It has 175 billion parameters, thus, about 500GiB for floating point parameters. If these are bfloat16, it is mere 300GiB. We know CPU is about 50 times slower than GPU for transformer models, hence, we are looking at 4 t
757.
by liuliu 6y ago
Have we reached herd immunity against other coronaviruses?
758.
by liuliu 6y ago
If you are already familiar with Bazel or don't mind spending a week to learn, sure. I manage all my personal projects in Bazel as well. Otherwise, many people have positive experience with XcodeGen and that is more close to Xcode proj
759.
by liuliu 6y ago
Xcode project (pbxproj) is simply ill-equipped for proper code reviews. Any modest size team should remove their Xcode projects from their codebase and move to either Bazel, Swift Package Manager or Xcodegen. It is really not that difficult
760.
by liuliu 6y ago
We knew hand-crafted program in limited domains can work for NLP, computer vision and voice recognition long time ago. The challenge is always, that the limited domain can be extremely limited, and to get anything practically interesting re
761.
by liuliu 6y ago
They sourced a lot of chips from MediaTek for their mid-end / low-end phones. It is likely that they are going to collaborate with MediaTek for high-end chips or source from Qualcomm.
762.
Who Should Buy ARM?(drz.today)
4 points by liuliu 6y ago|0 comments
763.
by liuliu 6y ago
DMCA is for one. I am not sure for "national security" reasons what options would be. It is definitely challenging to come up with excuses within rules of law. I would imagine that some combinations of failures to meet FCC regulat
764.
by liuliu 6y ago
Just echo back on U.S. doesn't have precedents of blocking software in the article. I don't think that's true. U.S. has both means (file request to Apple and Google to pull software off the shelf, remove the domain from root
765.
by liuliu 6y ago
You cannot have massive job program without state financial going into red. And states have to balance their book every year.
766.
by liuliu 6y ago
I disagree. Yes, it is just a decoder of transformer. But it looks like we are really close, with some tweaks on the network structure, reward function design and inputs / outputs. On the same time, GPT-3 also points how far away we ar
767.
by liuliu 6y ago
Agree in principle (you shouldn't rely on shutdown sequence to do correct thing). It is still nice to have a grace shutdown option. For some cases, it is just a little bit "nicer". Imagining you have one service instance need
768.
by liuliu 6y ago
Why Apple cannot use discrete graphics? We don't know that yet. NVIDIA already starts to have support for aarch64 so you can expect AMD not that far away.
769.
by liuliu 6y ago
That is a long chain. Internal promo doesn't work that way. That's why it is easier to get promoted when you work directly on ads.
770.
by liuliu 6y ago
Interesting. I thought that OT implementations require peers to have at least one authoritative view of the transformations somewhere, and then distribute different operation sequences (after transformed) to different node. That, in simples
771.
by liuliu 6y ago
The talk makes it clear, "no conflict" in CRDT sense is that states update, even ordered differently will converge to the same state. For practical sense, it is last-write-win baked in the the CRDT design (you can choose alternati
772.
by liuliu 6y ago
And most SDKs are provided as static frameworks. You have to be technically competent to know how to wrap a static framework as dynamic, and then use dlopen to load them properly with a rather high one-time runtime cost. A mess.
773.
by liuliu 6y ago
The truth is, 1st-party apps are well-supported but 3rd-party SDKs provided by these companies, especially the ones don't generate revenues (Facebook SDK, and many others) are simply not well-supported within the organization. They are
774.
by liuliu 6y ago
Yeah. To make the matter worse, they have updated libnccl-dev from their apt repo to be CUDA-11 based a few weeks ago. That breaks my CUDA app (because it is still on 10.2 and not compatible) in interesting ways. apt-hold libnccl-dev for a
775.
by liuliu 6y ago
It all sounds funny but the final solution is not far away except the JSON bit. FriendFeed 11 years ago first popularized the concept of storing schema-less data in MySQL: https://news.ycombinator.com/item?id=496946 . Uber d
776.
Dflat: SQLite \heart FlatBuffers(github.com)
2 points by liuliu 6y ago|0 comments
777.
by liuliu 6y ago
We use Flexbuffer to serialize aribitary Objective-C objects through reflection (rather than NSCoding). It is faster and stable for us in a few years actually (we implemented that in early 2019).
778.
by liuliu 6y ago
I use a 32" monitor at 4K with 1x scaling. It is a productivity booster for me. More horizontal space is just much nicer to host debug sessions and jump between code sections. At the same time (of debugging), I can tile web browser to
779.
by liuliu 6y ago
Agreed. For me, I think the internal fragmentation is well-understood. I would love to know more about the external fragmentation critique they had in this article though.
780.
by liuliu 6y ago
WAL mode can help large part of the fsync issues, but it does break the "one file" niceties SQLite has. If you don't care about losing some recent commits, WAL is definitely the way to go: https://www.sqlite.org&#x

More ›