itamarst

27 ms·

271.
by itamarst 7y ago
The article, at least, doesn't say anything about HDF5 not be chunked. I recommended Zarr on the basis that threading seems to work much better and compression options are better.
272.
Loading NumPy arrays from disk: mmap() vs. Zarr/HDF5(pythonspeed.com)
119 points by itamarst 7y ago|52 comments
273.
Where’s that log file? Debugging failed Docker builds(pythonspeed.com)
1 points by itamarst 7y ago|0 comments
274.
Reducing Pandas memory usage: Reading in chunks(pythonspeed.com)
2 points by itamarst 7y ago|0 comments
275.
by itamarst 7y ago
That was mostly about image build time (and it's pretty Python specific), this article is more about runtime performance.
276.
by itamarst 7y ago
Threads per core is 1, according lscpu. This is a Xeon desktop. Here's a documented example of an OpenBLAS threading issue from someone rather more knowledgeable than I am: https://github.com/xianyi/OpenBLAS/i
277.
The Parallelism Blues: when faster code is slower(pythonspeed.com)
2 points by itamarst 7y ago|2 comments
278.
by itamarst 7y ago
Probably just the wrong company. Bad management can make any career awful. And yes, there are lots of bad companies. But there are also good companies. I have worked at companies where people were very careful to test their software, and wh
279.
Book Launch: Just Enough Docker Packaging(pythonspeed.com)
1 points by itamarst 7y ago|0 comments
280.
by itamarst 7y ago
On Alpine it doesn't even download the wheel at all, it just downloads the source code.
281.
Alpine makes Python Docker builds slower, and images larger(pythonspeed.com)
306 points by itamarst 7y ago|148 comments
282.
Stuck on Docker packaging? Get up to speed quickly(pythonspeed.com)
1 points by itamarst 7y ago|0 comments
283.
by itamarst 7y ago
Iraq used chemical weapons in its war against Iran. But the US was supporting Iraq at the time. https://en.wikipedia.org/wiki/Iraqi_chemical_attacks_against...
284.
by itamarst 7y ago
1. It's not perfect if it's insecure. You really don't want to run your Docker images as root: https://pythonspeed.com/articles/root-capabilities-docker-se... 2. It's not perfect to include gcc; it&
285.
by itamarst 7y ago
Perhaps it's worth quoting Larry Page and Sergey Brin, in the original paper on the Google search engine: > "The goals of the advertising business model do not always correspond to providing quality search to users. ...we expec
286.
by itamarst 7y ago
Switching away from problematic terminology is also a good opportunity to make it more accurate and specific. In some situations it's "process A allows writes, process B only allow reads", in which case "writer/read
287.
by itamarst 7y ago
The easiest way is negotiating at your current job. I realize this hasn't worked for you, but it is possible. The next easiest way is negotiating at new job. You don't tell them upfront you want part-time job, you just get an offe
288.
Activating a Conda Environment in Your Dockerfile(pythonspeed.com)
1 points by itamarst 7y ago|0 comments
289.
by itamarst 7y ago
https://en.wikipedia.org/wiki/Array_programming might be starting point.
290.
by itamarst 7y ago
Even if your API provides both choices, you can use the pattern described in the article to make the copying variation more memory efficient.
291.
by itamarst 7y ago
Original author here: to reduce confusion I changed the article to use a new term, "hidden mutability".
292.
by itamarst 7y ago
I'm using ConvertKit for newsletter signups, and choices are either: 1. Validate emails with recaptcha (which is why you see that thing in the corner.) 2. no email validation at all, so random people get spammed with confirma-you-want-
293.
by itamarst 7y ago
Are you familiar with data-oriented design? The game-developer equivalent of the array-processing scientific programmers do. I suspect if you look at how they do things you'd find some domain-specific design patterns. https:/&#x2
294.
Copying data is wasteful, mutating data is dangerous(pythonspeed.com)
2 points by itamarst 7y ago|0 comments
295.
Reducing NumPy memory usage with lossless compression(pythonspeed.com)
1 points by itamarst 7y ago|0 comments
296.
by itamarst 7y ago
Alternatives if you don't want to risk missing some file that only gets loaded 10 minutes in: 1. Start with a small base image, e.g. for Python there's "python:3.7-slim". For Python I'm not a fan of Alpine, but for
297.
by itamarst 7y ago
Multi-GB images still seem... big. Though to be fair in many situations those 60 seconds are meaningless, so might not be worth the time to optimize. Often images that large are the result of including build toolchains, which you can omit w
298.
by itamarst 7y ago
In general, the bottom layer is a small Linux distro (Ubuntu, Debian, Alpine), just enough to get you going. Ideally you choose a base image that is getting security updates on regular basis, and that doesn't have significant changes o
299.
Installing system packages in Docker with minimal bloat(pythonspeed.com)
19 points by itamarst 7y ago|3 comments
300.
The Myth of Talent(codewithoutrules.com)
2 points by itamarst 7y ago|1 comments

More ›