daviesliu

12 ms·

91.
by daviesliu 7y ago
Have you publish the workload tracing from Spotify? We are very interesting on the results you got, looking for a way to replicate that using our setup.
92.
by daviesliu 8y ago
> Also, Huawei in their early ages stole one of Cisco router's source code and priced out Cisco with an exact same product compatible with Cisco systems, one Chinese academic invovled in modifying admitted it in an article. One of t
93.
by daviesliu 8y ago
I have done the things before. Before returning the work laptop to employer on the last day, I cleaned out all the company stuff to left my personal things (this is my only laptop with my personal stuff in it), then move them out and clean
94.
Show HN: JuiceSync – Move your data between any clouds or regions(github.com)
3 points by daviesliu 8y ago|0 comments
95.
by daviesliu 8y ago
In my opion, Cephfs and GlusterFS have much better support for Pandas and others, you don't need change any code between local development and train an distruted way, rather than a path. Second, the Python library for HDFS provides onl
96.
by daviesliu 8y ago
The rbd image can't be used as a shared storage for database.
97.
by daviesliu 9y ago
JuiceFS [1] could be a replacement for S3QL, which has better support for B2, can also be mounted on multiple machines in the same time. If you don't have more than 1TiB, JuiceFS is totally free. Disclaimer: Founder of JuiceFS here.
98.
by daviesliu 9y ago
MooseFS have a highly optimized service to load all the metadata into memory (similar to Redis).
99.
by daviesliu 9y ago
My friend also got a real big bill for S3 PUT requests, they are just too expensive. When we built JuiceFS [1], a POSIX filesystem on top of S3, there is feature in the roadmap to combine multiple small files together before uploading into
100.
by daviesliu 10y ago
After reading the paper, I felt disappointed that: 1) Bizur is used for building a scalable object storage rather than a filesystem, which does not even support atomic rename for single key (the hash of key will change, it will be moved to
101.
by daviesliu 10y ago
Have you looked into MooseFS [1]? It seems like moving towards the direction you may like. [1] http://moosefs.org/
102.
by daviesliu 10y ago
ObjectiveFS does provide a flat fee ($15 per client), is that what you what? If the $ per byte of storage is relatively small comparing to the cost of S3, I actually like that model, pay as you go.
103.
Ask HN: Will you hope S3 become a real file system?
5 points by daviesliu 10y ago|6 comments
104.
by daviesliu 10y ago
BFS has a very limited FUSE client. There is another distributed file system that support full POSIX semantics with well tuned FUSE client, called MooseFS [1]. My ex-employer used that in production for about 8 years, the biggest cluster ha
105.
Aevy use my face without authorization and make fake story(twitter.com)
6 points by daviesliu 10y ago|3 comments
106.
by daviesliu 10y ago
Right now, vectorization is ONLY used in Parquet reader and in-memory cache, other operators still work in one-row-at-a-time mode, but compiled into single (or a few) Java function(s), the intermediate results of these operators could be ho
107.
by daviesliu 12y ago
Could your confirm that the MOV is copying PTEs? I doubt that maybe it's copying the heap, because of copy-on-write. After fork(), the parent write to heap randomly, so it will copy more data with jemalloc than libc (because of huge pa
108.
by daviesliu 12y ago
Have you tried to use huge page to reduce the time of fork()? There also a similar problem for master of MFS, it will be great if you have some experience in it. EDIT: https://www.kernel.org/doc/Documentation/vm&#x
109.
by daviesliu 13y ago
Cache miss metric in OProfile may be the better solution than this.