chrislusf

7 ms·

31.
by chrislusf 3y ago
Should not be a problem. One similar use case used Cassandra as SeaweedFS filer store, and created thousands of files per second in a temp folder, and moved the files to a final folder. It caused a lot of tombstones for the updates in Cassa
32.
by chrislusf 3y ago
Thanks for sharing! I work on SeaweedFS. SeaweedFS is built on top of a blob storage based on Facebook's Haystack paper. The features are not fully developed yet, but what makes it different is a new way of programming for the cloud er
33.
by chrislusf 3y ago
I work on SeaweedFS. Ping me if you need some help to understand why SeaweedFS is much faster.
34.
by chrislusf 4y ago
It is still growing. This is not the case since many months ago. Also, welcome to make a PR.
35.
by chrislusf 4y ago
Another research suggests rubbing eyes also increase risk for Alzheimers. The virus getting into the eyes will slowly eat the brain.
36.
by chrislusf 4y ago
Can someone please critique on the difference between SeaweedFS and Colossus? https://github.com/seaweedfs/seaweedfs Need some advice on how to make SeaweedFS more scalable.
37.
by chrislusf 4y ago
Thanks for the detailed clarification! I am too deep into the SeaweedFS low level details and am all ears on how to make it simpler to use. SeaweedFS has weekly releases and is constantly evolving. Depending on your case, you may need to ad
38.
by chrislusf 4y ago
SeaweedFS author here. Thanks for your candid answer. You do not need to use multiple SeaweedFS components. Just download the binary and run "weed server -s3". There are many other components, but you do not really need to use the
39.
by chrislusf 4y ago
Besides storage cost, S3 API access cost can also be high if frequently accessed. And latency is unpredictable. You can use SeaweedFS Remote Object Store Gateway to cache S3 (or any S3 API compatible vendors) to local servers, and access th
40.
SeaweedFS and YDB(github.com)
1 points by chrislusf 4y ago|0 comments
41.
by chrislusf 5y ago
In snowflake id generation mode, the "which volume is writable" information can be read from other follower masters.
42.
by chrislusf 5y ago
This youtuber needs to make a living. Roblox is great that it built up an ecosystem where people can contribute and get rewarded. It is a positive feedback loop. Not like open source software, where the financial loop is broken. I am pretty
43.
by chrislusf 5y ago
Your answer should be voted to the top! :) OSS contributors are rarely noticed or appreciated. Did HashiCorp ever sponsor you or share any revenue with you? The OSS ecosystem is broken.
44.
by chrislusf 5y ago
Thanks for the details! 2 days with 1.5TB vs 3 days with 300GB, that is good to know! :) btw: If you use "weed filer.copy", it should be much faster. Rsync needs to go through FUSE mount.
45.
by chrislusf 5y ago
This is a fair complaint. :) For filer metadata, you should just pick the one you are most familiar with. There is a wiki page for production setup. https://github.com/chrislusf/seaweedfs/wiki/Production-Setup
46.
by chrislusf 5y ago
No preference actually. SeaweedFS is portable, the data file and the metadata.
47.
by chrislusf 5y ago
Many systems look similar. Ceph, MooseFS, etc. The difference are in details. Not possible to benchmark everything. And each use case has different access pattern, CPU, memory, network. It's better to benchmark with your own use cases.
48.
by chrislusf 5y ago
Actually SeaweedFS does just that, append small objects into larger blocks during write or update, and run garbage collection during compaction.
49.
by chrislusf 5y ago
Aha, I replied above and you are saying the same thing! Small objects, 4K ~ 4MB, are very common in machine learning area(audio, video, text), or surveillance, game assets, etc. SeaweedFS saw a lot of usage in these areas.
50.
by chrislusf 5y ago
Usually you would benchmark on difficult tasks, not the easiest one. For computers, batch IO operations are much faster than random IO and can easily saturate the network. This benchmark uses large batch size, 64MB, to test. There is nothin
51.
by chrislusf 5y ago
SeaweedFS is very different from Nginx. It's just the names are so similar. There are 2 ways to cache: write through and write back. You are using write through, which needs to write to the remote storage before returning. Write back i
52.
by chrislusf 5y ago
Interesting that SeaweedFS also has a similar named "Gateway to Remote Object Storage" https://github.com/chrislusf/seaweedfs/wiki/Gateway-to-Remot... The difference is that SeaweedFS can support bo
53.
by chrislusf 5y ago
Polish cloth max will double the cleaning capability with 2x size, at $29.99, with unprecedented industrial leading price-to-area ratio.
54.
by chrislusf 5y ago
Nice work on qocache! I checked the code. Is it loading data as "map[string]*list.Element"?
55.
by chrislusf 5y ago
Nice work! Does Roapi needs to load the data file into memory first, and then answer the queries? Or does it handle the query while streaming through the data?
56.
by chrislusf 5y ago
There are simply too many object store vendors, with different storage pricing, egress pricing, network speed, and reliabilities. What about using SeaweedFS gateway to remote object store? https://github.com/chrislusf/s
57.
by chrislusf 5y ago
Most cloud object storage can be a good off-site backup. But accessing your data fast and cheap is not easy. That is why SeaweedFS added a gateway to remote object store. https://github.com/chrislusf/seaweedfs/wiki
58.
by chrislusf 5y ago
As a main contributor to an open source project ( https://github.com/chrislusf/seaweedfs ), I can confirm that this finding is so true. However, seems this research did not look into Apache projects, which basically main
59.
by chrislusf 5y ago
The option has 7 languages. So 7 == 80 ?
60.
by chrislusf 5y ago
"And in 2016, Harvard University scientists concluded the molasses disaster would have killed fewer people had it happened in the summer. They figured the winter air cooled and then thickened the goo, making it harder to rescue victims

More ›