10 ms·
ownCloud Infinite Scale: Go instead of PHP, microservices instead of LAMP
- veddox 6y agoHere's a more detailed (German) discussion of what the rewrite entails: https://www.heise.de//news/ownCloud-Infinite-Scale-Go-statt-PHP-Microservices-statt-LAMP-5029244.html https://www.heise.de//news/ownCloud-Infinite-Scale-Go-statt-... I wonder what this will mean for the competition with Nextcloud? So far, NC seemed to be more active and slightly better than OC, but this move may change the playing board quite drastically.
- uselpa 6y agoWhy would that change the competition? I don't think that at the technical underpinnings have been a problem to rollout one or the other. Honestly curious.
- 8fingerlouie 6y agoOther than possibly gaining performance from Go vs PHP, as well as less memory consumption. I have no idea how large scale Next/Own-cloud implementations behave, but i'd assume you just throw more hardware after it. What will change though is the relatively well tested and hardened code that is Owncloud today. Switching to Go doesn't just magically make your webapp secure.
- piaste 6y agoI run Nextcloud on a Pi 4 and it's a struggle to get acceptable performance out of it. The ecosystem is nice, but I would absolutely trade a few less plugin for a smoother file management experience. I will definitely be checking out OCIS, assuming I find no reason for concern from a license / privacy / security scale. Golang isn't an automatic guarantee of performance, and "microservices" has me pause in the context of a personal server, but it happens that most of the Golang-based applications I host tend to perform extremely well. This coming from someone who strongly dislikes the "simple is better than correct" design of golang.
- znpy 6y agoThe problem is probably the pi4. I run nextcloud 20 on a i5-3470 CPU @ 3.20GHz (2c/4t) and for a small use case it works very well. The storage is a rotational (magnetic) hard drive (2TB, seagate barracuda, with zfs 0.8.something)
- johnchristopher 6y agoFWIW I always had performance problems running nextcloud on cheap VPSes while owncloud behaves more smoothly. Anecdata though.
- znpy 6y agoMaybe I hadn't been sufficiently clear: the hardware I mentioned is physical hardware, running at home. No virtual OS or sharing the physical cpu with other tenants. Cheap vps provider often do overselling on the bet that most customers won't be actually using the cpus in user-space 100% of the time (and that's often true). If your provider isn't doing anything nasty, you should be able to see "stolen time" (often indicated with 'st') in top/htop: that's when your vm wasn't actually scheduled on the real cpu.
- johnchristopher 6y agoI am inclined to believe you. I am giving nextcloud another try this weekend on my pi (I just found out about nextcloudpi). edit: Aaaaand nextcloudpi is stuck in an endless loop after first login. What was I expecting. https://help.nextcloud.com/t/docker-install-says-initializing-nextcloudpi-for-the-first-time-forever/72687 https://help.nextcloud.com/t/docker-install-says-initializin... And I am not the only one. I'll try again in a year or two. edit2: doesn't explain why owncloud runs smoother than nextcloud though. They must be doing something right.
- johnchristopher 6y agoOn a whim I tried the vps docker setup on the pi. Runs much smoother (seems like the db was the bottleneck but). edit: Aaaaaaand now I can't login because I foolishly tried to setup webdav and nextcloud is adding 30 seconds to my login and it never actually cool downs. Why do I even bother.
- KingOfCoders 6y agoI have so many problems with core syncing with my NextCloud here - I wish I had used something else. But I guess "competition" is over who has more features.
- pestaa 6y agoWhat issues have you encountered? I can't imagine using anything else, it works so flawlessly for me on multiple devices across different platforms.
- KingOfCoders 6y agoFirst, how many people are working on those files? I have more problems with more people. The main problem are multi files that are all written together, e.g. Scrivener, Git, ... - I get sync conflicts which are hard to solve.
- pestaa 6y agoI'm not sure what your use case is, but syncing source code repositories to a shared folder doesn't sound like a great idea to me. Collaboration on individual files is also best solved using online office suites and similar tools. The remaining scenarios indeed benefit from file locking and may need conflict resolution from time to time.
- KingOfCoders 6y ago"Collaboration on individual files is also best solved using online office suites and similar tools." Yes, as I've said, I'd use something else, I've fallen for the "Share and collaborate on documents" marketing slogan of NextCloud on their homepage.
- mtgx 6y agoNC should one-up them and do the rewrite in Rust for 20x the performance.
- choeger 6y agoIt is obviously a good idea to use a safe language that compiles to machine-code for something performance-sensitive. But I do not quite understand why someone should choose go over rust for any large-scale project.
- switch007 6y agoWhy Rust over Go? Imagine I'm the CTO of an ownCloud-like company who is showing a strong preference for Go.
- fxtentacle 6y agoI'd also go with Go, just because of the amazing standard library. The ability to reuse part of Google's production code for HTTP / JSON is a huge win in my book. Also, Go has been around for long enough that you'll find libraries and tutorials even for rare issues. When I evaluated Rust, I was missing that. Their ecosystem, while highly motivated, isn't mature enough yet for infrastructure level code.
- choeger 6y agoYou should choose rust for the long term. The type system will pay off any initial investment. Trust me, I am maintaining a large python code base. The number of problems that appear "organically" and would have been caught with a good type system is staggering.
- hnlmorg 6y agoI don't like Python personally but even I can see the problem here is you're doing Python wrong. Look up "type annotations" and include them as part of your tox testing. Also, as I've commented elsewhere, if what you want is a static type system then there is still a plethora of better languages for web development than Rust. Rust might be a hugely impressive language but it's not yet ready for prime time web development.
- tmikaeld 6y agoIt has more to do with maturity and reliability. Go has mature and fast standard libraries included in the language for web-related things like http, gateway/api, proxies and above all encryption. For rust, you'll need quite a lot of crates with a lot of unknown variables to consider. Why do large businesses choose Webpack when there's a lot of faster, more feature-full alternatives? Because it's reliable and maintained.
- 8fingerlouie 6y agoI just shut down my Nextcloud instance yesterday. Not that there was anything wrong with it, but all i really need is a file synchronization "platform", and there are others that don't open quite as large an attack surface as a complete nextcloud instance does. Not saying Nextcloud is insecure, and i've never had any problems with it. Currently trying to decide if Seafile is the way for me, though i dislike it's on disk fileformat (IIRC it's some adaptation of Git). For now i use Resilio Sync.
- dmacvicar 6y agoI am curious if you have tried https://syncthing.net/ https://syncthing.net/ ?
- pqs 6y agoI made the transition from Resilio to Syncthing and I am very happy.
- paule89 6y agoWhat where the noteworthy features, which make Syncthing a better solution?
- 8fingerlouie 6y agoI have, but syncthing doesn't have iOS clients, or at least it didn't last time i checked. There was a client on the appstore called "f-sync", but that is gone now. Also, Resilio (with paid license) supports features that Syncthing didn't support like selective sync, and encrypted folders, which allows you to share a folder with someone for redundancy and have things stored encrypted on disk.
- gingerlime 6y agoaren’t you worried about Resilio being closed source?
- jojo2000 6y agogood and bold choice. but count at least a year to get visible results.
- encom 6y agoThis is terrible news. I hope they keep the PHP version up to date, or someone forks it. I've used Owncloud over Nextcloud, because it seems more focused on just file syncing, which is all I want. Maybe it's time to migrate.
- Aissen 6y agoThis reminds me of cozy cloud's rewrite from node.js to go (https://github.com/cozy/cozy-stack https://github.com/cozy/cozy-stack ); this seems to be a rewrite from php to go, as well as re-architecturing it: https://github.com/owncloud/ocis https://github.com/owncloud/ocis Pydio also rewrote itself from php to go with pydio cells: https://github.com/pydio/cells https://github.com/pydio/cells
- aneutron 6y agoAnyone has a link for the actual technical architecture ? This talks about "three-tier architecture" and "no database needed", which is obviously pure marketing spiel. Are they re-inventing the wheel by implementing a distributed database directly into ownCloud ? I don't get it.
- tmikaeld 6y agoSeems it has a modular storage back-end[0], the default uses a built-in storage engine local files with oCIS storage driver [1]. [0] https://owncloud.github.io/ocis/storage-backends/eos/ https://owncloud.github.io/ocis/storage-backends/eos/ [1] "The default oCIS storage driver deconstructs a filesystem to be able to efficiently look up files by fileid as well as path. It stores all folders and files by a uuid and persists share and other metadata using extended attributes. This allows using the linux VFS cache using stat syscalls instead of a database or key/value store. The driver implements trash, versions and sharing. It not only serves as the current default storage driver, but also as a blueprint for future storage driver implementations."
- tscherno 6y agoWith this design we aim for the storage to become the single-point of truth and scale-out as the metadata is spatially close to the files.
- ThePhysicist 6y agoThat sounds scary. Going from a monolith to a microservices architecture doesn't seem like a great move for a software like ownCloud. I think rewriting this as a Go application with a modern JS-based frontend might make sense, I just don't get why you need to throw microservices into the mix. But let's see if they can pull this off, historically such rewrites don't have a great track record though.
- jeandejean 6y agoI'm curious about past examples of failures, can you provide some? Thanks!
- bayindirh 6y agoThe thing with Owncloud/Nextcloud is the complexity of the system. When many users are working with the files at the same time, there's a lot of messaging going on in the backbone. If people are working in shared folders, that traffic goes visibly up. Messaging between systems is hard. When things are async, it's harder. When there's async concurrency, it becomes spicy. I'm managing a Nextcloud instance at the office. Even with this relatively mature state, it can bork itself pretty decently. Also, it has separation of responsibilities and a system admin is not an admin the traditional sense so, a lot of things are hidden between logs and other layers. Complicating this further by adding microservices is a brave idea at least. We're going to see what's going to happen. I hope they succeed but it's not a piece of cake.
- fundatus 6y agoNetscape for example: https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ https://www.joelonsoftware.com/2000/04/06/things-you-should-...
- rimliu 6y agoIt is telling when a single example is used for 20+ years.
- danpalmer 6y ago> ownCloud Infinite Scale neither requires Apache nor any PHP infrastructure I don't know much about production web services in Go, but is it true that they wouldn't need Apache? I work on a site using Python/Gunicorn and Node behind Nginx, and Nginx is really doing a lot of critical stuff for us, normalising requests, preventing certain classes of security issues, DoS attacks, etc. I wouldn't expect that sort of protection to be built into Go's standard HTTP server?
- ThePhysicist 6y agoGo has good support for TLS and HTTP serving, so you can directly expose a Golang service to the Internet without a web server in front. Many people still choose to put nginx or caddy in front as a load balancer and TLS termination though.
- vbsteven 6y agoAs with most builtin language HTTP servers it is advised to run Nginx/Apache/Caddy in front of it. I think what they mean with PHP infrastructure is the specific configuration required to make Apache work with PHP, in the old days this was with an Apache plugin, or more recently the php-fpm project.
- kissgyorgy 6y agoForget ownCloud, Nextcloud is orders of magnitude better either as a company and as software: https://nextcloud.com/ https://nextcloud.com/ It is my favorite self-hosted software of all time. It's really a dream to use! The AGPL license makes it impossible to disappear or becoming closed source, which is a huge plus!
- teekert 6y agoI also like and run NextCloud, it is the "cloud" back-end for my entire family. That said, it seems to me that this means that OwnCloud en NextCloud are now truly diverging and OwnCloud is not just a less feature rich version of NextCloud. I think it is very interesting to see what happens to OwnCloud now. I mean, imagine OpenOffice being rewritten with different technologies from LibreOffice... I'd be interested once again!
- alpaca128 6y agoNo, thanks. I decided to stop using it after Nextcloud already got confused by a few of my Git repos, overwriting my local updates with the older version from the server. Not to mention the "don't sync hidden files" default setting that's simply not a good idea, to put it mildly. This new ownCloud product promises to handle that better("with millions of small files or thousands with hundreds of GB each, ownCloud Infinite Scale runs smoothly"), but I'll believe it when I see it. So far I haven't found a sync tool that I feel like I can trust. I don't care about user group management or online office suites, but I absolutely want ~100% reliable file sync that doesn't randomly do nothing for 15 minutes without me knowing why(I care about my files and if I don't know what your software does with them I won't use it), or like NextCloud, falsely detect merge conflicts when 100% of changes came from the same client machine.
- diarrhea 6y agoYou have naked git repos in synced cloud folders, without using git remote functionalities? That would be careless and you are actively begging for things to break that way, so I'm not surprised. In fact I'd bet any other service, Dropbox/GDrive/..., would also suffer from such issues.
- jbverschoor 6y agoGood move. I'd never consider owncloud on LAMP.
- mangecoeur 6y agoThey mention the involvement of research institutions - the Swiss universities use a common OwnCloud system (Switch Drive) for researchers to store and share files within and between institutions; which generally works very well. So seems likely they are involved since they must be one of the biggest OwnCloud users in the world.
- joshspankit 6y agoHuge respect for this direction: I’d put ownCloud on the “too old to seriously consider” pile, but they’ve potentially not just started fresh, but brought forward everything they’ve learned in all the time since they started. Done right, this means they are a serious, safe, and secure option.
- veddox 6y agoI'm pretty sure this is an attempt to regain some of the market share they lost against Nextcloud. IIRC, the latter's popularity has been growing continuously, while OC has been stagnating/shrinking. Let's see if they can pull it off.
- ThinkBeat 6y agoThere is a lot of hate for PHP, and monoliths. The great thing about WordPress, mediaWiki, Drupal, nextCloud, and many other open source web solutions is that you can get up a running for $2 -> with little computer knowledge. A lot of cheap consumer hosting services offer managed PHP/MySql stack very cheap. A lot of them have variations of one click installs of various kinds. You pay the monthly fee and do not have to know about Linux, PHP or MySql. If you are familiar with PHP, MySql finding a place to host it is still really cheap and available. A common argument would be that you can get a VPS for $5 maybe even less. Then you have to know about what a database is, what version of PHP, when to do upgrades of all of them. Managed VPS solutions are usually fairly pricy. With the new version of ownCloud, based on Go and micrnosercivices the barrier of entry is much higher. The best bet will be to buy managed nextCloud hosting from someone. While PHP is often scoffed at, the ecosystem that is ready and waiting to help host it for each is amazing. It enables a lot more people to be able to get involved and get inspired and maybe learn more and get start contributing. If you are writing software that needs hosting but want as many people as possible to be able to use it then PHP is still the best bet, even if not the sexy bet
- dosethree 6y agoIMO there is not a lot of hate for monoliths. Service oriented architectures and their successor microservices have experienced a lot of push back from teams that have tried it and it doesn't work out for all applications. PHP - I mean, it gets the job done. I would never start a new company and choose it, but if you're already on it, its good enough for most things. Go can be far superior depending on your needs.
- hedora 6y agoThe big issue I have with deploying PHP is the sprawling, historically low-quality ecosystem. Maybe it’s gotten better, but last I checked, running RATS (which might not even exist any more) against random, popular PHP code bases would find piles of zero day vulnerabilities. It’s been over a decade, so I might be being unfair.
- LargoLasskhyfv 6y agoHannisch wat verpasst, oder war dat jezz allet op deutsch?