17 ms·

ServiceFabric: a distributed platform for building microservices in the cloud

142 points by deegles 8y ago

wenc 8y ago
It's worth noting that while Service Fabric as a technology does power a lot of offerings at Microsoft (as noted in the article), strategically it has been superseded somewhat by Azure Kubernetes Service (AKS). Most new development should really be on AKS. Even Azure Container Instances (ACI), a recently GA'ed Azure service, runs Kubernetes underneath. Edit: As noted in the comments, SF is on Github. The following paragraph is incorrect and I retract what I said; leaving this here for context. "Also, by going the K8S route, you potentially avoid being locked into a proprietary offering -- SF is only available from one vendor, whereas K8S is available almost anywhere."
- manigandham 8y ago
  Service Fabric is different from Kubernetes. It's not a container orchestration platform but a full-stack distributed state and processing system with components that can serve as the foundation for building something like Kubernetes itself. The naming is confusing and there is a Service Mesh offering from Azure that lets you run your own applications easily if they're using the Service Fabric framework already, but SF can run containerized code in any language, and it has bindings for C# and Java (for now) if you want to use the primitives directly. SF is also now open-source and you can run it all yourself in whatever environment you want: https://github.com/Microsoft/service-fabric https://github.com/Microsoft/service-fabric
  - dewiz 8y ago
    SF has been opensourced very late though [1], to get some attention and trying to survive. Legitimate but very different from being an open source project. The cross-platform support is very weak [2][3], far from K8S, Mesos and Swarm developer friendliness. [1] https://blogs.msdn.microsoft.com/azureservicefabric/2018/03/14/service-fabric-is-going-open-source https://blogs.msdn.microsoft.com/azureservicefabric/2018/03/... [2] https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-get-started-mac https://docs.microsoft.com/en-us/azure/service-fabric/servic... [3] https://social.msdn.microsoft.com/Forums/en-US/b5045fe1-51a2-4c58-ae90-53c145d3ef59/using-mac-with-azure-service-fabric https://social.msdn.microsoft.com/Forums/en-US/b5045fe1-51a2...
    - manigandham 8y ago
      Trying to survive? It already runs all of Microsoft's cloud services and does just fine. Open-sourcing was more of a good-will move so that the community can also use it, similar to how Apple released FoundationDB. The links you point to are about developing with SF, which doesn't natively run on Mac OSX. It can run on Linux but has fewer features for now, but this is being worked on. It's not the same as K8S/Mesos/Swarm though so if container orchestration is all you need then you shouldn't use SF.
    - polskibus 8y ago
      I don't think it's buildable on Windows yet. It's been like this for a couple of months now, not sure if MS is really trying to get others involved.
      - dewiz 8y ago
        MS is going where developers are, and outside MS there is no community around SF. Hired one of K8S founders, built new products around k8s, advocates docker left and right at conferences, these are clear signals that the tech doesn’t have a future
        deleted 8y ago
        [deleted]
        youdontknowtho 8y ago
        SF can also manage docker containers. They are doing lots of work with k8s, but this system has a huge amount of internal usage and has provably been used to build global scale services. Its strengths are different than k8s and the distributed system space has lots of options. You (and others) seem to be enjoying some schadenfruede about SF not "winning" some competition. I don't think the world works that way. Lots of people will use it for big applications. More people will probably use something else. That's about it. The type of comment you made doesn't get marked by the moderators for some reason even though it doesn't actually provide any insight. The article had detailed information about the internals of how it worked. That's the interesting thing here. The Ziff Davis style dev-adoption-horse race is pretty pointless.
- s2g 8y ago
  Service fabric is open source. Not locked to one vendor. Http://github.com/Microsoft/service-fabric
- benaadams 8y ago
  > SF is only available from one vendor, whereas K8S is available almost anywhere. SF is open source on GitHub and you can run it on-premise or on any cloud, Windows or Linux. The tooling is probably easier on Azure; but you can also happily run it on AWS; as well as the Service Mesh variant which is sort of "Serverless Clusters" or "Clusterless Clusters", applying the strange naming of "Serverless".
ossinnameonly 8y ago
In my estimation, now would be a rather poor time to adopt SF. 1. It's poorly positioned against K8s. The "it can do containers" people are really not helping their case; SF's value proposition is clearly not in running arbitrary containers. Unless you're writing a new .NET app and will go all in on their SDK and replicated state model, it just doesn't make sense. 1a. It has no community. It has no presence outside of MS doc pages, the repo they threw over the wall, and the one external user that I'm aware of (who is already in this thread). 2. SF was operationally not-fun. I'm being nice. Though, neither is Kubernetes, most days. 3. SF is losing to Kubernetes, even internally, and even despite pressure from high-up-ish people. 4. The rumors about SFs fate would not seem to lend themselves toward adopting it for new projects.
- tannhaeuser 8y ago
  I don't know the first thing about SF (the product), and you might be absolutely right. But I find it discomforting to base engineering decisions on the equivalent of Facebook likes and Web presences/Googling rather than merit. For all I know, the media presence of the space of cloud orchestration software is highly manipulated by huge PR budgets (for example by Docker) and by media heavyweights competing in this space.
  - codeflo 8y ago
    I agree with the sentiment, but what’s sometimes overlooked is that the community around a technology is in many ways more important than the technology itself. Can I expect to find add-on libraries that help integrating with the rest of my technology stack? Will I find documentation, blog posts or StackOverflow answers that solve my specific edge case? How likely is it that I’ll be the very first person to encounter a critical bug, because no one else uses it at that scale? Many of those signals are not easily faked, as Microsoft itself painfully learned when spending billions in an attempt to artificially prop up a developer community around Windows Phone. If, as the grandparent suggested, Service Fabric is losing out even internally at MS where they can apply management pressure in addition to marketing, that’s a very bad sign.
- youdontknowtho 8y ago
SBArbeit 8y ago
Lots of developers today just assume that microservices = containers, and if you're already in containers, you must need Kubernetes, but, really, Service Fabric is years ahead of Kubernetes in lots of ways. If I were starting a major distributed systems project today that required serious uptime and scalability, I'd start on Service Fabric. And, yes, I know... whatever feature I could name from Service Fabric has some sort of open-source thing for Kubernetes from someone that probably/almost/kind-of/sort-of/some-assembly-required does the same thing as Service Fabric. Or I could just use Service Fabric, and not play the Kubernetes equivalent of "what's in your node_modules directory?" Microsoft has an existential-level bet on Service Fabric. It's 1) not going anywhere, 2) only getting better, 3) is already a better container orchestrator than Kubernetes, if that's all you want to use it for.
- zamalek 8y ago
  We have stuff running on SF in prod, using the SDK side of it and not the container-like capabilities. I have some counterpoints (but my team is also divided on SF). First-off, you mentioned micro-services. The mechanisms that support service and application versioning seem well-intentioned and well-designed. In many ways the mechanisms run counter to some of the key benefits of a micro-service architecture (especially surrounding decoupling development teams). If your latency is high, deployment times out. We had to use a remote desktop session to a VM near the cluster to deploy manually (solved with CI/CD). This impedes teams who are trying it out, when they are remote. It has a pretty hostile development experience. Before you can even open a project in Visual Studio, you need the entire SDK installed. "Which SF installer did you use?" was a frequent question in Slack, because there are a bunch of places where it resides. Whenever you add a project to your solution, you have to ensure that it targets x64 (not AnyCpu). Strange compilation errors abound if you don't. We get it Microsoft: the runtime only supports x64 - you don't have to enforce it to this degree. AnyCpu runs just fine on x64. It sounds like nitpicking, but it gets horribly frustrating. They do (or used to do) strange things their .targets files. It doesn't behave like the C# build system, with a very clear happy-path. I spent about a week working around $(MSBuildProjectDir) not working. I would fix this by making the entire thing nothing more than a Nuget package. Anyone can understand having to have an emulator installed to debug, but the project system is hostile. It's a big pity because the tech is amazing. I've personally started championing Orleans+Docker as a more friendly alternative, but as I said: others on my team adore it.
sfinternal 8y ago
Not sure why this comment was banned, it has valid points worth the conversation —- In my estimation, now would be a rather poor time to adopt SF. 1. It's poorly positioned against K8s. The "it can do containers" people are really not helping their case; SF's value proposition is clearly not in running arbitrary containers. Unless you're writing a new .NET app and will go all in on their SDK and replicated state model, it just doesn't make sense. 1a. It has no community. It has no presence outside of MS doc pages, the repo they threw over the wall, and the one external user that I'm aware of (who is already in this thread). 2. SF was operationally not-fun. I'm being nice. Though, neither is Kubernetes, most days. 3. SF is losing to Kubernetes, even internally, and even despite pressure from high-up-ish people. 4. The rumors about SFs fate would not seem to lend themselves toward adopting it for new projects.
kerng 8y ago
I like some of the metrics, e.g. >>Azure SQL DB (100K machines, 1.82M DBs containing 3.48PB of data) Makes you realize how different the scale of the top cloud companies is, compared to the average service/site. Pretty amazing.
martindale 8y ago
This is pretty similar to our project, Fabric: https://fabric.fm https://fabric.fm — different tech stack, but similar name and seemingly project goals. Should we add you to the list of "other Fabrics" [0]? [0]: https://github.com/FabricLabs/fabric#other-fabrics https://github.com/FabricLabs/fabric#other-fabrics
- tannhaeuser 8y ago
  It's also not dissimilar to fabric3, a Java implementation of the Service Component Architecture (SCA). SCA is a polyglot OASIS spec for a portable service runtime from the height of the SOA craze, originally by David Chappell [1]. [1]: http://www.davidchappell.com/writing/Introducing_SCA.pdf http://www.davidchappell.com/writing/Introducing_SCA.pdf
- manigandham 8y ago
  It's not similar at all.
vshan 8y ago
Very interesting read.

142 points by deegles 8y ago

wenc 8y ago

It's worth noting that while Service Fabric as a technology does power a lot of offerings at Microsoft (as noted in the article), strategically it has been superseded somewhat by Azure Kubernetes Service (AKS). Most new development should really be on AKS. Even Azure Container Instances (ACI), a recently GA'ed Azure service, runs Kubernetes underneath. Edit: As noted in the comments, SF is on Github. The following paragraph is incorrect and I retract what I said; leaving this here for context. "Also, by going the K8S route, you potentially avoid being locked into a proprietary offering -- SF is only available from one vendor, whereas K8S is available almost anywhere."

manigandham 8y ago
Service Fabric is different from Kubernetes. It's not a container orchestration platform but a full-stack distributed state and processing system with components that can serve as the foundation for building something like Kubernetes itself. The naming is confusing and there is a Service Mesh offering from Azure that lets you run your own applications easily if they're using the Service Fabric framework already, but SF can run containerized code in any language, and it has bindings for C# and Java (for now) if you want to use the primitives directly. SF is also now open-source and you can run it all yourself in whatever environment you want: https://github.com/Microsoft/service-fabric https://github.com/Microsoft/service-fabric
- dewiz 8y ago
  SF has been opensourced very late though [1], to get some attention and trying to survive. Legitimate but very different from being an open source project. The cross-platform support is very weak [2][3], far from K8S, Mesos and Swarm developer friendliness. [1] https://blogs.msdn.microsoft.com/azureservicefabric/2018/03/14/service-fabric-is-going-open-source https://blogs.msdn.microsoft.com/azureservicefabric/2018/03/... [2] https://docs.microsoft.com/en-us/azure/service-fabric/service-fabric-get-started-mac https://docs.microsoft.com/en-us/azure/service-fabric/servic... [3] https://social.msdn.microsoft.com/Forums/en-US/b5045fe1-51a2-4c58-ae90-53c145d3ef59/using-mac-with-azure-service-fabric https://social.msdn.microsoft.com/Forums/en-US/b5045fe1-51a2...
  - manigandham 8y ago
    Trying to survive? It already runs all of Microsoft's cloud services and does just fine. Open-sourcing was more of a good-will move so that the community can also use it, similar to how Apple released FoundationDB. The links you point to are about developing with SF, which doesn't natively run on Mac OSX. It can run on Linux but has fewer features for now, but this is being worked on. It's not the same as K8S/Mesos/Swarm though so if container orchestration is all you need then you shouldn't use SF.
  - polskibus 8y ago
    I don't think it's buildable on Windows yet. It's been like this for a couple of months now, not sure if MS is really trying to get others involved.
    - dewiz 8y ago
      MS is going where developers are, and outside MS there is no community around SF. Hired one of K8S founders, built new products around k8s, advocates docker left and right at conferences, these are clear signals that the tech doesn’t have a future
      - deleted 8y ago
        [deleted]
      - youdontknowtho 8y ago
        SF can also manage docker containers. They are doing lots of work with k8s, but this system has a huge amount of internal usage and has provably been used to build global scale services. Its strengths are different than k8s and the distributed system space has lots of options. You (and others) seem to be enjoying some schadenfruede about SF not "winning" some competition. I don't think the world works that way. Lots of people will use it for big applications. More people will probably use something else. That's about it. The type of comment you made doesn't get marked by the moderators for some reason even though it doesn't actually provide any insight. The article had detailed information about the internals of how it worked. That's the interesting thing here. The Ziff Davis style dev-adoption-horse race is pretty pointless.
s2g 8y ago
Service fabric is open source. Not locked to one vendor. Http://github.com/Microsoft/service-fabric
benaadams 8y ago
> SF is only available from one vendor, whereas K8S is available almost anywhere. SF is open source on GitHub and you can run it on-premise or on any cloud, Windows or Linux. The tooling is probably easier on Azure; but you can also happily run it on AWS; as well as the Service Mesh variant which is sort of "Serverless Clusters" or "Clusterless Clusters", applying the strange naming of "Serverless".

ossinnameonly 8y ago

In my estimation, now would be a rather poor time to adopt SF. 1. It's poorly positioned against K8s. The "it can do containers" people are really not helping their case; SF's value proposition is clearly not in running arbitrary containers. Unless you're writing a new .NET app and will go all in on their SDK and replicated state model, it just doesn't make sense. 1a. It has no community. It has no presence outside of MS doc pages, the repo they threw over the wall, and the one external user that I'm aware of (who is already in this thread). 2. SF was operationally not-fun. I'm being nice. Though, neither is Kubernetes, most days. 3. SF is losing to Kubernetes, even internally, and even despite pressure from high-up-ish people. 4. The rumors about SFs fate would not seem to lend themselves toward adopting it for new projects.

tannhaeuser 8y ago
I don't know the first thing about SF (the product), and you might be absolutely right. But I find it discomforting to base engineering decisions on the equivalent of Facebook likes and Web presences/Googling rather than merit. For all I know, the media presence of the space of cloud orchestration software is highly manipulated by huge PR budgets (for example by Docker) and by media heavyweights competing in this space.
- codeflo 8y ago
  I agree with the sentiment, but what’s sometimes overlooked is that the community around a technology is in many ways more important than the technology itself. Can I expect to find add-on libraries that help integrating with the rest of my technology stack? Will I find documentation, blog posts or StackOverflow answers that solve my specific edge case? How likely is it that I’ll be the very first person to encounter a critical bug, because no one else uses it at that scale? Many of those signals are not easily faked, as Microsoft itself painfully learned when spending billions in an attempt to artificially prop up a developer community around Windows Phone. If, as the grandparent suggested, Service Fabric is losing out even internally at MS where they can apply management pressure in addition to marketing, that’s a very bad sign.
youdontknowtho 8y ago

SBArbeit 8y ago

Lots of developers today just assume that microservices = containers, and if you're already in containers, you must need Kubernetes, but, really, Service Fabric is years ahead of Kubernetes in lots of ways. If I were starting a major distributed systems project today that required serious uptime and scalability, I'd start on Service Fabric. And, yes, I know... whatever feature I could name from Service Fabric has some sort of open-source thing for Kubernetes from someone that probably/almost/kind-of/sort-of/some-assembly-required does the same thing as Service Fabric. Or I could just use Service Fabric, and not play the Kubernetes equivalent of "what's in your node_modules directory?" Microsoft has an existential-level bet on Service Fabric. It's 1) not going anywhere, 2) only getting better, 3) is already a better container orchestrator than Kubernetes, if that's all you want to use it for.

zamalek 8y ago
We have stuff running on SF in prod, using the SDK side of it and not the container-like capabilities. I have some counterpoints (but my team is also divided on SF). First-off, you mentioned micro-services. The mechanisms that support service and application versioning seem well-intentioned and well-designed. In many ways the mechanisms run counter to some of the key benefits of a micro-service architecture (especially surrounding decoupling development teams). If your latency is high, deployment times out. We had to use a remote desktop session to a VM near the cluster to deploy manually (solved with CI/CD). This impedes teams who are trying it out, when they are remote. It has a pretty hostile development experience. Before you can even open a project in Visual Studio, you need the entire SDK installed. "Which SF installer did you use?" was a frequent question in Slack, because there are a bunch of places where it resides. Whenever you add a project to your solution, you have to ensure that it targets x64 (not AnyCpu). Strange compilation errors abound if you don't. We get it Microsoft: the runtime only supports x64 - you don't have to enforce it to this degree. AnyCpu runs just fine on x64. It sounds like nitpicking, but it gets horribly frustrating. They do (or used to do) strange things their .targets files. It doesn't behave like the C# build system, with a very clear happy-path. I spent about a week working around $(MSBuildProjectDir) not working. I would fix this by making the entire thing nothing more than a Nuget package. Anyone can understand having to have an emulator installed to debug, but the project system is hostile. It's a big pity because the tech is amazing. I've personally started championing Orleans+Docker as a more friendly alternative, but as I said: others on my team adore it.

sfinternal 8y ago

Not sure why this comment was banned, it has valid points worth the conversation —- In my estimation, now would be a rather poor time to adopt SF. 1. It's poorly positioned against K8s. The "it can do containers" people are really not helping their case; SF's value proposition is clearly not in running arbitrary containers. Unless you're writing a new .NET app and will go all in on their SDK and replicated state model, it just doesn't make sense. 1a. It has no community. It has no presence outside of MS doc pages, the repo they threw over the wall, and the one external user that I'm aware of (who is already in this thread). 2. SF was operationally not-fun. I'm being nice. Though, neither is Kubernetes, most days. 3. SF is losing to Kubernetes, even internally, and even despite pressure from high-up-ish people. 4. The rumors about SFs fate would not seem to lend themselves toward adopting it for new projects.

kerng 8y ago

I like some of the metrics, e.g. >>Azure SQL DB (100K machines, 1.82M DBs containing 3.48PB of data) Makes you realize how different the scale of the top cloud companies is, compared to the average service/site. Pretty amazing.

martindale 8y ago

This is pretty similar to our project, Fabric: https://fabric.fm https://fabric.fm — different tech stack, but similar name and seemingly project goals. Should we add you to the list of "other Fabrics" [0]? [0]: https://github.com/FabricLabs/fabric#other-fabrics https://github.com/FabricLabs/fabric#other-fabrics

tannhaeuser 8y ago
It's also not dissimilar to fabric3, a Java implementation of the Service Component Architecture (SCA). SCA is a polyglot OASIS spec for a portable service runtime from the height of the SOA craze, originally by David Chappell [1]. [1]: http://www.davidchappell.com/writing/Introducing_SCA.pdf http://www.davidchappell.com/writing/Introducing_SCA.pdf
manigandham 8y ago
It's not similar at all.

vshan 8y ago

Very interesting read.