3 ms·

One thing stood out for me in the article: "As mentioned here before, SparkFun.com is really the tip of a much bigger iceberg called Sparkle. Sparkle is an int

One thing stood out for me in the article: "As mentioned here before, SparkFun.com is really the tip of a much bigger iceberg called Sparkle. Sparkle is an internal website where we hang every tool we need. It does everything but the core accounting." If you can share, what's the reason you folks went with this architecture? Beyond DB choices, I'd be very concerned about a design that tied so many important business functions into the one tech. stack, let alone the same application. I'd like to be able to do things like bring internal tools down for maintenance without hosing the public-facing CMS. Or change database technologies to suit whichever particular application I was optimising.

gnaritas 13y ago
The big ball of mud architecture is quite popular in the business world.
- Glyptodon 13y ago
  When you have limited developer resources modular balls of mud (AKA big glob of Buckyballs - like the magnetic desk toy) with single login identity and shared well structured DB are far better and easier to maintain than a half dozen+ separate applications with differing architectural choices, languages, and abilities to integrate with each other. You do not know hell if you've never encountered a medium sized organization without much in house tech talent who've somehow ended up with 5 different application with database contents and business functionality overlap and completely different architectures and languages with no ability to integrate with each other. This is actually also the main reason I've come to dislike Rails - for complex businesses with single small teams or single developers it's hard to encapsulate different site functionalities as modular components of a larger system (more like 'apps' in Django) as compared to running multiple rails sites. This same need is also why many business (prepare to shoot yourself) try to do everything with Drupal. The ability to meet a need quickly with something that can just plug in easily to an existing system far outweighs anything else in most non-giant non-massive-scale organizations. This is part of why Rails is so successful despite not making it easy to modularize your site functionality - the components and libraries of the application overall are highly modular and community package resources make it so stuff can be built quickly even it's hard to modularize within the Rails application itself. The big bad ball of mud is more an outgrowth of business trying to move quickly and demanding their flavor of the week without being able to understand, plan, or manage the quality of their developers, modularity of their code, or the development process, rather than complex single (but modular) systems being bad.
  - fleitz 13y ago
    Rails makes me think of the unix way, one task, one tool, and do it well. The key with rails is not to modularize the site, but modularization at a service level. Give your rails app a REST API and use it from the other apps. eg. Instead of shared authentication classes between sites, setup an auth app that supports oauth, and use an oauth module for devise.
  - duncan_bayne 13y ago
    "When you have limited developer resources modular balls of mud (AKA big glob of Buckyballs - like the magnetic desk toy) with single login identity and shared well structured DB are far better and easier to maintain than a half dozen+ separate applications with differing architectural choices, languages, and abilities to integrate with each other." Says you. My experience has been that a sensibly separated range of simple, single-purpose applications with intuitive APIs is actually far easier to work with, especially given limited resources.
    - Glyptodon 13y ago
      I feel like you're not talking about single purpose applications that have human users, but rather ones that have other systems as clients. If I told our users that I had written nice APIs for the things they do I'd probably be out of a job. Likewise, every time I've had to store the same data multiple times in separate databases it's eventually become a headache. I might be missing something, though. Perhaps it's that other situations have concerns without lots of overlaps in business logic and data, or that there's an easier way than I've found to deploy multiple applications transparently so that users don't notice or have to deal with multiple applications. I'm curious, though, how with a true multiple application architecture you deal with database duplication and fragmentation? This is always the single biggest issue in my experience. (I also might not be talking about the same thing as you - if I can make a single purpose application that handles back-end tasks and isn't user facing, doesn't need auth, and also doesn't need it's own DB, such as for background processing images, or generating compute intensive reports, I'm all for it. But that wasn't really what I was talking about at all.)
memracom 13y ago
I think you are reading too much into it. They never said that there would only ever be one PostgreSQL db with everything. As you point out there are advantages to separating certain functions into separate dbs, but for most businesses that go that route, it makes sense to use multiple instances of the same core tool. Also, wanting to have a single authoritative view of inventory and order data is NOT a big ball of mud. Nor is a single big db with everything. The bid ball of mud generally starts with the link editor and developers who think that the only way to integrate functionality is to link it all into one big binary. For all we know they are planning to use RabbitMQ as their main integration tool, and build lots of small single purpose apps that work together like an orchestra.
strlen 13y ago
This sounds counter-intuitive, but the best (from the technical angle) companies I worked at -- or know friends at -- are muddle the "internal enterprise tools" and "public facing applications" divide quite a bit, but (and here's a catch) when their perceived and actual primary area of competence is building applications that at least somewhat resemble the internal tools in question. Sparkfun is in an odd place, they could call themselves a store, but then again so could Amazon -- it's not clear that selling technology is requisite to technology being a primary area of competence.
brennen 13y ago
> If you can share, what's the reason you folks went with this architecture? In its present state it's a fairly sane architecture for our purposes, but the only honest & concise answer to this question is that we didn't know what we were doing and it just kind of worked out that way. Not coincidentally, this also describes the architecture of the business as a whole.
- duncan_bayne 13y ago
  Thanks for the reply - & I'm glad it's working out :)

gnaritas 13y ago

The big ball of mud architecture is quite popular in the business world.

Glyptodon 13y ago
When you have limited developer resources modular balls of mud (AKA big glob of Buckyballs - like the magnetic desk toy) with single login identity and shared well structured DB are far better and easier to maintain than a half dozen+ separate applications with differing architectural choices, languages, and abilities to integrate with each other. You do not know hell if you've never encountered a medium sized organization without much in house tech talent who've somehow ended up with 5 different application with database contents and business functionality overlap and completely different architectures and languages with no ability to integrate with each other. This is actually also the main reason I've come to dislike Rails - for complex businesses with single small teams or single developers it's hard to encapsulate different site functionalities as modular components of a larger system (more like 'apps' in Django) as compared to running multiple rails sites. This same need is also why many business (prepare to shoot yourself) try to do everything with Drupal. The ability to meet a need quickly with something that can just plug in easily to an existing system far outweighs anything else in most non-giant non-massive-scale organizations. This is part of why Rails is so successful despite not making it easy to modularize your site functionality - the components and libraries of the application overall are highly modular and community package resources make it so stuff can be built quickly even it's hard to modularize within the Rails application itself. The big bad ball of mud is more an outgrowth of business trying to move quickly and demanding their flavor of the week without being able to understand, plan, or manage the quality of their developers, modularity of their code, or the development process, rather than complex single (but modular) systems being bad.
- fleitz 13y ago
  Rails makes me think of the unix way, one task, one tool, and do it well. The key with rails is not to modularize the site, but modularization at a service level. Give your rails app a REST API and use it from the other apps. eg. Instead of shared authentication classes between sites, setup an auth app that supports oauth, and use an oauth module for devise.
- duncan_bayne 13y ago
  "When you have limited developer resources modular balls of mud (AKA big glob of Buckyballs - like the magnetic desk toy) with single login identity and shared well structured DB are far better and easier to maintain than a half dozen+ separate applications with differing architectural choices, languages, and abilities to integrate with each other." Says you. My experience has been that a sensibly separated range of simple, single-purpose applications with intuitive APIs is actually far easier to work with, especially given limited resources.
  - Glyptodon 13y ago
    I feel like you're not talking about single purpose applications that have human users, but rather ones that have other systems as clients. If I told our users that I had written nice APIs for the things they do I'd probably be out of a job. Likewise, every time I've had to store the same data multiple times in separate databases it's eventually become a headache. I might be missing something, though. Perhaps it's that other situations have concerns without lots of overlaps in business logic and data, or that there's an easier way than I've found to deploy multiple applications transparently so that users don't notice or have to deal with multiple applications. I'm curious, though, how with a true multiple application architecture you deal with database duplication and fragmentation? This is always the single biggest issue in my experience. (I also might not be talking about the same thing as you - if I can make a single purpose application that handles back-end tasks and isn't user facing, doesn't need auth, and also doesn't need it's own DB, such as for background processing images, or generating compute intensive reports, I'm all for it. But that wasn't really what I was talking about at all.)

memracom 13y ago

I think you are reading too much into it. They never said that there would only ever be one PostgreSQL db with everything. As you point out there are advantages to separating certain functions into separate dbs, but for most businesses that go that route, it makes sense to use multiple instances of the same core tool. Also, wanting to have a single authoritative view of inventory and order data is NOT a big ball of mud. Nor is a single big db with everything. The bid ball of mud generally starts with the link editor and developers who think that the only way to integrate functionality is to link it all into one big binary. For all we know they are planning to use RabbitMQ as their main integration tool, and build lots of small single purpose apps that work together like an orchestra.

strlen 13y ago

This sounds counter-intuitive, but the best (from the technical angle) companies I worked at -- or know friends at -- are muddle the "internal enterprise tools" and "public facing applications" divide quite a bit, but (and here's a catch) when their perceived and actual primary area of competence is building applications that at least somewhat resemble the internal tools in question. Sparkfun is in an odd place, they could call themselves a store, but then again so could Amazon -- it's not clear that selling technology is requisite to technology being a primary area of competence.

brennen 13y ago

> If you can share, what's the reason you folks went with this architecture? In its present state it's a fairly sane architecture for our purposes, but the only honest & concise answer to this question is that we didn't know what we were doing and it just kind of worked out that way. Not coincidentally, this also describes the architecture of the business as a whole.

duncan_bayne 13y ago
Thanks for the reply - & I'm glad it's working out :)