5 ms·

"To bring Octopus Cloud to market quickly, we did the simplest thing possible; we took our self-hosted Octopus Server product and bundled it into an EC2 instanc

"To bring Octopus Cloud to market quickly, we did the simplest thing possible; we took our self-hosted Octopus Server product and bundled it into an EC2 instance for each customer that signed up. We had to make changes to the product, but mostly around permissions." I have used octopus and I could have told them that they can't lift-and-shift. The amount of requests the app makes is ridiculous, the way they wrote their database is appalling (actually crazily dumb https://octopus.com/blog/sql-as-document-store https://octopus.com/blog/sql-as-document-store). Everything about it would mean they can't scale in the cloud without wasting a ton of money. "Cloud stuff can be really expensive" - yes if you write dog shit, putting that in the cloud will be expensive.

teddyuk 7y ago
From the "sql-as-document-store" post, I re-read it every now and then to realise how actually terrible it is: "For a many-to-many, we do something a bit naughty: we store the values with | characters between each value:" Not naughty, stupid.
- swiley 7y ago
  Why would you bother with a relational database at that point? That sounds like it’s not normalized. I haven’t read the post... maybe I’m missing something.
  - pcr0 7y ago
    For ACID guarantees?
    - jfindley 7y ago
      ACID guarantees have nothing to do with relational vs non-relational. FoundationDB is one example of a non-relational key value store that none the less provides ACID guarantees.
  - dragonwriter 7y ago
    > Why would you bother with a relational database at that point? That sounds like it’s not normalized. It's non-1NF, though they sometimes (based on expected cardinality) normalize that aspect via a trigger, using the application-accessed table as, in effect, something of a write-through denormalized materialized view. It's not an entirely unreasonablev way to use an RDBMS in and of itself.
    - teddyuk 7y ago
      except they used sql server and sql server performance with this is awful, oracle would have been able to handle it much better
      - tfehring 7y ago
        It’s much better now since SQL Server 2017 added a new string split function. But I actually had to do basically the same thing in an earlier version of SQL Server (in my case, it was because I was interacting with a legacy application that spits out pipe-delimited strings) and the string parsing increased the execution time by several orders of magnitude.
  - teddyuk 7y ago
    because they didn't know what they were doing - I can say this as this is what the blog shows, they don't understand databases and customers (including me) have suffered this
    - jcadam 7y ago
      I've learned much too late in my career that competent software engineering is completely orthogonal to creating a successful(from a market perspective) software product.
      - swiley 7y ago
        Well yeah most of it’s just CRUD and that’s pretty straightforward even at a large scale (ex stack overflow.)
- brodouevencode 7y ago
  Agreed. Lifting and shifting is hard. It's harder when you do it in a careless way.
- phamilton 7y ago
  The approach seemed reasonable enough until I hit that point.
  - time0ut 7y ago
    I was like, ya, I do that sometimes too... Wait wtf?
- LeonM 7y ago
  > Not naughty, stupid. Why didn't they just make 1 table with 1 column and put a JSON object in it with all customer data? This would fit their design objective "Avoid the need for joins as much as possible"! It's brilliant! We'll call it: noSQL, because we don't even have to write that pesky SQL! Can't wait for the next blog post where they discover another naughty trick called primary keys to increase their NoSQL performance even more!
eightysixfour 7y ago
While I agree with you that it seems like they make some serious engineering mistakes, it was a successful MVP in that it told there there was sufficient demand to justify a rewrite for the cloud.
paulstovell 7y ago
None of this had anything to do with how the database is used.
omouse 7y ago
The best part is the "simplest thing possible". They actually didn't do that at all. I hate to dog-pile on this team, but damn, 100k spend in a month isn't simple at all!

teddyuk 7y ago

From the "sql-as-document-store" post, I re-read it every now and then to realise how actually terrible it is: "For a many-to-many, we do something a bit naughty: we store the values with | characters between each value:" Not naughty, stupid.

swiley 7y ago
Why would you bother with a relational database at that point? That sounds like it’s not normalized. I haven’t read the post... maybe I’m missing something.
- pcr0 7y ago
  For ACID guarantees?
  - jfindley 7y ago
    ACID guarantees have nothing to do with relational vs non-relational. FoundationDB is one example of a non-relational key value store that none the less provides ACID guarantees.
- dragonwriter 7y ago
  > Why would you bother with a relational database at that point? That sounds like it’s not normalized. It's non-1NF, though they sometimes (based on expected cardinality) normalize that aspect via a trigger, using the application-accessed table as, in effect, something of a write-through denormalized materialized view. It's not an entirely unreasonablev way to use an RDBMS in and of itself.
  - teddyuk 7y ago
    except they used sql server and sql server performance with this is awful, oracle would have been able to handle it much better
    - tfehring 7y ago
      It’s much better now since SQL Server 2017 added a new string split function. But I actually had to do basically the same thing in an earlier version of SQL Server (in my case, it was because I was interacting with a legacy application that spits out pipe-delimited strings) and the string parsing increased the execution time by several orders of magnitude.
- teddyuk 7y ago
  because they didn't know what they were doing - I can say this as this is what the blog shows, they don't understand databases and customers (including me) have suffered this
  - jcadam 7y ago
    I've learned much too late in my career that competent software engineering is completely orthogonal to creating a successful(from a market perspective) software product.
    - swiley 7y ago
      Well yeah most of it’s just CRUD and that’s pretty straightforward even at a large scale (ex stack overflow.)
brodouevencode 7y ago
Agreed. Lifting and shifting is hard. It's harder when you do it in a careless way.
phamilton 7y ago
The approach seemed reasonable enough until I hit that point.
- time0ut 7y ago
  I was like, ya, I do that sometimes too... Wait wtf?
LeonM 7y ago
> Not naughty, stupid. Why didn't they just make 1 table with 1 column and put a JSON object in it with all customer data? This would fit their design objective "Avoid the need for joins as much as possible"! It's brilliant! We'll call it: noSQL, because we don't even have to write that pesky SQL! Can't wait for the next blog post where they discover another naughty trick called primary keys to increase their NoSQL performance even more!

eightysixfour 7y ago

While I agree with you that it seems like they make some serious engineering mistakes, it was a successful MVP in that it told there there was sufficient demand to justify a rewrite for the cloud.

paulstovell 7y ago

None of this had anything to do with how the database is used.

omouse 7y ago

The best part is the "simplest thing possible". They actually didn't do that at all. I hate to dog-pile on this team, but damn, 100k spend in a month isn't simple at all!