6 ms·

First we built an API, then we built a CMS

102 points by JoelSutherland 16y ago

dools 16y ago
Getting the API right would be the only way a hosted CMS could acheive the flexibility that developers love about non-hosted platforms. This is so true! We've also developed a SaaS CMS (decalcms.com - still pre-release) and one of the major things I've noticed while looking around at other hosted solutions is they all have a pretty sharp "cut-off" past which you just can't do what you need to. VerbCMS solves this by letting you run your own code on their servers which we didn't want to do because it makes things less secure. Webvanta solves this by just saying "we'll build it for you" but that's just not scalable at all. A cracking API and integration model is the only real way that a hosted CMS can become a serious "platform of choice" for a lot of web developers/designs/agencies.
mark_l_watson 16y ago
Not being critical, just saying that it is common/best practice to implement dependent web services and back end functionality before writing a web UI, right? Nothing wrong with doing some prototype simple UIs, but having the back end stuff implemented and tested makes the first real UI implementation go much faster. So, they are following good development practices.
- mathgladiator 16y ago
  It's the difference between building stuff bottom-up versus top-down. It all depends on the product.
  - aaronblohowiak 16y ago
    The product and the stability of the feature definition.
- JoelSutherland 16y ago
  (I work on HiFi) You're 100% right. What we're describing sounds a lot like waterfall. Here are a couple more details: * Our team of devs have collectively built over 450 websites. We know the domain well. * We also did a lot of UI design early on that guided some some of our work. That said, we didn't just design and give birth to a great API. We have iterated on a it a lot. Our clients were going on HiFi 6 months prior to the public release. We learned a lot and iterated on the API quite a bit during that period. What ended up being nice was the ability to make sweeping changes at one point (the API) that benefited the entire application. This is a huge advantage.
  - StavrosK 16y ago
    Can you post a few more details, such as how you handled authentication, how views pass the session to the server (or do you use cookies for all that?), how the calls are being made (does each page do XHR calls and then render the results using logic itself?), etc?
    - JoelSutherland 16y ago
      I'm not the right person to answer all of these (the post author is). I'll do my best on what I can. For permissions, there are five levels of users. System, Agency, Site Owner, Member, Public. These are hierarchical like everything else. Group members and individual users can have varying read/write/create permissions for different areas of the content tree. This is all at the API level. So someone logged in will have the same permissions whether they are browsing the frontend of the site, the administration UI or using the API. This is nice, tidy and easy to understand. As for XHR vs not... In the Admin UI, we're doing most work through the API via XHR and JS templates. On the public site of things, everything happens server side just like any other CMS.
      - StavrosK 16y ago
        Thanks, I'm just wondering because, even though this approach had fantastic advantages, there also seem to be some things which are done differently in an API. For example, the API might use OAuth after logging the user in with credentials, so this is a case where things are done differently in the two approaches. In this case, does one simply write two ways of doing things? You might also not want to let the users do things with the api that they can do on the site, or give different permissions to the site versus the API. How do you handle these things?
- jamesbritt 16y ago
  " ... it is common/best practice to implement dependent web services and back end functionality before writing a web UI, right?" Well, there's UI-first development. http://www.sapdesignguild.org/editions/edition8/ui_first.asp http://www.sapdesignguild.org/editions/edition8/ui_first.asp "{Developers] usually create the user interface only after the underlying technical entities are available. Since the user interface sits on top of functions in the program logic and a database, why not create them early in the process and see what kind of interface will evolve? This approach inevitably leads to a user interface that mirrors database structures and to a user interaction that is determined by the flow of the program logic. It should be influenced; instead, by the way users act and think."
pak 16y ago
Ahhh! Invalid JSON in your examples! Please double quote all string values in your JSON, both keys and values, otherwise one day you will use a reserved word in JavaScript as an unquoted key and some client out there is going to be very sad. It seems strict, but JSON is a subset of JavaScript's object literal syntax. See http://json.org/ http://json.org/
- JoelSutherland 16y ago
  We use valid JSON in our documentation. Personally, I don't have a problem using invalid JSON in some places. For example, our templates. It saves keystrokes and will never cause problems since we're not evaluating those as Javascript anyway. I do understand that reasonable people would disagree with this position however. edit: It isn't clear here that I am not talking about our API or JSON really for that matter. The API requires valid JSON. But just as jQuery translates an object literal in it's AJAX call, our JS library (not API!) translates object literals to valid JSON before sending it to the API. Object literals are not JSON. For clarity we've edited the original post.
  - aaronblohowiak 16y ago
    IMHO, framework/library examples should be viewed as instruction as much as they are documentation. It helps to increase the understanding of junior devs.
    - JoelSutherland 16y ago
      In the face of good information I'm always willing to change my mind. Should we enforce valid JSON? This seems like a lot to ask of front-end devs that will be using our templates. For what it's worth, Kris on our team is very pro-JSON validity.
      - thwarted 16y ago
        If frontend devs can produce valid HTML, they can produce valid JSON. They aren't idiots. Be strict in what you produce and liberal in what you accept is nice and all, but that's how we got into the mess that is different implementations of HTML and "quirks mode" in different browsers today.
powertower 16y ago
From my own experience building server and desktop applications (with C# & .NET) I've found that the codebase, the user/dev API, and the UI have to be built and evolved together ... as all three are dependent on each other in ways that you won't truly understand until it's all complete.
jordanlev 16y ago
I first found out about you guys via the Recess PHP Framework ( http://www.recessframework.org/ http://www.recessframework.org/ ). Are you still using that under the hood for HiFi? Any plans to update the framework or continue developing it for public consumption?
- KrisJordan 16y ago
  HiFi and its API are running on Recess. The Recess database code is fundamental to making the data model / API possible. Recess has been somewhat dormant while focusing on product. I've slowly and infrequently been readying the 5.3 branch. 5.3 has so much upside and has now been out for over a year I see very little value in doing new work. The 5.3 branch packages modules for individual consumption rather than taking a kitchen sink approach. It feels more functional, in the lambda-y sense, given 5.3's anonymous function support. I've started a new SQL generation layer that is in the same spirit as Recess' SQLBuilder but much more powerful and flexible with heavy inspiration from Arel. I don't want to make any promises but its my full intention to package up the lessons of the last year or so into a great set of 5.3 libraries of Recess lineage.
dexen 16y ago
A co-worker (a web-developer) read through the article and guessed it's a spoof, or a joke, because of countless buzzwords and somewhat strange tech selection (schema-free on MySQL holding compressed JSON). Just saying...

102 points by JoelSutherland 16y ago

dools 16y ago

Getting the API right would be the only way a hosted CMS could acheive the flexibility that developers love about non-hosted platforms. This is so true! We've also developed a SaaS CMS (decalcms.com - still pre-release) and one of the major things I've noticed while looking around at other hosted solutions is they all have a pretty sharp "cut-off" past which you just can't do what you need to. VerbCMS solves this by letting you run your own code on their servers which we didn't want to do because it makes things less secure. Webvanta solves this by just saying "we'll build it for you" but that's just not scalable at all. A cracking API and integration model is the only real way that a hosted CMS can become a serious "platform of choice" for a lot of web developers/designs/agencies.

mark_l_watson 16y ago

Not being critical, just saying that it is common/best practice to implement dependent web services and back end functionality before writing a web UI, right? Nothing wrong with doing some prototype simple UIs, but having the back end stuff implemented and tested makes the first real UI implementation go much faster. So, they are following good development practices.

mathgladiator 16y ago
It's the difference between building stuff bottom-up versus top-down. It all depends on the product.
- aaronblohowiak 16y ago
  The product and the stability of the feature definition.
JoelSutherland 16y ago
(I work on HiFi) You're 100% right. What we're describing sounds a lot like waterfall. Here are a couple more details: * Our team of devs have collectively built over 450 websites. We know the domain well. * We also did a lot of UI design early on that guided some some of our work. That said, we didn't just design and give birth to a great API. We have iterated on a it a lot. Our clients were going on HiFi 6 months prior to the public release. We learned a lot and iterated on the API quite a bit during that period. What ended up being nice was the ability to make sweeping changes at one point (the API) that benefited the entire application. This is a huge advantage.
- StavrosK 16y ago
  Can you post a few more details, such as how you handled authentication, how views pass the session to the server (or do you use cookies for all that?), how the calls are being made (does each page do XHR calls and then render the results using logic itself?), etc?
  - JoelSutherland 16y ago
    I'm not the right person to answer all of these (the post author is). I'll do my best on what I can. For permissions, there are five levels of users. System, Agency, Site Owner, Member, Public. These are hierarchical like everything else. Group members and individual users can have varying read/write/create permissions for different areas of the content tree. This is all at the API level. So someone logged in will have the same permissions whether they are browsing the frontend of the site, the administration UI or using the API. This is nice, tidy and easy to understand. As for XHR vs not... In the Admin UI, we're doing most work through the API via XHR and JS templates. On the public site of things, everything happens server side just like any other CMS.
    - StavrosK 16y ago
      Thanks, I'm just wondering because, even though this approach had fantastic advantages, there also seem to be some things which are done differently in an API. For example, the API might use OAuth after logging the user in with credentials, so this is a case where things are done differently in the two approaches. In this case, does one simply write two ways of doing things? You might also not want to let the users do things with the api that they can do on the site, or give different permissions to the site versus the API. How do you handle these things?
jamesbritt 16y ago
" ... it is common/best practice to implement dependent web services and back end functionality before writing a web UI, right?" Well, there's UI-first development. http://www.sapdesignguild.org/editions/edition8/ui_first.asp http://www.sapdesignguild.org/editions/edition8/ui_first.asp "{Developers] usually create the user interface only after the underlying technical entities are available. Since the user interface sits on top of functions in the program logic and a database, why not create them early in the process and see what kind of interface will evolve? This approach inevitably leads to a user interface that mirrors database structures and to a user interaction that is determined by the flow of the program logic. It should be influenced; instead, by the way users act and think."

pak 16y ago

Ahhh! Invalid JSON in your examples! Please double quote all string values in your JSON, both keys and values, otherwise one day you will use a reserved word in JavaScript as an unquoted key and some client out there is going to be very sad. It seems strict, but JSON is a subset of JavaScript's object literal syntax. See http://json.org/ http://json.org/

JoelSutherland 16y ago
We use valid JSON in our documentation. Personally, I don't have a problem using invalid JSON in some places. For example, our templates. It saves keystrokes and will never cause problems since we're not evaluating those as Javascript anyway. I do understand that reasonable people would disagree with this position however. edit: It isn't clear here that I am not talking about our API or JSON really for that matter. The API requires valid JSON. But just as jQuery translates an object literal in it's AJAX call, our JS library (not API!) translates object literals to valid JSON before sending it to the API. Object literals are not JSON. For clarity we've edited the original post.
- aaronblohowiak 16y ago
  IMHO, framework/library examples should be viewed as instruction as much as they are documentation. It helps to increase the understanding of junior devs.
  - JoelSutherland 16y ago
    In the face of good information I'm always willing to change my mind. Should we enforce valid JSON? This seems like a lot to ask of front-end devs that will be using our templates. For what it's worth, Kris on our team is very pro-JSON validity.
    - thwarted 16y ago
      If frontend devs can produce valid HTML, they can produce valid JSON. They aren't idiots. Be strict in what you produce and liberal in what you accept is nice and all, but that's how we got into the mess that is different implementations of HTML and "quirks mode" in different browsers today.

powertower 16y ago

From my own experience building server and desktop applications (with C# & .NET) I've found that the codebase, the user/dev API, and the UI have to be built and evolved together ... as all three are dependent on each other in ways that you won't truly understand until it's all complete.

jordanlev 16y ago

I first found out about you guys via the Recess PHP Framework ( http://www.recessframework.org/ http://www.recessframework.org/ ). Are you still using that under the hood for HiFi? Any plans to update the framework or continue developing it for public consumption?

KrisJordan 16y ago
HiFi and its API are running on Recess. The Recess database code is fundamental to making the data model / API possible. Recess has been somewhat dormant while focusing on product. I've slowly and infrequently been readying the 5.3 branch. 5.3 has so much upside and has now been out for over a year I see very little value in doing new work. The 5.3 branch packages modules for individual consumption rather than taking a kitchen sink approach. It feels more functional, in the lambda-y sense, given 5.3's anonymous function support. I've started a new SQL generation layer that is in the same spirit as Recess' SQLBuilder but much more powerful and flexible with heavy inspiration from Arel. I don't want to make any promises but its my full intention to package up the lessons of the last year or so into a great set of 5.3 libraries of Recess lineage.

dexen 16y ago

A co-worker (a web-developer) read through the article and guessed it's a spoof, or a joke, because of countless buzzwords and somewhat strange tech selection (schema-free on MySQL holding compressed JSON). Just saying...