13 ms·

REST Servers in Go: Part 1 – standard library

257 points by tutfbhuf 6y ago

samuelroth 6y ago
Nice article! This is an interesting approach, much less likely to make Go devs' blood boil over unnecessary libraries. My only question is why the server / HTTP handlers have to deal with the Mutex. That seems like a "leak" from the `TaskStore` abstraction, which otherwise I really like. (Thank you for not using channels in that interface!)
- jrockway 6y ago
  I think it's necessary to leak the details of the mutex until you have some sort of transaction object to abstract that away. In a concurrent workload, these two things are different: store.Lock() store.WriteKey("foo", "bar") x := store.ReadKey("foo") store.Unlock() // x is always "bar" And: store.Lock() store.WriteKey("foo", "bar") store.Unlock() store.Lock() x := store.ReadKey("foo") store.Unlock() // x could be whatever another goroutine set "foo" to, not the "bar" that you just wrote. In a more complicated app, you'll have library that acts as the datastore, with transaction objects that abstract away the actual mutex (which will be something more complicated): var x string err := db.DoTx(func(tx *Tx) { tx.Write("foo", "bar") x = tx.Read("foo") }) if err != nil { ... } // what x is depends on the details of your database; maybe you're running at "read uncommitted", maybe you're running at "serializable". But, even in the simple examples, it's worth thinking about the difference between lock { write; read } and lock { write }; lock { read }.
jrockway 6y ago
Good introduction. A few thoughts: 1) Be careful with locks in the form "x.Lock(); x.DoSomething(); x.Unlock()". If DoSomething panics, you will still be holding the lock, and that's pretty much the end of your program. ("x.Lock(); defer x.Unlock(); x.DoSomething()" avoids this problem, but obviously in the non-panic case, the lock is released at a different time than in this implementation. Additional tweaking is required.) Generally I don't like locks in the request critical path because waiting for a lock is uncancelable, but in this very simple case it doesn't matter. For more complicated concurrency requirements, consider the difference between x.Lock()/x.Do()/x.Unlock vs. select { case x := <-ch: doSomethingWithX(x); case <-request.Context().Done(): error(request.Context().Err()) }. The channel wait can be cancelled when the user disconnects, or hits the stop button in the error, or the request timeout is reached. 2) Long if/else statements are harder to read than a switch statement. Instead of: if(foo == "bar") { // Bar } else if (foo == "baz") { // Baz } else { // Error } You might like: switch(foo) { case "bar": // Bar case "baz": // Baz default: // Error } These are exactly semantically equivalent, and neither protect you at compile-time from forgetting a case, but there is slightly less visual noise. Worth considering. 3) I have always found that error handling in http.HandlerFunc-tions cumbersome. The author runs into this, with code like: foo, err := Foo() if err != nil { http.Error(w, ...) return } bar, err := Bar() if err != nil { http.Error(w, ...) return } Basically, you end up writing the error handling code a number of times, and you have to do two things in the "err != nil" block, which is annoying. I prefer: func DoTheActualThing() ([]byte, error) { if everythingIsFine { return []byte(`{"result":"it worked and you are cool"}`), nil } return nil, errors.New("not everything is okay, feels sad") } Then in your handler function: func ServeHTTP(w http.ResponseWriter, req *http.Request) { result, err := DoTheActualThing() if err != nil { http.Error(w, ...) return } w.Header().Set("content-type", "application/json") w.WriteHeader(http.StatusOK) w.Write(result) } In this simple example, it doesn't matter, but when you do more than one thing that can cause an error, you'll like it better.
- klohto 6y ago
  Would you please expand more on your first point regarding using channels instead of Locks? It’s hard for me to wrap a head around it without practical example.
  - deleted 6y ago
    [deleted]
  - Philip-J-Fry 6y ago
    Not the OP but basically imagine that instead of locking a mutex to handle synchronised writes, you spawn a goroutine which just reads from a channel and writes the data. If that goroutine hasn't finished processing then the channel will be blocked, just like a mutex. So in your handler you can use a select statement to either write to the channel OR read from the request.Context().Done(). The request context only lives as long as the request. So if the connection drops or times out then the context gets cancelled and a value is pushed onto the done channel and your read is unblocked. Because you use a select statement then which ever operation unblocks first is what happens. If the write channel unblocks then you get to write your value. If your request context gets cancelled then you can report an error. The request context will always get cancelled eventually, unlike a mutex which will wait forever.
- makeworld 6y ago
  > Be careful with locks in the form "x.Lock(); x.DoSomething(); x.Unlock()". If DoSomething panics, you will still be holding the lock, and that's pretty much the end of your program. Interesting, thanks. But isn't panicking the end of your program anyway? Could you provide another example where no using defer causes problems?
  - acrispino 6y ago
    Not necessarily. Panics can be recovered and the stdlib http server recovers panics from handlers.
- AWebOfBrown 6y ago
anderspitman 6y ago
I think I've managed to get by with less dependencies in Go than any other language. It somehow walks the line between JavaScript leftpad and Python "stdlib is where modules go to die". I don't think there's been a single instance where I've thought "why can't stdlib do this?" nor "why the heck is this in stdlib?"
- laverya 6y ago
  > I don't think there's been a single instance where I've thought "why can't stdlib do this?" I've had this a few times, most recently with "how do I add this data file to my binary". At least that one made it to master now, and will be in 1.16! Another gripe is the lack of a proper parallel safe map (no, map[interface{}]interface{} like sync.Map is just not acceptable) which would be a godsend and should honestly just be the normal map implementation. Maybe eventually with generics... (This is more of a language gripe than a stdlib gripe, though, and I'm sure that once generics come out we'll see a sync.Map that doesn't use interface {})
  - swirepe 6y ago
    >I've had this a few times, most recently with "how do I add this data file to my binary". At least that one made it to master now, and will be in 1.16! Wait, how?? I've done some unholy things.
    - deleted 6y ago
      [deleted]
    - laverya 6y ago
      The best example for the new '//go:embed' directive I've seen so far is this: package main import ( "embed" "net/http" ) //go:embed assets/* var assets embed.FS func main() { fs := http.FileServer(http.FS(assets)) http.ListenAndServe(":8080", fs) } For the next month, the list of options here [0] will have to suffice. 0: https://go.googlesource.com/proposal/+/master/design/draft-embed.md#background https://go.googlesource.com/proposal/+/master/design/draft-e...
tptacek 6y ago
With respect to DRY'ing the JSON code, isn't something like this workable: err = json.NewEncoder(w).Encode(&task) I know there used to be a reason why this was disfavored but thought it had been addressed in the stdlib.
tekstar 6y ago
Nice demo. A few things you could add to make this more realistic to something that gets shipped: -CORS support. Deploy this to a non-local domain and try to reach it from a web browser and it will fail. I like https://github.com/rs/cors https://github.com/rs/cors. I had rolled my own but then moved to that library. - input validation. I like go-playground/validator. The other big issue is the locking by hand around the task store. In reality usually there would be a database to handle concurrent read/writes. I use SQLite in production. I know this is just a demo and you want to use just stdlib, but serializing all data access is sort of unacceptable as a solution in a concurrent language like Go. When I'm not handling concurrency with SQLite I like to implement The actor pattern, having a persistent goroutine listen and respond to "taskstore" requests via channels.
- permille42 6y ago
  You are implying that "realistic" uses would be using multiple domains and have browser calls between them. Why would that be?
  - cwackerfuss 6y ago
    This is common. Web apps frequently fetch data from other domains, whether internal or third party.
    - permille42 6y ago
      Agreed, but a common extension does not mean it is required, nor that it should be demanded that an example of making a REST service needs to show how to do it. The example is simply the beginning. It can be extended as needed depending on your use case. This is how most things Golang work. You start with the basics and add what you need on a case by case basis. If you want something with every bell and whistle out of the box, Golang is going to be a disappointment generally as that is not the Golang mentality. There are of course sufficient frameworks in Golang these days to provide such things though. Hence the article clearly says "standard library" to indicate that it isn't the last word on how to setup a REST service in Go.
dastx 6y ago
Is there a reason to prevent concurrent reads? Seems rather .. Silly considering the db should handle all that stuff. Also, one thing I'm still waiting for is a decent non-complex di that works using go generate. I'm aware of google's wire (?) But I've looked at it multiple times and I still struggle to understand it.
- throwaway894345 6y ago
  I've never understood the value proposition of a DI framework. Why would I want one when I can initialize my objects in main()? Is XML or JSON or whatever really that much more pleasant than wiring together Go objects?
  - orisho 6y ago
    I'm with you on this. Dependency injection seems to me to replicate some of the features of interfaces while introducing complexity because the injection is indirect -- instead of having code that initializes a different object, it all happens dynamically during runtime using reflection. An antipattern as far as I'm concerned. It seems to achieve little or no gain at a very high cost.
    - throwaway894345 6y ago
      It sounds like people just want a little bit of dynamic typing to assemble their object graph or something, but that's such a small (possibly negative) value add for such a steep price (everyone who might touch this software--new team members, system administrators, whoever takes over maintenance, etc--now needs to know the DI framework for even the most basic troubleshooting).
    - isbvhodnvemrwvn 6y ago
      It doesn't have to happen at runtime. For instance in Java you have annotation processors which generate the boilerplate for you (in DI space I'm only aware of Dagger 2 that does it). It's a bit inconvenient but you can easily audit what's going on. That being said I somehow doubt a similar tool is used with go.
alchermd 6y ago
I'm an API developer working with Python and Django. I did dabble with Golang for quite some time, but I just can't seem to justify the effort (in terms of lines of codes and static typing) of writing a ReST API with Go when I can build a similar one with Django and co (DRF, Swagger, etc). Can someone chime in? There must be an obvious advantage that I might be missing.
- lmarcos 6y ago
  I guess it's subjective. For me the clear advantage of building HTTP endpoints using Go (framework-less) over DRF, Swagger and others is: no magic, no annotations, no dependencies; I know exactly what's happening and I can build a tailored system that is adjusted to my needs. This leads to very performant systems that are highly independent of the current trends and, most importantly, to a high degree of ownership of the systems you are building. The act of writing these kind of systems is IMHO, somehow, liberating.
- ascotan 6y ago
  1. simpler architecture - python typically requires multiple systems to deal with concurrency (ioloops, celery, etc) 2. execution speed - Django's ORM comes to mind 3. concurrency That being said there are alot of great features with python web frameworks that come out of the box. You need to a lightweight project like gin/echo or something to get these features. Naked net/http is sorta like the 'erector set' of web frameworks.
- mhh__ 6y ago
  Static typing buys you correctness now. Go isn't really a poster child of language design, to be blunt - but it's better to find out now rather than in production. Compiled code is also significantly faster in many cases.
  - volkk 6y ago
    i see your point and i know OP asked about why Go instead of python, but having a statically typed language isn't limited to only Go. you could replace everything you wrote with Java/C++/C# and still have a similar answer (minus language design i guess which is a different argument)
alisausa 6y ago
Top burny busty chicks only on this site! Follow the link, and you won’t be sorry! - https://adultlove.life https://adultlove.life
ankurpatel 6y ago
Looking at such articles makes me feel we are going back in time rather than improving efficiencies for developers to build RESTful server. If you look at Ruby on Rails you can build the server shown here in one min that is scalable and backed with database. I know people will complain about speed of execution of language and framework but do you really care if you are not expecting Google like traffic.
- geodel 6y ago
  I think most people are expecting zero traffic so they do not need to waste even one min to write any REST server.
- skjfdoslifjeifj 6y ago
  He could build it in Go much faster just by using a couple of additional libraries but he's purposely limiting himself to just the standard library for the purpose of these articles. > I know people will complain about speed of execution of language and framework but do you really care if you are not expecting Google like traffic. If you don't care that's fine, but there are lots of us that do. This is subjective, but I find the memory use and overall performance of Rails/Django/Spring Boot applications to be completely unacceptable and there are far too many breaking changes.
  - lenartowski 6y ago
    What languages/framework do you use when performance/memory usage is key?
    - theshrike79 6y ago
      Not OP, but I'd go with Go or a fresh version of .NET Core depending on where it would need to integrate to. If we're talking real time sub-millisecond performance, then Rust.
  - ptr 6y ago
    It doesn’t make sense to say Rails etc are “Completely unacceptable” without giving a context. How can it be subjective? Memory is also rather arbitrary, why focus on that and not user friendliness? Or even GPU or cache efficiency? “Breaking changes” might not matter either if what you’re writing is a one-off.
bpicolo 6y ago
Defining them all on a single server struct means once you're past a handful you start having a hard-as-heck to organize folder of handlers (or a bunch of massive code files). How are folk managing Go endpoint as apps scale across number of endpoints? I'm skeptical that one struct in one folder leads to good app structure, and have seen this start to break down in a few cases in practice
- oppositelock 6y ago
  It depends on what the struct contains. I have developed many Go API's professionally at several companies since Go 1.1, and all my servers and up looking like a server struct with only a few fields - a database, AWS client object, and some prometheus metrics. The logic is typically split among many files, all implementing receivers on that struct. If you have independent, different elements in that API, you break them out into separate "servers" but still register the endpoints on the same HTTP handler. I know that people don't like external libraries too much, but I'd like to plug my own here. You declare your API in OpenAPI 3.0 (aka, Swagger) and it generates your server and models for you, so all you need to do is write the business logic. (https://github.com/deepmap/oapi-codegen https://github.com/deepmap/oapi-codegen)
  - bpicolo 6y ago
    Thanks for the link - I've had trouble finding pretty much exactly this in the Go ecosystem after a few brief looks, and codegen is really the only sensible way to do openapi in go. I wouldn't be surprised to use this soon
hit8run 6y ago
I’ve written a full stack web app with payment integration multi tenancy etc. in plain go. I learned a lot on that journey and what I learned made me value full stack frameworks like rails again. If you want to build something competitive you will need to write lots of things in plain go that you wouldn’t need to bother with when using rails or something similar with sane conventions.
- lenartowski 6y ago
  Out of curiosity, is your app shared on github?
- heofizzy 6y ago
  +1 on that. I recently had to write a small web app that has to do some data processing and interact with database on request. We chose go for this because it was faster than php for this one particular data intensive task. I liked working with go, but frameworks like rails, laravel or django have a huge advantage in terms of developer productivity when compared to go.
zemo 6y ago
usually for stores like this I just do this: type Store interface { GetTask(*Task) error } instead of having "GetTaskByID" and "GetTaskByTag" and whatnot. Then in the caller you just do this: task := store.Task{ID: 5} if err := db.GetTask(&task); err != nil { // wahtever } ^ that gets the task by ID task := store.Task{Tag: "foo"} if err := db.GetTask(&task); err != nil { // wahtever } ^ that gets the task by tag.
- fpopa 6y ago
  This makes sense, did you implement this alongside grpc / protobuf? I'm curios about the way you handled zero values, field masks could be a solution, but I think it would get bloaty.
  - zemo 6y ago
    func (db *actualStoreImplementation) GetTask(t *Task) error { if (t.ID != 0) { // query by ID, mutate the parameter, return nil } if (t.Tag != "") { // query by tag, mutate the parameter, return nil } return ErrWhatever } usually I have some other package that defines all of the types that can appear on the wire (which I often call `wire` because `proto` is taken by protobuf), define some exported interface in that package with an unexported method so that no other packages can define new types for that interface, and then have a method on my db structs that returns the wire types, like this: func (t Task) Public() wire.Value { return wire.Task{ // explicitly generate what you want } }
- hellcow 6y ago
  The downside to this approach is that you're unable to know what fields you can use in your query without reading the GetTask function, and changes to the GetTask function can silently break all callers, since it's now a runtime error.
  - zemo 6y ago
    tbh I've been using this method for years and the first one has never been an issue, because practically speaking you should only expect to give a single struct with a field or two filled in if that field or combination of fields is unique. You probably need that level of domain knowledge about what you're working on elsewhere anyway, so it has never been a problem. I mean ... for the second problem that's broadly true of making any changes to your data access layer since by definition your Go compiler is not going to, for example, check the validity of a SQL query. So ... yes? but that's not unique to this approach, that's true generally.
alisause 6y ago
I was a really bad girl. Punish me with your dick in my mouth. - https://adultlove.life https://adultlove.life
0xbkt 6y ago
I would probably go with gRPC + grpc-gateway[1] instead. Declaring your services and models in proto files, annotating your services with google.api.http to help grpc-gateway scaffold your HTTP base. Then just implement your services from the interface generated by grpc-go. You can even register your gRPC services to grpc-gateway without actually bringing up a gRPC server. You finally end up having your exact data models injected to your service handlers. Thus, you don't have to repeat yourself preparing the groundwork to call into your services. This is mostly the way today services in big OSS projects are exposed outside for consumption in a RESTful style. One exception I know of is sourcegraph/sourcegraph. [1] https://github.com/grpc-ecosystem/grpc-gateway#usage https://github.com/grpc-ecosystem/grpc-gateway#usage
- yolo42 6y ago
  This is exactly what I started using in my new open source project (not announced yet) and the experience has been amazing. I was wary about it in the beginning but it quickly became clear that this is the easiest and probably the fastest way to get this job done. The community and tooling has come far ahead.
- whoisjohnkid 6y ago
  Have you looked into goa design[1]? You define your services with a contract via their DSL and it generates all code for you as well; has some nice validation options out of the box. - similar to grpc gateway you just need to conform to the generated code’s service interface. I believe it supports, http, websocket and grpc. [1]https://github.com/goadesign/goa https://github.com/goadesign/goa
- anderspitman 6y ago
  These things can be fun to play with, but my experience is most projects don't need them. I would rather spend my time implementing functionality than regenerating and recompiling code from proto files. The large projects you mention often have something most projects don't: a well-defined data model and interface that has been heavily iterated and evolved over time. At that point things like backwards compatibility, auto docs, and performance become more important, and gRPC can be more useful. You only get a certain number of complexity tokens and IMO it's not worth spending any here.
barefeg 6y ago
Is there a spec-first framework for go that also follows a middleware paradigm?
- yolo42 6y ago
  Yes. Please take a look at grpc-gateway. You write your specs as proto files and then get handlers and middlewares to implement the endpoints.

257 points by tutfbhuf 6y ago

Nice article! This is an interesting approach, much less likely to make Go devs' blood boil over unnecessary libraries. My only question is why the server / HTTP handlers have to deal with the Mutex. That seems like a "leak" from the `TaskStore` abstraction, which otherwise I really like. (Thank you for not using channels in that interface!)

jrockway 6y ago
I think it's necessary to leak the details of the mutex until you have some sort of transaction object to abstract that away. In a concurrent workload, these two things are different: store.Lock() store.WriteKey("foo", "bar") x := store.ReadKey("foo") store.Unlock() // x is always "bar" And: store.Lock() store.WriteKey("foo", "bar") store.Unlock() store.Lock() x := store.ReadKey("foo") store.Unlock() // x could be whatever another goroutine set "foo" to, not the "bar" that you just wrote. In a more complicated app, you'll have library that acts as the datastore, with transaction objects that abstract away the actual mutex (which will be something more complicated): var x string err := db.DoTx(func(tx *Tx) { tx.Write("foo", "bar") x = tx.Read("foo") }) if err != nil { ... } // what x is depends on the details of your database; maybe you're running at "read uncommitted", maybe you're running at "serializable". But, even in the simple examples, it's worth thinking about the difference between lock { write; read } and lock { write }; lock { read }.

jrockway 6y ago

Good introduction. A few thoughts: 1) Be careful with locks in the form "x.Lock(); x.DoSomething(); x.Unlock()". If DoSomething panics, you will still be holding the lock, and that's pretty much the end of your program. ("x.Lock(); defer x.Unlock(); x.DoSomething()" avoids this problem, but obviously in the non-panic case, the lock is released at a different time than in this implementation. Additional tweaking is required.) Generally I don't like locks in the request critical path because waiting for a lock is uncancelable, but in this very simple case it doesn't matter. For more complicated concurrency requirements, consider the difference between x.Lock()/x.Do()/x.Unlock vs. select { case x := <-ch: doSomethingWithX(x); case <-request.Context().Done(): error(request.Context().Err()) }. The channel wait can be cancelled when the user disconnects, or hits the stop button in the error, or the request timeout is reached. 2) Long if/else statements are harder to read than a switch statement. Instead of: if(foo == "bar") { // Bar } else if (foo == "baz") { // Baz } else { // Error } You might like: switch(foo) { case "bar": // Bar case "baz": // Baz default: // Error } These are exactly semantically equivalent, and neither protect you at compile-time from forgetting a case, but there is slightly less visual noise. Worth considering. 3) I have always found that error handling in http.HandlerFunc-tions cumbersome. The author runs into this, with code like: foo, err := Foo() if err != nil { http.Error(w, ...) return } bar, err := Bar() if err != nil { http.Error(w, ...) return } Basically, you end up writing the error handling code a number of times, and you have to do two things in the "err != nil" block, which is annoying. I prefer: func DoTheActualThing() ([]byte, error) { if everythingIsFine { return []byte(`{"result":"it worked and you are cool"}`), nil } return nil, errors.New("not everything is okay, feels sad") } Then in your handler function: func ServeHTTP(w http.ResponseWriter, req *http.Request) { result, err := DoTheActualThing() if err != nil { http.Error(w, ...) return } w.Header().Set("content-type", "application/json") w.WriteHeader(http.StatusOK) w.Write(result) } In this simple example, it doesn't matter, but when you do more than one thing that can cause an error, you'll like it better.

klohto 6y ago
Would you please expand more on your first point regarding using channels instead of Locks? It’s hard for me to wrap a head around it without practical example.
- deleted 6y ago
  [deleted]
- Philip-J-Fry 6y ago
  Not the OP but basically imagine that instead of locking a mutex to handle synchronised writes, you spawn a goroutine which just reads from a channel and writes the data. If that goroutine hasn't finished processing then the channel will be blocked, just like a mutex. So in your handler you can use a select statement to either write to the channel OR read from the request.Context().Done(). The request context only lives as long as the request. So if the connection drops or times out then the context gets cancelled and a value is pushed onto the done channel and your read is unblocked. Because you use a select statement then which ever operation unblocks first is what happens. If the write channel unblocks then you get to write your value. If your request context gets cancelled then you can report an error. The request context will always get cancelled eventually, unlike a mutex which will wait forever.
makeworld 6y ago
> Be careful with locks in the form "x.Lock(); x.DoSomething(); x.Unlock()". If DoSomething panics, you will still be holding the lock, and that's pretty much the end of your program. Interesting, thanks. But isn't panicking the end of your program anyway? Could you provide another example where no using defer causes problems?
- acrispino 6y ago
  Not necessarily. Panics can be recovered and the stdlib http server recovers panics from handlers.
AWebOfBrown 6y ago

anderspitman 6y ago

I think I've managed to get by with less dependencies in Go than any other language. It somehow walks the line between JavaScript leftpad and Python "stdlib is where modules go to die". I don't think there's been a single instance where I've thought "why can't stdlib do this?" nor "why the heck is this in stdlib?"

laverya 6y ago
> I don't think there's been a single instance where I've thought "why can't stdlib do this?" I've had this a few times, most recently with "how do I add this data file to my binary". At least that one made it to master now, and will be in 1.16! Another gripe is the lack of a proper parallel safe map (no, map[interface{}]interface{} like sync.Map is just not acceptable) which would be a godsend and should honestly just be the normal map implementation. Maybe eventually with generics... (This is more of a language gripe than a stdlib gripe, though, and I'm sure that once generics come out we'll see a sync.Map that doesn't use interface {})
- swirepe 6y ago
  >I've had this a few times, most recently with "how do I add this data file to my binary". At least that one made it to master now, and will be in 1.16! Wait, how?? I've done some unholy things.
  - deleted 6y ago
    [deleted]
  - laverya 6y ago
    The best example for the new '//go:embed' directive I've seen so far is this: package main import ( "embed" "net/http" ) //go:embed assets/* var assets embed.FS func main() { fs := http.FileServer(http.FS(assets)) http.ListenAndServe(":8080", fs) } For the next month, the list of options here [0] will have to suffice. 0: https://go.googlesource.com/proposal/+/master/design/draft-embed.md#background https://go.googlesource.com/proposal/+/master/design/draft-e...

tptacek 6y ago

With respect to DRY'ing the JSON code, isn't something like this workable: err = json.NewEncoder(w).Encode(&task) I know there used to be a reason why this was disfavored but thought it had been addressed in the stdlib.

tekstar 6y ago

Nice demo. A few things you could add to make this more realistic to something that gets shipped: -CORS support. Deploy this to a non-local domain and try to reach it from a web browser and it will fail. I like https://github.com/rs/cors https://github.com/rs/cors. I had rolled my own but then moved to that library. - input validation. I like go-playground/validator. The other big issue is the locking by hand around the task store. In reality usually there would be a database to handle concurrent read/writes. I use SQLite in production. I know this is just a demo and you want to use just stdlib, but serializing all data access is sort of unacceptable as a solution in a concurrent language like Go. When I'm not handling concurrency with SQLite I like to implement The actor pattern, having a persistent goroutine listen and respond to "taskstore" requests via channels.

permille42 6y ago
You are implying that "realistic" uses would be using multiple domains and have browser calls between them. Why would that be?
- cwackerfuss 6y ago
  This is common. Web apps frequently fetch data from other domains, whether internal or third party.
  - permille42 6y ago
    Agreed, but a common extension does not mean it is required, nor that it should be demanded that an example of making a REST service needs to show how to do it. The example is simply the beginning. It can be extended as needed depending on your use case. This is how most things Golang work. You start with the basics and add what you need on a case by case basis. If you want something with every bell and whistle out of the box, Golang is going to be a disappointment generally as that is not the Golang mentality. There are of course sufficient frameworks in Golang these days to provide such things though. Hence the article clearly says "standard library" to indicate that it isn't the last word on how to setup a REST service in Go.

dastx 6y ago

Is there a reason to prevent concurrent reads? Seems rather .. Silly considering the db should handle all that stuff. Also, one thing I'm still waiting for is a decent non-complex di that works using go generate. I'm aware of google's wire (?) But I've looked at it multiple times and I still struggle to understand it.

throwaway894345 6y ago
I've never understood the value proposition of a DI framework. Why would I want one when I can initialize my objects in main()? Is XML or JSON or whatever really that much more pleasant than wiring together Go objects?
- orisho 6y ago
  I'm with you on this. Dependency injection seems to me to replicate some of the features of interfaces while introducing complexity because the injection is indirect -- instead of having code that initializes a different object, it all happens dynamically during runtime using reflection. An antipattern as far as I'm concerned. It seems to achieve little or no gain at a very high cost.
  - throwaway894345 6y ago
    It sounds like people just want a little bit of dynamic typing to assemble their object graph or something, but that's such a small (possibly negative) value add for such a steep price (everyone who might touch this software--new team members, system administrators, whoever takes over maintenance, etc--now needs to know the DI framework for even the most basic troubleshooting).
  - isbvhodnvemrwvn 6y ago
    It doesn't have to happen at runtime. For instance in Java you have annotation processors which generate the boilerplate for you (in DI space I'm only aware of Dagger 2 that does it). It's a bit inconvenient but you can easily audit what's going on. That being said I somehow doubt a similar tool is used with go.

alchermd 6y ago

I'm an API developer working with Python and Django. I did dabble with Golang for quite some time, but I just can't seem to justify the effort (in terms of lines of codes and static typing) of writing a ReST API with Go when I can build a similar one with Django and co (DRF, Swagger, etc). Can someone chime in? There must be an obvious advantage that I might be missing.

lmarcos 6y ago
I guess it's subjective. For me the clear advantage of building HTTP endpoints using Go (framework-less) over DRF, Swagger and others is: no magic, no annotations, no dependencies; I know exactly what's happening and I can build a tailored system that is adjusted to my needs. This leads to very performant systems that are highly independent of the current trends and, most importantly, to a high degree of ownership of the systems you are building. The act of writing these kind of systems is IMHO, somehow, liberating.
ascotan 6y ago
1. simpler architecture - python typically requires multiple systems to deal with concurrency (ioloops, celery, etc) 2. execution speed - Django's ORM comes to mind 3. concurrency That being said there are alot of great features with python web frameworks that come out of the box. You need to a lightweight project like gin/echo or something to get these features. Naked net/http is sorta like the 'erector set' of web frameworks.
mhh__ 6y ago
Static typing buys you correctness now. Go isn't really a poster child of language design, to be blunt - but it's better to find out now rather than in production. Compiled code is also significantly faster in many cases.
- volkk 6y ago
  i see your point and i know OP asked about why Go instead of python, but having a statically typed language isn't limited to only Go. you could replace everything you wrote with Java/C++/C# and still have a similar answer (minus language design i guess which is a different argument)

alisausa 6y ago

Top burny busty chicks only on this site! Follow the link, and you won’t be sorry! - https://adultlove.life https://adultlove.life

ankurpatel 6y ago

Looking at such articles makes me feel we are going back in time rather than improving efficiencies for developers to build RESTful server. If you look at Ruby on Rails you can build the server shown here in one min that is scalable and backed with database. I know people will complain about speed of execution of language and framework but do you really care if you are not expecting Google like traffic.

geodel 6y ago
I think most people are expecting zero traffic so they do not need to waste even one min to write any REST server.
skjfdoslifjeifj 6y ago
He could build it in Go much faster just by using a couple of additional libraries but he's purposely limiting himself to just the standard library for the purpose of these articles. > I know people will complain about speed of execution of language and framework but do you really care if you are not expecting Google like traffic. If you don't care that's fine, but there are lots of us that do. This is subjective, but I find the memory use and overall performance of Rails/Django/Spring Boot applications to be completely unacceptable and there are far too many breaking changes.
- lenartowski 6y ago
  What languages/framework do you use when performance/memory usage is key?
  - theshrike79 6y ago
    Not OP, but I'd go with Go or a fresh version of .NET Core depending on where it would need to integrate to. If we're talking real time sub-millisecond performance, then Rust.
- ptr 6y ago
  It doesn’t make sense to say Rails etc are “Completely unacceptable” without giving a context. How can it be subjective? Memory is also rather arbitrary, why focus on that and not user friendliness? Or even GPU or cache efficiency? “Breaking changes” might not matter either if what you’re writing is a one-off.

bpicolo 6y ago

Defining them all on a single server struct means once you're past a handful you start having a hard-as-heck to organize folder of handlers (or a bunch of massive code files). How are folk managing Go endpoint as apps scale across number of endpoints? I'm skeptical that one struct in one folder leads to good app structure, and have seen this start to break down in a few cases in practice

oppositelock 6y ago
It depends on what the struct contains. I have developed many Go API's professionally at several companies since Go 1.1, and all my servers and up looking like a server struct with only a few fields - a database, AWS client object, and some prometheus metrics. The logic is typically split among many files, all implementing receivers on that struct. If you have independent, different elements in that API, you break them out into separate "servers" but still register the endpoints on the same HTTP handler. I know that people don't like external libraries too much, but I'd like to plug my own here. You declare your API in OpenAPI 3.0 (aka, Swagger) and it generates your server and models for you, so all you need to do is write the business logic. (https://github.com/deepmap/oapi-codegen https://github.com/deepmap/oapi-codegen)
- bpicolo 6y ago
  Thanks for the link - I've had trouble finding pretty much exactly this in the Go ecosystem after a few brief looks, and codegen is really the only sensible way to do openapi in go. I wouldn't be surprised to use this soon

hit8run 6y ago

I’ve written a full stack web app with payment integration multi tenancy etc. in plain go. I learned a lot on that journey and what I learned made me value full stack frameworks like rails again. If you want to build something competitive you will need to write lots of things in plain go that you wouldn’t need to bother with when using rails or something similar with sane conventions.

lenartowski 6y ago
Out of curiosity, is your app shared on github?
heofizzy 6y ago
+1 on that. I recently had to write a small web app that has to do some data processing and interact with database on request. We chose go for this because it was faster than php for this one particular data intensive task. I liked working with go, but frameworks like rails, laravel or django have a huge advantage in terms of developer productivity when compared to go.

zemo 6y ago

usually for stores like this I just do this: type Store interface { GetTask(*Task) error } instead of having "GetTaskByID" and "GetTaskByTag" and whatnot. Then in the caller you just do this: task := store.Task{ID: 5} if err := db.GetTask(&task); err != nil { // wahtever } ^ that gets the task by ID task := store.Task{Tag: "foo"} if err := db.GetTask(&task); err != nil { // wahtever } ^ that gets the task by tag.

fpopa 6y ago
This makes sense, did you implement this alongside grpc / protobuf? I'm curios about the way you handled zero values, field masks could be a solution, but I think it would get bloaty.
- zemo 6y ago
  func (db *actualStoreImplementation) GetTask(t *Task) error { if (t.ID != 0) { // query by ID, mutate the parameter, return nil } if (t.Tag != "") { // query by tag, mutate the parameter, return nil } return ErrWhatever } usually I have some other package that defines all of the types that can appear on the wire (which I often call `wire` because `proto` is taken by protobuf), define some exported interface in that package with an unexported method so that no other packages can define new types for that interface, and then have a method on my db structs that returns the wire types, like this: func (t Task) Public() wire.Value { return wire.Task{ // explicitly generate what you want } }
hellcow 6y ago
The downside to this approach is that you're unable to know what fields you can use in your query without reading the GetTask function, and changes to the GetTask function can silently break all callers, since it's now a runtime error.
- zemo 6y ago
  tbh I've been using this method for years and the first one has never been an issue, because practically speaking you should only expect to give a single struct with a field or two filled in if that field or combination of fields is unique. You probably need that level of domain knowledge about what you're working on elsewhere anyway, so it has never been a problem. I mean ... for the second problem that's broadly true of making any changes to your data access layer since by definition your Go compiler is not going to, for example, check the validity of a SQL query. So ... yes? but that's not unique to this approach, that's true generally.

alisause 6y ago

I was a really bad girl. Punish me with your dick in my mouth. - https://adultlove.life https://adultlove.life

0xbkt 6y ago

I would probably go with gRPC + grpc-gateway[1] instead. Declaring your services and models in proto files, annotating your services with google.api.http to help grpc-gateway scaffold your HTTP base. Then just implement your services from the interface generated by grpc-go. You can even register your gRPC services to grpc-gateway without actually bringing up a gRPC server. You finally end up having your exact data models injected to your service handlers. Thus, you don't have to repeat yourself preparing the groundwork to call into your services. This is mostly the way today services in big OSS projects are exposed outside for consumption in a RESTful style. One exception I know of is sourcegraph/sourcegraph. [1] https://github.com/grpc-ecosystem/grpc-gateway#usage https://github.com/grpc-ecosystem/grpc-gateway#usage

yolo42 6y ago
This is exactly what I started using in my new open source project (not announced yet) and the experience has been amazing. I was wary about it in the beginning but it quickly became clear that this is the easiest and probably the fastest way to get this job done. The community and tooling has come far ahead.
whoisjohnkid 6y ago
Have you looked into goa design[1]? You define your services with a contract via their DSL and it generates all code for you as well; has some nice validation options out of the box. - similar to grpc gateway you just need to conform to the generated code’s service interface. I believe it supports, http, websocket and grpc. [1]https://github.com/goadesign/goa https://github.com/goadesign/goa
anderspitman 6y ago
These things can be fun to play with, but my experience is most projects don't need them. I would rather spend my time implementing functionality than regenerating and recompiling code from proto files. The large projects you mention often have something most projects don't: a well-defined data model and interface that has been heavily iterated and evolved over time. At that point things like backwards compatibility, auto docs, and performance become more important, and gRPC can be more useful. You only get a certain number of complexity tokens and IMO it's not worth spending any here.

barefeg 6y ago

Is there a spec-first framework for go that also follows a middleware paradigm?

yolo42 6y ago
Yes. Please take a look at grpc-gateway. You write your specs as proto files and then get handlers and middlewares to implement the endpoints.