8 ms·

Back to basics: Writing an application using Go and PostgreSQL

129 points by benhoyt 5y ago

akshayshah 5y ago
Great article! These days, I find myself reaching for sqlc whenever I need to access Postgres from Go. My experience with the project has been excellent: I get type-safe APIs and control over exactly which queries are executed without much fuss. https://sqlc.dev/ https://sqlc.dev/
- Ingon 5y ago
  Yes! I’ve been using sqlc both at work and on side projects and it works awesome. For me it strikes the right balance between “I want to work with sql/db abstractions” and “I don’t need orm, just automated mapping”. Has a few rough edges when advanced types needs to get mapped, but it’s getting there.
  - iknowstuff 5y ago
    >I don’t need orm, just automated mapping” Check out https://github.com/launchbadge/sqlx#sqlx-is-not-an-orm https://github.com/launchbadge/sqlx#sqlx-is-not-an-orm for an alternative approach. struct Country { country: String, count: i64 } let countries = sqlx::query_as!(Country, "SELECT country, count FROM countries WHERE organization = ?", organization) .fetch_all(&db_pool) .await?;
- henvic 5y ago
  Thanks! I love the concept of sqlc, but their support for pgx with its native PostgreSQL interface was quite limited when I checked. I imagine it's better nowadays, and will try again soon!
  - lonk11 5y ago
    The support for native pgx in sqlc is good now. I migrated my project from database/sql to pgx recently and posted the instructions at the end of this post: https://linklonk.com/item/1948319982853095424 https://linklonk.com/item/1948319982853095424
  - sa46 5y ago
    You might like pggen (I’m the author) which only supports Postgres and pgx. https://github.com/jschaf/pggen https://github.com/jschaf/pggen pggen occupies the same design space as sqlc but the implementations are quite different. Sqlc figures out the query types using type inference in Go which is nice because you don’t need Postgres at build time. Pggen asks Postgres what the query types are which is nice because it works with any extensions and arbitrarily complex queries.
    - henvic 5y ago
      Thank you. I'll take a look in the upcoming weeks. It seems promising!
- sdevonoes 5y ago
  Dumb question: how do you write dynamic queries using sqlc? From the doc I see: -- name: ListAuthors :many SELECT * FROM authors ORDER BY name; But what if I want to get a list of authors by date? Shall I do: -- name: ListAuthors :many SELECT * FROM authors ORDER BY name; -- name: ListAuthorsByDate :many ... And what if later I need to get a list of authors by date and name? Should I write yet another query? I'm used to write one generic (composable) query that is very handy in situations where one need to retrieve rows by many different filtering criteria. Is that possible in sqlc?
  - sa46 5y ago
    Dynamic queries are difficult to model when starting from a static query since SQL doesn't compose well. The workarounds I've used: - Use multiple queries that share the same output type. - Push the predicate into the query directly. However, you can't pass an expression for an order by column since the order by clause takes a name, not an expression. Postgres doesn't allow using names as arguments to a prepared query so that leaves either adding an annotation like sqlc.order_by that's dynamically added to a query string, or by getting more creative with the structure of the query: SELECT * FROM AUTHORS WHERE sqlc.arg('by_date') ORDER BY DATE UNION ALL SELECT * FROM AUTHORS WHERE NOT sqlc.arg('by_date')
pstuart 5y ago
Postgres is awesome technology but for a lot of applications SQLite will do the job just fine. This gets you a single binary with zero external dependencies.
- w-j-w 5y ago
  I keep seeing this comment on HN, but I never really get it. In the world of platform as a service, launching a database instance is far easier than managing disk based storage yourself. It also has a very obvious route to horizontally scaling the web process. Deployment is easier too with an external database, there's no data on the disk of the machine running the web process that needs to be carried over.
  - tekstar 5y ago
    > launching a database instance is far easier than managing disk based storage yourself Look, it comes down to what you, the developer are comfortable with. If you know and are comfortable with running something like postgres in the cloud, then go for it. Some of us prefer to not. Similarly, not all of us nuke the entire filesystem on every deploy :)
  - mtlynch 5y ago
    I had the same objection to SQLite, and then I heard about Litestream, and it won me over.[0] Litestream watches your SQLite database and then streams changes to a cloud storage provider (e.g., S3, Backblaze). You get the performance and simplicity of writing SQLite to the local filesystem, but it's syncing to the cloud. And the cool part is that you don't have to change any of your application code to do it - as far as your app is concerned, it's writing to a local SQLite file. I wrote a little log uploading utility for my business that uses Litestream, and it's been fantastic.[1] It essentially carries around its data with it, so I can deploy my app to Heroku, blow away the instance and then launch it on fly.io, and it pops up with the exact same data.[2] I'm currently in the process of rewriting an open-source AppEngine app to use SQLite + Litestream instead of Google Firestore.[2] It's such a relief to get away from all the complexity of GCP and Firestore and get back to simple SQLite. [0] https://litestream.io/ https://litestream.io/ [1] https://mtlynch.io/litestream/ https://mtlynch.io/litestream/ [2] https://asciinema.org/a/I2HcYheYayeh7aHj23QSY9Vyf/embed?size=medium&speed=2.0&cols=80 https://asciinema.org/a/I2HcYheYayeh7aHj23QSY9Vyf/embed?size... [3] https://github.com/mtlynch/whatgotdone/pull/639 https://github.com/mtlynch/whatgotdone/pull/639
candiddevmike 5y ago
The state of Go PostgreSQL drivers is pretty bad IMO. Everyone recommends pgx, but it has a few rough edges (like how it handles dates/time) and a lot of surface area. The other driver, lib/pq, isn't really being maintained (for no good reason, it's quite good, simple and works well with sqlx).
- benhoyt 5y ago
  I haven't used pgx -- can you give some specifics of the issues with dates/times or other rough edges?
  - candiddevmike 5y ago
    The way it parses dates using Unix time causes issues when you compare the values returned due to Go setting the location property: https://github.com/jackc/pgx/issues/863 https://github.com/jackc/pgx/issues/863 pgx doesn't seem to handle reconnects the same way lib/pg does, but I haven't tested this recently: https://github.com/jackc/pgx/issues/672 https://github.com/jackc/pgx/issues/672
  - henvic 5y ago
    The rough edges I've faced are either edge case bugs or using more obscure APIs. Nothing too much worrisome.
- henvic 5y ago
  I don't know the state of drivers for other ecosystems, but even though pgx has a few rough edges, the quality of the driver is quite good and the maintainer (jackc) is also super responsive and helpful. I've had a positive experience with the project.
- Ingon 5y ago
  Our issue with lib/pq is that it doesn’t respect context deadlines/cancellation - e.g. a long running query will keep running till its done (in postgres), even when the context and all the callers have been long gone. We had quite a few issues with this and RDS eventually exhausting (burst) balances, driving the performance of the rest of the system to the ground.
armchairhacker 5y ago
> I’ve had a bad experience with ORM in the recent past, and I generally recommend against introducing this sort of abstraction to code. Yeah I don’t get ORMs. Most ORMs advertise “type safety” and not having to learn SQL. But actually you’re just writing SQL code in a non-SQL language. Usually the type-safety is weak, and a good IDE will provide type-checking and even data-source checking to raw SQL anyways. Converting complex object operations into relational operations is non-trivial. ORM objects auto-query and auto-update from the database, but these queries/updates are often unoptimal or happen at invalid times. It’s much easier to write and execute the SQL queries manually, and then auto-convert the results to/from regular objects.
- davidw 5y ago
  > Most ORMs advertise “type safety” and not having to learn SQL. But actually you’re just writing SQL code in a non-SQL language I think you're doing it wrong if you end up doing really complex queries in your ORM. It's kind of a judgement call, but at a certain point, you need to switch over to SQL. Good ORM's let you do that without too much hassle.
  - dboreham 5y ago
    ORM selling point: you can bypass it.
    - davidw 5y ago
      Well, sure. It makes it easy to do simple stuff and gets out of the way when you want to bypass it rather than being from the "sorry Dave, I can't let you do that" school of interface design.
  - piyh 5y ago
    I wrote a django app using postgres full text search. I ended up going with direct sql calls for anything more than a single record get because of all the specialized syntax. It's nice to keep stuff like new record creation clean, but once you get more than one join deep it's time to break out the best domain specific language ever created.
koeng 5y ago
Dockertest makes it really easy to test Go+Postgresql projects. I use it a lot. https://github.com/ory/dockertest https://github.com/ory/dockertest
- henvic 5y ago
  I dislike this because you're assuming I've Docker on my system and you also have to deal with the startup time of the container. I'd rather manage the database separately, and just have it configured for my tests to use them. This is why I kind of pushed towards a solution that didn't involve Docker. Someone might think: "oh, but what [web] developer doesn't have Docker installed?" Well, I dod, but I always have problems with it, so I barely use it. I also prefer to run my database on a homelab server (https://henvic.dev/posts/homelab/ https://henvic.dev/posts/homelab/) instead of on my laptop, especially because I also need OpenSearch or Elasticsearch running, and it's better if I just offload the performance|battery hit. Funny is that when I joined the company back in February I was asked what computer I'd like to receive, and I really wanted a 13" MacBook with the new M1 processor, so I asked for it, but in the end I decided to go with Intel as I imagined using Docker would be very important. It turns out that the M1 would work just as fine for the type of work I've been doing since then, with some minor exceptions.
  - gmhafiz 5y ago
    iirc dockertest does not need docker itself to be installed. Container start up time otoh is instant.
jrockway 5y ago
One should be careful about these mega-interfaces that define an entire subsystem of your application (DB in this case). They are very painful to maintain, and add almost no value. Interfaces should always be defined by the consumer of the interface, not by the producer. And interfaces should never be used until there is a clear need. CodeReviewComments covers the basics: https://github.com/golang/go/wiki/CodeReviewComments#interfaces https://github.com/golang/go/wiki/CodeReviewComments#interfa... I wrote an insanely long rant about this with thorough examples using the structure of the article, but it was too long for HN. So I put it on my blog instead: https://jrock.us/posts/go-interfaces/ https://jrock.us/posts/go-interfaces/
- henvic 5y ago
  Fair criticism. I'm the author, and I hate using interfaces like this, but didn't comment on it probably because I got used to it for this kind of thing. The only reason why I defined inventory.DB (https://github.com/henvic/pgxtutorial/blob/main/internal/inventory/service.go#L18-L50 https://github.com/henvic/pgxtutorial/blob/main/internal/inv...) is to be able to test. Otherwise, I'd have skipped it. I haven't read your article yet, but will do so later and get back to this subject and tell my opinion by the end of the day (on vacation, and need to hurry to catch the train!).
  - jrockway 5y ago
    Testing is good, but probably not a good enough reason to add unbounded ongoing maintenance costs. The problem is that putting everything in one interfaces forces you to implement the test version of DB every time you add real functionality to the real implementation. That gets pretty expensive in terms of time, and forces you to do work that's not necessary. (You could always test using the real implementation, and that's the approach I'd recommend starting with. When it gets too slow, look for an alternative.) My blog post covers what unit testing is like with your interface, with a consumer-defined interface, and just putting some test data fields in the DBImpl struct. The latter is my preferred approach. It took me a long time to get there. I was working at Google writing some filesystem access code and wanted to test it. I designed it as interfaces so I could have a real implementation and a test implementation, and a member of the Go team reviewed the code and told me it was absolutely inappropriate to use interfaces there. I disagreed at the time, but I've slowly come to terms with this style. Separating out production and testing is "clean", but that cleanliness comes at the cost of a mental abstraction burden (every time you read the code you're confused about where the meat lives) and unbounded future maintenance costs (every time you want to add a new feature, you have to edit two files). I don't think having a mock framework do codegen to make generic mocks is that useful. Just keep the exact behavior close to the test. A working test harness for a particular unit is something small enough to write out in an HN comment, and keeping the test code close to the test (and not wrapped up in some complicated framework) makes it much easier to enhance and debug the tests. (I've had Mockery forced upon me before, and I'm miserable every time I have to interact with it. Have to get the right version. Have to update an interface after writing the real implementation. Have to regenerate the mocks. Then I have to refer to the docs to do even the simplest tests, and those tests end up not doing what I want (instead producing something useless like "assert that function was called with the right args"). It's a chore that isn't worth the cost.) I think the underlying psychology behind the mega-interfaces is probably that we come up with an idea for what the interface should look like, and want to write it down somewhere. Then we can disengage the brain and start filling out the method bodies. That is not a good enough reason to add a layer of abstraction to the code; you can always type out 'func RealImplementation() { panic("write this") }' if that's the real reason. My view shifted when I was trying to use the OpenTelemetry library (circa v0.6.0) to emit Prometheus metrics. I wanted to set the histogram buckets on a per-metric basis. I chased 100 layers of indirection (every function took and returned an interface, so you'd need to have like 5 code windows open to debug the flow of data between the interface level and the implementation level), and never figured out how to do it. I think I determined it wasn't possible. It was at that moment that I decided "wait, I think everyone is using interfaces wrong", and then found CodeReviewComments telling me "yup, it's wrong".
steeleduncan 5y ago
The article mentions that lib/pq is "effectively in maintenance mode". This surprised me, I've never had problems with it, and looking at the commits on github [1] it seems far from inactive. [1] https://github.com/lib/pq/commits/master https://github.com/lib/pq/commits/master
- phnofive 5y ago
  So says the readme: https://github.com/lib/pq#status https://github.com/lib/pq#status I wonder why, though?
  - henvic 5y ago
    If you search the issues, you're going to find its maintainer saying he himself was using pgx now, as it was more capable.
  - steeleduncan 5y ago
    Thanks, I missed that. It is a shame, because although pgx is more capable, it doesn't seem to work with Go's database/sql system, which it harder to support multiple database backends (e.g. Sqlite for testing).

129 points by benhoyt 5y ago

Great article! These days, I find myself reaching for sqlc whenever I need to access Postgres from Go. My experience with the project has been excellent: I get type-safe APIs and control over exactly which queries are executed without much fuss. https://sqlc.dev/ https://sqlc.dev/

Ingon 5y ago
Yes! I’ve been using sqlc both at work and on side projects and it works awesome. For me it strikes the right balance between “I want to work with sql/db abstractions” and “I don’t need orm, just automated mapping”. Has a few rough edges when advanced types needs to get mapped, but it’s getting there.
- iknowstuff 5y ago
  >I don’t need orm, just automated mapping” Check out https://github.com/launchbadge/sqlx#sqlx-is-not-an-orm https://github.com/launchbadge/sqlx#sqlx-is-not-an-orm for an alternative approach. struct Country { country: String, count: i64 } let countries = sqlx::query_as!(Country, "SELECT country, count FROM countries WHERE organization = ?", organization) .fetch_all(&db_pool) .await?;
henvic 5y ago
Thanks! I love the concept of sqlc, but their support for pgx with its native PostgreSQL interface was quite limited when I checked. I imagine it's better nowadays, and will try again soon!
- lonk11 5y ago
  The support for native pgx in sqlc is good now. I migrated my project from database/sql to pgx recently and posted the instructions at the end of this post: https://linklonk.com/item/1948319982853095424 https://linklonk.com/item/1948319982853095424
- sa46 5y ago
  You might like pggen (I’m the author) which only supports Postgres and pgx. https://github.com/jschaf/pggen https://github.com/jschaf/pggen pggen occupies the same design space as sqlc but the implementations are quite different. Sqlc figures out the query types using type inference in Go which is nice because you don’t need Postgres at build time. Pggen asks Postgres what the query types are which is nice because it works with any extensions and arbitrarily complex queries.
  - henvic 5y ago
    Thank you. I'll take a look in the upcoming weeks. It seems promising!
sdevonoes 5y ago
Dumb question: how do you write dynamic queries using sqlc? From the doc I see: -- name: ListAuthors :many SELECT * FROM authors ORDER BY name; But what if I want to get a list of authors by date? Shall I do: -- name: ListAuthors :many SELECT * FROM authors ORDER BY name; -- name: ListAuthorsByDate :many ... And what if later I need to get a list of authors by date and name? Should I write yet another query? I'm used to write one generic (composable) query that is very handy in situations where one need to retrieve rows by many different filtering criteria. Is that possible in sqlc?
- sa46 5y ago
  Dynamic queries are difficult to model when starting from a static query since SQL doesn't compose well. The workarounds I've used: - Use multiple queries that share the same output type. - Push the predicate into the query directly. However, you can't pass an expression for an order by column since the order by clause takes a name, not an expression. Postgres doesn't allow using names as arguments to a prepared query so that leaves either adding an annotation like sqlc.order_by that's dynamically added to a query string, or by getting more creative with the structure of the query: SELECT * FROM AUTHORS WHERE sqlc.arg('by_date') ORDER BY DATE UNION ALL SELECT * FROM AUTHORS WHERE NOT sqlc.arg('by_date')

pstuart 5y ago

Postgres is awesome technology but for a lot of applications SQLite will do the job just fine. This gets you a single binary with zero external dependencies.

w-j-w 5y ago
I keep seeing this comment on HN, but I never really get it. In the world of platform as a service, launching a database instance is far easier than managing disk based storage yourself. It also has a very obvious route to horizontally scaling the web process. Deployment is easier too with an external database, there's no data on the disk of the machine running the web process that needs to be carried over.
- tekstar 5y ago
  > launching a database instance is far easier than managing disk based storage yourself Look, it comes down to what you, the developer are comfortable with. If you know and are comfortable with running something like postgres in the cloud, then go for it. Some of us prefer to not. Similarly, not all of us nuke the entire filesystem on every deploy :)
- mtlynch 5y ago
  I had the same objection to SQLite, and then I heard about Litestream, and it won me over.[0] Litestream watches your SQLite database and then streams changes to a cloud storage provider (e.g., S3, Backblaze). You get the performance and simplicity of writing SQLite to the local filesystem, but it's syncing to the cloud. And the cool part is that you don't have to change any of your application code to do it - as far as your app is concerned, it's writing to a local SQLite file. I wrote a little log uploading utility for my business that uses Litestream, and it's been fantastic.[1] It essentially carries around its data with it, so I can deploy my app to Heroku, blow away the instance and then launch it on fly.io, and it pops up with the exact same data.[2] I'm currently in the process of rewriting an open-source AppEngine app to use SQLite + Litestream instead of Google Firestore.[2] It's such a relief to get away from all the complexity of GCP and Firestore and get back to simple SQLite. [0] https://litestream.io/ https://litestream.io/ [1] https://mtlynch.io/litestream/ https://mtlynch.io/litestream/ [2] https://asciinema.org/a/I2HcYheYayeh7aHj23QSY9Vyf/embed?size=medium&speed=2.0&cols=80 https://asciinema.org/a/I2HcYheYayeh7aHj23QSY9Vyf/embed?size... [3] https://github.com/mtlynch/whatgotdone/pull/639 https://github.com/mtlynch/whatgotdone/pull/639

candiddevmike 5y ago

The state of Go PostgreSQL drivers is pretty bad IMO. Everyone recommends pgx, but it has a few rough edges (like how it handles dates/time) and a lot of surface area. The other driver, lib/pq, isn't really being maintained (for no good reason, it's quite good, simple and works well with sqlx).

benhoyt 5y ago
I haven't used pgx -- can you give some specifics of the issues with dates/times or other rough edges?
- candiddevmike 5y ago
  The way it parses dates using Unix time causes issues when you compare the values returned due to Go setting the location property: https://github.com/jackc/pgx/issues/863 https://github.com/jackc/pgx/issues/863 pgx doesn't seem to handle reconnects the same way lib/pg does, but I haven't tested this recently: https://github.com/jackc/pgx/issues/672 https://github.com/jackc/pgx/issues/672
- henvic 5y ago
  The rough edges I've faced are either edge case bugs or using more obscure APIs. Nothing too much worrisome.
henvic 5y ago
I don't know the state of drivers for other ecosystems, but even though pgx has a few rough edges, the quality of the driver is quite good and the maintainer (jackc) is also super responsive and helpful. I've had a positive experience with the project.
Ingon 5y ago
Our issue with lib/pq is that it doesn’t respect context deadlines/cancellation - e.g. a long running query will keep running till its done (in postgres), even when the context and all the callers have been long gone. We had quite a few issues with this and RDS eventually exhausting (burst) balances, driving the performance of the rest of the system to the ground.

armchairhacker 5y ago

> I’ve had a bad experience with ORM in the recent past, and I generally recommend against introducing this sort of abstraction to code. Yeah I don’t get ORMs. Most ORMs advertise “type safety” and not having to learn SQL. But actually you’re just writing SQL code in a non-SQL language. Usually the type-safety is weak, and a good IDE will provide type-checking and even data-source checking to raw SQL anyways. Converting complex object operations into relational operations is non-trivial. ORM objects auto-query and auto-update from the database, but these queries/updates are often unoptimal or happen at invalid times. It’s much easier to write and execute the SQL queries manually, and then auto-convert the results to/from regular objects.

davidw 5y ago
> Most ORMs advertise “type safety” and not having to learn SQL. But actually you’re just writing SQL code in a non-SQL language I think you're doing it wrong if you end up doing really complex queries in your ORM. It's kind of a judgement call, but at a certain point, you need to switch over to SQL. Good ORM's let you do that without too much hassle.
- dboreham 5y ago
  ORM selling point: you can bypass it.
  - davidw 5y ago
    Well, sure. It makes it easy to do simple stuff and gets out of the way when you want to bypass it rather than being from the "sorry Dave, I can't let you do that" school of interface design.
- piyh 5y ago
  I wrote a django app using postgres full text search. I ended up going with direct sql calls for anything more than a single record get because of all the specialized syntax. It's nice to keep stuff like new record creation clean, but once you get more than one join deep it's time to break out the best domain specific language ever created.

koeng 5y ago

Dockertest makes it really easy to test Go+Postgresql projects. I use it a lot. https://github.com/ory/dockertest https://github.com/ory/dockertest

henvic 5y ago
I dislike this because you're assuming I've Docker on my system and you also have to deal with the startup time of the container. I'd rather manage the database separately, and just have it configured for my tests to use them. This is why I kind of pushed towards a solution that didn't involve Docker. Someone might think: "oh, but what [web] developer doesn't have Docker installed?" Well, I dod, but I always have problems with it, so I barely use it. I also prefer to run my database on a homelab server (https://henvic.dev/posts/homelab/ https://henvic.dev/posts/homelab/) instead of on my laptop, especially because I also need OpenSearch or Elasticsearch running, and it's better if I just offload the performance|battery hit. Funny is that when I joined the company back in February I was asked what computer I'd like to receive, and I really wanted a 13" MacBook with the new M1 processor, so I asked for it, but in the end I decided to go with Intel as I imagined using Docker would be very important. It turns out that the M1 would work just as fine for the type of work I've been doing since then, with some minor exceptions.
- gmhafiz 5y ago
  iirc dockertest does not need docker itself to be installed. Container start up time otoh is instant.

jrockway 5y ago

One should be careful about these mega-interfaces that define an entire subsystem of your application (DB in this case). They are very painful to maintain, and add almost no value. Interfaces should always be defined by the consumer of the interface, not by the producer. And interfaces should never be used until there is a clear need. CodeReviewComments covers the basics: https://github.com/golang/go/wiki/CodeReviewComments#interfaces https://github.com/golang/go/wiki/CodeReviewComments#interfa... I wrote an insanely long rant about this with thorough examples using the structure of the article, but it was too long for HN. So I put it on my blog instead: https://jrock.us/posts/go-interfaces/ https://jrock.us/posts/go-interfaces/

henvic 5y ago
Fair criticism. I'm the author, and I hate using interfaces like this, but didn't comment on it probably because I got used to it for this kind of thing. The only reason why I defined inventory.DB (https://github.com/henvic/pgxtutorial/blob/main/internal/inventory/service.go#L18-L50 https://github.com/henvic/pgxtutorial/blob/main/internal/inv...) is to be able to test. Otherwise, I'd have skipped it. I haven't read your article yet, but will do so later and get back to this subject and tell my opinion by the end of the day (on vacation, and need to hurry to catch the train!).
- jrockway 5y ago
  Testing is good, but probably not a good enough reason to add unbounded ongoing maintenance costs. The problem is that putting everything in one interfaces forces you to implement the test version of DB every time you add real functionality to the real implementation. That gets pretty expensive in terms of time, and forces you to do work that's not necessary. (You could always test using the real implementation, and that's the approach I'd recommend starting with. When it gets too slow, look for an alternative.) My blog post covers what unit testing is like with your interface, with a consumer-defined interface, and just putting some test data fields in the DBImpl struct. The latter is my preferred approach. It took me a long time to get there. I was working at Google writing some filesystem access code and wanted to test it. I designed it as interfaces so I could have a real implementation and a test implementation, and a member of the Go team reviewed the code and told me it was absolutely inappropriate to use interfaces there. I disagreed at the time, but I've slowly come to terms with this style. Separating out production and testing is "clean", but that cleanliness comes at the cost of a mental abstraction burden (every time you read the code you're confused about where the meat lives) and unbounded future maintenance costs (every time you want to add a new feature, you have to edit two files). I don't think having a mock framework do codegen to make generic mocks is that useful. Just keep the exact behavior close to the test. A working test harness for a particular unit is something small enough to write out in an HN comment, and keeping the test code close to the test (and not wrapped up in some complicated framework) makes it much easier to enhance and debug the tests. (I've had Mockery forced upon me before, and I'm miserable every time I have to interact with it. Have to get the right version. Have to update an interface after writing the real implementation. Have to regenerate the mocks. Then I have to refer to the docs to do even the simplest tests, and those tests end up not doing what I want (instead producing something useless like "assert that function was called with the right args"). It's a chore that isn't worth the cost.) I think the underlying psychology behind the mega-interfaces is probably that we come up with an idea for what the interface should look like, and want to write it down somewhere. Then we can disengage the brain and start filling out the method bodies. That is not a good enough reason to add a layer of abstraction to the code; you can always type out 'func RealImplementation() { panic("write this") }' if that's the real reason. My view shifted when I was trying to use the OpenTelemetry library (circa v0.6.0) to emit Prometheus metrics. I wanted to set the histogram buckets on a per-metric basis. I chased 100 layers of indirection (every function took and returned an interface, so you'd need to have like 5 code windows open to debug the flow of data between the interface level and the implementation level), and never figured out how to do it. I think I determined it wasn't possible. It was at that moment that I decided "wait, I think everyone is using interfaces wrong", and then found CodeReviewComments telling me "yup, it's wrong".

steeleduncan 5y ago

The article mentions that lib/pq is "effectively in maintenance mode". This surprised me, I've never had problems with it, and looking at the commits on github [1] it seems far from inactive. [1] https://github.com/lib/pq/commits/master https://github.com/lib/pq/commits/master

phnofive 5y ago
So says the readme: https://github.com/lib/pq#status https://github.com/lib/pq#status I wonder why, though?
- henvic 5y ago
  If you search the issues, you're going to find its maintainer saying he himself was using pgx now, as it was more capable.
- steeleduncan 5y ago
  Thanks, I missed that. It is a shame, because although pgx is more capable, it doesn't seem to work with Go's database/sql system, which it harder to support multiple database backends (e.g. Sqlite for testing).