5 ms·

API-diff: a tool for side-by-side evaluations of JSON APIs

107 points by blackmad 6y ago

sdfhbdf 6y ago
If anybody is thinking about testing APIs with this tool you might be interested in a concept called "Contract Testing" [0] or an implementation "Pact Contract Testing" [1]. [0]: https://martinfowler.com/bliki/ContractTest.html https://martinfowler.com/bliki/ContractTest.html [1]: https://docs.pact.io https://docs.pact.io
the_arun 6y ago
It will be very cool to show diff in the API contract with every new version or release. Visually implementing - “what is changed”
michaelbuckbee 6y ago
This is great. I work with a number of API providers who are sometimes less than consistent with their output.
arxpoetica 6y ago
Maybe a naive question, but how is this any different than diff tools found in Git / SVN / etc.?
- templaedhel 6y ago
  I’ve not used this tool before (am excited to in the future though because..) having done a lot of API diffing with text diff tools in the past, they break down very quickly on things like: JSON key order can be random, causing false positives. Some headers will always be different (any sort of timestamp based header for example) while others must not be different. URL query params can be in any order, etc. I think if you do api diffing at any reasonably large scale you’ll find yourself immediately building tools to help cut the signal from the noise, and this is an exciting step in that direction.
drewda 6y ago
Nice work by @blackmad! For those wanting to start in the "shallow end of the pool" tuning Pelias geocoder indexes and config, there's also a web-based UI that can hit multiple instances and visualize the various results: https://pelias.github.io/compare/ https://pelias.github.io/compare/ Made by the good people at https://geocode.earth/ https://geocode.earth/
- blackmad 6y ago
  Shhhh don't tell anyone I also completely rebuilt that tool from the ground up earlier this year ;-) (it looks roughly the same as it did before, but has a million tweaks to make the development lifecycle more pleasant)

107 points by blackmad 6y ago

If anybody is thinking about testing APIs with this tool you might be interested in a concept called "Contract Testing" [0] or an implementation "Pact Contract Testing" [1]. [0]: https://martinfowler.com/bliki/ContractTest.html https://martinfowler.com/bliki/ContractTest.html [1]: https://docs.pact.io https://docs.pact.io

the_arun 6y ago

It will be very cool to show diff in the API contract with every new version or release. Visually implementing - “what is changed”

michaelbuckbee 6y ago

This is great. I work with a number of API providers who are sometimes less than consistent with their output.

arxpoetica 6y ago

Maybe a naive question, but how is this any different than diff tools found in Git / SVN / etc.?

templaedhel 6y ago
I’ve not used this tool before (am excited to in the future though because..) having done a lot of API diffing with text diff tools in the past, they break down very quickly on things like: JSON key order can be random, causing false positives. Some headers will always be different (any sort of timestamp based header for example) while others must not be different. URL query params can be in any order, etc. I think if you do api diffing at any reasonably large scale you’ll find yourself immediately building tools to help cut the signal from the noise, and this is an exciting step in that direction.

drewda 6y ago

Nice work by @blackmad! For those wanting to start in the "shallow end of the pool" tuning Pelias geocoder indexes and config, there's also a web-based UI that can hit multiple instances and visualize the various results: https://pelias.github.io/compare/ https://pelias.github.io/compare/ Made by the good people at https://geocode.earth/ https://geocode.earth/

blackmad 6y ago
Shhhh don't tell anyone I also completely rebuilt that tool from the ground up earlier this year ;-) (it looks roughly the same as it did before, but has a million tweaks to make the development lifecycle more pleasant)