6 ms·

Quick list of Python libraries that help with application configuration: - Python application configuration -> https://github.com/edaniszewski/bison https://gi

Quick list of Python libraries that help with application configuration: - Python application configuration -> https://github.com/edaniszewski/bison https://github.com/edaniszewski/bison - Configuration with env variables for Python -> https://github.com/hynek/environ_config https://github.com/hynek/environ_config - Configuration library for python projects -> https://github.com/willkg/everett https://github.com/willkg/everett - Strict separation of config from code -> https://github.com/henriquebastos/python-decouple https://github.com/henriquebastos/python-decouple This is from my personal notes. See also: https://github.com/vinta/awesome-python#configuration https://github.com/vinta/awesome-python#configuration Anything else that we've missed ?

andyljones 6y ago
The two big ones in machine learning at least are Google's Gin: https://github.com/google/gin-config https://github.com/google/gin-config and Facebook's Hydra: https://hydra.cc/ https://hydra.cc/
- kyawzazaw 6y ago
  the name usage of "gin" is quite common, it seems: https://github.com/gin-gonic/gin https://github.com/gin-gonic/gin
ungawatkt 6y ago
Marshmallow. You can use its schema validation for any dict/json, which makes it a nice fit for validating json config files (which mitigates some of the json concerns from the article). Just immediately move the json.reads through a schema validate, build some classes around it for different config files. marshmallow.readthedocs.io/en/
- mianos 6y ago
  I use a similar library called Schema at https://pypi.org/project/schema/ https://pypi.org/project/schema/ I love the expressive nature of it. Just this week I was validating both yaml and json configuration data with it. One thing missing from this article is to always use a proper external key value store for everything but the stage (dev test prod etc) and the connections to the kv store. Config files on disk and even in environment variables suck for anything but the most trivial platforms.
- albi_lander 6y ago
  dataclass_json is also very useful for schema validation. It combines python's native dataclass objects with marshmallow's schema to provide additional functionalities simply through a @dataclass_json decorator on your dataclass. https://lidatong.github.io/dataclasses-json/ https://lidatong.github.io/dataclasses-json/
  - madman_bob 6y ago
    I made the similar dataclasses_serialization library. It doesn't require a special decorator on your classes, and is extensible for custom classes, and custom serialization methods (JSON and BSON provided by default). https://github.com/madman-bob/python-dataclasses-serialization https://github.com/madman-bob/python-dataclasses-serializati...
  - mands 6y ago
    I do something similar with dataclasses using the `dacite` (https://github.com/konradhalas/dacite https://github.com/konradhalas/dacite) library as a constructor for python dicts to dataclasses with runtime type-checking. Works really well with marshmallow for additional validation.
GlennS 6y ago
Pydantic https://pydantic-docs.helpmanual.io/usage/settings/ https://pydantic-docs.helpmanual.io/usage/settings/
- sandGorgon 6y ago
  Yes. Fastapi also uses the same - https://fastapi.tiangolo.com/advanced/settings/#pydantic-settings https://fastapi.tiangolo.com/advanced/settings/#pydantic-set...
BerislavLopac 6y ago
Here's my tiny contribution to the field: https://github.com/berislavlopac/figga https://github.com/berislavlopac/figga
- BerislavLopac 6y ago
  And I also like the .env approach: https://github.com/theskumar/python-dotenv https://github.com/theskumar/python-dotenv
pydry 6y ago
https://github.com/crdoconnor/strictyaml https://github.com/crdoconnor/strictyaml - typesafe YAML parser (solves the issues mentioned in 2 and 1).
lsorber 6y ago
Dynaconf [1], though I think this article's approach in combination with pydantic is better. [1] https://dynaconf.readthedocs.io/en/latest/ https://dynaconf.readthedocs.io/en/latest/
markus23 6y ago
https://www.libelektra.org/tutorials/python-bindings https://www.libelektra.org/tutorials/python-bindings
macintux 6y ago
Cerberus for schema validation. https://docs.python-cerberus.org/en/stable/ https://docs.python-cerberus.org/en/stable/
ak217 6y ago
https://github.com/kislyuk/tweak https://github.com/kislyuk/tweak
krut-patel 6y ago
There's also confuse[0], used by beets. I've found it to be a good way to handle configuration when the end-user is not a programmer. [0] - https://github.com/beetbox/confuse https://github.com/beetbox/confuse
rb808 6y ago
This and the list of other libraries in responses is a great reason to avoid Python. Its nearly as bad as Javascript, how are you supposed to live with this.
rochacbruno 6y ago
https://github.com/rochacbruno/dynaconf https://github.com/rochacbruno/dynaconf

andyljones 6y ago

The two big ones in machine learning at least are Google's Gin: https://github.com/google/gin-config https://github.com/google/gin-config and Facebook's Hydra: https://hydra.cc/ https://hydra.cc/

kyawzazaw 6y ago
the name usage of "gin" is quite common, it seems: https://github.com/gin-gonic/gin https://github.com/gin-gonic/gin

ungawatkt 6y ago

Marshmallow. You can use its schema validation for any dict/json, which makes it a nice fit for validating json config files (which mitigates some of the json concerns from the article). Just immediately move the json.reads through a schema validate, build some classes around it for different config files. marshmallow.readthedocs.io/en/

mianos 6y ago
I use a similar library called Schema at https://pypi.org/project/schema/ https://pypi.org/project/schema/ I love the expressive nature of it. Just this week I was validating both yaml and json configuration data with it. One thing missing from this article is to always use a proper external key value store for everything but the stage (dev test prod etc) and the connections to the kv store. Config files on disk and even in environment variables suck for anything but the most trivial platforms.
albi_lander 6y ago
dataclass_json is also very useful for schema validation. It combines python's native dataclass objects with marshmallow's schema to provide additional functionalities simply through a @dataclass_json decorator on your dataclass. https://lidatong.github.io/dataclasses-json/ https://lidatong.github.io/dataclasses-json/
- madman_bob 6y ago
  I made the similar dataclasses_serialization library. It doesn't require a special decorator on your classes, and is extensible for custom classes, and custom serialization methods (JSON and BSON provided by default). https://github.com/madman-bob/python-dataclasses-serialization https://github.com/madman-bob/python-dataclasses-serializati...
- mands 6y ago
  I do something similar with dataclasses using the `dacite` (https://github.com/konradhalas/dacite https://github.com/konradhalas/dacite) library as a constructor for python dicts to dataclasses with runtime type-checking. Works really well with marshmallow for additional validation.

GlennS 6y ago

Pydantic https://pydantic-docs.helpmanual.io/usage/settings/ https://pydantic-docs.helpmanual.io/usage/settings/

sandGorgon 6y ago
Yes. Fastapi also uses the same - https://fastapi.tiangolo.com/advanced/settings/#pydantic-settings https://fastapi.tiangolo.com/advanced/settings/#pydantic-set...

BerislavLopac 6y ago

Here's my tiny contribution to the field: https://github.com/berislavlopac/figga https://github.com/berislavlopac/figga

BerislavLopac 6y ago
And I also like the .env approach: https://github.com/theskumar/python-dotenv https://github.com/theskumar/python-dotenv

pydry 6y ago

https://github.com/crdoconnor/strictyaml https://github.com/crdoconnor/strictyaml - typesafe YAML parser (solves the issues mentioned in 2 and 1).

lsorber 6y ago

Dynaconf [1], though I think this article's approach in combination with pydantic is better. [1] https://dynaconf.readthedocs.io/en/latest/ https://dynaconf.readthedocs.io/en/latest/

markus23 6y ago

https://www.libelektra.org/tutorials/python-bindings https://www.libelektra.org/tutorials/python-bindings

macintux 6y ago

Cerberus for schema validation. https://docs.python-cerberus.org/en/stable/ https://docs.python-cerberus.org/en/stable/

ak217 6y ago

https://github.com/kislyuk/tweak https://github.com/kislyuk/tweak

krut-patel 6y ago

There's also confuse[0], used by beets. I've found it to be a good way to handle configuration when the end-user is not a programmer. [0] - https://github.com/beetbox/confuse https://github.com/beetbox/confuse

rb808 6y ago

This and the list of other libraries in responses is a great reason to avoid Python. Its nearly as bad as Javascript, how are you supposed to live with this.

rochacbruno 6y ago

https://github.com/rochacbruno/dynaconf https://github.com/rochacbruno/dynaconf