3 ms·

Thanks for your answers. By physical unit, I meant metres, seconds, nanoseconds, kilograms. Since I'm more in the physical sciences, units are important to me.

Thanks for your answers. By physical unit, I meant metres, seconds, nanoseconds, kilograms. Since I'm more in the physical sciences, units are important to me. Keeping track of physical units is a kind of provenance. I see now though that WSL is more like a textual relational database rather than improved CSV. I still stand by my NULL comment though, I totally understand why you want things to be as you've described (along with the emphasis on having not too many columns per table) but the problem will be other people and how they will inevitably use the format. Null is always problematic though and a source of arguments and bugs. Have you ever seen recutils? It's a similar concept although I have no experience of it, at a glance I prefer WSL.

jstimpfle 10y ago
I would say it's a relational database more than only an improved CSV: you can absolutely use it as that. While it was meant to model whole databases, I don't think there are any disadvantages if you only have one table. You can even easily make the library parse lines without the table prefix (which is unnecessary if there is only one table). However that use case is more trivial and might not justify depending on a library. Physical units are absolutely in scope. In fact thanks, I hadn't thought of that, if we can find a reasonable default implementation and syntax I might add them to the built-ins. (But you can also add them yourself as a user of the python API). Depending on your taste, you could let them have a unit suffix. % DOMAIN kilograms Kilograms precision=3 suffix % DOMAIN seconds Seconds precision=3 suffix % DOMAIN metres Metres precision=2 suffix % TABLE Example kilograms seconds meters Example 1.0kg 5.042s 4.42m Or % DOMAIN laptime Time infixunits % TABLE Laptime laptime Laptime 4h05m03s > Have you ever seen recutils? It's a similar concept although I have no experience of it, at a glance I prefer WSL. Yes, in fact I gave it a closer look. The scope is different; as the name says it's more record or even hierarchy oriented: - Records are written on paragraphs instead of lines; each member on its own line - Multi-valued members - Field names are explicit in each record. It doesn't really try to be a clean interpretation of the relational model. While it is also a query language and there are some sort of joins, the multi-values member notation combined with joins seems to quickly lead to cases where it's not clear how to interpret the resulting data. Also, due to the more complex multi-line data format, it's much better suited for consumption by humans than machines. WSL instead opts for unnamed records on single lines to be very easily consumable by machines as well, and is designed to enable very easy to use reader and writer APIs. Note that the python library is rather slow; on my old machine, the 220KB example database needs about 200ms to parse. With a C implementation, I reccon there could be a 10x speedup. However if you care very much about speed and do not always read in the data completely, sqlite3 or one of the big iron databases are a better fit anyway. WSL trades speed for semantics and plain text representation.

jstimpfle 10y ago

I would say it's a relational database more than only an improved CSV: you can absolutely use it as that. While it was meant to model whole databases, I don't think there are any disadvantages if you only have one table. You can even easily make the library parse lines without the table prefix (which is unnecessary if there is only one table). However that use case is more trivial and might not justify depending on a library. Physical units are absolutely in scope. In fact thanks, I hadn't thought of that, if we can find a reasonable default implementation and syntax I might add them to the built-ins. (But you can also add them yourself as a user of the python API). Depending on your taste, you could let them have a unit suffix. % DOMAIN kilograms Kilograms precision=3 suffix % DOMAIN seconds Seconds precision=3 suffix % DOMAIN metres Metres precision=2 suffix % TABLE Example kilograms seconds meters Example 1.0kg 5.042s 4.42m Or % DOMAIN laptime Time infixunits % TABLE Laptime laptime Laptime 4h05m03s > Have you ever seen recutils? It's a similar concept although I have no experience of it, at a glance I prefer WSL. Yes, in fact I gave it a closer look. The scope is different; as the name says it's more record or even hierarchy oriented: - Records are written on paragraphs instead of lines; each member on its own line - Multi-valued members - Field names are explicit in each record. It doesn't really try to be a clean interpretation of the relational model. While it is also a query language and there are some sort of joins, the multi-values member notation combined with joins seems to quickly lead to cases where it's not clear how to interpret the resulting data. Also, due to the more complex multi-line data format, it's much better suited for consumption by humans than machines. WSL instead opts for unnamed records on single lines to be very easily consumable by machines as well, and is designed to enable very easy to use reader and writer APIs. Note that the python library is rather slow; on my old machine, the 220KB example database needs about 200ms to parse. With a C implementation, I reccon there could be a 10x speedup. However if you care very much about speed and do not always read in the data completely, sqlite3 or one of the big iron databases are a better fit anyway. WSL trades speed for semantics and plain text representation.