4 ms·

I have been working for years in the NLP and one of my main difficulties during all these years (I will not give any figures, some may think that I belong to a

I have been working for years in the NLP and one of my main difficulties during all these years (I will not give any figures, some may think that I belong to a species that has long since disappeared: -) ) was the preparation of the corpora. The corpora are big dirty vicious beasts full of mistakes and traps. The first problem I encountered was the mixing of encodings where a piece of UTF8 mixes with some LATIN leftovers. Tamgu provides quite sophisticated means to solve this problem. The second problem was the parsing of the files to extract the relevant information. That's when I made the decision to offer automatic conversion elements. On the other hand, it should be noted that Tamgu is a typed language and that the interpretation of an object is done in context. When I write: string s = 10; s += 20; What I define is an operation within a string. The context is defined by the recipient variable. In the same way: int i = 20; i += "30"; The variable "i" defines an integer context and therefore string 30 is considered in this case as a number.

zimbatm 7y ago
Automatic conversion hides type mismatches in all the program, even the parts that are not concerned with input validation and cleanup. So that makes static typing a bit less useful. As a source code reader I would much prefer to see at least a keyword or function call to make the casting explicit. string s = cast 10; s += cast 20; (I had the same reaction as the parent and think the language looks really cool otherwise)
- clauderoux 7y ago
  Thank you for your comments. Actually, it is possible in Tamgu to have an explicit cast: i = int(s)+10; Each type can be used as a "function" to create the object you like, which is basically what a cast is...
dkersten 7y ago
I get that you made it to scratch an itch you had and solve a problem you had with other languages and that's cool, but as an external person looking in, who doesn't have that exact itch, its not for me, despite the rest of the language really looking quite awesome, because for the kind of code I'm writing, this seems like it will cause a lot more frustrating bugs than convenience. I don't deal with a lot of ad-hoc data, but even when I do, that data usually needs strict validation, as it tends to not be very clean. Having things silently convert hides this and while it may make the code simpler, that code should have validation anyway, so it really won't be any more work in the long run (for me). Slight errors in data cause billion dollar bugs, after all. Maybe not in NLP though, so, again, we have different use cases. But that's just me and obviously my priorities differ from yours. In any case, congratulations on building this, that's quite an achievement.
- clauderoux 7y ago
  Thank you. Actually, we used this programming language and its ancestors (this is the fifth iteration since 2010) in many evaluation campaigns and it proved very useful. For instance, XML is often the way dataset are exposed and I have encapsulated libxml2 in order to read these files properly. I also provide a kind of BNF grammar compiler to read complex formats and split them into meaningful components. The conversion is really done at the very end of the process, which prevents most errors. Furthermore, Tamgu requires each variable to be explicitly typed in function declarations and for global and local variables, which prevents many of the problems that I used to have with Python. The fact that types are explicit transform many Python runtime errors into compile errors in Tamgu.
  - dkersten 7y ago
    > The fact that types are explicit transform many Python runtime errors into compile errors in Tamgu. Nice!
hoytech 7y ago
Any reason why you don't give addition and concatenation distinct operators? Perl does this and it's a lot cleaner IMO. Whenever you see a "+" you can consider it an explicit numeric conversion of both arguments: https://codespeaks.blogspot.com/2007/09/ruby-and-python-overload-operator-for.html https://codespeaks.blogspot.com/2007/09/ruby-and-python-over... In fact, there is no dedicated numeric conversion operator in perl. The canonical "operator" for this is called the "0+" operator. Python-style dynamic-typing and strictness is sort of the worst-of-both-worlds approach. Many times I've lost rarely-seen log messages to "cannot concatenate 'str' and 'int' objects" errors.
mamcx 7y ago
I also deal a lot with data transformations and also think auto-conversion is BAAAAAD...... BUT, is a signal that most languages (all?) are pretty bad at handle this concern. I'm building one too in Rust and there exist a very nice concept: https://doc.rust-lang.org/std/convert/trait.From.html https://doc.rust-lang.org/std/convert/trait.From.html Is the first time I see a blessed way into a lang to define conversions. For example, to parse strings into int you need to say (in pseudo-code): convert::From(String) -> Int //Or better convert::TryFrom and suddenly you get the way to convert to strings into ints EVERYWHERE. Is amazing. The key here is that is for much more than just basic stuff. You can do a lot of very convoluted things here, like Auto-Convert a vector of ints into a Zip compressed stream in one go. ---- The second things required is how avoid the boilerplate when dealing with data. If I read a file and need a lot of parsing and validation and need to be relaxed about this... how about the same thing between eager and lazy? - Have "strict" by default and need to manually do 1->Str + "hello" + today->Str + ... calls - Have a "auto-convert" when need to chain a lot of steps: auto { 1 + "hello" + today + ... } this is nice because you can see where the " big dirty vicious beasts full of mistakes and traps" are in your code!
- clauderoux 7y ago
  Tamgu offers something very similar to that. I borrowed a lot of elements from Haskell, including data management but also declarations. In Tamgu functional elements are enclosed in <..> In this example, I implement a join that takes a vector and transforms it into a string. //type declaration <joining:: vector -> string> //The function itself <joining(v) = x | x <- v> println(joining(['a'..'z']));

zimbatm 7y ago

Automatic conversion hides type mismatches in all the program, even the parts that are not concerned with input validation and cleanup. So that makes static typing a bit less useful. As a source code reader I would much prefer to see at least a keyword or function call to make the casting explicit. string s = cast 10; s += cast 20; (I had the same reaction as the parent and think the language looks really cool otherwise)

clauderoux 7y ago
Thank you for your comments. Actually, it is possible in Tamgu to have an explicit cast: i = int(s)+10; Each type can be used as a "function" to create the object you like, which is basically what a cast is...

dkersten 7y ago

I get that you made it to scratch an itch you had and solve a problem you had with other languages and that's cool, but as an external person looking in, who doesn't have that exact itch, its not for me, despite the rest of the language really looking quite awesome, because for the kind of code I'm writing, this seems like it will cause a lot more frustrating bugs than convenience. I don't deal with a lot of ad-hoc data, but even when I do, that data usually needs strict validation, as it tends to not be very clean. Having things silently convert hides this and while it may make the code simpler, that code should have validation anyway, so it really won't be any more work in the long run (for me). Slight errors in data cause billion dollar bugs, after all. Maybe not in NLP though, so, again, we have different use cases. But that's just me and obviously my priorities differ from yours. In any case, congratulations on building this, that's quite an achievement.

clauderoux 7y ago
Thank you. Actually, we used this programming language and its ancestors (this is the fifth iteration since 2010) in many evaluation campaigns and it proved very useful. For instance, XML is often the way dataset are exposed and I have encapsulated libxml2 in order to read these files properly. I also provide a kind of BNF grammar compiler to read complex formats and split them into meaningful components. The conversion is really done at the very end of the process, which prevents most errors. Furthermore, Tamgu requires each variable to be explicitly typed in function declarations and for global and local variables, which prevents many of the problems that I used to have with Python. The fact that types are explicit transform many Python runtime errors into compile errors in Tamgu.
- dkersten 7y ago
  > The fact that types are explicit transform many Python runtime errors into compile errors in Tamgu. Nice!

hoytech 7y ago

Any reason why you don't give addition and concatenation distinct operators? Perl does this and it's a lot cleaner IMO. Whenever you see a "+" you can consider it an explicit numeric conversion of both arguments: https://codespeaks.blogspot.com/2007/09/ruby-and-python-overload-operator-for.html https://codespeaks.blogspot.com/2007/09/ruby-and-python-over... In fact, there is no dedicated numeric conversion operator in perl. The canonical "operator" for this is called the "0+" operator. Python-style dynamic-typing and strictness is sort of the worst-of-both-worlds approach. Many times I've lost rarely-seen log messages to "cannot concatenate 'str' and 'int' objects" errors.

mamcx 7y ago

I also deal a lot with data transformations and also think auto-conversion is BAAAAAD...... BUT, is a signal that most languages (all?) are pretty bad at handle this concern. I'm building one too in Rust and there exist a very nice concept: https://doc.rust-lang.org/std/convert/trait.From.html https://doc.rust-lang.org/std/convert/trait.From.html Is the first time I see a blessed way into a lang to define conversions. For example, to parse strings into int you need to say (in pseudo-code): convert::From(String) -> Int //Or better convert::TryFrom and suddenly you get the way to convert to strings into ints EVERYWHERE. Is amazing. The key here is that is for much more than just basic stuff. You can do a lot of very convoluted things here, like Auto-Convert a vector of ints into a Zip compressed stream in one go. ---- The second things required is how avoid the boilerplate when dealing with data. If I read a file and need a lot of parsing and validation and need to be relaxed about this... how about the same thing between eager and lazy? - Have "strict" by default and need to manually do 1->Str + "hello" + today->Str + ... calls - Have a "auto-convert" when need to chain a lot of steps: auto { 1 + "hello" + today + ... } this is nice because you can see where the " big dirty vicious beasts full of mistakes and traps" are in your code!

clauderoux 7y ago
Tamgu offers something very similar to that. I borrowed a lot of elements from Haskell, including data management but also declarations. In Tamgu functional elements are enclosed in <..> In this example, I implement a join that takes a vector and transforms it into a string. //type declaration <joining:: vector -> string> //The function itself <joining(v) = x | x <- v> println(joining(['a'..'z']));