4 ms·

I'd rather like to have a service that allows me to construct my own dictionaries. There have got to be some standard OpenSource/GNU-like tools that give a bud

by mmjaa 9y ago

I'd rather like to have a service that allows me to construct my own dictionaries. There have got to be some standard OpenSource/GNU-like tools that give a budding lexicographer the things they need to construct dictionaries - does anyone know what these tools are, and how effective they are at creating custom dictionaries?

deleted 9y ago
[deleted]
Nition 9y ago
One thing I always wanted was a paper dictionary that excluded the really common words. It'd be a lot faster to look up thaumaturge if I didn't have to sift through the likes of that and the. Take say the 5000 most common words in English and leave them out. Sure on rare occasions you'd look up something that was left out, but that already happens in the other direction (towards rare words) because that and the are taking up space where more difficult words could have been included. Smaller dictionaries often don't have the word I wanted, but do have a ton of words I'd never need to look up.
- my_first_acct 9y ago
  According to someone on Stack Exchange, quoting the NY Times, quoting the Chief Editor of the OED [1], 'there are for the verb-form alone of “run” no fewer than 645 meanings'. Other common words with huge numbers of meanings include "put" and "set". [1] https://english.stackexchange.com/questions/42480/words-with-most-meanings https://english.stackexchange.com/questions/42480/words-with...
  - Nition 9y ago
    And I'm sure some of those are obscure meanings, but the likelihood is I'm never going to look them up. Exclude those many definitions of "run" and either make the dictionary a bit smaller (easier to search) or include several rare word definitions in the same space.
    - dragonwriter 9y ago
      Removing a handful of thousands of common words won't make the OED noticeably shorter, and it's not space constrained anyway. And I don't think a rare use of a common word is any less likely to be looked up than a rare word.
      - Nition 9y ago
        I don't mean the full OED. Personally I have a Pocket Oxford and a Concise Oxford. Both are space constrained. The Pocket Oxford in particular has around 60,000 words so leaving out the most common 5-10K would make a noticeable difference. Since it's the complex words that get left out in small dictionaries, as you decrease dictionary size, the size saving that you'd get from removing common words proportionally increases. But you might be right that it wouldn't make enough of a difference, since small dictionaries can also fit less words in total. It might never be enough to matter much.
- haZard_OS 9y ago
  I know this isn't exactly what you meant but I have found it surprisingly useful (and fun!): https://www.amazon.com/Thinkers-Thesaurus-Sophisticated-Alternatives-Expanded/dp/0393351254 https://www.amazon.com/Thinkers-Thesaurus-Sophisticated-Alte...
- mmjaa 9y ago
  I want exactly the opposite - to be able to construct dictionaries where there are no words used in definitions that don't also have their own entries. To me this is a key feature of a dictionary - definition completeness. And its one of the reasons I'd like to know about the toolset used to construct dictionaries - I'd give anything to be able to throw a word set at these tools, have a custom dictionary constructed from it, and repeat until unity...
  - Nition 9y ago
    I actually assumed that was already a general rule, because any word I've ever cross referenced in a dictionary has always been there. In fact, in the diminutive "Little" Oxford[1] there are few enough words available for definitions that you can get stuck in a loop between two words! Although there's always at least one other synonym word in the definitions. [1] https://www.amazon.com/Little-Oxford-English-Dictionary/dp/B00A2KP7GA https://www.amazon.com/Little-Oxford-English-Dictionary/dp/B...
    - mmjaa 9y ago
      I'd really love to know the technology behind lexicography, as a state of the art. If anyone works in this field, any pointers you might provide are highly appreciated. I understand that unity is a difficult thing to attain in dictionaries, but perhaps there are tools and methods used by lexicographers that I need to learn about, as an interested party.
shullbitt0r 9y ago
Your Brain? I know it deteriorates if not used frequently and especially if you can rely on look up tables, but really, there is no replacement. Only supplements. Collections of cards in shoe cartons are used by libraries. What else do you need? A hypergraph of word-vertexes and relation-edges animated in webgl, layered by categories and streamed from an elastic back end? That's your brain.
WalterGR 9y ago
I'd rather like to have a service that allows me to construct my own dictionaries. In printed form, or is web-based okay?
- mmjaa 9y ago
  I'd really prefer something not web-based, but if you know of any tool like this, I'm all ears anyway. What I'd really want is to be able to construct lexicographic sets and save the in SQLite databases, for use in apps of course - but that may be a tall order. You know of tools like this?
  - WalterGR 9y ago
    You know of tools like this? I don't. I do have a background in dictionaries, though: I created The Online Slang Dictionary (http://onlineslangdictionary.com/ http://onlineslangdictionary.com/) which is the eldest slang dictionary and thesaurus on the web. I've given some minutes of thought to turning the underlying code into a product... What sort of data would you like to capture in the DB, and what would apps do with it?
    - mmjaa 9y ago
      I would like to have a kind of API that allows me to add a new word entry, add its definitions, describe the derivations (if any) of the word, and have a field for examples of the words usage. Some sort of system that would allow me to take a corpus, deconstruct the terms and words used, create a definition for each word, and so on. The reason is that I'm fascinated with dictionaries - the physical kind - in general, and would like to incorporate some sort of dictionary/glossary system in most of my apps as a means of documenting the technical terms for the subject those apps address. Another thing is that I'm a complete newbie to the subject of lexicography, and I'd love to learn more from like-minded individuals, on what kind of tooling and methodology is out there. I suppose I could go and look at the sources for such things as the OpenDict, sdcv (https://wiki.archlinux.org/index.php/Sdcv https://wiki.archlinux.org/index.php/Sdcv) and so on .. but surely there must be standards for the ways Websters and so on construct their lexicography. I'm just not familiar with these tools, so I'd hoped to have some insight from other like-minded individuals in this community, before I launch into a google-trail of my own investigation. With your site (wonderful, by the way) - did you invent your own schema for the database, or are you using some standard tooling on the backend? My ideal tooling would give me the ability to pipe a text file into it, construct a list of all un-defined terms/words/symbols, give the means of importing definitions from known sources for each word, and so on. Ultimately I'd love to be able to construct custom dictionaries for any given corpus, such that no term in that corpus is undefined; and to be even more meta about it, no term in the dictionary itself would be undefined. Obviously this is a highly iterative thing, as the undefined terms list grows with every definition added - which is why I think I have to understand the tooling better before I start constructing my own processes for this task.

by mmjaa 9y ago

deleted 9y ago

[deleted]

Nition 9y ago

One thing I always wanted was a paper dictionary that excluded the really common words. It'd be a lot faster to look up thaumaturge if I didn't have to sift through the likes of that and the. Take say the 5000 most common words in English and leave them out. Sure on rare occasions you'd look up something that was left out, but that already happens in the other direction (towards rare words) because that and the are taking up space where more difficult words could have been included. Smaller dictionaries often don't have the word I wanted, but do have a ton of words I'd never need to look up.

my_first_acct 9y ago
According to someone on Stack Exchange, quoting the NY Times, quoting the Chief Editor of the OED [1], 'there are for the verb-form alone of “run” no fewer than 645 meanings'. Other common words with huge numbers of meanings include "put" and "set". [1] https://english.stackexchange.com/questions/42480/words-with-most-meanings https://english.stackexchange.com/questions/42480/words-with...
- Nition 9y ago
  And I'm sure some of those are obscure meanings, but the likelihood is I'm never going to look them up. Exclude those many definitions of "run" and either make the dictionary a bit smaller (easier to search) or include several rare word definitions in the same space.
  - dragonwriter 9y ago
    Removing a handful of thousands of common words won't make the OED noticeably shorter, and it's not space constrained anyway. And I don't think a rare use of a common word is any less likely to be looked up than a rare word.
    - Nition 9y ago
      I don't mean the full OED. Personally I have a Pocket Oxford and a Concise Oxford. Both are space constrained. The Pocket Oxford in particular has around 60,000 words so leaving out the most common 5-10K would make a noticeable difference. Since it's the complex words that get left out in small dictionaries, as you decrease dictionary size, the size saving that you'd get from removing common words proportionally increases. But you might be right that it wouldn't make enough of a difference, since small dictionaries can also fit less words in total. It might never be enough to matter much.
haZard_OS 9y ago
I know this isn't exactly what you meant but I have found it surprisingly useful (and fun!): https://www.amazon.com/Thinkers-Thesaurus-Sophisticated-Alternatives-Expanded/dp/0393351254 https://www.amazon.com/Thinkers-Thesaurus-Sophisticated-Alte...
mmjaa 9y ago
I want exactly the opposite - to be able to construct dictionaries where there are no words used in definitions that don't also have their own entries. To me this is a key feature of a dictionary - definition completeness. And its one of the reasons I'd like to know about the toolset used to construct dictionaries - I'd give anything to be able to throw a word set at these tools, have a custom dictionary constructed from it, and repeat until unity...
- Nition 9y ago
  I actually assumed that was already a general rule, because any word I've ever cross referenced in a dictionary has always been there. In fact, in the diminutive "Little" Oxford[1] there are few enough words available for definitions that you can get stuck in a loop between two words! Although there's always at least one other synonym word in the definitions. [1] https://www.amazon.com/Little-Oxford-English-Dictionary/dp/B00A2KP7GA https://www.amazon.com/Little-Oxford-English-Dictionary/dp/B...
  - mmjaa 9y ago
    I'd really love to know the technology behind lexicography, as a state of the art. If anyone works in this field, any pointers you might provide are highly appreciated. I understand that unity is a difficult thing to attain in dictionaries, but perhaps there are tools and methods used by lexicographers that I need to learn about, as an interested party.

shullbitt0r 9y ago

Your Brain? I know it deteriorates if not used frequently and especially if you can rely on look up tables, but really, there is no replacement. Only supplements. Collections of cards in shoe cartons are used by libraries. What else do you need? A hypergraph of word-vertexes and relation-edges animated in webgl, layered by categories and streamed from an elastic back end? That's your brain.

WalterGR 9y ago

I'd rather like to have a service that allows me to construct my own dictionaries. In printed form, or is web-based okay?

mmjaa 9y ago
I'd really prefer something not web-based, but if you know of any tool like this, I'm all ears anyway. What I'd really want is to be able to construct lexicographic sets and save the in SQLite databases, for use in apps of course - but that may be a tall order. You know of tools like this?
- WalterGR 9y ago
  You know of tools like this? I don't. I do have a background in dictionaries, though: I created The Online Slang Dictionary (http://onlineslangdictionary.com/ http://onlineslangdictionary.com/) which is the eldest slang dictionary and thesaurus on the web. I've given some minutes of thought to turning the underlying code into a product... What sort of data would you like to capture in the DB, and what would apps do with it?
  - mmjaa 9y ago
    I would like to have a kind of API that allows me to add a new word entry, add its definitions, describe the derivations (if any) of the word, and have a field for examples of the words usage. Some sort of system that would allow me to take a corpus, deconstruct the terms and words used, create a definition for each word, and so on. The reason is that I'm fascinated with dictionaries - the physical kind - in general, and would like to incorporate some sort of dictionary/glossary system in most of my apps as a means of documenting the technical terms for the subject those apps address. Another thing is that I'm a complete newbie to the subject of lexicography, and I'd love to learn more from like-minded individuals, on what kind of tooling and methodology is out there. I suppose I could go and look at the sources for such things as the OpenDict, sdcv (https://wiki.archlinux.org/index.php/Sdcv https://wiki.archlinux.org/index.php/Sdcv) and so on .. but surely there must be standards for the ways Websters and so on construct their lexicography. I'm just not familiar with these tools, so I'd hoped to have some insight from other like-minded individuals in this community, before I launch into a google-trail of my own investigation. With your site (wonderful, by the way) - did you invent your own schema for the database, or are you using some standard tooling on the backend? My ideal tooling would give me the ability to pipe a text file into it, construct a list of all un-defined terms/words/symbols, give the means of importing definitions from known sources for each word, and so on. Ultimately I'd love to be able to construct custom dictionaries for any given corpus, such that no term in that corpus is undefined; and to be even more meta about it, no term in the dictionary itself would be undefined. Obviously this is a highly iterative thing, as the undefined terms list grows with every definition added - which is why I think I have to understand the tooling better before I start constructing my own processes for this task.