10 ms·

HTTP: , FTP:, and Dict:?

425 points by edent 2y ago

masswerk 2y ago
Nowadays, on macOS, "dict://Internet" will open the Dictionary app with the query "Internet". (Probably behind a security prompt.) Not sure, if there's similar functionality on other operating systems.
- dmd 2y ago
  What do you mean by "behind a security prompt"?
  - promiseofbeans 2y ago
    I tried it and firefox came up with a prompt confirming I wanted to open 'internet' in 'Dictionary'
    - WarOnPrivacy 2y ago
      > firefox came up with a prompt confirming I wanted to open 'internet' in 'Dictionary' My Ffx 131.0b9 wasn't so adept. It gave me: The address wasn’t understood Firefox doesn’t know how to open this address, because one of the following protocols (dict) isn’t associated with any program or is not allowed in this context. You might need to install other software to open this address.
      - wormius 2y ago
        Mine just opened the default search engine with the results, maybe it's how I have the address bar configured?
        masswerk 2y ago
        The behavior may be also vary depending on whether it's an actual link a document or direct input to the location bar. (I get different prompts on Firefox, while both will forward to the built-in system dictionary.) And, of course, any further configurations may alter this, e.g., another service may have been registered for this protocol.
      - thayne 2y ago
        Perhaps it only does that on Macs?
  - bobbylarrybobby 2y ago
    Browsers generally throw up a prompt before opening an external app
bubblesnort 2y ago
Never heard of the dict command? The author went through the trouble of figuring out the protocol but never bothered to just run dict. Okay.
- jmillikin 2y ago
  What do you think the post would have contained if he had run dict? Here's a hint: macbook% dict zsh: command not found: dict desktop$ dict bash: dict: command not found You'd have to be pretty into retro-computing before you'll find an OS that ships /usr/bin/dict .
  - politelemon 2y ago
    Not really... FWIW, addressing the 'hint' you're giving specifically, this is on Ubuntu. $ dict Command 'dict' not found, but can be installed with: sudo apt install dict After installing, $ dict example 6 definitions found ...
    - jmillikin 2y ago
      So the causal chain would be: 1. Notice a URL scheme dict:// 2. Try to type 'dict' into a terminal, on the off chance there's a command-line tool with the same name (would you do this for https:// https:// and expect the same outcome?) 3. Be running a distribution that modifies the user's shell environment to suggest packages related to unknown commands 4. Actually install and run that command 5. Be running tcpdump or wireshark at the same time to notice that the `dict` command is reaching out to the network, as opposed to doing some sort of local lookup in /usr/share/dict 6. Figure out from the network traffic that the tool is using a dictionary-specific protocol as opposed to just making an HTTP request to dictionary.com or whatever. -- Nah, the only way someone would know (or even suspect!) that dict:// is somehow related to an ancient Unix command-line tool is prior knowledge, and it's unreasonable to expect the article author to have somehow intuited such an idea.
      - deleted 2y ago
        [deleted]
hkt 2y ago
I've been aware of dict for a while since I wrapped up an esperanto to english dictionary for KOReader in a format KOReader could understand. What I'd really have liked is a format like this: dict://<server/<origin language>/<definition language>/<word> Still, it is pretty cool that dict servers exist at all, so no complaints here.
im3w1l 2y ago
I admire these old protocols that are intentionally built to be usable both by machines and humans. Like the combination of a response code, and a human readable explanation. A help command right in the protocol. Makes me think it's a shame that making a json based protocol is so much easier to whip up than a textual ebnf-specified protocol. Like imagine if in python there was some library that you gave an ebnf spec maybe with some extra features similar to regex (named groups?) and you could compile it to a state machine and use it to parse documents, getting a Dict out.
- orf 2y ago
  Unfortunately, in practice they are a nightmare. Look at the WHOIS protocol for an example. Humans don’t look at responses very much, so you should optimise for machines. If you want a human-readable view, then turn the JSON response into something readable.
  - poincaredisk 2y ago
    Whois protocol has no grammar to speak of (the response body is not defined at all, just a text blob) which makes it a nightmare to parse. Having a proper response format would solve this. Though I agree, I prefer my responses in JSON.
    - tephra 2y ago
      If you want WHOIS with a better defined format we have a protocol for that! It's called RDAP https://en.wikipedia.org/wiki/Registration_Data_Access_Protocol https://en.wikipedia.org/wiki/Registration_Data_Access_Proto...
  - astrobe_ 2y ago
    The next logical step is to use a machine-friendly format instead; that is a binary protocol. Even HTML and XML which were designed for readability and manual writing eventually became 'not usable enough" ("became" because I think part of it is that their success made them exposed to less technical populations), and now we have markdown everywhere which most of the times is converted to HTML. So if you are going to use a tool more sophisticated than Ed/Edlin to read and write (rich) text in a certain format, it could be more efficient to focus on making the job of the machine - and of the programmer, easier. If you look at a binary protocol such as NTP, the binary format leaves very little room for Postel's principle [1], so it is straightforward to make a program that queries a server and display the result. [1] https://en.wikipedia.org/wiki/Robustness_principle https://en.wikipedia.org/wiki/Robustness_principle
mogoh 2y ago
hmmm $>curl dict://dict.org/d:Internet curl: (1) Protocol "dict" not supported
- fallingsquirrel 2y ago
  Works for me. I bet your OS ships a crippled version of curl. $ curl --version curl 8.7.1 (x86_64-pc-linux-gnu) [...] $ curl dict://dict.org/d:Internet 220 dict.dict.org dictd 1.12.1/rf on Linux 4.19.0-10-amd64 <auth.mime> <370202891.28105.1727009645@dict.dict.org> 250 ok 150 1 definitions retrieved [...]
  - soraminazuki 2y ago
    Manual build with an explicit `--disable-dict` perhaps? Because it's not Debian, Fedora, Homebrew, Nix, Alpine, Arch, or Gentoo, judging by their package definitions.
    - mogoh 2y ago
      I am on Fedora Silverblue $>curl --version curl 8.6.0 (x86_64-redhat-linux-gnu) libcurl/8.6.0 OpenSSL/3.2.2 zlib/1.3.1.zlib-ng libidn2/2.3.7 nghttp2/1.59.0 Release-Date: 2024-01-31 Protocols: file ftp ftps http https ipfs ipns Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz SPNEGO SSL threadsafe UnixSockets I am not sure I I understand you correctly. Should it work on Fedora?
      - SushiHippie 2y ago
        The description on this page at least lists the dict protocol https://src.fedoraproject.org/rpms/curl/ https://src.fedoraproject.org/rpms/curl/ Only the minimal build disables the dict protocol, maybe you have installed the curl-minimal package? https://src.fedoraproject.org/rpms/curl/blob/rawhide/f/curl.spec https://src.fedoraproject.org/rpms/curl/blob/rawhide/f/curl....
        2y ago
kragen 2y ago
dict and the relevant dictionaries are things i pretty much always install on every new laptop. gcide in particular includes most of the famous 1913 webster dictionary with its sparkling prose: : ~; dict glisten 2 definitions found From The Collaborative International Dictionary of English v.0.48 [gcide]: Glisten \Glis"ten\ (gl[i^]s"'n), v. i. [imp. & p. p. {Glistened}; p. pr. & vb. n. {Glistening}.] [OE. glistnian, akin to glisnen, glisien, AS. glisian, glisnian, akin to E. glitter. See {Glitter}, v. i., and cf. {Glister}, v. i.] To sparkle or shine; especially, to shine with a mild, subdued, and fitful luster; to emit a soft, scintillating light; to gleam; as, the glistening stars. Syn: See {Flash}. [1913 Webster] it's interesting to think about how you would implement this service efficiently under the constraints of mid-01990s computers, where a gigabyte was still a lot of disk space and multiuser unix servers commonly had about 100 mips (https://netlib.org/performance/html/dhrystone.data.col0.html https://netlib.org/performance/html/dhrystone.data.col0.html) totally by coincidence i was looking at the dictzip man page this morning; it produces gzip-compatible files that support random seeks so you can keep the database for your dictd server compressed. (as far as i know, rik faith's dictd is still the only server implementation of the dict protocol, which is incidentally not a very good protocol.) you can see that the penalty for seekability is about 6% in this case: : ~; ls -l /usr/share/dictd/jargon.dict.dz -rw-r--r-- 1 root root 587377 Jan 1 2021 /usr/share/dictd/jargon.dict.dz : ~; \time gzip -dc /usr/share/dictd/jargon.dict.dz|wc -c 0.01user 0.00system 0:00.01elapsed 100%CPU (0avgtext+0avgdata 1624maxresident)k 0inputs+0outputs (0major+160minor)pagefaults 0swaps 1418350 : ~; gzip -dc /usr/share/dictd/jargon.dict.dz|gzip -9c|wc -c 556102 : ~; units -t 587377/556102 % 105.62397 nowadays computers are fast enough that it probably isn't a big win to gzip in such small chunks (dictzip has a chunk limit of 64k) and you might as well use a zipfile, all implementations of which support random access: : ~; mkdir jargsplit : ~; cd jargsplit : jargsplit; gzip -dc /usr/share/dictd/jargon.dict.dz|split -b256K : jargsplit; zip jargon.zip xaa xab xac xad xae xaf adding: xaa (deflated 60%) adding: xab (deflated 59%) adding: xac (deflated 59%) adding: xad (deflated 61%) adding: xae (deflated 62%) adding: xaf (deflated 58%) : jargsplit; ls -l jargon.zip -rw-r--r-- 1 user user 565968 Sep 22 09:47 jargon.zip : jargsplit; time unzip -o jargon.zip xad Archive: jargon.zip inflating: xad real 0m0.011s user 0m0.000s sys 0m0.011s so you see 256-kibibyte chunks have submillisecond decompression time (more like 2 milliseconds on my cellphone) and only about a 1.8% size penalty for seekability: : jargsplit; units -t 565968/556102 % 101.77413 and, unlike the dictzip format (which lists the chunks in an extra backward-combatible file header), zip also supports efficient appending even in python (3.11.2) it's only about a millisecond: In [13]: z = zipfile.ZipFile('jargon.zip') In [14]: [f.filename for f in z.infolist()] Out[14]: ['xaa', 'xab', 'xac', 'xad', 'xae', 'xaf'] In [15]: %timeit z.open('xab').read() 1.13 ms ± 16.2 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each) this kind of performance means that any algorithm that would be efficient reading data stored on a conventional spinning-rust disk will be efficient reading compressed data if you put the data into a zipfile in "files" of around a meg each. (writing is another matter; zstd may help here, with its order-of-magnitude faster compression, but info-zip zip and unzip don't support zstd yet.) dictd keeps an index file in tsv format which uses what looks like base64 to locate the desired chunk and offset in the chunk: : jargsplit; < /usr/share/dictd/jargon.index shuf -n 4 | LANG=C sort | cat -vte fossil^IB9xE^IL8$ frednet^IB+q5^IDD$ upload^IE/t5^IJ1$ warez d00dz^IFLif^In0$ this is very similar to the index format used by eric raymond's volks-hypertext https://www.ibiblio.org/pub/Linux/apps/doctools/vh-1.8.tar.gz https://www.ibiblio.org/pub/Linux/apps/doctools/vh-1.8.tar.g... or vi ctags or emacs etags, but it supports random access into the file strfile from the fortune package works on a similar principle but uses a binary data file and no keys, just offsets: : ~; wget -nv canonical.org/~kragen/quotes.txt 2024-09-22 10:44:50 URL:http://canonical.org/~kragen/quotes.txt [49884/49884] -> "quotes.txt" [1] : ~; strfile quotes.txt "quotes.txt.dat" created There were 87 strings Longest string: 1625 bytes Shortest string: 92 bytes : ~; fortune quotes.txt Get enough beyond FUM [Fuck You Money], and it's merely Nice To Have Money. -- Dave Long, <dl@silcom.com>, on FoRK, around 2000-08-16, in Message-ID <200008162000.NAA10898@maltesecat> : ~; od -i --endian=big quotes.txt.dat 0000000 2 87 1625 92 0000020 0 620756992 0 933 0000040 1460 2307 2546 3793 0000060 3887 4149 5160 5471 0000100 5661 6185 6616 7000 of course if you were using a zipfile you could keep the index in the zipfile itself, and then there's no point in using base64 for the file offsets, or limiting them to 32 bits
commandersaki 2y ago
I love dict/dictd but I had an issue using it in hostile networks that block the port/protocol. I've been tempted to revamp dict/dictd to shovel the dict protocol over websokets so I can use it over the web. Just one of those ideas in the pipeline that I haven't revisited because I'm no longer dealing with that hostile network.
- gwervc 2y ago
  The dict protocol really show it's age, notably the stateful connection part. Having a new protocol based on HTTP and JSON similar to LSP would be nice but there is no real interest. (I made and used my own nonetheless in a research project. It may even be deployed but desactivated in another one) This biggest issue isn't technical, it's the fact organizations having dictionary data don't want third-party to interact with it without paid licensing.
  - steve_taylor 2y ago
    I hate the fact that corporate IT collectively decided to block every port except 80 and 443, making it necessary to base new protocols on HTTP instead of TCP/IP.
    - jolux 2y ago
      In my experience HTTP is a better foundation for novel protocols in most cases.
      - LambdaComplex 2y ago
        Doesn't HTTP require binary data to be converted to base64 encoding, thereby increasing its size on the wire? That seems suboptimal for a lot of use cases
        kragen 2y ago
        no, it does not, neither in requests nor in replies. possibly you are thinking of smtp
praveen9920 2y ago
> in an age of low-size disk drives and expensive software, looking up data over a dedicated protocol seems like a nifty2 idea. Then disk size exploded, databases became cheap, and search engines made it easy to look up words. I love this particular part of history about How protocols and applications got build based on restrictions and got evolved after improvements. Similar examples exists everywhere in computer history. Projecting the same with LLMs, we will have AIs running locally on mobile devices or perhaps AIs replacing OS of mobile devices and router protocols and servers. In future HN people looking at the code and feeling nostalgic about writing code
- 38 2y ago
  Given that most current AI generated code is dogshit, I would say we are well off from that.
  - praveen9920 2y ago
    Not right now but we are seeing the accelerated trends of generated code. How long would it take for regular use cases to be completely AI generated on the go. Edge cases exists everywhere.
    - AStonesThrow 2y ago
      It seems grossly inefficient for coding to go from NL prompt -> LLM -> formal language -> compile/execute. I'd envision AIs that would construct apps directly from the prompts, without the fragile middleware. In fact, if it is optimal coding to interface existing libraries and frameworks, with minimal novel code, then just go full LEGO and pull in those dependencies, minimizing the error-prone originality.
- mycall 2y ago
  Imagine if dict://internet was renamed to agent://source, then agentic calls to model sources could interconnect with ease. With HTTP/3, one stream could be the streaming browser and the other streams could be multi-agent sessions.
zenoprax 2y ago
I recently began testing my own public `dictd` server. The main goal was to make the OED (the full and proper one) available outside of a university proxy. I figured I would add the Webster's 1913 one too. Unfortunately the vast majority of dictionary files are in "stardict" format and the conversion to "dict" has yielded mixed results. I was hoping to host _every_ dictionary, good and bad, but will walk that back now. A free VPS could at least run the OED.
- kragen 2y ago
  what's the stardict format? which edition of the oed are you hosting? i scanned the first edition decades ago but i don't think there's a reasonable plain-text version of it yet
  - cormorant 2y ago
    StarDict (a program/file format) is easily googlable. A bit of a rabbit hole is that it's been chased around hosting providers because its site (used to) offer downloads of copyrighted dictionaries, including the OED 2nd edition. I don't know how these files were originally obtained or produced. See: https://web.archive.org/web/20230718140437/http://download.huzheng.org/bigdict/ https://web.archive.org/web/20230718140437/http://download.h... Edit to add: Also, "i scanned the first edition decades ago" sounds like quite a story. 13 volumes? What project were you doing?
    - kragen 2y ago
      oh, i just thought it would be good for the public-domain dictionary to be available to the public: https://www.mail-archive.com/kragen-tol@canonical.org/msg00152.html https://www.mail-archive.com/kragen-tol@canonical.org/msg001...
  - zenoprax 2y ago
    It will be the 2nd edition which is freely available on the internet with all of the usual copyright concerns. And it's already in 'dict' format so I didn't need to convert.
anthk 2y ago
echo "define * hacker " | nc dict.org 2628 | less
cratermoon 2y ago
Oh yes, I remember dictionary servers. Also many other protocols. What happened to all of those other protocols? Everything got squished onto http(s) for various reasons. As mentioned in this thread, corporate firewalls blocking every other port except 80 and 443. Around the time of the invention of http, protocols were proliferating for all kinds of new ideas. Today "innovation" happens on top of http, which devolves into some new kind of format to push back and forth.
- giantrobot 2y ago
  I wouldn't place all the blame on corporate IT for low level protocols dying out. A lot of corporate IT filtering was a reaction to malicious traffic originating from inside their networks. I think filtering on university networks killed more protocols than corporate filtering. Corporate networks were rarely the place where someone stuck a server in the corner with a public IP hosting a bunch of random services. That however was very common in university networks. When university networks (early 00s or so) started putting NAT on ResNets and filtering faculty networks is when a lot of random Internet servers started drying up. Universities had huge IPv4 blocks and would hand out their addresses to every machine on their networks. More than a few Web 1.0 companies started life on a random Sun machine in dorm rooms or the corner of a university computer lab. When publicly routed IPs dried up so did random FTPs and small IRC servers. At the same time residential broadband was taking off but so were the sales of home routers with NAT. Hosting random raw socket protocols stopped being practical for a lot of people. By the time low cost VPSes became available a lot of old protocols had already died out.
wormius 2y ago
Wow, either I've forgotten this existed, or had no clue, I was around for this era, and I remember Veronica, Archie, WAIS, Gopher, etc, but never recall reading about a Dict protocol, nice find!
nunobrito 2y ago
Nice find, didn't knew the protocol either. The site lists all available dictionaries here: https://dict.org/bin/Dict?Form=Dict4 https://dict.org/bin/Dict?Form=Dict4 I'll then be writing a java server for DICT. Likely add more recent types of dictionaries and acronyms to help keeping it alive.
fitsumbelay 2y ago
super fascinating and potentially useful for future projects with or w/o AI. obviously makes me want to maintain my own dict service love this
dokyun 2y ago
Emacs includes a browsable client for this protocol; you can use it with `M-x dictionary`.
nashashmi 2y ago
I dont think dict is secure enough. We need a new version called dick. K is for encryption key. /s /rant (at sftp).
5117142749 2y ago
[flagged]
t-3 2y ago
dict is a must for me on any daily driver. Removing the friction of opening a web browser or specialized app and just looking up text from the terminal is just so nice. Just like bc, you don't miss it when you don't know it's there, but once you get used to using it you can't live without. Making custom dictionaries is not very well documented though.
kgbcia 2y ago
Isn't .mobi an ebook format?
- therein 2y ago
  .COM is also a file format.
sedatk 2y ago
There was also a translation server called Babylon that used a similar raw text protocol (like WHOIS, and DICT here) in 1998. I remember adding it to my IRC script, but it must have stopped working at some point that I had replaced it with "babelfish.altavista.com" :)
divbzero 2y ago
Regarding OP’s unanswered question: > 00. Are there any other Dictionary Servers still available on the Internet? There are a number of other dict: servers including ones for different languages: https://servers.freedict.org/ https://servers.freedict.org/
divbzero 2y ago
The DICT Development Group also provides a dedicated dict: client: sudo apt install dict brew install dict Which allows you to query dict://dict.org/ directly: dict foo

425 points by edent 2y ago

Nowadays, on macOS, "dict://Internet" will open the Dictionary app with the query "Internet". (Probably behind a security prompt.) Not sure, if there's similar functionality on other operating systems.

dmd 2y ago
What do you mean by "behind a security prompt"?
- promiseofbeans 2y ago
  I tried it and firefox came up with a prompt confirming I wanted to open 'internet' in 'Dictionary'
  - WarOnPrivacy 2y ago
    > firefox came up with a prompt confirming I wanted to open 'internet' in 'Dictionary' My Ffx 131.0b9 wasn't so adept. It gave me: The address wasn’t understood Firefox doesn’t know how to open this address, because one of the following protocols (dict) isn’t associated with any program or is not allowed in this context. You might need to install other software to open this address.
    - wormius 2y ago
      Mine just opened the default search engine with the results, maybe it's how I have the address bar configured?
      - masswerk 2y ago
        The behavior may be also vary depending on whether it's an actual link a document or direct input to the location bar. (I get different prompts on Firefox, while both will forward to the built-in system dictionary.) And, of course, any further configurations may alter this, e.g., another service may have been registered for this protocol.
    - thayne 2y ago
      Perhaps it only does that on Macs?
- bobbylarrybobby 2y ago
  Browsers generally throw up a prompt before opening an external app

bubblesnort 2y ago

Never heard of the dict command? The author went through the trouble of figuring out the protocol but never bothered to just run dict. Okay.

jmillikin 2y ago
What do you think the post would have contained if he had run dict? Here's a hint: macbook% dict zsh: command not found: dict desktop$ dict bash: dict: command not found You'd have to be pretty into retro-computing before you'll find an OS that ships /usr/bin/dict .
- politelemon 2y ago
  Not really... FWIW, addressing the 'hint' you're giving specifically, this is on Ubuntu. $ dict Command 'dict' not found, but can be installed with: sudo apt install dict After installing, $ dict example 6 definitions found ...
  - jmillikin 2y ago
    So the causal chain would be: 1. Notice a URL scheme dict:// 2. Try to type 'dict' into a terminal, on the off chance there's a command-line tool with the same name (would you do this for https:// https:// and expect the same outcome?) 3. Be running a distribution that modifies the user's shell environment to suggest packages related to unknown commands 4. Actually install and run that command 5. Be running tcpdump or wireshark at the same time to notice that the `dict` command is reaching out to the network, as opposed to doing some sort of local lookup in /usr/share/dict 6. Figure out from the network traffic that the tool is using a dictionary-specific protocol as opposed to just making an HTTP request to dictionary.com or whatever. -- Nah, the only way someone would know (or even suspect!) that dict:// is somehow related to an ancient Unix command-line tool is prior knowledge, and it's unreasonable to expect the article author to have somehow intuited such an idea.
    - deleted 2y ago
      [deleted]

hkt 2y ago

I've been aware of dict for a while since I wrapped up an esperanto to english dictionary for KOReader in a format KOReader could understand. What I'd really have liked is a format like this: dict://<server/<origin language>/<definition language>/<word> Still, it is pretty cool that dict servers exist at all, so no complaints here.

im3w1l 2y ago

I admire these old protocols that are intentionally built to be usable both by machines and humans. Like the combination of a response code, and a human readable explanation. A help command right in the protocol. Makes me think it's a shame that making a json based protocol is so much easier to whip up than a textual ebnf-specified protocol. Like imagine if in python there was some library that you gave an ebnf spec maybe with some extra features similar to regex (named groups?) and you could compile it to a state machine and use it to parse documents, getting a Dict out.

orf 2y ago
Unfortunately, in practice they are a nightmare. Look at the WHOIS protocol for an example. Humans don’t look at responses very much, so you should optimise for machines. If you want a human-readable view, then turn the JSON response into something readable.
- poincaredisk 2y ago
  Whois protocol has no grammar to speak of (the response body is not defined at all, just a text blob) which makes it a nightmare to parse. Having a proper response format would solve this. Though I agree, I prefer my responses in JSON.
  - tephra 2y ago
    If you want WHOIS with a better defined format we have a protocol for that! It's called RDAP https://en.wikipedia.org/wiki/Registration_Data_Access_Protocol https://en.wikipedia.org/wiki/Registration_Data_Access_Proto...
- astrobe_ 2y ago
  The next logical step is to use a machine-friendly format instead; that is a binary protocol. Even HTML and XML which were designed for readability and manual writing eventually became 'not usable enough" ("became" because I think part of it is that their success made them exposed to less technical populations), and now we have markdown everywhere which most of the times is converted to HTML. So if you are going to use a tool more sophisticated than Ed/Edlin to read and write (rich) text in a certain format, it could be more efficient to focus on making the job of the machine - and of the programmer, easier. If you look at a binary protocol such as NTP, the binary format leaves very little room for Postel's principle [1], so it is straightforward to make a program that queries a server and display the result. [1] https://en.wikipedia.org/wiki/Robustness_principle https://en.wikipedia.org/wiki/Robustness_principle

mogoh 2y ago

hmmm $>curl dict://dict.org/d:Internet curl: (1) Protocol "dict" not supported

fallingsquirrel 2y ago
Works for me. I bet your OS ships a crippled version of curl. $ curl --version curl 8.7.1 (x86_64-pc-linux-gnu) [...] $ curl dict://dict.org/d:Internet 220 dict.dict.org dictd 1.12.1/rf on Linux 4.19.0-10-amd64 <auth.mime> <370202891.28105.1727009645@dict.dict.org> 250 ok 150 1 definitions retrieved [...]
- soraminazuki 2y ago
  Manual build with an explicit `--disable-dict` perhaps? Because it's not Debian, Fedora, Homebrew, Nix, Alpine, Arch, or Gentoo, judging by their package definitions.
  - mogoh 2y ago
    I am on Fedora Silverblue $>curl --version curl 8.6.0 (x86_64-redhat-linux-gnu) libcurl/8.6.0 OpenSSL/3.2.2 zlib/1.3.1.zlib-ng libidn2/2.3.7 nghttp2/1.59.0 Release-Date: 2024-01-31 Protocols: file ftp ftps http https ipfs ipns Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz SPNEGO SSL threadsafe UnixSockets I am not sure I I understand you correctly. Should it work on Fedora?
    - SushiHippie 2y ago
      The description on this page at least lists the dict protocol https://src.fedoraproject.org/rpms/curl/ https://src.fedoraproject.org/rpms/curl/ Only the minimal build disables the dict protocol, maybe you have installed the curl-minimal package? https://src.fedoraproject.org/rpms/curl/blob/rawhide/f/curl.spec https://src.fedoraproject.org/rpms/curl/blob/rawhide/f/curl....
      - 2y ago

kragen 2y ago

dict and the relevant dictionaries are things i pretty much always install on every new laptop. gcide in particular includes most of the famous 1913 webster dictionary with its sparkling prose: : ~; dict glisten 2 definitions found From The Collaborative International Dictionary of English v.0.48 [gcide]: Glisten \Glis"ten\ (gl[i^]s"'n), v. i. [imp. & p. p. {Glistened}; p. pr. & vb. n. {Glistening}.] [OE. glistnian, akin to glisnen, glisien, AS. glisian, glisnian, akin to E. glitter. See {Glitter}, v. i., and cf. {Glister}, v. i.] To sparkle or shine; especially, to shine with a mild, subdued, and fitful luster; to emit a soft, scintillating light; to gleam; as, the glistening stars. Syn: See {Flash}. [1913 Webster] it's interesting to think about how you would implement this service efficiently under the constraints of mid-01990s computers, where a gigabyte was still a lot of disk space and multiuser unix servers commonly had about 100 mips (https://netlib.org/performance/html/dhrystone.data.col0.html https://netlib.org/performance/html/dhrystone.data.col0.html) totally by coincidence i was looking at the dictzip man page this morning; it produces gzip-compatible files that support random seeks so you can keep the database for your dictd server compressed. (as far as i know, rik faith's dictd is still the only server implementation of the dict protocol, which is incidentally not a very good protocol.) you can see that the penalty for seekability is about 6% in this case: : ~; ls -l /usr/share/dictd/jargon.dict.dz -rw-r--r-- 1 root root 587377 Jan 1 2021 /usr/share/dictd/jargon.dict.dz : ~; \time gzip -dc /usr/share/dictd/jargon.dict.dz|wc -c 0.01user 0.00system 0:00.01elapsed 100%CPU (0avgtext+0avgdata 1624maxresident)k 0inputs+0outputs (0major+160minor)pagefaults 0swaps 1418350 : ~; gzip -dc /usr/share/dictd/jargon.dict.dz|gzip -9c|wc -c 556102 : ~; units -t 587377/556102 % 105.62397 nowadays computers are fast enough that it probably isn't a big win to gzip in such small chunks (dictzip has a chunk limit of 64k) and you might as well use a zipfile, all implementations of which support random access: : ~; mkdir jargsplit : ~; cd jargsplit : jargsplit; gzip -dc /usr/share/dictd/jargon.dict.dz|split -b256K : jargsplit; zip jargon.zip xaa xab xac xad xae xaf adding: xaa (deflated 60%) adding: xab (deflated 59%) adding: xac (deflated 59%) adding: xad (deflated 61%) adding: xae (deflated 62%) adding: xaf (deflated 58%) : jargsplit; ls -l jargon.zip -rw-r--r-- 1 user user 565968 Sep 22 09:47 jargon.zip : jargsplit; time unzip -o jargon.zip xad Archive: jargon.zip inflating: xad real 0m0.011s user 0m0.000s sys 0m0.011s so you see 256-kibibyte chunks have submillisecond decompression time (more like 2 milliseconds on my cellphone) and only about a 1.8% size penalty for seekability: : jargsplit; units -t 565968/556102 % 101.77413 and, unlike the dictzip format (which lists the chunks in an extra backward-combatible file header), zip also supports efficient appending even in python (3.11.2) it's only about a millisecond: In [13]: z = zipfile.ZipFile('jargon.zip') In [14]: [f.filename for f in z.infolist()] Out[14]: ['xaa', 'xab', 'xac', 'xad', 'xae', 'xaf'] In [15]: %timeit z.open('xab').read() 1.13 ms ± 16.2 µs per loop (mean ± std. dev. of 7 runs, 1,000 loops each) this kind of performance means that any algorithm that would be efficient reading data stored on a conventional spinning-rust disk will be efficient reading compressed data if you put the data into a zipfile in "files" of around a meg each. (writing is another matter; zstd may help here, with its order-of-magnitude faster compression, but info-zip zip and unzip don't support zstd yet.) dictd keeps an index file in tsv format which uses what looks like base64 to locate the desired chunk and offset in the chunk: : jargsplit; < /usr/share/dictd/jargon.index shuf -n 4 | LANG=C sort | cat -vte fossil^IB9xE^IL8$ frednet^IB+q5^IDD$ upload^IE/t5^IJ1$ warez d00dz^IFLif^In0$ this is very similar to the index format used by eric raymond's volks-hypertext https://www.ibiblio.org/pub/Linux/apps/doctools/vh-1.8.tar.gz https://www.ibiblio.org/pub/Linux/apps/doctools/vh-1.8.tar.g... or vi ctags or emacs etags, but it supports random access into the file strfile from the fortune package works on a similar principle but uses a binary data file and no keys, just offsets: : ~; wget -nv canonical.org/~kragen/quotes.txt 2024-09-22 10:44:50 URL:http://canonical.org/~kragen/quotes.txt [49884/49884] -> "quotes.txt" [1] : ~; strfile quotes.txt "quotes.txt.dat" created There were 87 strings Longest string: 1625 bytes Shortest string: 92 bytes : ~; fortune quotes.txt Get enough beyond FUM [Fuck You Money], and it's merely Nice To Have Money. -- Dave Long, <dl@silcom.com>, on FoRK, around 2000-08-16, in Message-ID <200008162000.NAA10898@maltesecat> : ~; od -i --endian=big quotes.txt.dat 0000000 2 87 1625 92 0000020 0 620756992 0 933 0000040 1460 2307 2546 3793 0000060 3887 4149 5160 5471 0000100 5661 6185 6616 7000 of course if you were using a zipfile you could keep the index in the zipfile itself, and then there's no point in using base64 for the file offsets, or limiting them to 32 bits

commandersaki 2y ago

I love dict/dictd but I had an issue using it in hostile networks that block the port/protocol. I've been tempted to revamp dict/dictd to shovel the dict protocol over websokets so I can use it over the web. Just one of those ideas in the pipeline that I haven't revisited because I'm no longer dealing with that hostile network.

gwervc 2y ago
The dict protocol really show it's age, notably the stateful connection part. Having a new protocol based on HTTP and JSON similar to LSP would be nice but there is no real interest. (I made and used my own nonetheless in a research project. It may even be deployed but desactivated in another one) This biggest issue isn't technical, it's the fact organizations having dictionary data don't want third-party to interact with it without paid licensing.
- steve_taylor 2y ago
  I hate the fact that corporate IT collectively decided to block every port except 80 and 443, making it necessary to base new protocols on HTTP instead of TCP/IP.
  - jolux 2y ago
    In my experience HTTP is a better foundation for novel protocols in most cases.
    - LambdaComplex 2y ago
      Doesn't HTTP require binary data to be converted to base64 encoding, thereby increasing its size on the wire? That seems suboptimal for a lot of use cases
      - kragen 2y ago
        no, it does not, neither in requests nor in replies. possibly you are thinking of smtp

praveen9920 2y ago

> in an age of low-size disk drives and expensive software, looking up data over a dedicated protocol seems like a nifty2 idea. Then disk size exploded, databases became cheap, and search engines made it easy to look up words. I love this particular part of history about How protocols and applications got build based on restrictions and got evolved after improvements. Similar examples exists everywhere in computer history. Projecting the same with LLMs, we will have AIs running locally on mobile devices or perhaps AIs replacing OS of mobile devices and router protocols and servers. In future HN people looking at the code and feeling nostalgic about writing code

38 2y ago
Given that most current AI generated code is dogshit, I would say we are well off from that.
- praveen9920 2y ago
  Not right now but we are seeing the accelerated trends of generated code. How long would it take for regular use cases to be completely AI generated on the go. Edge cases exists everywhere.
  - AStonesThrow 2y ago
    It seems grossly inefficient for coding to go from NL prompt -> LLM -> formal language -> compile/execute. I'd envision AIs that would construct apps directly from the prompts, without the fragile middleware. In fact, if it is optimal coding to interface existing libraries and frameworks, with minimal novel code, then just go full LEGO and pull in those dependencies, minimizing the error-prone originality.
mycall 2y ago
Imagine if dict://internet was renamed to agent://source, then agentic calls to model sources could interconnect with ease. With HTTP/3, one stream could be the streaming browser and the other streams could be multi-agent sessions.

zenoprax 2y ago

I recently began testing my own public `dictd` server. The main goal was to make the OED (the full and proper one) available outside of a university proxy. I figured I would add the Webster's 1913 one too. Unfortunately the vast majority of dictionary files are in "stardict" format and the conversion to "dict" has yielded mixed results. I was hoping to host _every_ dictionary, good and bad, but will walk that back now. A free VPS could at least run the OED.

kragen 2y ago
what's the stardict format? which edition of the oed are you hosting? i scanned the first edition decades ago but i don't think there's a reasonable plain-text version of it yet
- cormorant 2y ago
  StarDict (a program/file format) is easily googlable. A bit of a rabbit hole is that it's been chased around hosting providers because its site (used to) offer downloads of copyrighted dictionaries, including the OED 2nd edition. I don't know how these files were originally obtained or produced. See: https://web.archive.org/web/20230718140437/http://download.huzheng.org/bigdict/ https://web.archive.org/web/20230718140437/http://download.h... Edit to add: Also, "i scanned the first edition decades ago" sounds like quite a story. 13 volumes? What project were you doing?
  - kragen 2y ago
    oh, i just thought it would be good for the public-domain dictionary to be available to the public: https://www.mail-archive.com/kragen-tol@canonical.org/msg00152.html https://www.mail-archive.com/kragen-tol@canonical.org/msg001...
- zenoprax 2y ago
  It will be the 2nd edition which is freely available on the internet with all of the usual copyright concerns. And it's already in 'dict' format so I didn't need to convert.

anthk 2y ago

echo "define * hacker " | nc dict.org 2628 | less

cratermoon 2y ago

Oh yes, I remember dictionary servers. Also many other protocols. What happened to all of those other protocols? Everything got squished onto http(s) for various reasons. As mentioned in this thread, corporate firewalls blocking every other port except 80 and 443. Around the time of the invention of http, protocols were proliferating for all kinds of new ideas. Today "innovation" happens on top of http, which devolves into some new kind of format to push back and forth.

giantrobot 2y ago
I wouldn't place all the blame on corporate IT for low level protocols dying out. A lot of corporate IT filtering was a reaction to malicious traffic originating from inside their networks. I think filtering on university networks killed more protocols than corporate filtering. Corporate networks were rarely the place where someone stuck a server in the corner with a public IP hosting a bunch of random services. That however was very common in university networks. When university networks (early 00s or so) started putting NAT on ResNets and filtering faculty networks is when a lot of random Internet servers started drying up. Universities had huge IPv4 blocks and would hand out their addresses to every machine on their networks. More than a few Web 1.0 companies started life on a random Sun machine in dorm rooms or the corner of a university computer lab. When publicly routed IPs dried up so did random FTPs and small IRC servers. At the same time residential broadband was taking off but so were the sales of home routers with NAT. Hosting random raw socket protocols stopped being practical for a lot of people. By the time low cost VPSes became available a lot of old protocols had already died out.

wormius 2y ago

Wow, either I've forgotten this existed, or had no clue, I was around for this era, and I remember Veronica, Archie, WAIS, Gopher, etc, but never recall reading about a Dict protocol, nice find!

nunobrito 2y ago

Nice find, didn't knew the protocol either. The site lists all available dictionaries here: https://dict.org/bin/Dict?Form=Dict4 https://dict.org/bin/Dict?Form=Dict4 I'll then be writing a java server for DICT. Likely add more recent types of dictionaries and acronyms to help keeping it alive.

fitsumbelay 2y ago

super fascinating and potentially useful for future projects with or w/o AI. obviously makes me want to maintain my own dict service love this

dokyun 2y ago

Emacs includes a browsable client for this protocol; you can use it with `M-x dictionary`.

nashashmi 2y ago

I dont think dict is secure enough. We need a new version called dick. K is for encryption key. /s /rant (at sftp).

5117142749 2y ago

[flagged]

t-3 2y ago

dict is a must for me on any daily driver. Removing the friction of opening a web browser or specialized app and just looking up text from the terminal is just so nice. Just like bc, you don't miss it when you don't know it's there, but once you get used to using it you can't live without. Making custom dictionaries is not very well documented though.

kgbcia 2y ago

Isn't .mobi an ebook format?

therein 2y ago
.COM is also a file format.

sedatk 2y ago

There was also a translation server called Babylon that used a similar raw text protocol (like WHOIS, and DICT here) in 1998. I remember adding it to my IRC script, but it must have stopped working at some point that I had replaced it with "babelfish.altavista.com" :)

divbzero 2y ago

Regarding OP’s unanswered question: > 00. Are there any other Dictionary Servers still available on the Internet? There are a number of other dict: servers including ones for different languages: https://servers.freedict.org/ https://servers.freedict.org/

divbzero 2y ago

The DICT Development Group also provides a dedicated dict: client: sudo apt install dict brew install dict Which allows you to query dict://dict.org/ directly: dict foo