7 ms·

J Has the Best Development Environment (2009)

84 points by epsylon 12y ago

minikomi 12y ago
I've been dabbling with learning J recently and, yes, it's really really humbling. However, I think it can be much easier. The thing I have the most trouble with is, predictably, unfamiliarity with the large amount of verbs. So, what to do? I'm beginning to think approaching it more like a foreign language will make more sense - i.e., reading & spaced repetition of phrases (short programs - J's specialty!) / vocab (useful hooks, forks) using a flash card system. It's a compelling language to learn (even just for the fact you can pretty much program using a pen and paper while waiting for the bus - short little rows of verbs you build out gradually as you reason about it or type them in your iphone), and I hope my approach has merit.
- qznc 12y ago
  There is a public Anki (spaced repetition app) deck available: https://ankiweb.net/shared/info/832417998 https://ankiweb.net/shared/info/832417998 That one still looks very cryptic, though.
  - minikomi 12y ago
    Ah, good find! I'll give it a whirl.
    - RBerenguel 12y ago
      Actually, I'm learning APL with a homemade Anki deck (APL is slightly easier for me to read than J, since it uses symbols)
- klibertp 12y ago
  I learned J in very much this way, although instead of reading J code I chose to write it. I worked my way through a cheatsheet (http://www.jsoftware.com/jwiki/HenryRich?action=AttachFile&do=view&target=J602_RefCard_color_letter_current.pdf http://www.jsoftware.com/jwiki/HenryRich?action=AttachFile&d...) inventing little toy problems to solve with particular verbs along the way. But words (verbs, gerunds, etc.) are not enough - you need to know the rules for composing them. Fortunately this is rather easy to learn. I'd say that you need to know about hooks, forks (trains in general) and ranks before starting to learn the vocabulary. But maybe you can learn those things along the vocabulary too.
- eggy 12y ago
  I have been toying with J for the past few years, and I think you only appreciate it when you see it as equivalent to mathematical notation, alien at first, but succinct and very useable when you know what the symbols mean. See Iverson's paper on 'Notation as a Tool of Thought' (http://www.jsoftware.com/papers/tot.htm http://www.jsoftware.com/papers/tot.htm). I am self-teaching mathematics, and even when trying Julia as a math tool, I find the basic unit of J, the array, as more amenable to my studies. It's interesting how there is all the talk about 'vectorization' and such in other languages, which J has been doing for decades. J is functional, and can have 'for' loops, but loopless programming is encouraged and easy. When dealing with images or large data sets that are memory mapped, the operations are succinct and powerful and eye-opening. My latest epiphany from moving away from using J as just a sophisticated calculator came after watching the 4 videos by Martin Saurer (https://www.youtube.com/watch?v=VSJpJt3c11c https://www.youtube.com/watch?v=VSJpJt3c11c). Iverson's books on arithmetic and calculus using J are mind-blowing. You are engaged, not distracted by the language. J is fast too. From another talk (https://www.youtube.com/watch?v=IKfJRyoiBlY&feature=youtu.be https://www.youtube.com/watch?v=IKfJRyoiBlY&feature=youtu.be), creating a 1x10^6 element array and summing over columns or rows is fast, and the code is simply: b=: ? 1000 1000 $ 10 +/+/b These two lines create a 1x10^6 element array and then sums the columns, and then sums the single row total. Fast. These guys have really distilled the libraries and made them work well with J (lapacke c interface to lapack). You get c speeds from an interpreter. I guess this is why q with kdb+ is used so much by the finance community despite how expensive it is. Anyway, I am currently staying with J, and I am writing my little tasks in J and julia to see how they fare against each other in speed and development efforts. Now that I have the knowledge to do Qt interfaces for J, I may just stick with J for the sheer joy.
  - oddthink 12y ago
    IMHO, q's (and K's) big advantage over J is that they have tables and dictionaries as an available data type. This makes basic tasks like reading csv files and merging different data sources much easier in q. q also has a nicer syntax for anonymous functions. Of course, q is also very expensive, and it tries to be too clever at times: I remember running in to all sorts of weird corner cases, with things like NAs being skipped in float arrays, but not in int arrays, etc.
adamnemecek 12y ago
Speaking of exotic languages, I'm somewhat intrigued by Rebol and Red. How bad of an idea is investing any time into learning either of these?
- erikb 12y ago
  Well, it's better than watching a movie instead, but not much.
  - k__ 12y ago
    Depends on the movie. The stuff from Jim Jarmusch is rather nice.
- purpletoned 12y ago
  Depends on what you want to do with it. If you're looking to develop mobile or desktop applications it's almost useless. The desktop GUI is stuck in the 80s and as far as I know even that isn't an option for mobile. If you're doing web apps, it's probably worth a try provided you don't mind writing your own libraries for a lot of stuff.
  - kbd 12y ago
    > The desktop GUI is stuck in the 80s and as far as I know even that isn't an option for mobile. It's a shame that REBOL was open sourced a decade and a half too late. Back when Perl was still king of scripting languages and Python/Ruby etc. hadn't taken over for Perl yet, REBOL could have had success. The community was fun and creative and the implementation seemed like it would be small and able to be improved through open source. Just checked, and the repository hasn't had a checkin for 7 months: https://github.com/rebol/rebol https://github.com/rebol/rebol
  - akavel 12y ago
    I believe REBOL and Red shouldn't be confused. The Red project seems to be going on quite lively, with the most recent commit some 9 days ago at the time I'm writing this (see https://github.com/red/red https://github.com/red/red ). That said, it's still not fully complete, but it does most certainly have more modern goals than REBOL -- see e.g. the Roadmap Status page (http://www.red-lang.org/p/roadmap.html http://www.red-lang.org/p/roadmap.html) for information about Android bridge (currently reported "90% complete", look for it at the bottom of the page).
chubot 12y ago
I like vectorized languages -- I know R fairly well, and used to use Matlab. J's syntax, where everything is a single character, kind of turns me off. Is that incidental or is the syntax essential to the language? (In the case of Lisp, the "weird" syntax isn't incidental but rather essential, since code is data) If it's incidental, I'd like to see some elegant J programs "written out" with some more friendly syntax. EDIT: If you want a laugh, download the source for J, and run wc -l .c .h. http://www.jsoftware.com/source.htm http://www.jsoftware.com/source.htm
- minikomi 12y ago
  Check out Tracy Harms's talk: http://www.infoq.com/presentations/j-language http://www.infoq.com/presentations/j-language Or, perhaps, this chapter of the (rather dense) "Learning J": http://www.jsoftware.com/help/learning/03.htm http://www.jsoftware.com/help/learning/03.htm
  - huehue 12y ago
    I guess it's a matter of taste. As someone versed in both APL and K/Kona, J's syntax puts me off as well.
    - Avshalom 12y ago
      J suffers vs APL/K. The verb trains; number of verbs vs number of available symbols; variable lengths and semantic baggage we bring to various symbols contribute to a much worse 'line barf problem'. I understand why Iverson went with ascii but frankly I don't even know APL beyond 'advanced calculator' level but every time I look at an APL program it just seems cleaner.
      - seanstickle 12y ago
        This is my difficulty with J as well. APL has a better mental "feeling" for me, more like mathematical symbols. Oftentimes, these are pretty direct. Want to drop the first element of a vector? APL: 1 ↓ 1 2 3 4 5 J: 1 }. 1 2 3 4 5 The APL version looks like a drop to me. The J version looks like two characters that I have to remember the meaning of. I'm sure if I was deeply proficient at J, the two-characters would stand out by their meaning, but it's hard for me to get that far when APL is so much cleaner for me to read.
wyc 12y ago
I have some thin (read: incomplete) notes that hint at how math-oriented the J verbs are: http://wycd.net/notes/jlang/jlang.pdf http://wycd.net/notes/jlang/jlang.pdf Among my favorite parts of the language is the sole set of vocabulary[0] that you have to learn to be proficient in J. It's daunting at first, but after this "alphabet" is internalized, you can understand most code. Just as our integral symbols save us from writing out "Integral()", J's operators are shorthand for fundamental computational functions. For example, here's a very naive kNN implementation in J, without pulling in any "jlearn" external libraries or anything: https://github.com/wyc/snippets/blob/master/j/knn.ijs https://github.com/wyc/snippets/blob/master/j/knn.ijs One of the stated purposes of J is the clear expression of algorithms. How are you supposed to clearly do that with layers upon layers of abstractions? The language excels at matrix and statistical operations on sets of homogeneous data. Many financial institutions deploy J's close sister, K (also a descendant of APL) coupled with KDB[1]. Does it have potential for use on "Big Data"? Maybe. It's not SMP, but the interpreter is rather light, so it can be spun up in multiple processes. Cutting up the data correctly is still a problem. I would also not use J to write a webapp; CRUD isn't its strong point. Some more examples of J: https://github.com/wyc/snippets/blob/master/j/jtalk/examples.txt https://github.com/wyc/snippets/blob/master/j/jtalk/examples... Come hang out at #jsoftware on freenode to find people much more knowledgeable than me. :P [0] http://www.jsoftware.com/help/dictionary/vocabul.htm http://www.jsoftware.com/help/dictionary/vocabul.htm [1] not the kernel debugger, but the time-series K database: http://kx.com/kdb-plus.php http://kx.com/kdb-plus.php
- agumonkey 12y ago
  It recently came to my mind that Math, J, lisp, ml are just anti-marketing artefacts. But on the intellectual/UX they're on point. When you have a good abstractions, and very consistent and orthogonal ones, appearance becomes almost irrelevant. They help you think and that's what matters.
- scottlocklin 12y ago
  FWIIW, the Jd database is pretty good (which works much like kdb+). It's not designed to work on distributed systems (kdb+ can do this), but it wouldn't be too difficult to make it do this. I recently wrote a discrete wavelet transform algorithm in J. It took 4 lines. I could probably have done it in 1. Granted it took me a while to write these 4 lines, but it's pretty cool being able to take the whole thing in at a glance. While I was at it, I wrote my own query system for a much larger than memory dataset; that part only took a few minutes. One of the great things about J: it teaches you to appreciate the matrix subsystems of things like R. You can do a lot of J in R if you stay away from data.frames. Oh yeah, while the IDE is pretty nice, I use it in emacs. Some resources: Minimal J for beginners: http://www.jsoftware.com/jwiki/DevonMcCormick/MinimalBeginningJ http://www.jsoftware.com/jwiki/DevonMcCormick/MinimalBeginni... A more wordy vocabulary for beginners: http://www.jsoftware.com/jwiki/NuVoc http://www.jsoftware.com/jwiki/NuVoc
S4M 12y ago
>> The IDE is more akin to Python's IDLE than monstrosities which may come to mind. There's a window for entering commands and seeing the results, and you can open up separate, syntax-colored editor windows, running the contents of each with a keypress. It's nothing groundbreaking, but it's something that most languages don't provide. What? I have considered that standard for all the languages that have an interpreter (which is roughly, everything except C,C++,C#, Java).
- mgraczyk 12y ago
  Both of the major C# distributions (mono and CLR with VS) come with a repl now: $ csharp Mono C# Shell, type "help;" for help Enter statements below. csharp> var seq = Enumerable.Range(0, 100); csharp> var mod17 = from num in seq > where num % 17 == 0 > select num csharp> foreach (var n in mod17) { > Console.WriteLine(n); > } 0 17 34 51 68 85 csharp>
  - S4M 12y ago
    Oh, that's nice, thanks - I haven't used C# since 2008. What you said just reinforces my point: the OP boasts over a feature that is very, very common.
    - Avshalom 12y ago
      it's wasn't just a repl it's a graphical repl (important on windows with its horrible terminal) with an integrated editor it was more like (wait for it) Dr.Ed or IDLE.
      - S4M 12y ago
        In that case something like a screenshot wouldn't have hurt to make his point. On a totally different topic, are you Avshalom as well on #emacs on freenode? If that's the case I once spoke to you about improving how ESS shows graphics - something I haven't worked on further, sadly.
james_hague 12y ago
I'd like to clarify that I was using J6.02 for Windows when I wrote that. You can still download it from jsoftware.com. Since then the priority has been for J to be platform independent, but it's resulted in some big steps backward for beauty and usability. That's for the environment only--not the language.
kevinSuttle 12y ago
I'd love to see screenshots or videos of the more visual parts of the environment. Especially the Labs.
ww2 12y ago
one hurdle of J is that the verb train approach is too raw.

84 points by epsylon 12y ago

I've been dabbling with learning J recently and, yes, it's really really humbling. However, I think it can be much easier. The thing I have the most trouble with is, predictably, unfamiliarity with the large amount of verbs. So, what to do? I'm beginning to think approaching it more like a foreign language will make more sense - i.e., reading & spaced repetition of phrases (short programs - J's specialty!) / vocab (useful hooks, forks) using a flash card system. It's a compelling language to learn (even just for the fact you can pretty much program using a pen and paper while waiting for the bus - short little rows of verbs you build out gradually as you reason about it or type them in your iphone), and I hope my approach has merit.

qznc 12y ago
There is a public Anki (spaced repetition app) deck available: https://ankiweb.net/shared/info/832417998 https://ankiweb.net/shared/info/832417998 That one still looks very cryptic, though.
- minikomi 12y ago
  Ah, good find! I'll give it a whirl.
  - RBerenguel 12y ago
    Actually, I'm learning APL with a homemade Anki deck (APL is slightly easier for me to read than J, since it uses symbols)
klibertp 12y ago
I learned J in very much this way, although instead of reading J code I chose to write it. I worked my way through a cheatsheet (http://www.jsoftware.com/jwiki/HenryRich?action=AttachFile&do=view&target=J602_RefCard_color_letter_current.pdf http://www.jsoftware.com/jwiki/HenryRich?action=AttachFile&d...) inventing little toy problems to solve with particular verbs along the way. But words (verbs, gerunds, etc.) are not enough - you need to know the rules for composing them. Fortunately this is rather easy to learn. I'd say that you need to know about hooks, forks (trains in general) and ranks before starting to learn the vocabulary. But maybe you can learn those things along the vocabulary too.
eggy 12y ago
I have been toying with J for the past few years, and I think you only appreciate it when you see it as equivalent to mathematical notation, alien at first, but succinct and very useable when you know what the symbols mean. See Iverson's paper on 'Notation as a Tool of Thought' (http://www.jsoftware.com/papers/tot.htm http://www.jsoftware.com/papers/tot.htm). I am self-teaching mathematics, and even when trying Julia as a math tool, I find the basic unit of J, the array, as more amenable to my studies. It's interesting how there is all the talk about 'vectorization' and such in other languages, which J has been doing for decades. J is functional, and can have 'for' loops, but loopless programming is encouraged and easy. When dealing with images or large data sets that are memory mapped, the operations are succinct and powerful and eye-opening. My latest epiphany from moving away from using J as just a sophisticated calculator came after watching the 4 videos by Martin Saurer (https://www.youtube.com/watch?v=VSJpJt3c11c https://www.youtube.com/watch?v=VSJpJt3c11c). Iverson's books on arithmetic and calculus using J are mind-blowing. You are engaged, not distracted by the language. J is fast too. From another talk (https://www.youtube.com/watch?v=IKfJRyoiBlY&feature=youtu.be https://www.youtube.com/watch?v=IKfJRyoiBlY&feature=youtu.be), creating a 1x10^6 element array and summing over columns or rows is fast, and the code is simply: b=: ? 1000 1000 $ 10 +/+/b These two lines create a 1x10^6 element array and then sums the columns, and then sums the single row total. Fast. These guys have really distilled the libraries and made them work well with J (lapacke c interface to lapack). You get c speeds from an interpreter. I guess this is why q with kdb+ is used so much by the finance community despite how expensive it is. Anyway, I am currently staying with J, and I am writing my little tasks in J and julia to see how they fare against each other in speed and development efforts. Now that I have the knowledge to do Qt interfaces for J, I may just stick with J for the sheer joy.
- oddthink 12y ago
  IMHO, q's (and K's) big advantage over J is that they have tables and dictionaries as an available data type. This makes basic tasks like reading csv files and merging different data sources much easier in q. q also has a nicer syntax for anonymous functions. Of course, q is also very expensive, and it tries to be too clever at times: I remember running in to all sorts of weird corner cases, with things like NAs being skipped in float arrays, but not in int arrays, etc.

adamnemecek 12y ago

Speaking of exotic languages, I'm somewhat intrigued by Rebol and Red. How bad of an idea is investing any time into learning either of these?

erikb 12y ago
Well, it's better than watching a movie instead, but not much.
- k__ 12y ago
  Depends on the movie. The stuff from Jim Jarmusch is rather nice.
purpletoned 12y ago
Depends on what you want to do with it. If you're looking to develop mobile or desktop applications it's almost useless. The desktop GUI is stuck in the 80s and as far as I know even that isn't an option for mobile. If you're doing web apps, it's probably worth a try provided you don't mind writing your own libraries for a lot of stuff.
- kbd 12y ago
  > The desktop GUI is stuck in the 80s and as far as I know even that isn't an option for mobile. It's a shame that REBOL was open sourced a decade and a half too late. Back when Perl was still king of scripting languages and Python/Ruby etc. hadn't taken over for Perl yet, REBOL could have had success. The community was fun and creative and the implementation seemed like it would be small and able to be improved through open source. Just checked, and the repository hasn't had a checkin for 7 months: https://github.com/rebol/rebol https://github.com/rebol/rebol
- akavel 12y ago
  I believe REBOL and Red shouldn't be confused. The Red project seems to be going on quite lively, with the most recent commit some 9 days ago at the time I'm writing this (see https://github.com/red/red https://github.com/red/red ). That said, it's still not fully complete, but it does most certainly have more modern goals than REBOL -- see e.g. the Roadmap Status page (http://www.red-lang.org/p/roadmap.html http://www.red-lang.org/p/roadmap.html) for information about Android bridge (currently reported "90% complete", look for it at the bottom of the page).

chubot 12y ago

I like vectorized languages -- I know R fairly well, and used to use Matlab. J's syntax, where everything is a single character, kind of turns me off. Is that incidental or is the syntax essential to the language? (In the case of Lisp, the "weird" syntax isn't incidental but rather essential, since code is data) If it's incidental, I'd like to see some elegant J programs "written out" with some more friendly syntax. EDIT: If you want a laugh, download the source for J, and run wc -l .c .h. http://www.jsoftware.com/source.htm http://www.jsoftware.com/source.htm

minikomi 12y ago
Check out Tracy Harms's talk: http://www.infoq.com/presentations/j-language http://www.infoq.com/presentations/j-language Or, perhaps, this chapter of the (rather dense) "Learning J": http://www.jsoftware.com/help/learning/03.htm http://www.jsoftware.com/help/learning/03.htm
- huehue 12y ago
  I guess it's a matter of taste. As someone versed in both APL and K/Kona, J's syntax puts me off as well.
  - Avshalom 12y ago
    J suffers vs APL/K. The verb trains; number of verbs vs number of available symbols; variable lengths and semantic baggage we bring to various symbols contribute to a much worse 'line barf problem'. I understand why Iverson went with ascii but frankly I don't even know APL beyond 'advanced calculator' level but every time I look at an APL program it just seems cleaner.
    - seanstickle 12y ago
      This is my difficulty with J as well. APL has a better mental "feeling" for me, more like mathematical symbols. Oftentimes, these are pretty direct. Want to drop the first element of a vector? APL: 1 ↓ 1 2 3 4 5 J: 1 }. 1 2 3 4 5 The APL version looks like a drop to me. The J version looks like two characters that I have to remember the meaning of. I'm sure if I was deeply proficient at J, the two-characters would stand out by their meaning, but it's hard for me to get that far when APL is so much cleaner for me to read.

wyc 12y ago

I have some thin (read: incomplete) notes that hint at how math-oriented the J verbs are: http://wycd.net/notes/jlang/jlang.pdf http://wycd.net/notes/jlang/jlang.pdf Among my favorite parts of the language is the sole set of vocabulary[0] that you have to learn to be proficient in J. It's daunting at first, but after this "alphabet" is internalized, you can understand most code. Just as our integral symbols save us from writing out "Integral()", J's operators are shorthand for fundamental computational functions. For example, here's a very naive kNN implementation in J, without pulling in any "jlearn" external libraries or anything: https://github.com/wyc/snippets/blob/master/j/knn.ijs https://github.com/wyc/snippets/blob/master/j/knn.ijs One of the stated purposes of J is the clear expression of algorithms. How are you supposed to clearly do that with layers upon layers of abstractions? The language excels at matrix and statistical operations on sets of homogeneous data. Many financial institutions deploy J's close sister, K (also a descendant of APL) coupled with KDB[1]. Does it have potential for use on "Big Data"? Maybe. It's not SMP, but the interpreter is rather light, so it can be spun up in multiple processes. Cutting up the data correctly is still a problem. I would also not use J to write a webapp; CRUD isn't its strong point. Some more examples of J: https://github.com/wyc/snippets/blob/master/j/jtalk/examples.txt https://github.com/wyc/snippets/blob/master/j/jtalk/examples... Come hang out at #jsoftware on freenode to find people much more knowledgeable than me. :P [0] http://www.jsoftware.com/help/dictionary/vocabul.htm http://www.jsoftware.com/help/dictionary/vocabul.htm [1] not the kernel debugger, but the time-series K database: http://kx.com/kdb-plus.php http://kx.com/kdb-plus.php

agumonkey 12y ago
It recently came to my mind that Math, J, lisp, ml are just anti-marketing artefacts. But on the intellectual/UX they're on point. When you have a good abstractions, and very consistent and orthogonal ones, appearance becomes almost irrelevant. They help you think and that's what matters.
scottlocklin 12y ago
FWIIW, the Jd database is pretty good (which works much like kdb+). It's not designed to work on distributed systems (kdb+ can do this), but it wouldn't be too difficult to make it do this. I recently wrote a discrete wavelet transform algorithm in J. It took 4 lines. I could probably have done it in 1. Granted it took me a while to write these 4 lines, but it's pretty cool being able to take the whole thing in at a glance. While I was at it, I wrote my own query system for a much larger than memory dataset; that part only took a few minutes. One of the great things about J: it teaches you to appreciate the matrix subsystems of things like R. You can do a lot of J in R if you stay away from data.frames. Oh yeah, while the IDE is pretty nice, I use it in emacs. Some resources: Minimal J for beginners: http://www.jsoftware.com/jwiki/DevonMcCormick/MinimalBeginningJ http://www.jsoftware.com/jwiki/DevonMcCormick/MinimalBeginni... A more wordy vocabulary for beginners: http://www.jsoftware.com/jwiki/NuVoc http://www.jsoftware.com/jwiki/NuVoc

S4M 12y ago

>> The IDE is more akin to Python's IDLE than monstrosities which may come to mind. There's a window for entering commands and seeing the results, and you can open up separate, syntax-colored editor windows, running the contents of each with a keypress. It's nothing groundbreaking, but it's something that most languages don't provide. What? I have considered that standard for all the languages that have an interpreter (which is roughly, everything except C,C++,C#, Java).

mgraczyk 12y ago
Both of the major C# distributions (mono and CLR with VS) come with a repl now: $ csharp Mono C# Shell, type "help;" for help Enter statements below. csharp> var seq = Enumerable.Range(0, 100); csharp> var mod17 = from num in seq > where num % 17 == 0 > select num csharp> foreach (var n in mod17) { > Console.WriteLine(n); > } 0 17 34 51 68 85 csharp>
- S4M 12y ago
  Oh, that's nice, thanks - I haven't used C# since 2008. What you said just reinforces my point: the OP boasts over a feature that is very, very common.
  - Avshalom 12y ago
    it's wasn't just a repl it's a graphical repl (important on windows with its horrible terminal) with an integrated editor it was more like (wait for it) Dr.Ed or IDLE.
    - S4M 12y ago
      In that case something like a screenshot wouldn't have hurt to make his point. On a totally different topic, are you Avshalom as well on #emacs on freenode? If that's the case I once spoke to you about improving how ESS shows graphics - something I haven't worked on further, sadly.

james_hague 12y ago

I'd like to clarify that I was using J6.02 for Windows when I wrote that. You can still download it from jsoftware.com. Since then the priority has been for J to be platform independent, but it's resulted in some big steps backward for beauty and usability. That's for the environment only--not the language.

kevinSuttle 12y ago

I'd love to see screenshots or videos of the more visual parts of the environment. Especially the Labs.

ww2 12y ago

one hurdle of J is that the verb train approach is too raw.