3 ms·
>> First, the tool asks for an input-output example of the desired tensor transformation. Then, it runs a combinatorial search to find TensorFlow expressions th
by YeGoblynQueenne 6y ago
>> First, the tool asks for an input-output example of the desired tensor
transformation. Then, it runs a combinatorial search to find TensorFlow
expressions that perform that transformation.
Ouch! The combinatorial space of arbitrary programs is huge. This can't work
for programs of any but the smallest size.
Indeed:
>> There are limitations to TF-Coder. It can currently find solutions involving
3-4 operations within a minute of searching, but solutions involving 6 or more
operations are too complex to find in a reasonable amount of time.
Btw, my PhD research is pretty much on something like this but learning Prolog
programs from examples, rather than Python. The problem is extremly hard and
all solutions so far look at ways to restrict the large hypothesis search
spaces, mainly by imposing some kind of strong inductive or language bias
(i.e. restricting what can be learned and how) but there is always a point
beyond which any way one tries to search for an arbitrary program will fall
into a huge search space and disappear down a myriad winding paths that lead
nowhere. So while it's interesting to see this effort (taking hints from
natural language is a nice touch) program syntehsis or program learning will
not be solved by any search-based algorithm.
Edit: Actually, the bit about how solutions involving 6 or more operations
take forever to return, is eerily familiar. An algorithm I was working on at
the start of my PhD was pretty much limited to programs of 5 clauses (roughly,
5 function calls). I'm sure it's a coincidence but, golly.
- im3w1l 6y agoI think the intended usage is not to find your whole program from example. Instead the programmer breaks the program down into simple operations, and uses this tool to remember what the name of that operation is. And finding short compositions is a bonus.
- YeGoblynQueenne 6y agoThere is certainly a lot of use that one can make out of a program with this kind of limitation, but it is a limitation- if it wasn't, there wouldn't be a warning about it in the article. In any case, if there was no such limitation most people here would be out of a job now, so it's an important limitation. >> And finding short compositions is a bonus. You mean because they can be easily inspected and debugged and edited by hand if required? If so, yes, I agree. But, for example, the algorithm I worked with, that I discuss above, a big criticism of the underlying technique has always been that _in practice_ it's limited to learning small programs.
- im3w1l 6y agoNo, I mean finding short compositions is a bonus feature over finding a single function from example.
- YeGoblynQueenne 6y agoOK, I don't understand what you mean. Can you explain?
- im3w1l 6y agoBeing able to find one_hot from input/output example is already useful, because you might skip over that name thinking it's some weird machine learning thing and you just want to set element values. You could easily lose an hour building and testing a solution based on stacking your desired indices with ranges and feeding it to scatter_nd.
- YeGoblynQueenne 6y agoI think I understand now, but just to be sure- you mean that being able to find expressions that do what you want fom examples of what you want, is a big bonus because it speeds up your programming? i.e. a sort of enhanced search function?
- im3w1l 6y agoYes exactly. But only for tensorflow. Not for programming languages in general.
- ghj 6y agoI can give another example use case. Say I know math but don't know tf api. I already know how to matrix multiply with `@` but don't know the function for matrix power. Then in theory it should be fairly quick for me to create a random matrix and feed it `mat @ mat`, `mat @ mat @ mat`, `mat @ mat @ mat @ mat`, etc until it spits out the correct tf function I should call for matrix_power. This is even better than googling since it guarantees that the function it found is correct for your example (googling returns tf.pow tf.exp tf.expm, none of which are actually what you want!)