4 ms·

I wonder, if at same point we will have just domain specific models (or do we have them already?). For instance: - Computer Science Models that can complete co

I wonder, if at same point we will have just domain specific models (or do we have them already?). For instance: - Computer Science Models that can complete code - Wikipedia-Style Models, which can infer knowledge - Fantasy Models, if you want to hear a new story every day - Naughty Models, for your daily dose of dirty talk Maybe it's not necessary from a technical standpoint, but maybe it's beneficial from a societal standpoint. There is nothing wrong with all the different kinks, but you would not ask your librarian for some dirty talk, or would expect to see explicit images outside the red district.

extasia 3y ago
There's a theory that these LLMs are learning these specialized internal structures anyways. If you could distill _those_, now that would be useful! (cheaper inference, ability to deploy on mobile etc.)
techdragon 3y ago
Small flaw, where does the “naughty” start… we aren’t really doing ourselves any favours with an “I’ll know it when I see it” approach to restricted free form generative models that can be steered by users. The mistake I see in a lot of arguments getting made online about all this is treating the AI/ML models as “whole systems” … we shouldn’t be trying to make very hard to analyse “black box” heart of the model, the bit where we’re turning statistical weights into data… this should not be assumed to do “everything”. If you want a naughty word filter, use a rejection filter from a dictionary, or bolt on a second AI/ML system tuned to catch stuff too similar to what you don’t want… but assuming all functionality can be somehow shoved into the one model is just foolish. We should have some unfettered models out there for us to use as either red-team tools to help generate content we can test safety and sanitising systems on, or otherwise use as desired…
- MrGilbert 3y ago
  Well, it was just my train of thoughts released into the wild. I have no specialized AI/ML background, so I cannot weight anything I write. I'm just making educated guesses. There are some rules in every society that at least deal with the question about what is appropriate and when, so these rules should serve as a boundary. But I agree, there will always be an area of uncertainty. Maybe we should just stop to think about how to eliminate these. Also, these rules differ from society to society. Maybe the greater mistake is to assume that there is a generative model that fits the needs of every society.
  - techdragon 3y ago
    The difference between societal standards and expectations is one of the reasons I think the efforts to build such content filtering/restrictions into the models themselves is a bad idea. Between uncertainty and the variation around the world we would be much better off if we embraced modular pipelines and tooling to make them easier, plug my American culture value filter in after my global human text generation tool, and serve that up to my American customers, plug my French culture value filter into the global model and serve it up to my French customers… much better than pretending there are global standards of morality for written content … there are obviously some standards people agree on in the real world of physical interaction between humans and other humans, animals, private property and so on … but what constitutes offensive text is nowhere near as universal.
api 3y ago
Then there’s the crossover market, like naughty code completion or fantasy knowledge inference.
- MrGilbert 3y ago
  That one dirty line of code everyone is looking for...
  - logicchains 3y ago
    All those people with an S&M kink for writing extra-verbose enterprise Java.
    - sirsinsalot 3y ago
      My safe word is netbeans
redox99 3y ago
The trend has been going from domain specific models to things like ChatGPT.
armchairhacker 3y ago
I think we'll have personal models where you can fine-tune a model to align with your attitudes and writing style. So you can have the assistant better pose as "you" when replying to emails, doing your tasks, writing novels etc. and have one which writes stories and comes up with ideas you'll be interested in. And because most people are a bit narcissistic, they'll prefer a model which acts more like them.
- noisy_boy 3y ago
  Another one that "watches" a movie would and "rates" it for you - so you can decide if you want to watch it for real or not. Bunch of competing personalized models as services - PMaaS - all fighting to impersonate you better. Then one of them gets hacked.
moffkalast 3y ago
There's at least one more group that comes to mind: logical reasoning models for time-critical accurate problem solving in robotic applications, that take natural language + sensor context and output movement commands. A "fly 100 m up and take bird's eye photos of this area" or "take this drill and make 3 holes spaced evenly into this wall" or "drive along the plants in this field and water them all" kind of application.

extasia 3y ago

There's a theory that these LLMs are learning these specialized internal structures anyways. If you could distill _those_, now that would be useful! (cheaper inference, ability to deploy on mobile etc.)

techdragon 3y ago

Small flaw, where does the “naughty” start… we aren’t really doing ourselves any favours with an “I’ll know it when I see it” approach to restricted free form generative models that can be steered by users. The mistake I see in a lot of arguments getting made online about all this is treating the AI/ML models as “whole systems” … we shouldn’t be trying to make very hard to analyse “black box” heart of the model, the bit where we’re turning statistical weights into data… this should not be assumed to do “everything”. If you want a naughty word filter, use a rejection filter from a dictionary, or bolt on a second AI/ML system tuned to catch stuff too similar to what you don’t want… but assuming all functionality can be somehow shoved into the one model is just foolish. We should have some unfettered models out there for us to use as either red-team tools to help generate content we can test safety and sanitising systems on, or otherwise use as desired…

MrGilbert 3y ago
Well, it was just my train of thoughts released into the wild. I have no specialized AI/ML background, so I cannot weight anything I write. I'm just making educated guesses. There are some rules in every society that at least deal with the question about what is appropriate and when, so these rules should serve as a boundary. But I agree, there will always be an area of uncertainty. Maybe we should just stop to think about how to eliminate these. Also, these rules differ from society to society. Maybe the greater mistake is to assume that there is a generative model that fits the needs of every society.
- techdragon 3y ago
  The difference between societal standards and expectations is one of the reasons I think the efforts to build such content filtering/restrictions into the models themselves is a bad idea. Between uncertainty and the variation around the world we would be much better off if we embraced modular pipelines and tooling to make them easier, plug my American culture value filter in after my global human text generation tool, and serve that up to my American customers, plug my French culture value filter into the global model and serve it up to my French customers… much better than pretending there are global standards of morality for written content … there are obviously some standards people agree on in the real world of physical interaction between humans and other humans, animals, private property and so on … but what constitutes offensive text is nowhere near as universal.

api 3y ago

Then there’s the crossover market, like naughty code completion or fantasy knowledge inference.

MrGilbert 3y ago
That one dirty line of code everyone is looking for...
- logicchains 3y ago
  All those people with an S&M kink for writing extra-verbose enterprise Java.
  - sirsinsalot 3y ago
    My safe word is netbeans

redox99 3y ago

The trend has been going from domain specific models to things like ChatGPT.

armchairhacker 3y ago

I think we'll have personal models where you can fine-tune a model to align with your attitudes and writing style. So you can have the assistant better pose as "you" when replying to emails, doing your tasks, writing novels etc. and have one which writes stories and comes up with ideas you'll be interested in. And because most people are a bit narcissistic, they'll prefer a model which acts more like them.

noisy_boy 3y ago
Another one that "watches" a movie would and "rates" it for you - so you can decide if you want to watch it for real or not. Bunch of competing personalized models as services - PMaaS - all fighting to impersonate you better. Then one of them gets hacked.

moffkalast 3y ago

There's at least one more group that comes to mind: logical reasoning models for time-critical accurate problem solving in robotic applications, that take natural language + sensor context and output movement commands. A "fly 100 m up and take bird's eye photos of this area" or "take this drill and make 3 holes spaced evenly into this wall" or "drive along the plants in this field and water them all" kind of application.