10 ms·

Shasta: AI-powered audio recording and editing

139 points by nonoesp 4y ago

muttantt 4y ago
Well, there goes Andrew Mason's startup Descript...
- preommr 4y ago
  inb4 Adobe continues their streak of demonstrating they're incapable of building software and spends another couple of billion to buy out the competition.
  - luckydata 4y ago
    it's not like it's a particularly out there idea, anything logical will have competition. I welcome that a large company with resources builds to market useful tools that I can expect to exist beyond the next round of funding.
- adroitboss 4y ago
  Descript was the first thing I thought of. Since it does audio and video, I am not sure this can be called a competitor. Descript has such a head start with features I don't think Adobe will be able to catch up. I wonder if they made Descript an offer before starting this project. Although, the base problem of editing audio or video with text isn't that difficult. They probably thought it could easily be cloned. The devil is in the details however, and the rabbit hole runs deep.
  - CharlesW 4y ago
    > Descript has such a head start with features I don't think Adobe will be able to catch up. What specifically is Descript doing that Adobe couldn't do relatively easily, or isn't doing already in its other NLE products (Premiere, Audition, Rush, After Effects)?
    - adroitboss 4y ago
      I think everything associated with their lyrebird acquisition. I think it would probably not be as simple as hiring decent engineers to throw at the problem when it comes to Descripts machine learning offerings.
      - steakknife 4y ago
        Adobe already has a big AI research team and Shasta is already incorporating AI features. Check out http://shasta.adobe.com/enhance http://shasta.adobe.com/enhance
nonoesp 4y ago
I discovered today that Adobe is introducing Project Shasta, a web-based AI-powered audio recording and editing tool. You can request access now [1]. It seems their intention is not only to provide AI-based audio editing and recording but also to cover the ground of what Riverside already offers. Remote recording - Recording with others is as easy as sharing a link. Everyone’s audio is recorded in high quality locally, then Project Shasta syncs it back together in the cloud automatically. "Soon to come features" list things we can already find in Descript, such as filler word removal or speech enhancement. And features already available in the beta include microphone checking for optimal quality and microphone distance, AI-powered audio, remote recording with guests, and project templates. I was waiting to see what Adobe's take on AI-based audio editing and recording was, and it seems it's here. At the moment the project is referred to as Shasta; I imagine these capabilities either format a new product offering or are integrated with existing tools like Audition, Premiere, or After Effects. What do you think? [1] https://pages.adobe.com/shasta/request-invite https://pages.adobe.com/shasta/request-invite
- mrandish 4y ago
  As an old-school video editor who learned editing on analog VTRs (tape machines) and progressed to digital non-linear editing, I think the time saving potential of AI-assisted audio editing is substantial. I still think it will be a while before AI is ready to create more than a first draft edit but if it handles 95% of the easy cases that's still a huge time-saving - especially on long-form projects. I spent so, so many hours of soul-sucking, rote drudgery doing things like manual audio ducking back in the day.
  - dilap 4y ago
    Hopefully something like this catches on w/ smaller podcasters. Listening to shit-quality compressed internet audio in a podcast interviewing someone really interesting is just such a shame!
    - jamiek88 4y ago
      Yeah the volume button dance too. Whisper, whisper, mumble, mumble, EAR PIERCING LAUGH, whisper.
inasmuch 4y ago
I know everyone hates Adobe (I certainly share some of the grievances about their pricing structure) but I have been using Photoshop for something like 23 years, working across half that many creative disciplines, and cannot overstate how much I value the consistency, reliability, and neutron-star density of its feature set. You call it bloated, I call it a Swiss army knife. I don't want every tool to be that way, but I love that some are. (The problem with this analogy is that Photoshop actually /is/ the best tool for many of the things it can do, where the Swiss army knife is solidly in jack-of-all-trades territory.) Given this, I am very concerned about Adobe's continued march into web apps. For most of my UI and web design work, I use Figma, and quite like its feature set, but that I cannot use it offline and that I cannot work in a file that requires over 2GB of RAM due to its browser RAM cap is infuriating to me when Photoshop (the supposedly worse program) enables me to manually allocate as much of my system's 64GB of RAM as I please. It's difficult to imagine any web app stably and efficiently growing to Swiss army knife-levels of capability. Not just in terms of performance and resource management, but also GUI, keyboard shortcuts, etc. I've long assumed my CC subscription would die with me, unlike many (most?) of my peers, but if the web-based Photoshop somehow became the primary Photoshop, I may have to cancel my subscription, and then kill myself.
- quasarsunnix 4y ago
  The problem is the bloat. I have CC access via work, but still choose to go with Affinity because of the performance issues. I counted 20 processes running the last time I had PS installed. The apps all take forever to load, drain battery if working remotely, and are far more unstable than in the past. I too grew up in PS in the 90s, but let's not pretend it's not become a dumpster fire - albeit a very feature rich dumpster fire as you stated. I agree that Figma shares many of these same issues performance wise. Truthfully the only creative software I use that I truly enjoy these days is Blender. Hopefully one day the passion of Blender will make it to other areas in the creative software space.
  - inasmuch 4y ago
    I haven't used any of the Affinity apps. But are we talking about the same Photoshop? > I counted 20 processes running the last time I had PS installed I don't check how many background processes are running when I use Photoshop because I don't notice them at all. Almost every feature I use in it feels responsive, especially compared to how it used to be. If you're talking about telemetry, yeah, I hate that too, but I've more or less resigned to it as a necessary evil in all my professional tooling. > The apps all take forever to load Photoshop opens nearly instantly on my M1 Max Macbook Pro, as it did on my M1. It was slower on Intel, but what wasn't. I can open and close Photoshop three times before Discord even loads an empty window. > drain battery if working remotely I've never tried a remote workflow with Photoshop, so can't speak to that at all. Definitely does not seem like what the program is optimized for, and I would think battery drain would be the fault of whatever tool you're using to remote in, no? Or do you mean just working on an unplugged laptop? In which case, I find Electron apps that do one task poorly tend to be less efficient than even heavier desktop apps that 100 things well, Adobe or otherwise. > and are far more unstable than in the past. Sorry, what? I haven't had an Adobe app crash on me in probably eight years. I of course count that as good luck, but to say the current builds are less stable than older versions is just … insane to me. Thinking back to, say, the CS2/3 days, half my time in the program was spent Cmd+S-ing for fear of the inevitable hourly crash and loss of work. Photoshop crashes were a legitimate part of creative culture from like 1999 to 2012. Sorry if I sound like an Adobe shill. We've just clearly had very different experiences with their products of late. Where we are in agreement is that Blender is cool! I'm too much of a neophyte in 3D workflows to assess whether I think it's ultimately well designed (some bits seem to be … others not so much?), but it seems to run very well on the various machines I've tried it on and I have a great deal of respect for the nature of the program and the people behind it.
jannyfer 4y ago
This “Mic Check AI” - I actually just want to try this one feature. I never know what gain I should set on my XLR mic + USB audio interface and I also don’t know how Teams/Webex will do to my signal. Is it compressing my signal? Noise gating it? No idea!
- capableweb 4y ago
  You don't need AI, the one you have in your brain is enough, you just need to teach it what to look for :) There is bunch of introduction material about "gain staging" online, here's just one to get you started: https://www.audiotechnology.com/tutorials/microphones-levels-gain https://www.audiotechnology.com/tutorials/microphones-levels...
  - TheRealPomax 4y ago
    Or, if this is a solved problem with "AI" (a nonsense name for the most part, but if that's what they call it, that's what they call it) then don't and use the tools that get the job done. I have no reason to learn how a variable transmission engine works just to go shopping, and I have no reason to learn what to look for if tools do that for me. That's the whole point of good tooling: to turn conventional "you need to learn X before you can Y" into "this is archaic and no one needs to know this anymore". Or perhaps more in line with Adobe products: no one in their right mind is going to learn how to create uniform blurs with successive blends when they can just hit "filters -> blur -> gaussian blur". The tool made the traditional approach obsolete. Here's to many more things made obsolete.
- samdotdesign 4y ago
  You can try it now at https://shasta.adobe.com/miccheck https://shasta.adobe.com/miccheck
- steakknife 4y ago
  You can use it right now if you make a free account: http://shasta.adobe.com/miccheck http://shasta.adobe.com/miccheck Not mobile friendly yet though.
adzm 4y ago
Really excited to see the transcription based editing. There is a ton of work that this can simplify. I really hope some of this gets integrated into Audition or AE etc as well.
DocJade 4y ago
The audio denoising / filtering they have seems quite impressive, if only I could use it in real-time on teams calls
samdotdesign 4y ago
Hi! I’m Sam, lead designer of Shasta. You can check out the app at https://shasta.adobe.com https://shasta.adobe.com (OP's post links to one of our launch announcement pages). Our enhance speech and mic check features are already available for everyone to try.
- _HMCB_ 4y ago
  Thank you for stopping by and the link. I signed up for access using the OP’s link. Is that no longer required?
  - samdotdesign 4y ago
    Sorry for the confusion! Applying for access is still required to access Shasta, our recording + editing app. We’ve been slowly ramping up in getting people access to it… working hard on making sure it’s a super smooth experience. Buuuut in the meantime, we launched two of our AI features as standalone mini apps that anyone can use for free, and those are called Mic Check and Enhance Speech. They’re fun, try them out and let me know what you think.
    - Tepix 4y ago
      They both require me to sign in.
      - steakknife 4y ago
        They only require a free account, no subscription necessary.
- sullivanmatt 4y ago
  You have a very neat website (https://sam.design https://sam.design). Learned something new today.
  - samdotdesign 4y ago
owlbynight 4y ago
Signed up for early access. We currently use Zencastr for recording our podcast, which has been great, but curious how Shasta will stack up and it'd be nice to not have to pay for two subscriptions.
thomasqbrady 4y ago
Is it still called "Sherlock-ing" if Apple isn't the one doing it? Adobe XD : Sketch :: Shasta : Descript (https://www.descript.com https://www.descript.com), right down to the advertising style
- CharlesW 4y ago
  This is Adobe's Descript killer for sure. I personally don't think of making a copycat product as "Sherlocking". Wikipedia says Sherlocking is "releasing a feature that supplants or obviates third-party software". https://en.wikipedia.org/wiki/Sherlock_(software)#Sherlocked_as_a_term https://en.wikipedia.org/wiki/Sherlock_(software)#Sherlocked...
  - jfengel 4y ago
    Holy cow. I did not know about that. And the pricing seems extremely reasonable.
- ohadron 4y ago
  The Adobe XD :: Figma move didn’t work as well as they’ve hoped in the end…
relwin 4y ago
The denoising works very well. I found that mixing the enhanced audio with about -3 to -6 dB with the original voice audio works very well. (short clip at a brewery) https://youtu.be/M5pdHVoXQHE https://youtu.be/M5pdHVoXQHE I also tested with wind noise (outdoor speaker) and the audio "recovery" is amazing. It rescues bad field recordings that you normally can't use. I'm impressed!
- steakknife 4y ago
  Shasta dev here. We're releasing an update soon to the enhance feature that does exactly this!
Krasnol 4y ago
> While Project Shasta is in alpha, we only support Google Chrome on desktop. I hate that this came up AFTER I logged in...
- Tepix 4y ago
  Right now it shows this before i signed up. Also it only supports english.
- steakknife 4y ago
  Officially it's Google Chrome, but for practical purposes any Chromium-based browser seems to work fine, e.g. Brave, Edge.
jjcm 4y ago
As a Figma employee, there's obviously been a lot of shakeup recently for us. That said, this is one of the areas that actually makes me really excited to partner with Adobe. They've always been at the forefront of AI-esque assisted editing (content aware fill in photoshop was so far ahead of its time). Definitely excited to try this out and to see other AI projects by Adobe to help creatives.
osigurdson 4y ago
It is really cool but I think needs to sync with video. It seems that this would be relatively easy to do.

139 points by nonoesp 4y ago

muttantt 4y ago

Well, there goes Andrew Mason's startup Descript...

preommr 4y ago
inb4 Adobe continues their streak of demonstrating they're incapable of building software and spends another couple of billion to buy out the competition.
- luckydata 4y ago
  it's not like it's a particularly out there idea, anything logical will have competition. I welcome that a large company with resources builds to market useful tools that I can expect to exist beyond the next round of funding.
adroitboss 4y ago
Descript was the first thing I thought of. Since it does audio and video, I am not sure this can be called a competitor. Descript has such a head start with features I don't think Adobe will be able to catch up. I wonder if they made Descript an offer before starting this project. Although, the base problem of editing audio or video with text isn't that difficult. They probably thought it could easily be cloned. The devil is in the details however, and the rabbit hole runs deep.
- CharlesW 4y ago
  > Descript has such a head start with features I don't think Adobe will be able to catch up. What specifically is Descript doing that Adobe couldn't do relatively easily, or isn't doing already in its other NLE products (Premiere, Audition, Rush, After Effects)?
  - adroitboss 4y ago
    I think everything associated with their lyrebird acquisition. I think it would probably not be as simple as hiring decent engineers to throw at the problem when it comes to Descripts machine learning offerings.
    - steakknife 4y ago
      Adobe already has a big AI research team and Shasta is already incorporating AI features. Check out http://shasta.adobe.com/enhance http://shasta.adobe.com/enhance

nonoesp 4y ago

I discovered today that Adobe is introducing Project Shasta, a web-based AI-powered audio recording and editing tool. You can request access now [1]. It seems their intention is not only to provide AI-based audio editing and recording but also to cover the ground of what Riverside already offers. Remote recording - Recording with others is as easy as sharing a link. Everyone’s audio is recorded in high quality locally, then Project Shasta syncs it back together in the cloud automatically. "Soon to come features" list things we can already find in Descript, such as filler word removal or speech enhancement. And features already available in the beta include microphone checking for optimal quality and microphone distance, AI-powered audio, remote recording with guests, and project templates. I was waiting to see what Adobe's take on AI-based audio editing and recording was, and it seems it's here. At the moment the project is referred to as Shasta; I imagine these capabilities either format a new product offering or are integrated with existing tools like Audition, Premiere, or After Effects. What do you think? [1] https://pages.adobe.com/shasta/request-invite https://pages.adobe.com/shasta/request-invite

mrandish 4y ago
As an old-school video editor who learned editing on analog VTRs (tape machines) and progressed to digital non-linear editing, I think the time saving potential of AI-assisted audio editing is substantial. I still think it will be a while before AI is ready to create more than a first draft edit but if it handles 95% of the easy cases that's still a huge time-saving - especially on long-form projects. I spent so, so many hours of soul-sucking, rote drudgery doing things like manual audio ducking back in the day.
- dilap 4y ago
  Hopefully something like this catches on w/ smaller podcasters. Listening to shit-quality compressed internet audio in a podcast interviewing someone really interesting is just such a shame!
  - jamiek88 4y ago
    Yeah the volume button dance too. Whisper, whisper, mumble, mumble, EAR PIERCING LAUGH, whisper.

inasmuch 4y ago

I know everyone hates Adobe (I certainly share some of the grievances about their pricing structure) but I have been using Photoshop for something like 23 years, working across half that many creative disciplines, and cannot overstate how much I value the consistency, reliability, and neutron-star density of its feature set. You call it bloated, I call it a Swiss army knife. I don't want every tool to be that way, but I love that some are. (The problem with this analogy is that Photoshop actually /is/ the best tool for many of the things it can do, where the Swiss army knife is solidly in jack-of-all-trades territory.) Given this, I am very concerned about Adobe's continued march into web apps. For most of my UI and web design work, I use Figma, and quite like its feature set, but that I cannot use it offline and that I cannot work in a file that requires over 2GB of RAM due to its browser RAM cap is infuriating to me when Photoshop (the supposedly worse program) enables me to manually allocate as much of my system's 64GB of RAM as I please. It's difficult to imagine any web app stably and efficiently growing to Swiss army knife-levels of capability. Not just in terms of performance and resource management, but also GUI, keyboard shortcuts, etc. I've long assumed my CC subscription would die with me, unlike many (most?) of my peers, but if the web-based Photoshop somehow became the primary Photoshop, I may have to cancel my subscription, and then kill myself.

quasarsunnix 4y ago
The problem is the bloat. I have CC access via work, but still choose to go with Affinity because of the performance issues. I counted 20 processes running the last time I had PS installed. The apps all take forever to load, drain battery if working remotely, and are far more unstable than in the past. I too grew up in PS in the 90s, but let's not pretend it's not become a dumpster fire - albeit a very feature rich dumpster fire as you stated. I agree that Figma shares many of these same issues performance wise. Truthfully the only creative software I use that I truly enjoy these days is Blender. Hopefully one day the passion of Blender will make it to other areas in the creative software space.
- inasmuch 4y ago
  I haven't used any of the Affinity apps. But are we talking about the same Photoshop? > I counted 20 processes running the last time I had PS installed I don't check how many background processes are running when I use Photoshop because I don't notice them at all. Almost every feature I use in it feels responsive, especially compared to how it used to be. If you're talking about telemetry, yeah, I hate that too, but I've more or less resigned to it as a necessary evil in all my professional tooling. > The apps all take forever to load Photoshop opens nearly instantly on my M1 Max Macbook Pro, as it did on my M1. It was slower on Intel, but what wasn't. I can open and close Photoshop three times before Discord even loads an empty window. > drain battery if working remotely I've never tried a remote workflow with Photoshop, so can't speak to that at all. Definitely does not seem like what the program is optimized for, and I would think battery drain would be the fault of whatever tool you're using to remote in, no? Or do you mean just working on an unplugged laptop? In which case, I find Electron apps that do one task poorly tend to be less efficient than even heavier desktop apps that 100 things well, Adobe or otherwise. > and are far more unstable than in the past. Sorry, what? I haven't had an Adobe app crash on me in probably eight years. I of course count that as good luck, but to say the current builds are less stable than older versions is just … insane to me. Thinking back to, say, the CS2/3 days, half my time in the program was spent Cmd+S-ing for fear of the inevitable hourly crash and loss of work. Photoshop crashes were a legitimate part of creative culture from like 1999 to 2012. Sorry if I sound like an Adobe shill. We've just clearly had very different experiences with their products of late. Where we are in agreement is that Blender is cool! I'm too much of a neophyte in 3D workflows to assess whether I think it's ultimately well designed (some bits seem to be … others not so much?), but it seems to run very well on the various machines I've tried it on and I have a great deal of respect for the nature of the program and the people behind it.

jannyfer 4y ago

This “Mic Check AI” - I actually just want to try this one feature. I never know what gain I should set on my XLR mic + USB audio interface and I also don’t know how Teams/Webex will do to my signal. Is it compressing my signal? Noise gating it? No idea!

capableweb 4y ago
You don't need AI, the one you have in your brain is enough, you just need to teach it what to look for :) There is bunch of introduction material about "gain staging" online, here's just one to get you started: https://www.audiotechnology.com/tutorials/microphones-levels-gain https://www.audiotechnology.com/tutorials/microphones-levels...
- TheRealPomax 4y ago
  Or, if this is a solved problem with "AI" (a nonsense name for the most part, but if that's what they call it, that's what they call it) then don't and use the tools that get the job done. I have no reason to learn how a variable transmission engine works just to go shopping, and I have no reason to learn what to look for if tools do that for me. That's the whole point of good tooling: to turn conventional "you need to learn X before you can Y" into "this is archaic and no one needs to know this anymore". Or perhaps more in line with Adobe products: no one in their right mind is going to learn how to create uniform blurs with successive blends when they can just hit "filters -> blur -> gaussian blur". The tool made the traditional approach obsolete. Here's to many more things made obsolete.
samdotdesign 4y ago
You can try it now at https://shasta.adobe.com/miccheck https://shasta.adobe.com/miccheck
steakknife 4y ago
You can use it right now if you make a free account: http://shasta.adobe.com/miccheck http://shasta.adobe.com/miccheck Not mobile friendly yet though.

adzm 4y ago

Really excited to see the transcription based editing. There is a ton of work that this can simplify. I really hope some of this gets integrated into Audition or AE etc as well.

DocJade 4y ago

The audio denoising / filtering they have seems quite impressive, if only I could use it in real-time on teams calls

samdotdesign 4y ago

Hi! I’m Sam, lead designer of Shasta. You can check out the app at https://shasta.adobe.com https://shasta.adobe.com (OP's post links to one of our launch announcement pages). Our enhance speech and mic check features are already available for everyone to try.

_HMCB_ 4y ago
Thank you for stopping by and the link. I signed up for access using the OP’s link. Is that no longer required?
- samdotdesign 4y ago
  Sorry for the confusion! Applying for access is still required to access Shasta, our recording + editing app. We’ve been slowly ramping up in getting people access to it… working hard on making sure it’s a super smooth experience. Buuuut in the meantime, we launched two of our AI features as standalone mini apps that anyone can use for free, and those are called Mic Check and Enhance Speech. They’re fun, try them out and let me know what you think.
  - Tepix 4y ago
    They both require me to sign in.
    - steakknife 4y ago
      They only require a free account, no subscription necessary.
sullivanmatt 4y ago
You have a very neat website (https://sam.design https://sam.design). Learned something new today.
- samdotdesign 4y ago

owlbynight 4y ago

Signed up for early access. We currently use Zencastr for recording our podcast, which has been great, but curious how Shasta will stack up and it'd be nice to not have to pay for two subscriptions.

thomasqbrady 4y ago

Is it still called "Sherlock-ing" if Apple isn't the one doing it? Adobe XD : Sketch :: Shasta : Descript (https://www.descript.com https://www.descript.com), right down to the advertising style

CharlesW 4y ago
This is Adobe's Descript killer for sure. I personally don't think of making a copycat product as "Sherlocking". Wikipedia says Sherlocking is "releasing a feature that supplants or obviates third-party software". https://en.wikipedia.org/wiki/Sherlock_(software)#Sherlocked_as_a_term https://en.wikipedia.org/wiki/Sherlock_(software)#Sherlocked...
- jfengel 4y ago
  Holy cow. I did not know about that. And the pricing seems extremely reasonable.
ohadron 4y ago
The Adobe XD :: Figma move didn’t work as well as they’ve hoped in the end…

relwin 4y ago

The denoising works very well. I found that mixing the enhanced audio with about -3 to -6 dB with the original voice audio works very well. (short clip at a brewery) https://youtu.be/M5pdHVoXQHE https://youtu.be/M5pdHVoXQHE I also tested with wind noise (outdoor speaker) and the audio "recovery" is amazing. It rescues bad field recordings that you normally can't use. I'm impressed!

steakknife 4y ago
Shasta dev here. We're releasing an update soon to the enhance feature that does exactly this!

Krasnol 4y ago

> While Project Shasta is in alpha, we only support Google Chrome on desktop. I hate that this came up AFTER I logged in...

Tepix 4y ago
Right now it shows this before i signed up. Also it only supports english.
steakknife 4y ago
Officially it's Google Chrome, but for practical purposes any Chromium-based browser seems to work fine, e.g. Brave, Edge.

jjcm 4y ago

As a Figma employee, there's obviously been a lot of shakeup recently for us. That said, this is one of the areas that actually makes me really excited to partner with Adobe. They've always been at the forefront of AI-esque assisted editing (content aware fill in photoshop was so far ahead of its time). Definitely excited to try this out and to see other AI projects by Adobe to help creatives.

osigurdson 4y ago

It is really cool but I think needs to sync with video. It seems that this would be relatively easy to do.