That would be the logical solution, but "The javascript is dynamically created and hosted on S3. The keys, etc are safely on a server that runs to generate the

That would be the logical solution, but "The javascript is dynamically created and hosted on S3. The keys, etc are safely on a server that runs to generate the file" sounds a lot like they have a server where they build a .js file that is capable of querying Slack, then store that file on S3 for the website to pull whenever someone registers. In that case, the client is pulling a .js file that probably has the Slack API key in it.