otterk10

6 ms·

1. 1.▲
   Show HN: Realistic Synthetic Conversations for Testing LLMs(github.com)

   2 points by otterk10 1y ago|1 comments
2. 2.▲
   Show HN: Product Analytics for LLMs(github.com)

   3 points by otterk10 1y ago|0 comments
3. 3.▲
   Show HN: Open-Source Conversational Analytics(github.com)

   2 points by otterk10 1y ago|0 comments
4. 4.▲
   by otterk10 2y ago

   I've spent the past 2 years as a fractional AI engineer, and this totally resonates with the advice I've given clients. They usually hire me after they've gained some initial traction and are stuck on how to improve their LLM
5. 5.▲
   Show HN: GPT Fine-Tuning Notebook Boosts Classification Accuracy from 69% to 94%(github.com)

   1 points by otterk10 2y ago|1 comments
6. 6.▲
   by otterk10 2y ago

   OpenAI is offering free fine-tuning until September 23rd! To help people get started, I've created an end-to-end example showing how to fine-tune GPT-4o mini to boost the accuracy of classifying customer support tickets from 69% to 94%
7. 7.▲
   by otterk10 7y ago

   Thanks for the feedback! I totally agree about observational studies being suggestive but not replacing A/B tests - that’s why the main use case I listed in the blog (and how current customers have used the product so far) is “prioriti
8. 8.▲
   by otterk10 7y ago

   Thanks for the feedback! I totally agree about observational studies being suggestive but not replacing A/B tests - that’s why the main use case I listed in the blog (and how current customers have used the product so far) is “prioriti
9. 9.▲
   by otterk10 7y ago

   Thanks! Yes, the concerns you mentioned would also apply to PCA. What we've actually done to help alleviate this is a union of components from y-aware[1] and normal PCA to capture variables that are correlated to both the dependent var
10. 10.▲
    by otterk10 7y ago

    We didn’t explore causal graphs because doing so would require manually creating a causal graph for each relationship that you wish to explore. Our goal was to create an automated approach that could provide an estimate of the treatment eff
11. 11.▲
    by otterk10 7y ago

    Yes, you're correct that the underlying algorithm used is very close to OLS. What allows the regression to provide an estimate for average treatment effects is how it is structured. Namely, adding in pre-treatment confounders as well
12. 12.▲
    by otterk10 7y ago

    You can definitely do an AB test to verify the causal relationship - in fact, that's the preferred method! Our platform is for situations where you didn't run an A/B test - either because you can't run as many as you&#x2
13. 13.▲
    by otterk10 7y ago

    Our analysis runs over our user’s customer data (usually collected through either a tag manager or a CDP such as Segment), which is a few petabytes of data for some of our larger customers. The reason for using Spark is to quickly transform
14. 14.▲
    by otterk10 7y ago

    Thanks! You're correct in surmising that our approach was heavily influenced by Judea Pearl's research. And yes, the timing of the blog post isn't a coincidence, we actually filed a patent last week :)
15. 15.▲
    by otterk10 7y ago

    Good to hear! In my experience, BigQuery ML (and other cloud ml products) is great for creating basic models out of the box, but don't provide a ton of flexibility for non-standard ML use-cases. For example, our approach to causal anal
16. 16.▲
    by otterk10 7y ago

    Scott here from ClearBrain - the ML engineer who built the underlying model behind our causal analytics platform. We’re really excited to release this feature after months of R&D. Many of our customers want to understand the causal impa
17. 17.▲
    by otterk10 9y ago

    It’s fascinating to read about the power that Facebook’s advertising apis provide. I’ve always wondered whether Facebook ads are just based off of facebook’s internal data or whether businesses can combine facebook’s data with their own int