7 ms·

Launch HN: Trellis (YC W24) – AI-powered workflows for unstructured data

234 points by macklinkachorn 2y ago

Hey HN — We're Jacky and Mac from Trellis (https://runtrellis.com/ https://runtrellis.com/). We’re building AI-powered ETL for unstructured data. Trellis transforms phone calls, PDFs, and chats into structured SQL format based on any schema you define in natural language. This helps data and ops teams automate manual data entry and run SQL queries on messy data. There’s a demo video at https://www.youtube.com/watch?v=ib3mRh2tnSo https://www.youtube.com/watch?v=ib3mRh2tnSo and a sandbox to try out (no sign-in required!) at https://demo.runtrellis.com/ https://demo.runtrellis.com/. An interesting historical archive of unstructured data we thought it would be interesting to run Trellis on top of are old Enron emails which famously took months to review. We’ve created a showcase demo here: https://demo.runtrellis.com/showcase/enron-email-analysis https://demo.runtrellis.com/showcase/enron-email-analysis, with some documentation here: https://docs.runtrellis.com/docs/example-email-analytics https://docs.runtrellis.com/docs/example-email-analytics. Why we built this: At the Stanford AI lab where we met, we collaborated with many F500 data teams (including Amazon, Meta, and Standard Chartered), and repeatedly saw the same problem: 80% of enterprise data is unstructured, and traditional platforms can’t handle it. For example, a major commercial bank I work with couldn’t improve credit risk models because critical data was stuck in PDFs and emails. We realized that our research from the AI lab could be turned into a solution with an abstraction layer that works as well for financial underwriting as it does for analysis of call center transcripts: an AI-powered ETL that takes in any unstructured data source and turns it into a schematically correct table. Some interesting technical challenges we had to tackle along the way: (1) Supporting complex documents out of the box: We use LLM-based map-reduce to handle long documents and vision models for table and layout extraction. (2) Model Routing: We select the best model for each transformation to optimize cost and speed. For instance, in data extraction tasks, we could leverage simpler fine-tuned models that are specialized in returning structured JSONs of financial tables. (3) Data Validation and Schema Guarantees: We ensure accuracy with reference links and anomaly detection. After launching Trellis, we’ve seen diverse use cases, especially in legacy industries where PDFs are treated as APIs. For example, financial services companies need to process complex documents like bonds and credit ratings into a structured format, and need to speed up underwriting and enable pass-through loan processing. Customer support and back-office operations need to accelerate onboarding by mapping documents across different schema and ERP systems, and ensure support agents follow SOPs (security questions, compliance disclosures, etc.). And many companies today want data preprocessing in ETL pipelines and data ingestion for RAG. We’d love your feedback! Try it out at https://demo.runtrellis.com/ https://demo.runtrellis.com/. To save and track your large data transformations, you can visit our dashboard and create an account at https://dashboard.runtrellis.com/ https://dashboard.runtrellis.com/. If you’re interested in integrating with our APIs, our quick start docs are here: https://docs.runtrellis.com/docs/getting-started https://docs.runtrellis.com/docs/getting-started. If you have any specific use cases in mind, we’d be happy to do a custom integration and onboarding—anything for HN. :) Excited to hear about your experience wrangling with unstructured data in the past, workflows you want to automate, and what data integration you would like to see.

localfirst 2y ago
looks like more solutions looking for a problem that can be solved at the vendor level
- hamsterbooster 2y ago
  In many use cases, like flagging documents for compliance issues or processing customer emails, it's challenging to manage this at the vendor level because end customers want the ability to apply business logic and run different analyses. For data ingestion and mapping, I agree that in an ideal world, we would all have first-party API integrations. However, many industries still rely on PDFs and CSV files to transfer data.
  - localfirst 2y ago
    perhaps im misunderstanding the product offering here, isn't this just throwing PDFs (which also has unparsable content like formulas, symbols and large tables even with OCR) on an LLM with structured outputs and running SQL queries? isn't it obvious that this would be a problem that will eventually be solved by the LLM providers themselves including the ability to flag and apply business logic on top of the structured outputs? Like I'm not sure if this is well known but LLM providers have huge pressure to turn a profit and will not hesitate to copy any downstream wrappers out of existence rather than acquiring them outright. Its like selling wrapping tape around the shovel handle for better grip and expecting the shovel makers to not release their new shovels with it in the near future. The shovel makers don't even need to do any market research or product development and the buyers don't have any incentive to seek or pay a dedicated third party for what their vendors will release for free and at lower costs if that makes sense.
    - constantinum 2y ago
      This misunderstanding is valid. Another example is why subscription/recurring billing software exists when payment gateways can solve this problem themselves. The elephant in the room is the complexities involved down the funnel that need very specific focus/solutions.

localfirst 2y ago

looks like more solutions looking for a problem that can be solved at the vendor level

hamsterbooster 2y ago
In many use cases, like flagging documents for compliance issues or processing customer emails, it's challenging to manage this at the vendor level because end customers want the ability to apply business logic and run different analyses. For data ingestion and mapping, I agree that in an ideal world, we would all have first-party API integrations. However, many industries still rely on PDFs and CSV files to transfer data.
- localfirst 2y ago
  perhaps im misunderstanding the product offering here, isn't this just throwing PDFs (which also has unparsable content like formulas, symbols and large tables even with OCR) on an LLM with structured outputs and running SQL queries? isn't it obvious that this would be a problem that will eventually be solved by the LLM providers themselves including the ability to flag and apply business logic on top of the structured outputs? Like I'm not sure if this is well known but LLM providers have huge pressure to turn a profit and will not hesitate to copy any downstream wrappers out of existence rather than acquiring them outright. Its like selling wrapping tape around the shovel handle for better grip and expecting the shovel makers to not release their new shovels with it in the near future. The shovel makers don't even need to do any market research or product development and the buyers don't have any incentive to seek or pay a dedicated third party for what their vendors will release for free and at lower costs if that makes sense.
  - constantinum 2y ago
    This misunderstanding is valid. Another example is why subscription/recurring billing software exists when payment gateways can solve this problem themselves. The elephant in the room is the complexities involved down the funnel that need very specific focus/solutions.