joewferrara

6 ms·

1.
PrivacyBench: An open benchmark for de-identifying text that scores synthesis(huggingface.co)
4 points by joewferrara 3mo ago|0 comments
2.
by joewferrara 1y ago
I think you mean x*x = -1, for which I agree with your point.
3.
by joewferrara 2y ago
right now we support txt, csv, tsv, docx, xlsx, pdf, png, tif, tiff, jpg, and jpeg filetypes. we support either local files or aws s3 as the document store where the files are read from. so we can work with your messy files in s3 as they ar
4.
by joewferrara 2y ago
that's a good question. for this use case, we have a custom models feature where the user can define an entity type (types) they're interested in and then a custom ner model is trained to identify the specified type. for this, a c
5.
by joewferrara 2y ago
The built in static, I'm not sure if it's purposeful or not, but I like it. If it is purposeful, it's a nice touch.
6.
by joewferrara 2y ago
kevin buzzard is a very good(usually math) expositor with a bent towards programming. love it
7.
by joewferrara 3y ago
I tried out Amazon Bedrock, and used Tonic Validate to do a head to head comparison of very simple RAG system's built using embedding and text models available in Amazon Bedrock. I compared Amazon Titan's embedding and text models
8.
Validating the RAG Performance of Amazon Titan vs. Cohere Using Amazon Bedrock(tonic.ai)
1 points by joewferrara 3y ago|1 comments
9.
Open Language Model: OLMo(allenai.org)
1 points by joewferrara 3y ago|0 comments
10.
by joewferrara 3y ago
Is the staff (which includes you) is all volunteers, there’s so reason to expect or want people to have a “get shit done” attitude. You should all appreciate that you’re putting in any time together towards a shared goal.
11.
by joewferrara 3y ago
Tonic's RAG evaluation platform is Tonic Validate, which has open source RAG metrics https://github.com/TonicAI/tonic_validate , and a web app for tracking and monitoring RAG performance https://www.toni
12.
Tonic.ai and LlamaIndex join forces to help developers build RAG systems(tonic.ai)
1 points by joewferrara 3y ago|1 comments
13.
by joewferrara 3y ago
They show that a decoder only transformer (which gpts are) are rnns with infinite hidden state size. Infinite hidden state size is a pretty strong thing! Sounds interesting to me.
14.
by joewferrara 3y ago
Where we are today is a world where people do not generally worry about nuclear bombs being dropped. So seems like a pretty good outcome in that example.
15.
by joewferrara 3y ago
Yeah but for non profits it’s worse than for profits and that’s the point. At for profit companies everyone (the board included) is trying to make money. That’s the end goal, and most decisions go towards that goal so even if employees don’
16.
by joewferrara 3y ago
Every prime is odd and 2 is the oddest of them all!
17.
by joewferrara 3y ago
Amazing how simple the proof is. 37 is my new favorite prime number, haha.
18.
by joewferrara 3y ago
The interesting point/question in this is whether Open AI will invest in hardware to build a ChatGPT interface in the future that allows for less friction of use than just “being an app on an IPhone”. I think it’s amazing and would be
19.
by joewferrara 3y ago
Gauss died in 1855 and his work still influences modern mathematics. Someone told me while in grad school that in number theory we’re still figuring out stuff that Gauss already knew.
20.
by joewferrara 3y ago
I enjoy Sharp Tech (Stratechery’s podcast), though unfortunately it’s mostly paywalled but has free episodes. It’s good for the business side of big tech and the two people on it, Ben Thompson and Andrew Sharp keep it pretty light and fun,
21.
by joewferrara 3y ago
Author here. The linked Jupyter notebook shows how to use tvalmetrics, a RAG metrics Python package, to evaluate RAG query engines built using LlamaIndex on a dataset of Paul Graham essays about startup founders. The RAG parameters chunk si
22.
by joewferrara 3y ago
It's true that there are not a lot of datasets for benchmarking RAG. RAG applications are so tailored to the specific data being used as well as the use case, that a benchmark dataset is not useful across different RAG applications. Th
23.
by joewferrara 3y ago
tvalmetrics is similar to ragas for sure, and we really like ragas. tvalmetrics has structural differences as well as differences in the specific metrics when compared to ragas. With regards to the metrics, we have an end to end metric that
24.
by joewferrara 3y ago
Joe here. This is a good question, and always a challenge when using a test set to measure the performance of an LLM/AI/ML application. The answer is to make the test dataset consist of questions that come from users of your app -
25.
by joewferrara 3y ago
Joe here. It's difficult to evaluate natural language responses that come from LLM applications - there are not hard metrics to measure performance like there are in say supervised machine learning tasks. For RAG, you have the response
26.
by joewferrara 3y ago
To counter point, the Mona Lisa makes so much money from people going to see it, it’s not surprising that some money could be found to do this research. And the write up on the composition of the Mona Lisa paint would get exponentially many
27.
by joewferrara 3y ago
Learn complex analysis contour integrals! There super fun for doing physics integrals. I'm not sure if these are too basic for what your asking about or not, but thought I'd mention them. Junjiro Noguchi's Introduction to Com
28.
Will Computers Outsmart Mathematicians? (2021) [video](youtube.com)
2 points by joewferrara 3y ago|1 comments
29.
by joewferrara 3y ago
It's kind of the opposite scenario (a mathematical result that people are widely skeptical of, but that could be correct), but this reminded me, I wonder what's going on with Mochizuki's proof of the ABC conjecture.
30.
by joewferrara 3y ago
Best advice for quality of life for a PhD student living off the PhD stipend - live in a low cost of living area. In math (what I got a PhD in), you get paid the same amount (about $20,000 a year when I started in 2013) if you live the SF B

More ›