joshhart

9 ms·

31.
by joshhart 3y ago
You are totally right - both more and better matters. There are many good papers on the importance of data quality, Textbooks Are All You Need is one that comes to mind - https://arxiv.org/abs/2306.11644
32.
by joshhart 3y ago
From a modeling perspective, I am impressed with the effects of training on 2T tokens rather than 1T. Seems like this was able to get LLAMA v2 7b param models equivalent to LLAMA v1's 13b performance, and the 13b similar to 30b. I wond
33.
by joshhart 3y ago
One of the “words” is a stop token, which represents the ending of text. So you can say the thing that maximizes coherence is to stop right then.
34.
by joshhart 3y ago
I'm the director of engineering for Databricks' model serving product. It is serverless, meaning it autoscales to & from zero. If you are a Databricks customer or willing to be, you can reach out about enrolling in the GPU pre
35.
by joshhart 4y ago
7B is a sweet spot where you can do something with limited resources both for training and inference. Going beyond that you spill out of an A100 without tricks. We will continue iterating on this with other models.
36.
by joshhart 4y ago
And interestingly, the value of binance coin (BNB-USD) is actually up quite a bit YTD. https://finance.yahoo.com/quote/BNB-USD/
37.
by joshhart 4y ago
The 3 hours is the instruction fine-tuning. The base foundational model is GPT-J which was already provided by Eleuther-AI and has been around for a couple of years. Note: I work at Databricks and am familiar with this project but didn&#x27
38.
by joshhart 4y ago
I don't have a good solution for this. The reality is a lot of people can't be the quarterback to build new shiny things so promoting those who can is reasonable. But at the same time, it does incentivize this path more than neede
39.
by joshhart 4y ago
Congrats Stefan, from someone working at a competitor - always good to see more tools for production ML.
40.
Databricks Announces General Availability of Model Serving(databricks.com)
26 points by joshhart 4y ago|5 comments
41.
by joshhart 4y ago
Many in the investment community are touting ChatGPT as a Google-killer. I don't think it's the case, but I do think the expectation is that Sundar will drop everything to personally treat it like an existential threat. If it were
42.
by joshhart 4y ago
Sorry to hear that. I’m a parent and that sounds scary and awful
43.
by joshhart 4y ago
I was an employee of LinkedIn at the time that Dylan was an intern. His talent and drive were really something special. He wasn’t content to just do the project he was given. He was making demos and presenting at company all hands. I rememb
44.
by joshhart 4y ago
[Disclaimer: Databricks employee] There's also a lot of value in DBSQL, Unity catalog (data management), and serverless for autoscaling that can all save money in terms of just running raw Spark. But if you want to operate Spark yourse
45.
by joshhart 4y ago
The median household and the median homebuyer aren’t the same. Median household includes a whole swath of people who need to live with relatives or packed in with a large number of roommates. So the median homebuyer already represents proba
46.
by joshhart 5y ago
60M barrels is nothing. It only gets interesting if they release hundreds of millions.
47.
by joshhart 5y ago
Nobody remembers the MangoDB spoof where they made fun of mongodb reliability by writing to /dev/null ? https://github.com/dcramer/mangodb If this project is real they chose the worst name possible.
48.
by joshhart 5y ago
I’ve heard of culture problems from a few folks and similar comments are here today. My friend who works in Quantum Computing says the joke is “I regretti joining this company”
49.
by joshhart 5y ago
Why not have certified users who have a clear history of buying things on the platform that can provide "trusted" reviews? Certainly there might still be a problem with sellers spamming people to ask for reviews in money but I thi
50.
by joshhart 5y ago
Another idea I think is interesting. If we reach that level of usability and tech with an iPad, then it seems plausible that we could build tech where you just need to bring an iPhone near a monitor, keyboard, and mouse and you instantly ha
51.
by joshhart 6y ago
I think we need more funding outside of large tech companies and OpenAI for these kinds of things. I wonder if there is a way to crowdsource donations to rent the hardware to train big versions of these things in an open manner.
52.
by joshhart 6y ago
would wait for people who are higher risk
53.
by joshhart 8y ago
Completely agreed. If we assume the lifetime of a system is ~30 years, then the amount of labor per coal worker is between 2x and 3x, not 79x. I also note they used the terminology "coal worker" which probably does not include all
54.
by joshhart 8y ago
I went through the same a few years ago with hospice. She had emphysema and real trouble breathing. She was mentally checked out after a few days in the hospital and opted for hospice. They started morphine and for a couple of hours she was
55.
by joshhart 9y ago
According to Wikipedia the worldwide consumption of electricity is about 21k Tera watt hours. That's 21,000,000 gw so it's .001% right? Also I think cost is now about a quarter per watt
56.
by joshhart 9y ago
I'm not so sure. I bet the suburbs will spread out into rural areas when people don't have to drive and can just hang out and watch Netflix on the way to work.
57.
by joshhart 9y ago
My company, LinkedIn, has started an in-company program called Reach to do exactly that. https://careers.linkedin.com/reach Rather than hire a traditional Stanford CS graduate, we find people who have self-taught to some ba
58.
by joshhart 9y ago
Need to account for both age, location, and salary as variables in the model, then see what's left over for occupation. We know that occupation is correlated with salary which is correlated with divorce rates. Need to account for this!
59.
by joshhart 9y ago
Good article. It is important to point out that GWT comes with more than just a Java -> Javascript compiler. It is also a set of tools and components to make web development feel more native. Over time, emphasis on these has lessened but
60.
by joshhart 9y ago
Java people just remember PECS - Producer Extends, Consumer Super

More ›