11 ms·
Okay, this is awesome (and easy to miss with a just a cursory skim): "5.3 Delicious Chocolate Chip Cookies Vizier is also used to solve complex black–box o
by kevlar1818 8y ago
Okay, this is awesome (and easy to miss with a just a cursory skim):
"5.3 Delicious Chocolate Chip Cookies
Vizier is also used to solve complex black–box optimization
problems arising from physical design or logistical problems.
Here we present an example that highlights some additional
capabilities of the system: finding the most delicious chocolate chip cookie recipe from a parameterized space of recipes...
We provided recipes to contractors responsible for providing desserts for Google employees. The head chefs among
the contractors were given discretion to alter parameters if
(and only if) they strongly believed it to be necessary, but
would carefully note what alterations were made. The cookies
were baked, and distributed to the cafes for taste–testing.
Cafe goers tasted the cookies and provided feedback via a
survey. Survey results were aggregated and the results were
sent back to Vizier. The “machine learning cookies” were
provided about twice a week over several weeks...
The cookies improved significantly over time; later rounds
were extremely well-rated and, in the authors’ opinions, delicious."
- rnjs 8y agoI noted this section when I read the paper as well... This is cute, but "in the authors' opinions, delicious" does not contribute anything to scientific research. Yes, you can use black-box optimization for cookie recipes. No, you should not make any sort of performance claims nor talk about "significant" improvement unless you back it up.
- marksomnian 8y agoThat's the authors' sense of humour. The more relevant part for science is "extremely well-rated".
- ec109685 8y agoTheir back up: later rounds were extremely well-rated
- rnjs 8y agoHow about "My brand new image recognition architecture performs extremely well on benchmarks" But it's worse than the last 100 SOTA models...
- pm90 8y ago> in the authors' opinions, delicious Can there be/Is there an objective way to measure deliciousness? If not what can they really say?
- dekhn 8y agoTo the extent that deliciousness is a physiologically measurable phenomenon, or there are proxy measures that correlated with it, yes, there is an objective way. You give large numbers of people cookies and have them rate it. Tastes vary but generally if you make something with umami, crunch, fat, and sugar, people will say it is delicious.
- rnjs 8y ago> Cafe goers tasted the cookies and provided feedback via a survey. Survey results were aggregated and the results were sent back to Vizier. This, from the paper, is a great start on measuring deliciousness. But if you go to that effort, why not provide a benchmark? Such as the existing chocolate chip cookie recipe or the average scores from 10 random chocolate chip cookie recipes? As a reader, I might suspect that performance was not that great compared to baselines but that the authors were able to mask it by making this qualitative claim ("the cookies were delicious") -- which I'm sure was still true!
- AstralStorm 8y agoThis technique is known as anchoring and should be used alongside a descriptive scale when making subjective comparisons.
- tripzilch 8y agoYep. There's a lot of pre-existing science that this research ignored ...
- tripzilch 8y agoBut it really is not a very great start on measuring deliciousness. Like said elsewhere in this thread, if after a few batches the algorithm optimizes into a local maximum gully that some people don't like, but others really love, then the ones that dislike it will stop taking the cookies and thus stop rating them. But does that mean they're less delicious? Nobody knows! Because certainly the first and foremost great start on measuring deliciousness, is doing research on what is deliciousness, and are there any good methods to measure deliciousness that can be compared with other literature and research, and if that fails, either measure something else or defend why you chose a target metric that can't be measured properly. The latter being valuable if your way of measuring is novel and you think it's better than what has been used so far in the field. But you gotta provide a good argument for that, and the method used in this experiment wasn't particularly novel, simply flawed. I mean you can be all giggly-googly about it, but food science is a thing.
- halbritt 8y agoDid they post the recipe?
- halbritt 8y agoThey didn't. I am very disappointed. I, for one, welcome our new robot overlords and also want to try their chocolate chip cookies.
- joshuamorton 8y agoHere you go: https://www.blog.google/technology/research/makings-smart-cookie/ https://www.blog.google/technology/research/makings-smart-co...
- pliny 8y agoThese are not the vizier cookies.
- joshuamorton 8y agoOh darn you're right (to clarify, I'm pretty sure those are vizier cookies, just not the vizier cookies). I have the recipe sitting on my counter at home. I thought it was public, but it doesn't' seem to exist online anywhere.
- deleted 8y ago[deleted]
- sitkack 8y agoThe secret formula is love!
- pliny 8y agohttps://research.google.com/pubs/archive/46507.pdf https://research.google.com/pubs/archive/46507.pdf
- dekhn 8y agoIt's great that they published this - I wasn't sure if I could share this externally. I see they left the ingredient that really improved the scores unexpectedly out, presumbly for proprietary reasons.
- hn_throwaway_99 8y agoWeed?
- sp332 8y agoYou're not thinking of these gluten-free cardamom cookies? https://www.blog.google/technology/research/makings-smart-cookie/ https://www.blog.google/technology/research/makings-smart-co...
- patejam 8y agoThat ingredient list is missing the "secret ingredient" that your parent post is talking about, so nah.
- Itsdijital 8y agoThey most likely are holding it back for a bigger media event where they will show them off and every tech blog on the internet will run a story about them.
- joefkelley 8y agoAre you talking about orange extract? or cayenne pepper? The post is here: https://ai.google/research/pubs/pub46507 https://ai.google/research/pubs/pub46507 and direct link to paper is here: https://storage.googleapis.com/pub-tools-public-publication-data/pdf/f52ac33bc9d1adecd3a8037a7009b185fd934f0e.pdf https://storage.googleapis.com/pub-tools-public-publication-... And the recipe I found internally is "The Real California Cookie" in that paper: Bake at 163C (325◦ F) until brown: • 167 grams of all-purpose flour. • 245 grams of milk chocolate chips. • 0.60 tsp. baking soda. • 0.50 tsp. salt. • 0.125 tsp. cayenne pepper. • 127 grams of sugar (31% medium brown, 69% white). • 25.7 grams of egg. • 81.3 grams of butter. • 0.12 tsp. orange extract. • 0.75 tsp. vanilla extract.
- jcims 8y agoGood application for Ben Krasnow's Cookie Perfection machine - http://benkrasnow.blogspot.com/2014/01/cookie-perfection-machine.html http://benkrasnow.blogspot.com/2014/01/cookie-perfection-mac...
- meinstream 8y agoI worked for Google at the time they went through the cookie iterations in the cafes. Personally I disliked all the cookies that ever came out of this project, mainly because they contained spices, I heard the same thing from many others. After you got handed a cookie you were asked to fill out a feedback form on a tablet, which almost always is annoying. But more importantly, the process of optimizing the ingredients left out many parts of what makes food enjoyable, temperature and texture for example. I know this was meant to be a fun showcase of ML but to me this is still my favourite example of explaining misuse of ML technology where a simpler statistical model supported by expert opinion would have outperformed the taste of the cookies. Eg, any cookie expert might know that the number of people who like spicy cookies is only a small subset of all cookie lovers.
- joshuamorton 8y agoIf I recall, the spices were an intentional choice of a kind of pathological input to the optimizer, and iirc it did what you might expect (minimize the amount of cayenne to the smallest in the acceptable range).
- shmageggy 8y agoSounds like you got the less tasty side of the explore/exploit tradeoff.
- oh-kumudo 8y agoI think this is exactly the case how ML could go wrong. I would assume the people who hate the cookies probably will not bother to submit a survey and try again. The sample might be biased and not representative of the general population, thus overfitting, performs badly on a boarder test set.
- whazor 8y agoOur dystopian future where we will be forced fed cookies in order for accurate optimization.
- 8y ago
- iamgopal 8y agoWhy I have an idea about receipe optimisation as a service ?
- dallen33 8y agoSeems to be missing the key ingredient of the best tasting chocolate chip cookies: espresso powder.
- sannee 8y agoIs this possibly related to the cookie machine by Ben Krasnow from several years ago? [1] [1] https://www.youtube.com/watch?v=8YEdHjGMeho https://www.youtube.com/watch?v=8YEdHjGMeho