3 ms·
I've been wanting to make my students do, as a side project, a free-food searcher for Stanford. Wouldn't have to be hard at all: Download (or even wget) the CS/
by danso 10y ago
I've been wanting to make my students do, as a side project, a free-food searcher for Stanford. Wouldn't have to be hard at all: Download (or even wget) the CS/Engineering calendar, do the minimal scraping needed to figure out if the words "food", "meal", "dinner", "will be served" "refreshments" appear anywhere in the text, then return True.
That's the main thing, the rest is just gravy: returning the event serialized as JSON/CSV (event name, date/time, URL) to be used in a web app or notification system -- so a simple web scraper can lead to exploring web dev (even just a simple Flask app) or fun APIs like AWS SNS and Twilio. You could even fit in a good "cache invalidation is a hard problem" lesson.
I never get around to assigning it because most people don't think of it as "serious work". Also, I'm afraid the CS dept. will obfuscate their calendar if random people start showing to things for free food. But I keep telling students, the only/best way to learn coding is to do something that directly affects your life or your bottom-line. It's the best way to put constraints on a project, i.e. think of things as the MVP that improves your life.
I learned Ruby and Ruby on Rails much faster than I had any right to, when my new job in NYC required it. I practiced not by writing Ruby on the job, but writing Ruby to scrape Craigslist apartment listings and feeding them into a spreadsheet.
I've thought about creating "personal data/programming projects"...in which the data comes from the student. Such as the SQLite that stores their Chrome/Firefox/Safari history. Or the parseable HTML dump that Facebook gives you when you request your records. Or your Twitter data dump. Or your Google search history.
But I've been reticent to do this. Partly because it's not a guarantee that every student uses Google or Facebook or has an Instagram. And partly because I'm deeply paranoid students (especially those who are novices about programming and operating systems) will accidentally upload or otherwise expose this sensitive data dump.
- captn3m0 10y agoSomeone else in this thread did it at MIT: https://news.ycombinator.com/item?id=13809108 https://news.ycombinator.com/item?id=13809108
- jerf 10y agoAll you need to do is drop a hint and put the idea in their head, however obliquely you want to do it. When I was at University, there was a public directory of employees and students available internally. You could opt-out, but very few people did. You could search by last name, but you had to use at least two letters and the responses cut off after 50 or something like that, although those 50 were in order. When I was in my senior year, somebody breathlessly ran to the student newspaper to loudly proclaim that they had scraped it and had a full copy of the database. I took a little impromptu survey at my next computer science class... "Who here hasn't at least mentally designed how to scrape the student database?" The answer was, nobody. The student body at large was at least modestly impressed but that guy won no points with the computer science students that day. Oh, and nobody had to "suggest" this to us.
- tedmiston 10y ago> I practiced not by writing Ruby on the job, but writing Ruby to scrape Craigslist apartment listings and feeding them into a spreadsheet. On a related note, PadMapper [1] is an amazing scraper / browser for saving time apartment hunting on Craigslist. [1]: https://www.padmapper.com https://www.padmapper.com
- SmellTheGlove 10y agoGravy, you say? > I never get around to assigning it because most people don't think of it as "serious work". Also, I'm afraid the CS dept. will obfuscate their calendar if random people start showing to things for free food. It's a practical problem to solve, and the calendar obfuscation angle is a real world issue. If the information itself is valuable and you're trying to aggregate it, it's naturally going to become a moving target. I think it's a solid lesson.
- Abundnce10 10y agoI learned Ruby and Ruby on Rails much faster than I had any right to, when my new job in NYC required it. I practiced not by writing Ruby on the job, but writing Ruby to scrape Craigslist apartment listings and feeding them into a spreadsheet. I started teaching myself computer programming in 2011/2012: I bought a bunch of Head First books published by O'Reilly, watched the entry-level CS video lectures that Stanford provided for free, participated in the first few Udacity courses, and I tried a couple Coursera classes. But I think I learned the most (in the shortest amount of time) when I went through your online book, 'The Bastards Book of Ruby'[0] I loved the real-world examples you used: fetching Tweets through the Twitter API, web scraping with Nokogiri, and manipulating images with ImageMagick/RMagick are a few that stick out to me. I'm sure you're doing great things at Stanford but I'm also confident that you could come out with a new book (covering the topics you spoke of above) that would help motivate people who are on the fence on whether or not they should start/continue learning computer programming. I'd be happy to collaborate on something but I would argue that Python would be a better language to use than Ruby. Here are some of the topics I'd like to share with people (which I've found to be useful in my career/side-projects): reading data from CSV/Excel files (xlrd[1]), fetching data from APIs (requests[2]), web scraping (BeautifulSoup[3], Selenium[4]), connecting to a SQL database within a Python script (psycopg[4]), complex mathematical computations (numpy), and downloading videos/metadata from YouTube (youtube-dl[6]) are a few that come to mind. [0] http://ruby.bastardsbook.com/ http://ruby.bastardsbook.com/ [1] https://github.com/python-excel/xlrd https://github.com/python-excel/xlrd [2] http://docs.python-requests.org/en/master/ http://docs.python-requests.org/en/master/ [3] https://www.crummy.com/software/BeautifulSoup/ https://www.crummy.com/software/BeautifulSoup/ [4] http://selenium-python.readthedocs.io/ http://selenium-python.readthedocs.io/ [5] http://initd.org/psycopg/ http://initd.org/psycopg/ [6] http://www.numpy.org/ http://www.numpy.org/ [7] https://rg3.github.io/youtube-dl/ https://rg3.github.io/youtube-dl/
- danso 10y agoThanks for the kind comment. I have been thinking of putting together a Python and SQL book since I teach those primarily and no current book fits my needs. I hope to have the same appeal except being a much better programmer these days :)