10 ms·
Python For The Web
- nashequilibrium 15y agogreat post, I enjoyed reading it and following along!!!
- jQueryIsAwesome 15y agoYou got downvoted for a simple compliment? Many people here are really smart... but many of them are not nice at all.
- deleted 15y ago[deleted]
- nashequilibrium 15y agoI am surprised that some members actually took the effort to down vote my comment because I expressed my honest reaction to the author and did not add the extra flair to seem super smart.
- ez77 15y agoimport requests r = requests.get('http://gun.io') print r.content That's it! In only three lines of python, you can grab a whole webpage and print it to the screen. Awesome! Well, that's a poor selling point that unduly diminishes the credibility of the article. Your audience is likely to know snappy alternatives such as curl gun.io
- blhack 15y agoOr even just urllib? import urllib foo = urllib.urlopen("http://gun.io).read() print foo I mean...maybe urllib (or urllib2) are old and unhip at this point, but they're heavily documented. I use urllib all over my projects and have never encountered any problems.
- phzbOx 15y agoIn fact, it's even better than what the author said: import requests print requests.get('http://gun.io').content Only two lines!
- blhack 15y agoWell...urllib is the same import urllib print urllib.urlopen('http://gun.io').read() And to be clear, you'd really never want to just print the content like that. You'd be doing something like. import urllib import simplejson req = urllib.urlopen("https://graph.facebook.com/some_id") response = req.read() json_data = simplejson.loads(response) (Python is syntactically pretty, imho)
- kenneth_reitz 15y agoimport requests; print requests.get('http://gun.io').content my god, python is cloud scale.
- rbonvall 15y agoIn one statement: print __import__('requests').get('http://gun.io').content
- kenneth_reitz 15y ago+1
- jay_kyburz 15y agoBut how to I get Python in my browser. Python on the back and front end is my dream.
- llambda 15y agoFirst, why is the author recommending SimpleJSON? Just use json, it's built into the standard lib! (Also if I'm not mistaken the json implementation included in Python may even be based on simplejson, the APIs are very similar even if not.) Second, I disagree that Django is the best web framework. It might be the best web framework, but it depends on what you're doing. I've come to prefer Flask for its simplicity and overall the way it feels more Pythonic. That said, requests cannot be recommended enough! It is an awesome package that should not be missed if you're doing web programming in Python.
- pyre 15y agorequests comes close to Perl's LWP::UserAgent in terms of usability, but LWP::UserAgent has been around for years. I don't know how that makes Python the 'best language for interacting with the web.'
- draegtun 15y agoAnd also LWP::UserAgent has two younger whipper snappers biting at its heels! * HTTP::Tiny - lightweight useragent that comes with 5.14 use HTTP::Tiny; print HTTP::Tiny->new->get('http://gun.io')->{content}; * And the all singing and dancing Mojo::UserAgent use Mojo::UserAgent; print Mojo::UserAgent->new ->get('https://github.com/timeline.json') ->res->json->{repository}->{name};
- pjscott 15y agoSimplejson is completely API-compatible with standard library json, but an order of magnitude faster.
- Mizza 15y agoI wrote this for a friend who is just getting started out in python/web stuff. It might be a little n00bly for lots of people here, but it could be interesting for people who aren't pythonic who are thinking about making the switch!
- sixtofour 15y agoNicely done, very clear and concise. A question: is lxml what we would use today instead of Beautiful Soup?
- waterside81 15y agolxml is definitely faster, but I've found BSoup to be more forgiving with poorly formatted DOMs
- kenneth_reitz 15y agoBeautifulSoup is poorly maintained — you have to be very specific with which version you're using. Note: Lxml has a number of repair modes that allows it to parse virtually anything. Cpu cycles and memory go up quite a bit when they're activated, but it's still better than BeautifulSoup.
- ericflo 15y agoThankfully lxml has a slower-but-more-forgiving mode that you can use when interacting with poorly formatted HTML, which takes advantage of BeautifulSoup http://lxml.de/elementsoup.html http://lxml.de/elementsoup.html
- sudont 15y agoI’ve found the exact opposite. BSoup will choke on invalid tags in the DOM, such as: <div id=“content”><content>…</content></div> If I try to return the innerHTML of #content, I get '<div id=“content”><content>’ as a string, nothing else. While I know that’s inexcusable markup, it’s nothing I have control over. lxml (if it builds on the target system) has been much better for my scripts.
- 15y ago
- mccutchen 15y agoNote: In Python 2.6+, you can just `import json` instead of needing to install and import simplejson.
- stock_toaster 15y agoI believe that the python2.6 json module (basically forked form simplejson) is much slower than the current simplejson version. The speedups (simplejson rewrite) didn't get included until 2.7 as I recall (memory is vague on this point).
- pjscott 15y agoAccording to my un-reproducible benchmark, simplejson was 27x faster than built-in json. It's not a small difference.
- stock_toaster 15y agoYeah, I remembered it being significant, but wow. That is larger than I recalled it being. Probably due to a very unscientific test at the time. ;)
- sethish 15y agoIn my experience I have to reference the local path to a python file when using manage.py `python manage.py runserver` becomes `python ./manage.py runserver` But your mileage may vary.
- rglullis 15y agoAre you doing anything funky with the file or the file system? Windows, Linux or Mac OS? There should be no need to do what you are saying. Perhaps you are trying to run "./manage.py runserver" without calling python? In this case you'd be right that you need to have "./" as part of the command call, but you still would have to add the shebang to manage.py and make it executable.
- deleted 15y ago[deleted]
- 5partan 15y agoalso you have to write: from lxml import html to import the html-parser
- Mizza 15y agoNoted!
- _mattb 15y agoFlask is also awesome for python-based web development. I found it very easy to jump in as the docs are great: http://flask.pocoo.org/ http://flask.pocoo.org/ Here's the post's Django example written using Flask: https://gist.github.com/1296926 https://gist.github.com/1296926
- mccutchen 15y agoNice. It does seem like one of the lighter-weight Python web frameworks might fit in better with the overall theme of this intro, and Flask is a great choice. I thought about doing the same thing as I read through it.
- Mizza 15y agoThis is really, really cool. Thanks for making this. I really dig the Flask project (love the website and the docs, and everybody who uses it raves about it) and I hope to play with it more in the future, but I don't think I'd recommend it to somebody who is a first timer, largely because of one issue: Data. Flask leaves you to sort it out on your own, which is great if you're capable of that, but Django holds your hand, which is more appropriate for a beginner.
- akavlie 15y agoData? Can you explain that? Not sure what you mean.
- Acorn 15y agoI think Mazzo is referring to a lack of Django's ORM.
- anthonyb 15y agoI'm pretty sure he means that there's no database support built in, ie. you need to go download something like SQLAlchemy, SQLObject, Storm, etc. If you're already going to do that anyway it's not so much of an issue, but for people who are just getting their feet wet, it's yet another obstacle to overcome.
- hartror 15y agoI was a little surpised that lxml is used in this example over Beautiful Soup. Any reason?
- pjscott 15y agoIt's much faster than Beautiful Soup, better maintained, and does an excellent job of handling crazy broken HTML.
- benatkin 15y ago> Python is the best language in the world for interacting with the web, and I'm going to show you why. I disagree. I didn't need to read it, but I did, and I still feel that way. The statement says language, but the actual points made are about the libraries and the community (i. e. the ecosystem). I think the ruby ecosystem is actually better, for a few reasons: * Deep integration with JavaScript (ExecJS and TheRubyRacer are way ahead of what is available in Python, Perl, and PHP) * Many different excellent choices for talking to HTTP APIs, including the highly innovative Faraday * A top-notch JVM implementation (JRuby has had more traction than Jython over the last five years) * Lots of good auth libraries: OAuth (for authenticating with a plethora of outside services), Devise (pre-built auth with confirmation emails, password resets), and OAuth2 provider (server) implementations * Ruby has evented programming too. Goliath is pretty good. Python is great but I take issue with saying that it's the best web dev language out there.
- pyre 15y agoIt also depends on what you need to do with the web. Perl is currently the best language for interacting with Unicode, for example. If you web needs include Unicode, you might have trouble with Ruby or Python, unless you're willing to accept half-ass, partial, or just plain wrong Unicode implementations.
- ashrust 15y agoI agree, I love Python but I've spent untold hours dealing with its unicode issues.
- codygman 15y agoEverything unencoded is unicode in 3 isn't it?
- lloeki 15y agoStrings are unicode. Bytes are encoded data. With Python 2 strings you have to be proactive about it and - always use u"" unless you know what you're doing - segregate non-unicode data to boundaries by decode()ing early and encode()ing late, or you're guaranteed to shoot yourself in the foot sooner or later but if you do it it's quite smooth sailing. Also, #/usr/bin/env python # -*- encoding: utf-8 -*- helps a lot. All that just brings you closer to Python 3 anyway and really helps when using 2to3.
- phzbOx 15y agoTo parse Html and XML, BeautifulSoup is awesome.
- mekoka 15y agoVery nice article and thanks for the intro to Requests. Though, I'm not sure we would agree about Django being the best thing to learn in the long run. I undertook to move to Python for web development about 8 months ago and embarked on a thorough quest for the one true framework. Obviously Django was the first and most recurrent recommendation. Having briefly toyed with it in the past, I really could see myself commit to it for the long run. The various raving reviews also made the choice all that simple. I was all ready to give it my seal of approval, when I encountered my first complain, which was so compelling that it raised an eyebrow. It was about an experienced Python programmer explaining that as time went and as he progressed with Django, he found himself increasingly swapping parts out of it for external libraries. SQLAlchemy, WTForms, Jinja2. In the end he had only the routing module and the admin, which wasn't that big a deal for him. He was asking what was the point of using a full-stack framework not necessarily designed with interchangeability in mind, if you end up just using it like a glue mini-framework? As I dug deeper, I found more similar complains, all from similarly experienced developers, who all ended up adopting something else with a light plugable base approach. I heard of Repose.bfg, Pyramid, Werkzeug and a slew of other ones, that allow you to get down and dirty fast, while still allowing you to get big in the long run. Just as you, I was recently asked by a friend wanting to get into web applications development to recommend a platform to work from. I also did point to Django, but explained that it wasn't because it's necessarily the best, but rather because it's the gentler introduction. It comes with batteries, crutches, first aid kit and a nice box of goodies, perfect for someone who has no clue what they're doing. Note that I'm not dissing Django or relegating it as an amateurish framework. I agonized on my decision and still sometimes experience some Django envy (FYI I adopted Flask and don't regret it one bit). Nonetheless, it's hard to deny that it does a particularly good job of introducing newbies to good concepts fast, while at the same time being notorious for getting in the way of more experienced developer than some other frameworks.
- beaumartinez 15y agoDjango is where the jobs are. If you want to profit from your skills, the smart bet is in learning technologies that are in demand.
- 15y ago
- gbrindisi 15y agoWhy on earth I didn't heard about Requests before? How does it compares against urllib2?
- mekoka 15y agoWhere have you been indeed? It's been posted on HN at least twice and made page 1 both times: http://news.ycombinator.com/item?id=2882301 http://news.ycombinator.com/item?id=2882301 http://news.ycombinator.com/item?id=3094695 http://news.ycombinator.com/item?id=3094695
- pjscott 15y agoRequests is really good. urllib2 is overly complicated, with a loathsome API. The world would be better of more people used Requests.
- jroseattle 15y agoThis is an article professing love for a specific library in a given language. Move along, nothing to see here.
- eliben 15y agoI'm not sure why the avoiding of the Python standard library. Why import an external json library, when there's one built in?
- tripzilch 15y agoThe whole article is a bit heavy on the keyword stuffing, isn't it?