5 ms·
I'm the original author of the post. If anyone has any questions about AppEngine or what it's like to develop on that platform, feel free to post here.
by mmastrac 16y ago
I'm the original author of the post. If anyone has any questions about AppEngine or what it's like to develop on that platform, feel free to post here.
- tosh 16y agoI am really glad someone wrote a response from another POV. I was thinking about doing the same if I get the time on the weekend. We have been very happy appengine campers and the 1.4.0 SDK release looks very promising. There are and have been some rough edges but overall it is a great platform and it was well worth it for us to invest time to evaluate and embrace the constraints.
- krosaen 16y agoNice writeup. I'd like to hear more about how you manage using S3 for video in conjunction with running the rest on GAE.
- newobj 16y agoHi, I work with Matt at Gripe :) Video is in development, but the system is pretty straightforward. We're integrated with Zencoder. Our clients (e.g. mobile/web) request an upload endpoint from our server (GAE). The endpoint is a pre-signed S3 upload url computed by the server. The client uploads to that url and then triggers a "process" API on our server which issues a request to Zencoder on the backend to pick up the videos from S3 and begin transcoding them. Zencoder sends back a response that allows us to associate the "video" entity in our datastore with job ids/output locations. You can set up a callback so that Zencoder will notify GAE when transcoding is complete. The output is available in S3, including thumbnails. So I guess this is also an endorsement of Zencoder: it rocks and was very easy to integrate with!
- krosaen 16y agoThanks! our startup (realtimefarms.com) also runs on GAE (python though), and we use the blob store image serving to great benefit, but have been toying with supporting video uploads, helps to hear how you guys are doing it.
- jon_dahl 16y agoThanks! And great writeup by Matt. You should do a blog post on your video workflow - GAE and a transcoding API like Zencoder. :)
- kevindication 16y agoMy first impression of GAE (admittedly, many months ago) was that it was very painful to deal with loading or otherwise manipulating data in BigTable. Do you have any shortcuts or advice here? Has it improved? Annoying Example: I realized that I needed to delete a lot of records from the table. I created a URL that I could ping periodically to delete a bunch, then get killed because it was taking too long, repeat.
- mmastrac 16y agoCheck out the mapper API. They've wrapped up all the difficult details behind a hadoop-like interface: http://googleappengine.blogspot.com/2010/07/introducing-mapper-api.html http://googleappengine.blogspot.com/2010/07/introducing-mapp...
- kevindication 16y agoThat looks exactly like what I needed. I will have to come back and give them a try again. Thanks.
- mootothemax 16y agoDo you know of any good links for GAE development patterns? I've read tips in various comments on various websites, such as use filters instead of GQL, using task queues and handling storage exceptions, but very little in one place :)
- mmastrac 16y agoI'm more familiar with GAE/J than the Python stuff myself. I don't have specific links to patterns, but I highly recommend watching the AppEngine videos from Google I/O. I watched nearly a dozen hours of video during development: http://www.youtube.com/watch?v=o3TuRs9ANhs http://www.youtube.com/watch?v=o3TuRs9ANhs http://www.youtube.com/watch?v=AgaL6NGpkB8 http://www.youtube.com/watch?v=AgaL6NGpkB8 Here's some random tips I can give: It's worth spending some time and writing the ORM stuff by hand for your first application. The low-level datastore APIs are well-written and you'll learn a lot about how stuff works by being close to the metal. AppEngine rarely throws storage or memcache exceptions outside of maintenance periods anymore. Unless there's someone here who's had a different experience, I would consider them rare. Use as few abstractions as possible, no matter which language you are choosing. If you don't need something that interacts with an external system done right away, stick it in a task queue, no matter how quick you think it will run. Assume that it'll succeed in your code (it probably will, eventually).
- mootothemax 16y agoGreat tips, thanks! :-D
- znt 16y agoI'd love to hear about your view on which frameworks to use for python (tipfy, webapp, tornado, django ports etc) and their advantages over each other. I've developed a webapp using django-nonrel, it was pretty quick to have it going but I have this uneasy feeling in my gut that something may be wrong.
- mmastrac 16y agoI'm not as strong with GAE/Python as I am with GAE/Java, but my feeling is that your AppEngine experience will be far more productive if you use AppEngine's native Datastore APIs rather than any abstraction layer. There's a lot of detail at the low level that you miss by using any layers of abstraction (on both Python and Java platforms). That detail is important for getting maximum performance out of your application and correctly dealing with AppEngine's unique transaction model. The choice of web frameworks is tougher. Startup time is important right now for AppEngine (although slightly less important with some of the new features rolling down). You'll want to use the lightest-weight framework you can find. On Python, I've had great luck with webapp + Django templates + raw Datastore APIs. On Java, Guice + Guice servlets + raw Datastore API is really lightweight and fast. We've also developed our own Django-like templating system for Java that we hope to open-source at some point. Anyone else with more GAE/Python experience want to chime in?
- adn37 16y ago> We've also developed our own Django-like templating system for Java Any chance you could list viable alternatives, please? (I'm considering GAE/Java). Thanks for this interesting thread.
- mmastrac 16y agoI did a study of the bigger templating systems available on Java earlier this year and the landscape wasn't that great: JSP, Freemarker, Velocity and a number of smaller projects. We decided that it was worth some time developing a system inspired by Django's lightweight syntax. If you send me a note (matthew@mastracci.com), I can let you know when we get our project up on github. It's rendering every page on gri.pe right now and has been stable for a few months now.
- nl 16y agoNice write-up Matt. One thing I'm curious about the is continuous deployment. AppEngine has a 1000 deployment limit (http://code.google.com/appengine/docs/quotas.html#Deployments http://code.google.com/appengine/docs/quotas.html#Deployment...) Is that a worry for you?
- mmastrac 16y agoThanks, Nick! That 1,000 deployment limit is daily, AFAICT. Our quota dashboard shows 9/1000 deployments, which matches up with the number of builds we've pushed live today.
- redstripe 16y agoHow are you doing backups of your database? How often do you do them? How do you do full text search? The other two GAE topics posted this week had quite a few reports about datastore time outs. Has this been a problem for you?
- deleted 16y ago[deleted]
- mmastrac 16y agoFor backups, we do them once a night using the Python SDK + remote API (even though our app is Java, it still works). Full-text search is done on Solr/EC2. We had issues with the datastore for a short period in late September/early October (everyone did). There was some maintenance work done by the AppEngine team shortly after that which resolved the issues we've been having. There haven't been any problems lately.
- Radix 16y agoIf you want to see the answer to your question turn show dead on. Someone made a mistake.