Ask HN: Advice on Alpha testing?

Hey guys, Just wondering if any HN'rs have any advice for undertaking, collecting, and "sorting through" alpha feedback? I know this is a pretty generic question, but I thought there might be something obvious I can learn from. My startup Setkick (http://setkick.com - an online film/tv pre-production and scheduling tool) is ready for some alpha feedback, and my basic idea was: * Sit down with some existing users (of pre-production software) and get them to run through the whole program from A to Z * Screen record the whole session for reference * Maybe create a short questionnaire at the end, 'what stood out', 'what was lacking' etc I'm debating whether to allow alpha testers to play via the web (not under my supervision), as I want to make sure I can get all the feedback (even down to user mumblings whist using Setkick, facial expressions etc) - btu I guess at the same time I need to get a wide array of feedback. Anyways, any advice would be great, Cheers guys, Chris.