scumola

20 ms·

151.
by scumola 18y ago
Sure. http://mediawombat.com does that very thing. The website is stored on a hosted machine somewhere in Florida and the actual media is stored on Amazon's S3 and loaded via the browser on demand.
152.
by scumola 18y ago
* Still trying to improve the http://mediawombat.com crawler * Working on better pre-caching scripts/SQL for http://mediawombat.com search results * Working on how to charge a company that wants to use the http://mediawombat.com API *
153.
by scumola 18y ago
libmicrohttpd
154.
by scumola 18y ago
I still write C code and GUIs for stuff, but doing it in PHP,SQL,Perl is just so much quicker and easier for things that my C programming is not as frequent as it used to be and my C skills are getting stale. I'm not in need of a client-si
155.
by scumola 18y ago
I used to love a small business in my local town that used to do this. He sold cheap, korean PC hardware and charged by the hour to fix problems for people. He had a real store front and was in business for at least 5 years before Best Bu
156.
by scumola 18y ago
Ok, http://mediawombat.com (Flash Search Engine) is officially developing an iPhone Flash Search app now.
157.
by scumola 18y ago
I knew a guy who was a graphic artist for a game company and they worked in a fantastically beautiful room, soft lighting, Star Wars movies playing in the background. Everything was made to inspire the artists to do better work. I wonder
158.
Why isn't there a Linux distro out there that is made for huge web 2.0 infrastructures?(badcheese.com)
23 points by scumola 18y ago|35 comments
159.
by scumola 18y ago
dbserver swf # ps auxw | grep crawl root 6984 3.0 0.5 215832 5168 pts/5 Sl+ 12:59 0:00 ./crawl I'm using a linked-list in C to store all of the queued-up URLs so all of the threads can mark them as being worked on or whatever.
160.
by scumola 18y ago
Page size varies, but when I was doing the initial speed tests, pages were around 20k each (guestimate). Yea, the crawler uses libcurl, which supports gzip compressed content. My comcast link is (theoretically) 7Mbit down, 385Kbit up.
161.
by scumola 18y ago
Yea, fetching one URL at a time was a major issue, but things like the executable size in memory was also a problem. Remember, this is an old machine. Mysql takes 400M for its index for my data, my other processes run on the same machine
162.
by scumola 18y ago
kingkongrevenge - I didn't mean to sound like I replaced all of my DB functionality with flat files. I have several stages that my data flows through (fetching, ripping, sanitizing, indexing, uploading) and I replaced the constant calls to
163.
Shoestring Budget? Starting to feel growth issues on your back-end? Embrace unix and C
140 points by scumola 18y ago|120 comments
164.
by scumola 18y ago
A couple of years ago, my setup looked like this. Since I've moved to LCD screens and a little nicer setup, but no picture as of yet. http://badcheese.com/~steve/gallery/albums/userpics/10003/no...
165.
by scumola 18y ago
Amazon S3 & UnBox. Google Adwords. Playstation Network. GoDaddy. Tivo.
166.
by scumola 18y ago
Sysadmin for a startup. Bring a hammer and a power screwdriver too! :)
167.
by scumola 18y ago
I kind of agree. I'd rather be able to use a site knowing that it was in beta (as long as it worked mostly) and if there were any data loss, at least I know that it's in beta and wouldn't be too pissed if some data was lost. I like to use
168.
by scumola 18y ago
Yea, 'sex' is actually a popular query, but for some reason, none of the SWF files that we've indexed contain the string 'sex'. Believe me, there are plenty of porn-related SWF files out there, so the material is not lacking, just categori
169.
by scumola 18y ago
Yea, I use flashblock myself. I'm of the mind of making a non-flash front-end just because it's not very light-weight. Javascript and CSS could provide the same experience without the weight of the flash interface, but it kinda makes sens
170.
by scumola 18y ago
Hey, I'd just like to thank everyone for helping out and providing your comments and suggestions about our project. We got a lot of exposure just from this YC post and we're starting to shake hands with people who have offered assistance.
171.
by scumola 18y ago
Yea, we only submitted our site to one VC group so-far, and the competition was fierce (380 submissions and only 10 were chosen for funding). I've never dealt with Investors of any kind before, so it's kind of a new game to me. I'm comfor
172.
by scumola 18y ago
We initially put about 5 days (2 people working almost 24 hours a day it seemed) into getting the concept from paper to a working site in time to make a deadline for a VC funding submission (which we didn't get). The crawler, ripper, index
173.
by scumola 18y ago
We got permission from bluebison.net and she even offered to make a custom logo for us, but we're waiting to see if the idea is even worth investing money into at this point. We bought a logo off ebay (not suggested) that's on our wiki pag
174.
by scumola 18y ago
I won't know what they put together - as far as I know, there's nothing concrete on the Google or Yahoo end, but you know PR - just being in talks with Adobe gives the PR department to promise a product. It's also very possible that we cou
175.
by scumola 18y ago
Hi there. I'm the other founder of the site (Troy made the original post on this thread) and I sent a "Hey, we're doing that too" email to the VP of Yahoo search and the VP of Google search this morning regarding SWF search. Still haven't
176.
by scumola 19y ago
MLS. :)
177.
by scumola 19y ago
Also, there should be some way of getting MSLP (?) data from a service via RSS or something somewhere - which is a TON nicer than crawling several peoples' webpages.
178.
by scumola 19y ago
You can build a generic crawler that pulls pages from sites quickly and then process the pages offline with whatever lnguage you'd like. It's better to have a distributed way of doing things. Plus, there are standards that you need to com