matsche

4 ms·

1.
Show HN: Cargo-memex – compile rust code into a meme and run it(github.com)
61 points by matsche 5y ago|5 comments
2.
by matsche 5y ago
paperoni is awesome! I once implemented the crossref api in rust in order to search for publications from the command line https://github.com/mattsse/crossref-rs
3.
by matsche 6y ago
Yes, you can either limit how many request can be send concurrently or enforce a fixed or random delay in between consecutive requests. Basically this a just a futures_timer::Delay [0] that is reset after each request, which is non blocking
4.
by matsche 6y ago
Thanks, you're right. I fixed it just now by also requiring the tokio feature for all the tests in the Cargo.toml.
5.
by matsche 6y ago
The guys from corridor crew did a great break down of that amazing scene [0] [0] https://youtu.be/pTl42COCNaM?t=255
6.
by matsche 6y ago
I haven't used scraper too much. I personally find the predicate approach of select.rs [0] easier to use. However in this case the selector approach just made more sense. Standalone html5ever can be a bit cumbersome to work with direct
7.
Show HN: Voyager – write a web crawler/scraper as a state machine in Rust(github.com)
110 points by matsche 6y ago|11 comments
8.
by matsche 6y ago
tokio support just landed https://github.com/mattsse/chromiumoxide
9.
by matsche 6y ago
Unfortunately this crate has been abandoned by its author for quite some time now. The main differences are that are that chromiumoxide uses async rust by design and that all the devtools protocol types are generated. rust-headless-chrome w
10.
by matsche 6y ago
> The pdl files are the canonical protocol definitions and are maintained manually by the devtools team [0]. The JSON protocol definition files are then in fact being generated from those pdl files. You find them mirrored in the devtools
11.
by matsche 6y ago
I started that way, but that was more work than writing a generator. Luckily I found https://github.com/chromedp/pdlgen
12.
by matsche 6y ago
It uses async-tungstenite [0] for the websocket part. At the moment configured only with async-std-runtime but since async-tungstenite also supports tokio, the option to use tokio instead of async-std will be added as well. [0] https:&#x2F
13.
by matsche 6y ago
It only supports the chrome devtools protocol[0]. chromiumoxide is therefor more like a (limited) puppeteer rust port. Support for javascript page injection is currently being worked on [0] https://chromedevtools.github.io/d
14.
Show HN: Chromiumoxid – An Async Headless Chrome API in Rust(github.com)
127 points by matsche 6y ago|19 comments
15.
Show HN: Extrablatt – customizable news article scraping in Rust and WebAssembly(mattsse.github.io)
4 points by matsche 6y ago|0 comments
16.
by matsche 6y ago
it baffles me that it had to come this far. There were warning signs all over it. The German BaFin messed up so badly that one must either fundamentally doubt theircompetence or question their integrity.