10 ms·
Show HN: A cryptocurrency news scraper inspired by Hacker News
- howiem 9y agoI built this using the webhose api to scrape and filter for cryptocurrency related topics from reputable news sources and trending forum posts. I'm a terrible designer so I eyeballed the hacker news layout and finger painted some CSS over it. Hope people find it interesting :P
- showkiller 9y agoOverall the site looks nice and functions well. The ticker a the top is hard to use as I could not see that you could scroll. I like the concept of the 101 section, could use some more info.
- howiem 9y agoAwesome thanks for the feedback. I’ll feed some more content for the 101s :) id appreciate any interesting posts or links anyone recommends explaining crypto currency and block chain in plan English
- sharemywin 9y agomight add twitter.
- howiem 9y agoGreat idea, I'll put that on the list
- sharemywin 9y agowould be nice to add some kind of json feed. maybe add some kind of account login to get the feed. some kind of database to collect all this would be neat for some kind of price predictor
- lsadam0 9y agoAny plans to add a comment system? Edit: I'm excited by the idea of a crypto focused HN-ish site, I'd love if there was a comment system :)
- vthallam 9y agoI thought that was Reddit is for. The /r/Cryptocurrency subreddit exactly serves this purpose.
- sanefive 9y agoNot everybody can post in this subreddit though. And it's always good to have different places to share about crypto :)
- QML 9y ago/r/Cryptocurrency is literally all price-talk and shilling -- very few posts of self-reflection or technicals.
- lsadam0 9y agoI'm looking for comments and discussion with quality on par with HN. Reddit, generally, does not provide that.
- zencash 9y agoLose the tickers. Add discussions.
- lawnchair_larry 9y agoThat is not what “crypto” means.
- mhluongo 9y agoLanguage isn't prescriptive, get over it.
- sethgecko 9y agoIt doesn't mean cryptography either, it just means hidden.
- lawnchair_larry 9y agoHah, fair enough!
- akkat 9y agoAny plans on making the inverse?
- shepardrtc 9y agoI like the ticker, and I like it overall. Good job!
- deleted 9y ago[deleted]
- Xeoncross 9y agoIt would be really great to be able to login 1) save stuff and 2) discuss articles. It doesn't even need to be threaded or have voting at first.
- howiem 9y agoThanks. I'm working on this now. I've built the authentication system and user models and will be working on comments next
- colecut 9y agohttp://hodlfeed.com http://hodlfeed.com
- howiem 9y agoI've just added the ability to save interesting posts. You will see a button that says "favorite" under each post. You will be prompted to sign up and then it will save it your account to access at a later stage. Next up comments and discussions :)
- sickcodebruh 9y agoOne of the reasons I love Hacker News is because of the high quality of content and discussion. Looking like HN is nice but the fact that your top post includes a nice little "(((bankers)))" remark from /u/BigNuts881 was a rough reminder that this isn't to be a much better than just looking through the results of a broad Twitter search.
- petercooper 9y agoThat is a good part of Hacker News, but I'd suspect if it's like any other community site maybe 10% of people bother reading comments and 1% bother to make them. There's plenty of room catering for the 90% who just want the links Drudge-style, and kicking off a community that actually has discussions is super hard work.
- optimuspaul 9y agothat's funny because I read the comments on far more posts that I actually follow the links to.
- bringtheaction 9y agoMe too. I thought most people on HN came here for the discussion -- even if that means primarily reading comments and not necessarily speaking up.
- kaycebasques 9y agoI spent about an hour yesterday learning all of these amazing insights about the solar energy industry, thanks to Hacker News comments.
- Ftuuky 9y agoI can't open the links for the original posts, is this happening with you all as well?
- Moter8 9y agoI can't middleclick links. Unusable for me. Also, please don't attach /?sort=new (or remove it) to reddit threads. It overrides the user's sorting.
- deleted 9y ago[deleted]
- dmm 9y agoHN works great without js. This site I can't even click the links?
- czechdeveloper 9y agoInspired by HN without all the thinks that makes HN great.
- kiddico 9y agoUse the link to the right of the title (shown as the name of the site it's on). Clicking on titles expands/collapses.
- notamy 9y agoThat's pretty counter-intuitive, compared to reddit/hn/similar...
- optimuspaul 9y agowhat does a middleclick do?
- jtokoph 9y agoIn most browsers, middle clicking a link will open in a new tab.
- optimuspaul 9y agoI don't have a middle button so it's news to me. I have a modifier key for that or a right click and select "open in new tab"
- nulagrithom 9y agoWhy does a 0-points, downvoted Reddit thread with an (((echo))) have 640 points on this scraper?
- Geee 9y agoI think the vote count is obviously manipulated. Stay away from this site.
- a13n 9y agoI would visit this daily if the content quality were better. Maybe start with scraping, then let people post themselves, and gradually move towards no scraping. I like the ticker. Saves me from needing to check elsewhere.
- galfarragem 9y agoYes, the ticker is great! I would love a chrome extension that could attach a ticker like this to HN.
- howiem 9y agoThanks. I've simply used the coinmarketcap API to get the currency feeds, super straight forward. It fetches from the client side. Anyone can use it here https://coinmarketcap.com/api/ https://coinmarketcap.com/api/
- phoneboy 9y agoI'd like to see a sentiment for each article as well. If you add that, then add a green/red indicator on the price chart as well for when the news came out.
- howiem 9y agoGreat idea, I actually had this before. I was feeding the articles into google's natural language processing API to get a sentiment index from -1 to 1 (-1 negative, 0 neutral +1 positive). The issue I was having that it wasn't really that accurate. Google NPL has trouble analysing contextual news articles. e.g. if a headline says "Bubble in cryptocurrency markets", it doesn't recognise that bubble is negative in news contexts. I have been looking for some different NPL libraries that might be better including AWS Rekognition. I'm not a data scientist so don't think I'll do very well building my own machine learning algorithm. Any suggestions of good libraries people have used for news sentiment would be greatly appreciated
- Retr0spectrum 9y agoSide note, I find the font very hard to read. The letters are too thin and grey. Removing the "font-weight: 300;" style from the body makes it infinitely more readable.
- IncRnd 9y agoThis is A cryptocurrency news scraper inspired by Hacker News. Reading the HN title, and looking at the site name listed on HN, I had expected there to be information relating to cryptography, which there is not.
- howiem 9y agoSorry for the confusion I've gotten used to referring to cryptocurrency as crypto as shorthand
- nijaru 9y agoThis is not crypto news, but cryptocurrency news.
- phoneboy 9y agoI'd like to see other terms as well, incl. distributed ledger stuff.
- colecut 9y agoIn a world where 'literally' can be defined as 'figuratively', I'm not sure why we bother splitting hairs on terminology... what is accepted is what is accepted.
- dingaling 9y ago> what is accepted is what is accepted Is it accepted? I would agree only when it appears in the NYT or The Economist with its 'new' meaning. It's a subset of lazy people in tech who are abusing 'crypto' at this point in time, not Valley Girls. There's an inflection point here to interdict the abuse, before it gains mass-usage.
- colecut 9y agoI see it as hopelessly yelling at kids to stop walking on your lawn
- topbarcolor 9y agoWhat is actually happening is that someone tells other people on the lawn that they wish kids wouldn't walk (and poop) on it, and then some of those other people play kiddo advocate and yell at that person. The kids are aren't providing any active resistance at all compared to that.
- andrewcmyk 9y agoPlenty of cryptocurrency chatter on HN these days anyway.
- blueprint 9y agoGah. Crypto news? More like cryptocurrency. Wait. No. More like news for cryptocurrency traders.
- FLUX-YOU 9y agoHorizontal scroll bar on the ticker covers the Mktcap row.
- Vervious 9y agoWhat is all the metamask code for?
- hanniabu 9y ago3 suggestions. - when I click on the link to go to the side, I'd much rather prefer it to open as a new page because it allows for quicker loading when you hit the back button, which would then close that page and bring you back to your site without needing it to reload. - when visiting a link and then going back, I need to tap it twice. Just tapping once brings me to the redirect and then it just goes to the link again. This doesn't need time be fixed if the link is opened in a New tab though - the tickers up top should give priority to a watchlist we could build and store in local storage so sign up isn't needed. Should be pretty trivial to implement
- justboxing 9y agoCongrats on shipping! What's the tech stack you used? I see from your comments buried at the bottom that you are not a designer, so will give you a pass on the usability issues that others are whining about. Would suggest you buy a HTML template off of wrapbootstrap or themeforest and customize it. If you can't do that, hire someone from upwork to do the customization for you. PM if you wanna chat more. Email in profile.
- notnull 9y agoCan you say more about how you built this? How you did the scraping, etc. ?
- howiem 9y agoFor scraping I've used the webhose api (I am still on the free account hence when u click on the link it has a 5 second redirect). I've found it is an excellent resource and easy to use API, I highly recommend it http://webhose.io/ http://webhose.io/ For the tech stack its just node.js and express on the backend and vanilla JS on the front end using webpack for bundling. I built this project really as a means to teach myself to write unit tests better and mock integration tests. In the first iterations I fed the news articles after they were scraped into Google's Natural Language Processing library to get the sentiment of the article. I scrapped this after it became too inaccurate.
- deleted 9y ago[deleted]
- eggbrain 9y agoIf you model your site after Hacker News, people are going to try to use it like Hacker News, and the expected behavior of your site is incredibly not-hacker-news-like. 1. I click on a story expecting for it to take me to the site, but nothing happens. After feeling confused, I realize it actually added some text content after the story. What? It was only after looking in the comments here that I realized I had to click the link in parentheses (which on Hacker News shows you submissions from that website, not links you to the content) 2. I see a lot of "points", so I try to understand how things are upvoted - is it accounts? No, no accounts for your site. Is it based on the links popularity (eg trending tweets, hot news, hot reddit stories, etc)? No, there are some stories with hundreds of points, but linking to a reddit post with 2 upvotes. It's only then do I realize by clicking on the title, or the link, I'm "up-voting" it, even though it might not be good content (and in fact, each time I click it adds a point). 3. I search for the ability to comment, but realize that's not there, but again since it looks like Hacker News I think there's comments and go to click on something that doesn't exist. In general, I think your site could be really useful, and there's a lot of cool stuff you did, but I'd either change your design so people don't treat your site (interaction wise) like the site your referencing, or try to at least link up expected behavior in a more consistent way.
- howiem 9y agoThanks for your comments. This is very much an early prototype and I appreciate your comments. I will be looking to add the ability to comment next. Regarding adding text at the bottom I thought it might be easier for people to read a glimpse of text before deciding to open the link, I’ve found this useful in my own browsing. The actual link has a 5 second redirect because I am using the free tier currently for webhose.io api (a great news scraping provider). if I get enough traction I may look at getting a paid account, but this is very much a side project
- shit-show 9y agoGood stuff, I highly recommend applying as a researcher to the top leading forecast reporting firm called cryptonaire.com. They have many clients that are large investment groups that will be using their research when the full app release is out. It would be nice to have more researchers that love crypto rather then investment. Give it a try, they are the first research firm in the industry.
- metrue 9y agoWill you open the API like HackerNews ?
- howiem 9y agoI may, in the meantime you can access it via my backend service. It's a mongodb store https://cryptoserver.now.sh/threads https://cryptoserver.now.sh/threads
- skeleton 9y agoUnless I'm reading the cryptocurrency market table incorrectly, the market cap row appears to be a horizontal scroll bar.
- nexus2045 9y agoNeat site, something I would use, since I just started looking into cryptocurrency news, and have both a discovery problem (I didn't know til recently that r/ethtrader existed) and a content filtering problem (but what if most of r/ethtrader is a circlejerk?). Maybe in the future users could also upvote credible news with small amounts of bitcoin, like giving gold in Reddit.
- numismatex 9y agoExpanding stories that require me to Google search the title are shit.
- txsh 9y agoWould you mind adding sentiment analysis to this? Ex: bitcoin: negative; ethereum: neutral; bitcoin cash: very positive I would love you forever.
- kristianp 9y agoNot sure how this one got in there at number 33: "Iranian woman reported missing after waving headscarf in public without wearing a hijab". Possibly a spam comment about cryptos got parsed.
- howiem 9y agoOnce again I’d appreciate any great 101 resources people have found explaining blockchain, cryptocurrencies and other related topics. I will look at adding more content to the 101 page. I know very little about the topic and just chose this as a random topic for news scraping. I am very much learning :)
- howiem 9y agoThank you everyone for the great feedback. Please feel free to share any helpful links that explain blockchain, cryptocurrencies, ICOs etc. and I will post on the 101 resource page. I am very new to this and only a curious observer of the cryptocurrency wave
- techaddict009 9y agoI am not able to click any story :(
- howiem 9y agoI’ve now added the ability to comment and am fine tuning replies :) thanks again for all the great feedback.