8 ms·

Ask HN: How do you search for $string if a webpage “supports” infinite scroll?

• whb07 7y ago
  reddit has a JSON api you can access with your credentials.
• niceworkbuddy 7y ago
  Maybe try google with `site:` keyword. You can use also advance searching tools, so eg. you can search value to date.
  • OJFord 7y ago
    > And to make it clear: I don't want to search for stuff via $string site:reddit.com via some search engine
• klaustopher 7y ago
  What I have done in the past: Scroll down a couple of days/pages/whatever and then use Ctrl+F. It‘s annoying as hell, but that usually works as most pages just add stuff at the bottom and don‘t unload the previous content on top
  • mijoharas 7y ago
    As mentioned above, there are some sites that page out the content that is no longer in the viewport after you've gone suitably far from it. I still remember my consternation when I first noticed this, and it was because I tried exactly what you had suggested, but a previous match further up the page disappeared upon subsequent CTRL+F-ing.
• gcmrtc 7y ago
  You'd have the same problem with pagination, you'd need to Ctrl+f on each page. Solution? Use the website's search form.
  • deanebarker 7y ago
    I agree with this. I feel like infinite scroll gets a bad rap. Pagination is no better. There are some sites and use cases where infinite scroll is a huge usability improvement. (I know -- pagination gives you a URL to a specific page of results. Who cares? When was the last time you deep-linked into a specific index page? Doing so is probably a bad idea, since when new content gets added, the contents of a specific index pagination will change.)
    • leeoniya 7y ago
      > Doing so is probably a bad idea, since when new content gets added, the contents of a specific index pagination will change unless you paginate in chronological order rather than reverse chronological. but no one does.
      • efreak 7y ago
        It's easy enough. Just request the next X results after the last result on the current page, like so: `?resultsperpage=50&resultsafter=postid750`. If you're not sorting by time, then you should be able to add a time constraint to the search backend and include that parameter as well. I've also seen forums that cache the results for a certain amount of time, and only return the search ID and page number in the query string.
    • xorcist 7y ago
      Pagination is a lot better. In the example presented, most of the time the user just wants to find something on the front page. Nobody uses ctrl-F to search all of reddit.
      • deanebarker 7y ago
• reportgunner 7y ago
  1. Click somewhere on the page so the scrolling works 2. Hold down Ctrl + End until a desired amount of pages loads 3. Ctrl + F to find $string
  • egeozcan 7y ago
    Which is useless if elements outside the viewport are garbage collected because you are browsing a virtual list. Instagram is an example but they don't have text in the elements so it's not a huge hindrance.
  • Aachen 7y ago
    That's what I used to do but sites like Twitter unload content to not make your browser tab super slow (which is a good thing in general), which breaks ctrl+f.
• luistricks 7y ago
  Hello friend, this is pretty simple, just change your user agent for this one: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) It's ie 6 user agent with no support of js, so bye bye infinite scroll.
• mcv 7y ago
  The naive way is to scroll down a ridiculous amount and then do a simple ctrl-F. It would be nice if infinite-scroll sites provided their own search function that searched through all potential content for that page. (And then probably provide those search results on an infinite-scroll page, requiring another search function to search through those search results. It's infinite scroll all the way down.)
  • egeozcan 7y ago
    It's exactly after realizations like this that one (unreasonably, to be very explicit) decides all the GUIs are leaky abstractions and should be avoided. Then you want to do some math and realize how amazing visual spreadsheets are. What I mean is, there's no perfect solution and it all depends on what you're trying to achieve.
• combatentropy 7y ago
  Other problems: - Scroll, scroll, scroll, and then click a link. Now click back. You have lost your place. - How can anyone link to a section far down? - Impossible to read or use anything in the footer. I suppose if you install infinite scrolling, you remove the footer. But I have heard a story where they forgot.
  • Aachen 7y ago
    Twitter seems to have the back thing down somehow. I don't know how it works (my best guess is making a <div style=height:scrollTop-200px> and only loading a few tweets around the scroll position and making that work with bfcache) but so it is apparently possible to make that work fine. Linking to things in the middle of a page was solved in the 90s with the hash part of the URL. Aside from legal terms or privacy policies, I haven't yet seen a site with excessively tall pages that don't have some mechanism (hash part or direct linking an entry) of linking to in-the-middle content. Most sites with infinite scroll don't have a footer, though I have seen a few. Inspect element works well enough there imo, if you're tech savvy of course.
    • mstade 7y ago
      > Linking to things in the middle of a page was solved in the 90s with the hash part of the URL. And then promptly broken again in the mid-00s more or less till now, by way of sub-par client side routing, giving rise to SPAs and breaking back buttons, history management and deep linking all over the place. Things are getting slightly better since the history api came about, but you still see loads of sites with non-existent or broken deep linking, and crappy history management, such as reddit. This is of course because the browser behavior of fragment links, history etc. on static content is well defined, but for dynamic content in most cases have to be considered and implemented in the application logic, and it’s never a priority. Hands up everyone who’s been in planning sessions for your project and right from the start you’ve considered deep linking and history management! Anyone..? No, ok. Truth is, in my near 15 years of web dev experience, I’ve never once seen this be part of the requirements, because you kind of just expect it to work. Yet it’s incredibly easy to break, unintentionally even, and three weeks before release it’s usually incredibly difficult to fix because of all the corners we’ve painted ourselves into. So then it becomes a thing you fix after the fact, and spend tons of time tryin to figure out how to wrangle your router or framework or what have you to work it out, but turns out you’ll probably have to rewrite it all. But of course you won’t, because it’s silly, so it just becomes one of those things you paper over where possible and quietly ignore otherwise... sigh Sorry for the rant.
• bishalb 7y ago
  Go to Inspect element and do control + F to search for the string.
  • inetknght 7y ago
    That assumes the site doesn't do something funny with tags to get around the user's desire to not risk infection from malvertisements.
  • bilekas 7y ago
    This still wont solve for lazy loading data on scroll, which is actually how its supposed to be done, to reduce load time, and dom draw together.
    • bishalb 7y ago
      I don't understand what you mean but if the infinite scroll content has already been loaded, the HTML for the same will update in the element inspector as well and you can search for the string there. Of course it's hacky but I use this technique all the time.
  • muzani 7y ago
    This is no different than scrolling down and ctrl+F. Infinite scroll doesn't load content until you scroll down.
    • bishalb 7y ago
      Ok I was assuming they meant that if they did control + F on the webpage and the string was towards the bottom area, the page would start infinite scrolling automatically after locating that string via search(and they wanted to avoid this). Heres a similar example, if I wanted to get the contact page link(located on the footer) on an infinite scroll page, I would never be able to get to the bottom because of infinite scrolling. But I could do a Ctrl +f on the inspector to locate the "contact" string and copy the contact page link from there.
• cronin101 7y ago
  I see lots of replies suggesting scrolling for a long time and then using CTRL+F. However, this won't work if the page is using virtualised scrolling (common with React et al. SPA for performance reasons, to avoid huge DOM trees as the page expands). The majority of content that is outside of the visible window will simply be unmounted from the DOM. I'm not sure what the best-practice for a webapp designer is here? Perhaps intercepting Ctrl+F and displaying a custom search that will do the correct filtering on the back-end and update/retarget the view? Azure DevOps does this but it's still frustrating if your focused element is not within the capture point for the event.
  • lonelappde 7y ago
    Google Docs implements custom search. Unless web frameworks start to expose some basic controls to the user, like pre-walled-garden Operating Systems, that's what we have to wait for
  • Hello71 7y ago
    that's what discourse does. it's frustrating because it's significantly slower than the built-in browser search, probably in part due to excessive network accesses, but also probably due to poor caching and inefficient implementation.
    • 1wd 7y ago
      The custom UI is also IMO terrible. It's completely different than the browser Ctrl-F UI. It does not simply highlight and move to the matches. Instead it shows a "preview", so you have to navigate to each match separately and back to the search UI. And it just navigates to the matching comment, but for very long comments I still have no idea where in the comment the match is. Thankfully you can press Ctrl-F twice to get the browser UI.
    • ekzy 7y ago
      I hate discourse's CTRL+F. When things like this happen I open the devtools and search in the dom, and it's a horrible experience too. I just don't want to learn each website's custom controls. Don't hijack my CTRL+F or my scrolling or anything else please!
• INTPenis 7y ago
  This is a real interesting rant. I've definitely had this problem and wondered about the benefits of infinite scroll at those times. Also bad UX design. I can't offer anything besides the fact that I love my mouse which has this free roll toggle button. When I press that I can roll the wheel really fast and long without anything slowing it down. But it's sad that certain websites force users to such measures.
• xorcist 7y ago
  For reddit specifically, the answer seems to be to use "old.reddit.com" instead of "www.reddit.com". The is no general solution. Infinite scroll style web apps are implementing their own content view, in essence a web browser inside a web browser. It will never behave as users expect.
  • luistricks 7y ago
    This is a good answer that works in reddit, but if you change the user agent, it will work in almost any website with support for ie. IE6 User Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1) https://addons.mozilla.org/es/firefox/addon/uaswitcher/ https://addons.mozilla.org/es/firefox/addon/uaswitcher/ This can be done with Selenium too. https://stackoverflow.com/questions/29916054/change-user-agent-for-selenium-driver https://stackoverflow.com/questions/29916054/change-user-age...
    • css 7y ago
      You can set a custom UA in Firefox without an extension by going to about:config and adding the new UA as a string called `general.useragent.override`.
      • luistricks 7y ago
        Yes you can, but with the addon it is easier, and you can set your custom user agents and store within, it is just for comfort.
  • blotter_paper 7y ago
    I personally prefer the interface of "i.reddit.com", but they semi-frequently let the cert expire for that one.
  • rococode 7y ago
    This add-on for Firefox is helpful for that (redirects all www to old): https://addons.mozilla.org/en-US/firefox/addon/old-reddit-redirect/ https://addons.mozilla.org/en-US/firefox/addon/old-reddit-re... I presume this is the corresponding Chrome extension: https://chrome.google.com/webstore/detail/old-reddit-redirect/dneaehbmnbhcippjikoajpoabadpodje https://chrome.google.com/webstore/detail/old-reddit-redirec...
• z3t4 7y ago
  I open dev tools (or dev proxy) to see all the connections the site makes, grab the url to the "page" AJAX call. Put ' (single quote) into the URL which then returns a SQL error. Then I make a SQL injection to make the page return everything. And search that.
• dyrl 7y ago
  More and more I feel like we should be simulating a human by macroing an actual typical web browser, including macroing the browser's dev tools. (When scraping is fragile and you may have to periodically maintain scripts anyway, this is my conclusion after trying to automate a React website using recently-mentioned lib taiko, and finding it to be hit & miss, although it probably speaks more to my inexperience in general.)
• dusted 7y ago
  Infinite scroll is one of those things that needs to die.
  • matt74827289 7y ago
    Nope, I love infinite scroll as a user. I even use extensions to add it to paginated websites. I wish those websites had infinite scroll as default or as an option cause infinite scroll extensions can be jankey. Not saying there aren't bad implementations of infinite scroll out there, those do need to die off, but they aren't that common.
    • dusted 7y ago
      I hate it as a user, it needs to die because it makes my user experience so much worse. I've never seen a not-broken implementation. Suddenly, the end key cannot take me to the bottom of the page, using the browsers native search function (which I will always chose over an in-page alternative) cannot do its thing, I can no longer get an idea of how far along I am in the list, and I cannot do a quick visual scan of many items because dragging the scrollbar does not behave naturally. Page up and page down is all but broken. A compromise may be a browser option that one can set globally to indicate whether one wants pagination (and how many results per page) or infini-scroll.
• luistricks 7y ago
  Hello friend, yeah, there is an option, just change your user agent, tested in several sites and works...
• kogir 7y ago
  I recall a time I scrolled through the whole page while using Fiddler or Burp as a proxy and then just searched the logged requests. Not pretty at all but it worked.
• drej 7y ago
  Unless there's a mobile site, I usually do this: I use my Logitech MX Master 2s, hit the scroller really hard to let it spin for a while thanks to inertia, triggering as many "page" events as possible :-)
• waltbosz 7y ago
  If reddit is your only use case, there is always https://old.reddit.com/ https://old.reddit.com/ ... until they take it down. I've see web apps with infinite scroll that will capture your Ctrl+F keypress and provides their own in-page text search tool (ex: https://i.imgur.com/BJPDDFw.png https://i.imgur.com/BJPDDFw.png) . I don't find it to be as easy/natural to use than the browsers build in text search, but it's better than broken text search.
  • ekzy 7y ago
    No, I prefer a broken text search. You can have a custom search, I'm fine with it, but don't overwrite Ctrl+F
    • waltbosz 7y ago
      Yes, I agree with you there. Don't overwrite my browser's shortcut keys.
• pier25 7y ago
  I had this exact problem yesterday while looking for someone in my list of Twitter people that I follow. Also on YouTube when searching for something on my list of liked videos. If you want to implement infinite scroll at least implement a damn search.
• kgwxd 7y ago
  I made a Firefox addon for myself that applies CSS to elements with text and/or attribute values that match a given regular expression. One of the first issues I ran into was that a lot of pages build their content via javascript so I added a delay to the scan. That mostly worked but the next issue (obvious in hindsight) was that can happen at any time, multiple times, for various reasons. That's when I learned about MutationObserver for the first time and it's worked pretty much perfectly since. Here's the addon source [0]. I mostly (only) use the addon to style elements with links to sites I know I never want to visit, but it should work for this type of thing. However, the current UI isn't convenient for adding an adhoc rule. [0] https://github.com/7w0/ssure/blob/4fd34677ad1c3f667ae85b939f08ea081a064e2c/content/content.js#L90 https://github.com/7w0/ssure/blob/4fd34677ad1c3f667ae85b939f...
• mwexler 7y ago
  Hey, just like when you want to get to all the information in the footer like Contact Us, but the moment it appears, it gets scrolled away. I feel like a cat chasing a laser pointer.
• codegeek 7y ago
  I cannot think of one single use case where infinite scroll makes sense compared to plain old data tables that are paginated with server side search/pagination built in.
  • asdff 7y ago
    Great for if you want your users to forget about how long they are spending on your site. Less friction and no jarring page loads, just a steady drip of dopamine.
• u801e 7y ago
  On another note, infinite scrolling pages start using more and more memory as you scroll, which eventually requires closing the browser tab.
• strangattractor 7y ago
  Rendering with JS causes many difficulties. It often simply makes the content less useful - how do we index such a page for efficient discovery (search) for example. Running JS locally to render pages is problematic and often does not work. I cannot tell you how many times I have gone to commerce sites that use infinite scroll - scroll down - click on and look at a product - back button to listing page returns to top of scrolling page making me scroll down again. What a waste of time.
• robbrown451 7y ago
  You can paste something like this into your javascript console: setInterval(function() { window.scrollTo(0, document.body.scrollHeight); }, 2000); That will scroll continuously, so you can let it run for a while and then do ctrl-F. It might take a while but you can do something else. I've used it successfully many times (usually when I wanted to scrape the DOM for some reason or another, rather than just do a search).
  • sfoley 7y ago
    I've tried this twice in the last year or so: once on twitter and once on dropbox. Neither time did it work. It _seemed_ that ctrl+f would not find anything that was not visible +/- a screen height or two. I couldn't tell if it was a shortcoming of Firefox (unlikely IMO) or both sites tried to be clever about dynamically unloading content when it was outside the view window (more likely).
    • robbrown451 7y ago
      Seems to be a Firefox thing. Works fine on Chromium browsers, only finds stuff near the top in Firefox.
  • leppr 7y ago
    Or just put something heavy on your keyboard's "End" key :)
    • robbrown451 7y ago
      Ha yeah that works, if you don't want to do anything else on your computer for a while as it does its thing....
• winrid 7y ago
  Use the search box, or figure out their API and use it directly.
• barry27 7y ago
  You dont. Infinite scroll is rubbish.
• wiseleo 7y ago
  Seems like there’s opportunity for an extension to grab all text as it appears, store in client side DB, search that. I am going to build that as a feature in an extension I am working on to track all of my upvotes as automated bookmarks.
• buboard 7y ago
  switch to old reddit? facebook, twitter etc have decided that their content is so low quality it's not even worth searching so they re just not supporting it
• tuananh 7y ago
  Discourse (forum) used to hijack Ctrl+F (Command+F) shortcut which makes it super irritating! Not sure if they have changed that.
  • xfalcox 7y ago
    That is only done if you are in a topic page where the amount of posts is higher than 20 replies. In that case Ctrl+f is redirected to a search that is scoped to the current topic and can search it all. Other pages don't have the search hijacked, and of they do you can press Ctrl+f twice to open the browser search.
    • tuananh 7y ago
      last i tried discourse was in 2016. things may have changed.
• mygo 7y ago
  I’ve been finding it increasingly difficult to find blog templates that don’t do infinite scrolling with no way to turn it off. It’s not impossible, and most blog themes still have an option to turn it off if they’re included, but I’ve noticed it’s increasingly difficult to find themes without it. It seems to be the cool thing to do. But for setting up a static blog using the template, it’s a non-starter. Plus for all the aforementioned reasons: CTRL+F, linking to content, following a link and then going back, SEO (although some do SSR — still this is a non-starter for a statically published blog that just generates the pages using the theme)...
• AznHisoka 7y ago
  I haven't tested if this works for Reddit for some sites, if you use Opera, somehow they show you an 'old' version of the site that doesn't have auto-scrolling. I'm not sure why - perhaps it's because there's some kink in Opera that doesn't support it, but it's worth a try?