3 ms·
I was poking through your source a little and I noticed a common inefficiency in the way Python for handling text is often written. Due to the immutable nature
by cmiller1 3y ago
I was poking through your source a little and I noticed a common inefficiency in the way Python for handling text is often written. Due to the immutable nature of strings in Python repeatedly concatenating onto one is very inefficient and not very kind to the gc, so it's much more efficient and pythonic to build up your string as a list of strings and then join them later. For example you could rewrite your do_info function a bit like this:
def do_info(self, line):
"""Display information about the current page."""
renderer = self.get_renderer()
def get_renderer_class():
return str(renderer.__class__).lstrip("<class '__main__.").rstrip("'>") if renderer else "None"
def get_page_lists(url):
return [l for l in self.list_lists() if self.list_has_url(url, l)]
def get_list_status(l):
if self.list_is_system(l):
return ""
status = "normal list"
if self.list_is_subscribed(l):
status = "subscription"
elif self.list_is_frozen(l):
status = "frozen list"
return f"({status})"
url, mode = unmode_url(self.current_url)
out = [
f"{renderer.get_page_title()}\n\n",
f"URL : {url}\n",
f"Mime : {renderer.get_mime()}\n",
f"Cache : {netcache.get_cache_path(url)}\n",
f"Renderer : {get_renderer_class()}\n\n"
]
lists = get_page_lists(url)
if lists:
out.append("Page appeared in the following lists:\n")
for l in lists:
out.append(f" • {l}\t{get_list_status(l)}\n")
return ''.join(out)
- BiteCode_dev 3y agoBecause it was such a common problem, CPython 3 tries to detect such string concatenation and handle it like a list of strings concatenation internally. Have you measured that your code is significantly faster than the original one?
- ploum 3y agoI was not aware of that problem. The do_info is trivial anyway, so performance is not an issue there. But in ansicat, that might speed up significantly the HTML rendering (which is quite expensive with images transformed as ANSI strings by Chafa). So I would be curious to have some benchmark and would happily accept patches improving performances in ansicat.
- BiteCode_dev 3y agoIf html rendering is really a big bottleneck, a bytearray could be even faster, provided you accept the compromise of manipulating encoded strings. And a rope for even more perfs. But yes, benchmarks are very much necessary.
- ploum 3y agoI’ve learned quite quickly that you never do any performance job without profiling. Never! Even if the optimisation looks straightforward.