19 ms·
Python’s new t-strings
- gnabgib 2y agoBig discussion (414 points, 10 days ago, 324 comments) https://news.ycombinator.com/item?id=43647716 https://news.ycombinator.com/item?id=43647716
- damnitbuilds 2y agoMaybe it's good to come back to a discussion after a few days, more informed and with a cooler head ? ;-)
- oulipo 2y agoNice, so it's a kind of "limited DSL" inside python that's easy to extend
- pauleveritt 2y agoThe PEP originally portrayed this as "for DSLs"
- avereveard 2y agowhat's the tldr difference between this and .format ?
- jitl 2y agoIt’s injection safe and compostable, and the resulting object retains the original values you interpolate in. This makes it useful for building SQL queries with prepared arguments for example.
- porridgeraisin 2y ago[flagged]
- pwdisswordfishz 2y agoCompostable? The Green New Deal has gone too far.
- earnestinger 2y agoIt’s custom .format implementation. (You access the placeholder and value and produce the string)
- hk__2 2y agoIt’s on a meta level: instead of formatting the string, it returns an object that contains both the format string and its argument. Library author can then implement whatever format function they want, for example one that escapes the interpolated strings.
- OJFord 2y agof-strings are syntactic sugar for .format, e.g.: f"foo is {foo} and {bar=}" "foo is {} and bar={}".format(foo, bar) are equivalent. t-strings are actually not strings, but Template objects, giving access to both the templating string and the parameters for processing. Sibling comments describe it as a custom .format implementation in that sense - it's f-string-like sugar where you're also allowed to take control of the .format function that it's sugar for.
- zahlman 2y agof-strings are actually translated into formatting and concatenation of the pieces at a bytecode level; they don't call `.format` under the hood and thus aren't what I'd call "syntactic sugar" in the traditional sense. But yes, the equivalence holds. t-strings, of course, (will) translate at the bytecode level into instantiation of the Template object (and whatever code is needed to compute the Interpolation values in-place).
- enescakir 2y agoNot sure about introducing yet another string prefix. Between f-strings, raw strings, and i18n stuff, it’s already getting crowded. Curious how readable this will be in large codebases.
- wodenokoto 2y agoI'm of the opposite opinion. Let's set the prefixes free! from sql import sql query = sql"SELECT user_id, user_name FROM {user_table_versioned} WHERE user_name = {user_name}"
- dmurray 2y agoHow would this be different from a function sql() that operates on one of these new t-strings? The syntactic sugar of changing it from sql(t"...") doesn't seem particularly valuable. The novel thing about t-strings is that they change the parsing at compile-time.
- stavros 2y agoIt wouldn't be different, but it would be more convenient because we no longer have to count the number of %s, you'd put the variable inline.
- nezirus 2y agoMaybe this could be useful to libraries like psycopg3 to use something more simple/natural instead of this: https://www.psycopg.org/psycopg3/docs/api/sql.html https://www.psycopg.org/psycopg3/docs/api/sql.html (while I also agree it gets crowded with yet another string prefix)
- dtech 2y agoThat's the exact use case. Basically these are syntactic sugar for a very similar function signature.
- pizza 2y agoLooks useful for embedding interpreters
- damnitbuilds 2y agoI enjoy f-strings, I guess some people need these. And I love Python but, having been through 2->3 ( occasionally still going through it! ) whenever I see a new language feature my first thought is "Thank goodness it doesn't break everything that went before it".
- stavros 2y agoYeah but it's been 17 years, maybe it's time to put the PTSD behind us. We're almost at a point where the current generation of programmers wasn't even programming when that happened.
- rglullis 2y ago> We're almost at a point where the current generation of programmers wasn't even programming when that happened I've been programming with Python since 2006, I think most of the systems were based on 2.4 at the time. I've been one of those who switched to Python 3 somewhat late, waiting for some major libraries to ship python 3 packages - celery and Twisted were one of the biggest holdouts - so I remember that the first project where all my dependencies were ready for python 3 was around 2015. This is to say: even seasoned developers who were conservative around the migration have spent more time working with Python 3 than Python 2. There simply is no reason anymore to be talking about python 2.
- kstrauser 2y agoThe last time I touched a large Py2 project was in 2018 when I ported it to Py3. So, I have 18 years of Py2, probably 6 years of overlap, and 7 years of pure Py3. That means I still have a lot more Py2 than Py3 time. Buuuttt, I'm so over the transition. It’s ancient now and I agree that we can stop fretting about it.
- nightfly 2y agoPython2 code didn't disappear when Python3 came out. At my work we're _still_ occasionally having to help people migrate code that was written for python2
- m2f2 2y agoIf this is just for sql queries ... it'd be overkill especially where you need to compare the usual PREPARE statements with the hassle of keeping everyone on 3.14 and above.
- bazoom42 2y agoCould also be used to prevent html injection.
- nhumrich 2y agoIt's for SQL, HTML, and shell. But idk how solving injection, a top on the OWASP list forever is considered "overkill".
- orthoxerox 2y agoIt's also for logging: log.debug(f"The value of counter was {counter}, the nonce was {nonce}") builds a new string every time the interpreter hits this line. Whereas log.debug(t"The value of counter was {counter}, the nonce was {nonce}") passes a Template to the debug() function that bails out if debug mode is not on and doesn't build a string.
- 7734128 2y agoSure, this avoids issues with SQL injections. However, I have a hard time imagining any developer who would both make such fundamental errors with f-strings currently and also switching to this option when it ships. Seems like a self selection which renders this meaningless, to some extent :/
- masklinn 2y ago> However, I have a hard time imagining any developer who would both make such fundamental errors with f-strings currently and also switching to this option when it ships. t-strings are a different type (both static and dynamic), f-strings are not. So t-strings can be mandated at the API level, forcing the developer into "proper" usage. That is, you need third-party tools to differentiate between some_call("safe literal string") and some_call(f"unsafe dynamically created string") That is not the case when it comes to t-strings, `some_call` can typecheck internally that it got a t-string, and reject regular strings entirely. Although some space probably needs to be made for composing t-strings together in case of e.g. runtime variation in items or their count. Facetting for instance. I don't know if that's a native affordance of t-strings.
- Timwi 2y agoBut that would require any SQL library you're currently using to make the breaking change of no longer allowing strings.
- baggiponte 2y agosqlalchemy doesn’t really accepts strings - if you do, you need to pass them into a “conn.execute(text(…))”, so end users should not face a breaking change.
- nhumrich 2y agoYep. Probably worth it. You could also do this with a monkeypatch method to "opt in" to this change.
- masklinn 2y ago
- runekaagaard 2y agoIt feels a bit like "cheating" that new x-string features are built-in only. It would be cool to be able to do: from foo import bar bar"zoop"
- masklinn 2y agoA t-string is a literal for a Template object which is a data holder, it doesn't actually do anything, so you would simply call bar(t"zoop")
- Timwi 2y agoTrue. Then you could use sql"..." html"..." for each of the given examples and achieve some runtime type safety.
- jbverschoor 2y agoAnd you'd end up with almost no improvement. If you pass a "t-string" to a framework, it can force escaping. What you suggest is to rely on escaping by the user (dev), who, if he was aware, would already escape. Unless you'd suggest that it would still return a template, but tagged with a language.
- mcintyre1994 2y agoFWIW the JS equivalent is a template but tagged with a language. It has all the benefits of this template, but IDEs can easily syntax highlight the string. That seems like it would be a bit trickier to do with the Python one which is a shame.
- Timwi 1y agoNo, you misunderstood that completely. They would still be like t-strings, but sql-strings are now a different type from html-strings. The escaping would be done by the library that offers the sql"..." functionality.
- thund 2y agoUse a function? bar(“zoop”) It’s just syntax, like we used to have print “foo” that later became print(“foo”)
- TekMol 2y agoWill this allow neat SQL syntax like the following? city = 'London' min_age = 21 # Find all users in London who are 21 or older: users = db.get(t' SELECT * FROM users WHERE city={city} AND age>{min_age} ') If the db.get() function accepts a template, it should, right? This would be the nicest way to use SQL I have seen yet.
- zelphirkalt 2y agoIsn't the actually proper way to use prepared statements anyway? If we are doing that properly, then what does this t string business buy us for SQL usage from Python?
- scott_w 2y agoBecause, as the article states, people aren’t using prepared statements. Instead, they pass f-strings because they’re more convenient.
- hedgehog 2y agof strings are syntax rather than a type, the resulting templates look like a reasonable way to specify a prepared statement.
- vultour 2y agoExcept to maintain backwards compatibility we're probably going to get new methods that only accept templates, completely circumventing any effort to stop people passing in strings. Prepared statements were the recommended way to run SQL queries when I was starting with PHP 15 years ago, anyone writing code vulnerable to SQL injection at this point should not be writing code.
- scott_w 2y agoWell yes but the alternative is to never make language improvements because legacy code exists.
- 2y ago
- sgt 2y agoWill it be a performance boost if Django's template engine started using t-strings internally?
- pwdisswordfishz 2y agoWhy would mere syntax make a difference?
- sgt 2y agoNo, I mean that if Django's template engine internally switched over its implementation to enable templating (as a functionality) to use these new t-strings. If t-strings are more highly optimized within Python, it could mean a performance boost.
- sashank_1509 2y agoWhy does this need to be a language feature. This could just be a separate library, we could use brackets instead of a letter before a string. I fear, Python is going down the path of C++
- jsnell 2y agoIt being a language feature gives you controlled access to the lexical scope, such that the template string can refer to variables by name rather than having to pass each value as a parameter. Doing it via parameters is repetitive and/or error-prone.
- linsomniac 2y agoYou CAN get access to the calling scope of a function. Something like: current_frame = inspect.currentframe() env = current_frame.f_back.f_locals.copy() I did it in uplaybook so that you can do things like: for module in ['foo', 'bar', 'baz']: ln(path="/etc/apache2/mods-enabled", src="/etc/apache2/mods-available/{{ module }}.load") This is an ansible-like tooling with a python instead of YAML syntax, hence the Jinja2 templating.
- dsego 2y agoBecause JS has it in the form of tagged template literals.
- chrisrodrigue 2y agoWe have a handful of ways to create strings now. Meanwhile, pytest is still not part of the standard library.
- perlgeek 2y agoIf it's not a language feature, there's always a risk of fragmentation. Some people won't use it because it adds another dependency, that means fewer programmers will be familiar with it. Others will come up with their own, slightly incompatible implementation. See for example Perl and its various Object Orientation frameworks (Moose, Mouse, Moo, Mo, M, Mojolicious comes with its own...) Other languages have a policy of prototyping such things out of core, and only adding it to the core language if it gains traction. Of course that works better if the language has a mechanism for extending the syntax out of core.
- toxik 2y agoSo should f strings now go away? They are just a special case of t strings. Also, don’t get me started on g strings.
- perlgeek 2y ago> So should f strings now go away? No > They are just a special case of t strings. Not really, because they produce a string right away instead of a template.
- nhumrich 2y agoNo. F-strings should still be the default. T-strings aren't usable directly, and don't have a `str` like API, so they aren't even compatible, intentionally.
- est 2y agohttps://peps.python.org/pep-0750/#approaches-to-lazy-evaluation https://peps.python.org/pep-0750/#approaches-to-lazy-evaluat... name = "World" template = t"Hello {(lambda: name)}" This looks cool
- fph 2y agoHow do these interact with i18n? Can I load a translated t-string with `_(t"")` from a .po file? Can it include variable names and arbitrary code inside lambdas?
- nhumrich 2y agoI suggest you read the PEP, there is a section on i18n. Short version is it's not designed for this use case. As for variables and arbitrary code/lambdas, yes: t-strings can do that, just like f-strings
- serbuvlad 2y agoAll things considered, this is pretty cool. Basically, this replaces db.execute("QUERY WHERE name = ?", (name,)) with db.execute(t"QUERY WHERE name = {name}") Does the benefit from this syntactic sugar outweigh the added complexity of a new language feature? I think it does in this case for two reasons: 1. Allowing library developers to do whatever they want with {} expansions is a good thing, and will probably spawn some good uses. 2. Generalizing template syntax across a language, so that all libraries solve this problem in the same way, is probably a good thing.
- pinoy420 2y agoNow instead of being explicit all it takes is someone unfamiliar with t strings (which will be almost everyone - still few know about f strings and their formatting capabilities) to use an f instead and you are in for a bad time.
- falcor84 2y agoThat is an issue, but essentially it boils down to the existing risk of unknowledgeable people not escaping untrusted inputs. The solution should be more education and better tooling (linters, SAST), and t-strings are likely to help with both.
- masklinn 2y agot-strings allow building APIs which don't accept strings at all (or require some sort of opt-in), and will always error on such. That's the boon. Having to write cr.execute(t"...") even when there's nothing to format in is not a big imposition.
- deleted 2y ago[deleted]
- mcintyre1994 2y agoAny sane library will just error when you pass a string to a function that expects a template though. And that library will have types too so your IDE tells you before you get that far.
- franga2000 2y agoI wish they added the same thing JS has, where this "string literal prefix thingy" can be user-defined. html`<p>${value}</p>` will actually run the function html(template). This means you can use this to "mark" a function in a way that can be detected by static analysis. Many editors will, for example, syntax highlight and lint any HTML marked this way, same with SQL, GraphQL and probably some others too.
- conartist6 2y agoFor the record the JS thing desugars to the exact same as the Python thing, so it is no more or less safe to do the syntax highlighting in Python as it is in JS.
- Timon3 2y agoIt desugars similarly, but the Python version doesn't have a name. Any t-string is a t-string, there's no HTML t-string or SQL t-string or anything like that. It's just a t-string you can pass to a function: html_string = t"<something />" sql_string = t"SELECT * FROM something" In JS, the string has a prefix that can differ between languages, e.g.: const htmlString = html`<something />` const sqlString = sql`SELECT * FROM something` and so on. See the difference?
- masklinn 2y agoExcept your labels are incorrect because neither `html_string` nor `sql_string` are strings, they're both Template objects, and the sink function is the one which processes it. No processing has happened to them by the end of the snippet, beyond creating the template object itself.
- Timon3 2y agoSure, choose different variable names, who cares. The essential difference is that the language is referenced at the declaration site, not the usage site, which makes the syntax highlighting far easier. Please engage with my point instead of criticizing trivialities.
- 0xFEE1DEAD 2y agoCall me a monarchist, but I think Python has changed for the worse ever since Guido van Rossum stepped down.
- pansa2 2y agoI don't think things would be very different if Guido were still BDFL. He's second-author of the t-strings PEP, and has been an author of most other major PEPs in recent releases (including the walrus operator, PEG parser, and pattern matching).
- nhumrich 2y agoI for one think python has never been better.
- ashwinsundar 2y agoIn what ways has it changed for the worse?
- sanderjd 2y agoIt's so fascinating to see so many people, when faced with a simple and useful programming language feature, get all up in arms about how it's the end of the world for that language. I honestly feel like a lot of people just seem bored and looking for stuff to be mad about.
- pansa2 2y ago> t-strings evaluate to a new type, `string.templatelib.Template` > To support processing, `Template`s give developers access to the string and its interpolated values before* they are combined into a final string.* Are there any use-cases where processing a Template involves something other than (i) process each value, then (ii) recombine the results and the string parts, in their original order, to produce a new string? In other words, is the `process_template` function ever going to be substantially different from this (based on `pig_latin` from the article)? def process_template(template: Template) -> str: result = [] for item in template: if isinstance(item, str): result.append(item) else: result.append(process_value(item.value)) return "".join(result) I haven't seen any examples where the function would be different. But if there aren't any, it's strange that the design requires every Template processing function to include this boilerplate, instead of making, say, a `Template.process` method that accepts a `process_value` function.
- rfoo 2y agoThere are a lot of examples about SQL in comments. In the SQL case you want something like: def process_template(template: Template) -> tuple[str, tuple]: sql_parts = [] args = [] for item in template: if isinstance(item, str): sql_parts.append(item) else: sql_parts.append("?") args.append(process_value(item.value)) return "".join(sql_parts), tuple(args) (of course it would be more nuanced, but I hope you get the point)
- pansa2 2y agoYes that makes sense, thanks. Also, my comment was about the amount of boilerplate required, but that can be vastly reduced by writing `process_template` in a more functional style instead of the highly-imperative (Golang-like?) style used in the article. The first `process_template` example is just: def process_template(template: Template) -> str: return ''.join(interleave_longest(template.strings, map(process_value, template.values))) And the second is something like: def process_template(template: Template) -> tuple[str, tuple]: return ( ''.join(interleave_longest(template.strings, ['?'] * len(template.values))), map(process_value, template.values) )
- nu11ptr 2y agoPersonally, this feels like a feature that is too focused on one problem to be a general feature. Python is getting huge. When people ask me if Python is easy and simple to learn I have to say "the basics, yes, but to to learn the whole language... not so much". I feel like in this sense Go really is interesting by rejecting almost every single feature. Honestly not sure generics were worth it as they add a lot of complexity, and while they are nice, I don't need them very much. The general idea to keep the language at its original focus is the right idea IMO. C++ would be the most extreme case where the language itself barely resembles what it started out as.
- murkt 2y agoThis is a pretty simple and useful feature. I wouldn’t say that it bloats the language too much. Descriptors and metaclasses are much more complicated and have a lot more implications and have been in the language for a veeeeery long time. Is it decades already?
- pansa2 2y agoYeah, Python hasn’t been a simple language for a long time, if ever. That’s probably the biggest misconception about the language - that its friendly syntax implies simple semantics. It’s not true at all.
- tetha 2y agoI would say python in it's entirety is one of, if not the deepest and potentially most complex language I know. C++ is the other contender. The things you could do with metaclasses, multiple inheritance and operator overloading are quite staggering. I'm just glad you don't have to think or even use this as a normal user of the language, most of the time or at all.
- nu11ptr 2y agoThis feature is not complicated, but one must keep every feature that can possibly be seen in code in their head. Even if it is familiar now, what happens when you use the feature in the one small section of code where it fits, nowhere else, and then read that code 2 years later? This is the problem with adding useful features that are only used in a few key places. I'm not saying Go is a perfect language, far from it, but limiting # of features as a general goal is something more languages should strive for IMO.
- Smithalicious 2y agoI really was on the side of being generally willing to accept new python features, but this is getting ridiculous. What an utterly pointless thing to bloat the language with. At this point my moving to clojure as my first line language of choice is only accelerating. This is of the category "things I wouldn't want to use even for the specific hyper niche things they're intended for". What even does a "t-string" represent? Because it's clearly not a string of any kind, it's a weird kind of function call notation. The programmer sees something that looks like string formatting, but the program executes some arbitrary procedure that might not return a string whatsoever.
- nhumrich 2y agoFor me, this is the best feature to land in python for 6 years. JS has had this and it allows my code to be completely safe from SQL injection, which is an absolutely incredible feature, given SQL injection has been the #1 vulnerability for a long time.
- pauleveritt 2y agoThanks Nick for this response and all the time you've spent explaining. It's funny, I looked back at the comments on f-strings before they landed. They also got similar complaints about bloat. And yet, my uneducated guess: very popular.
- Smithalicious 2y agoBut this doesn't prevent SQL injection, does it? It adds a grammar feature that you can then use to build SQL injection prevention following a novel idiom. Someone still needs to write the actual SQL building logic somewhere! I don't think this is the right idiom for doing this. Frankly I don't think SQL query generation should look like string templating at all! The sell seems to be "now you can write code that looks like an SQL injection vulnerability, without it actually being vulnerable!". I'd rather write code that isn't a vulnerability, and doesn't look like one, and doesn't have to bend the language grammar either.
- 1y ago
- pragma_x 2y agoDebate around the usefulness aside, are there any linter rules for warning about f-strings in light of this? I can easily see where mistaking one for the other would cause problems. For context, I'm thinking specifically about tools like Black and MyPy.
- sanderjd 2y agoThis will be trivial to implement (it is just a check against the type that t-strings return), so I'm sure it will be done, if it hasn't already.
- xixixao 2y agoThe runtime and static type of the template string is different to fstring (Template vs str). Unless you’re passing them somewhere that accepts strings, you can’t go wrong. I actually quite like the simplicity of this design over tagged literals in JS.
- youruinedpython 2y ago[flagged]
- russfink 2y agoI feel like this can be solved another way. S=f”my good code #### {potentially_evil_user_input} #### my good code again” then work around the ####. Of course, even better, S=evil_user_input and do a scrub on S first.
- youruinedpython 2y ago[flagged]
- JohnKemeny 2y agoAs they mention in PEP 750,"we expect many more developers will use t-strings than write t-string processing functions."
- morkalork 2y agoHonestly think this is a more useful feature and elegant solution than the walrus operator that was added. Formatting query strings has always felt messy especially with different DBs having their own non-standard ways of doing it.
- curtisszmania 2y ago[dead]
- dec0dedab0de 2y agoSeems pretty neat so far, but I don't understand the motivation behind not allowing you to call str(template) and get the template as a normal string. I could imagine it being very useful to be able to gather up the template itself in a string to do stringy things with. The only reason I could imagine, is if you are trying to protect developers from themselves, which kinda goes against the "we're all adults here" mentality that makes Python so great. I suppose it's easy enough to add that functionality, but come on.
- gaogao 2y agoThe traditional way at the megacorp for something like that is unsafe_str_DO_NOT_USE_OR_YOU_WILL_BE_FIRED(template)
- TheRealPomax 2y ago> If you’ve worked with JavaScript, t-strings may feel familiar. They are the pythonic parallel to JavaScript’s tagged templates. The syntax is template literals, not just "tagged templates". Which is a huge difference: template literals still act as real strings. They don't need a tag prefix to work, you have the option to tag them if and when needed. As far as I understand it, t-strings can't do that. They're not strings, and you can't even coerce them into strings, you have to run them through a processor before they become a string. So they're nothing like JS's template literals, they're syntactic sugar for forming "an instance of an object that needs to be passed into a function that returns a string". So I don't look forward to folks preferring f-strings over t-strings even when they really shouldn't, simply because "having to constantly convert them from not-a-string to a string is a hassle". If only they'd worked like JS template literals.. that would have been fantastic.
- haberman 2y agoTL;DR: like f-strings, all {foo} expressions in the t-string are evaluated immediately, but instead of immediately concatenating everything into a single result string, the t-string evaluation returns a Template object that keeps the interpolation results and the surrounding strings separate. This lets subsequent logic decide whether the interpolation results need any special escaping before concatenating them with the strings around them. In other words, t-strings are basically f-strings where the final concatenation is delayed. And indeed, you can trivially implement f-strings using t-strings by performing a simple, non-escaped concatenation step: https://peps.python.org/pep-0750/#example-implementing-f-strings-with-t-strings https://peps.python.org/pep-0750/#example-implementing-f-str... f'...' -> str t'...' -> Template foo(t: Template) -> str
- dheera 2y ago> This lets subsequent logic decide whether the interpolation results need any special escaping before concatenating them with the strings around them This sounds like unnecessary fluff in what was supposed to be a simple language. I'm worried Python is turning into C++42 with 65535 ways to do one simple thing. Why not just: f'SELECT * FROM `{esc(table)}` WHERE name = "{esc(name)}"' Nice and simple.
- pansa2 2y agoWhile your code is a valid alternative way to implement @haberman's description, the feature is actually much more flexible. The "subsequent logic" has full access to the interpolation results and strings. Not only can it escape the results, it can do whatever it wants to them. It can also do whatever it wants to the strings, and then combine everything in any way it likes - it's not even necessary that the final result is a string.
- pauleveritt 2y agoThe other PEP example shows generating HTML attributes from a passed-in dictionary. HTML has a number of places where this is helpful, if you have original data.
- mounir9912 2y agoWhat I really don't get is how it's any different than applying whatever function you would apply to the template, on the f-string variables. So instead of: evil = "<script>alert('bad')</script>" template = t"{evil}" safe = html(template) Why not just: evil = "<script>alert('bad')</script>" safe = f"{html(evil)}" Or even before creating the f-string. Is it just about not forgetting the sanitization/string manipulation part and forcing you to go through that?
- scott_w 2y agoPretty much, yeah. The article highlights that people were using f-strings directly, and they wanted to provide an alternative for lightweight template/interpolation.
- mounir9912 2y agoI feel like I'm still missing something when they're saying this about the example(s): "Neither of these examples is possible with f-strings. By providing a mechanism to intercept and transform interpolated values, template strings enable a wide range of string processing use cases." As far as I can see, anything you do with the template, you could do before building the f-string or inline as in my intial example.
- scott_w 2y agoYou wouldn’t really do your example, though. If you’re using an f-string, you’d just directly interpolate, because it’s convenient. You wouldn’t use an extra library to properly make it safe, otherwise you’d just use a proper template library and language. This gives you a convenient middle ground where you don’t need to learn a template library but still get safety. I can’t think of the code right now but I could see this being useful to pass in some dynamic HTML to, say, Django without having to remember to turn off escaping for that section. It can also be convenient for writing raw SQL without having to use prepared strings.
- davepeck 2y agoWith f-strings, you cannot write code to determine which portions of the resulting `str` were static and which were dynamic; with t-strings, you can. * (As to your initial example: it's worth considering what will happen as you compose multiple bits of HTML via nesting to generate a large final page. The developer experience may become... unideal.) * (barring undesirable hacks with inspect, etc.)
- mont_tag 2y agoHow does this Python docs example work with t-strings? cur.executemany("INSERT INTO movie VALUES(?, ?, ?)", data) Can SQLite3 cache the query as it does now?
- roywiggins 2y agoIt seems to me that it would be straightforward to parse a Template's parameters back into ? placeholders?
- mont_tag 2y agoNo, it would fail upstream before the template post-processing even begin. The template object itself could not be formed because the each name must be a visible variable name. This is the same error that you would get with an f-string that contained an undefined variable name.
- sanderjd 2y agoI'm intrigued but confused by your comment. I think it could be done like this: cur.executemany(t"INSERT INTO movie VALUES({data})") Then the Template object would have a parameter that is a list, and it could turn that into the right number of "?"s to pass into the database driver along with the data. What am I missing?
- nikisweeting 2y agoThis is pretty cool, if we're porting JS features do we get dictionary unpacking/destructuring next? I just want this so badly, it's the main reason I drift back to JS: >>> {a, b=45, c=None, **d} = {'a': 234, xzy: 32456} >>> print(a, b, c, d) 234 45 None {'xyz': 32456}
- enragedcacti 2y agojust use this very sane pattern /s: >>> match {'b': 45, 'c': None}|{'a': 234, 'xyz': 32456}: >>> case {'a': a, 'b': b, 'c': c, **d}: pass >>> print(a, b, c, d) 234 45 None {'xyz': 32456}
- zahlman 2y agoAnother alternative: >>> a, b, c, d = (lambda a, b=45, c=None, **d: (a, b, c, d))(**{'a': 234, 'xyz': 32456}) The parentheses for this are admittedly a bit tricky.
- kamikaz1k 2y agoby making it a generic `t` you lose explicit syntax highlighting. Where something like JS template`string` could determine which syntax to use based on the template value. I supposed when assigning it to a, variable: SyntaxRecognizableTemplate, you could give it the hint necessary. was this discussed in the PEP? *edit: reading the PEP-750[1] it doesn't seem like it.. [1] https://peps.python.org/pep-0750/#the-interpolation-type https://peps.python.org/pep-0750/#the-interpolation-type
- davepeck 2y agoWe pushed these questions out of the PEP to keep its scope constrained. (See https://peps.python.org/pep-0750/#mechanism-to-describe-the-kind-of-template https://peps.python.org/pep-0750/#mechanism-to-describe-the-...) But yes, the PEP leaves open an important question: how will tools decide to work with common types of content in t-strings, like HTML or SQL? There are simple approaches that can be taken in the short term (content sniffing) and more robust approaches (type annotations, perhaps) that will take time and the broader tooling community to develop.
- pauleveritt 2y agoIt was discussed in the first revision and discussion of the PEP. The decision was made to move that to follow-on work, as we discovered more about what tooling needs. As an example, I was excited about using `Annotated` on the function to indicate the language it expected. Turns out, a lot of linters know nothing about the type system.
- davepeck 2y agoHi! I wrote this. :-) I'm a little late to the conversation (and a bit surprised to see this trending on HN) but am happy to answer any questions; I'll try to pop in throughout the day.
- 18172828286177 2y agoThis is super cool, thank you.
- maxloh 2y agoHi. I come from a JavaScript background. I am wondering what is the reason behind not using a similar syntax to JavaScript? Seems simpler to me. # Compare this: template = t"<p>{evil}</p>" safe = html(template) # To this: safe = html"<p>{evil}</p>"
- davepeck 2y agoThe PEP originally started with a similar-to-javascript syntax but over time we decided it wasn't the right way to expose these ideas in Python. There's more detail about why this approach was rejected in the PEP: https://peps.python.org/pep-0750/#arbitrary-string-literal-prefixes https://peps.python.org/pep-0750/#arbitrary-string-literal-p...
- varunneal 2y agoWould be interested in inclusion of PEP 292 [1] in your discussion here, which introduced `string.Template`. Is this Template going to be deprecated? [1] https://peps.python.org/pep-0292/ https://peps.python.org/pep-0292/
- davepeck 2y agoPEP 292's `string.Template` will remain; there are no plans to deprecate it. PEP 750's `string.templatelib.Template` is a separate and unrelated type. Amongst many differences, unlike PEP 292, `Template` has a literal form too. I'm hopeful that the confusion will be minimal; in practice, PEP 292 (aka $-strings) is used only in specialized cases, like flufl.i18n, a really deep I18N framework.
- florbnit 2y ago> In addition, I hope that the tooling ecosystem will adapt to support t-strings. For instance, I’d love to see black and ruff format t-string contents, and vscode color those contents, if they’re a common type like HTML or SQL. This is such a strange take on t-strings. The only way for anything to infer that the template string is supposed to turn into valid HTML or SQL is to base it of the apparent syntax in the string, which can only be done in an ad-hoc fashion and has nothing to do with the template string feature. The way the feature has been designed there is no indication in the string itself what type of content it is or what it will eventually be converted to. It’s all handled by the converting function. As others have added, something like sql”select * from {table}” would have been able to do this, but there’s not even any guarantees that something that is in a template that will be converted into valid sql by a converting function should be any type of valid sql prior to that conversion. For all you know t“give me {table} but only {columns}” might be a converted into valid sql after the template is processed.
- pauleveritt 2y agoThe original PEP and the original discussion had this in scope. We removed it to let this emerge later. There are different ways to signal the language -- some more friendly to tooling, some more robust.
- pphysch 2y agoPython has had type annotations for a decade, and modern IDEs can interpret them. Writing `query: SQL = t"SELECT ..."` is a small price to pay for such a DX boost.
- florbnit 1y agoI mean yes, so t-strings add nothing. You could already do everything you are proposing t-strings will make easier with just strings or f-strings. ‘query: SQL “Select …”’ “Wait” you’ll say “then the function taking the string has to parse the string and will have to add ugly logic grabbing variables from globals()!” Ah yes. That’s such an ugly solution… let’s instead codify just that as the official best practice and sell people on it by prefixing a t to the string so they don’t notice…
- permo-w 2y agois this just a copy paste of the PEP announcement?
- metrognome 2y agoZen of Python in 2025: There should be one-- and preferably only one --obvious way to do it. Python String Formatting in 2025: - t-strings - f-strings - %-operator - +-operator - str.format()
- a_t48 2y agoThe last three generally shouldn't be used (`+` is sometimes fine, but not really for formatting), but I doubt we would ever get a py4 that removes them, given the stomachaches that py3 caused. It does feel odd that a t-string is just an f-string without args applied.
- alfalfasprout 2y agoIt's my understanding that they're still recommended for logging since with f-strings you always pay the formatting cost (but with str.format it's deferred).
- davepeck 2y agoAnd $-strings (PEP 292) as well. :-) I see each of these as distinct but overlapping; I'm (slowly) writing a guide to string formatting with all of these in mind, trying to emphasize when I'd choose one over the other. (fwiw I personally avoid % and + these days; $ is pretty uncommon in practice; f-, t-, and .format() all seem to have good unique uses.)
- sanderjd 2y agot-strings aren't really like these others. It's definitely true that those four string formatting techniques violate the "one obvious way" advice.
- hughw 2y agoI could scarcely believe this new t-string wasn't a joke. As an occasional, grudging Python programmer, I rue the accretion of string formatters over the years. My code bears the history of successive, imperfectly understood (by me)formatters. "Situation: There are 14 competing standards...." https://xkcd.com/927/ https://xkcd.com/927/
- feraidoon 2y agoI had something like this implemented for safely executing shell commands, by using metaprogramming: name="A$ron" z("echo Hello {name}") Note that this is not an f-string. The z function expands the variables by parsing this string and accessing its caller's local variables. https://github.com/NightMachinery/brish https://github.com/NightMachinery/brish
- mos_basik 2y agoLanding in 3.14? Nice, but also oof, that's probably not getting to my employer's codebase for a year or two. And it sounds like it could really solve some problems for us, too. Paging asottile - any plans to make a `future-tstrings`? :) `future-fstrings` (https://pypi.org/project/future-fstrings/ https://pypi.org/project/future-fstrings/) was a real QOL improvement for our team for a year or 2 around 2019 before we got onto Python 3.x.
- pauleveritt 2y agoSome prior art: https://pypi.org/project/tagged/ https://pypi.org/project/tagged/ In fact, the repo of a companion project from the author has the ticket that spawned the work leading to t-strings: https://github.com/jviide/htm.py/issues/11 https://github.com/jviide/htm.py/issues/11
- ayhanfuat 2y agoI asked asottile for his opinion on PEP 750 (or rather, a previous version of it), and he wasn’t really in favor. So I think it’s unlikely that we’ll see a future-tstrings from him. :)
- vFunct 2y agoWould like to see Django fully use these to replace a lot of it's own complicated template syntax.
- unit149 2y ago[dead]
- bk496 1y agoLowercase f is very visually similar to t. Hopefully no bugs occur from a typo
- pauleveritt 1y agoAs an example of t-strings helping SQL, Phil Jones published a package [1] that illustrates how it could work. [1] https://github.com/pgjones/sql-tstring https://github.com/pgjones/sql-tstring