6 ms·
I kinda hoped a formatting library designed to be small and able to print strings, and ints ought to be ~50 bytes... strings are ~4 instructions (test for null
by londons_explore 2y ago
I kinda hoped a formatting library designed to be small and able to print strings, and ints ought to be ~50 bytes...
strings are ~4 instructions (test for null terminator, output character, branch back two).
Ints are ~20 instructions. Check if negative and if so output '-' and invert. Put 1000000000 into R1. divide input by R1, saving remainder. add ASCII '0' to result. Output character. Divide R1 by 10. put remainder into input. Loop unless R1=0.
Floats aren't used by many programs so shouldn't be compiled unless needed. Same with hex and pointers and leading zeros etc.
I can assure you that when writing code for microcontrollers with 2 kilobytes of code space, we don't include a 14 kilobyte string formatting library...
- vient 2y agoIt is a featureful formatting library, not simply a library for slow printing of ints and strings without any modifiers. You can't create a library which is full of features, fast, and small simultaneously.
- jstimpfle 2y agoYou'd hope the unused stuff gets stripped out but I don't know much about this topic so not going to argue.
- vlovich123 2y agoFfunction-sections and fdata-sections would need at a minimum to be used to strip dead code. But even with LTO it’s highly unlikely this could be trimmed unless all format strings are parsed at compile time because the compiler wouldn’t know that the code wouldn’t be asked to format a floating point number at some point. There could be other subtle things that hide it from the compiler as dead code. The surest bet would be a compile time feature flag to disable floating point formatting support which it does have. Still, that’s 8kib of string formatting library code without floating point and a bunch of other optimizations which is really heavy in a microcontroller context
- CoastalCoder 2y agoI think this is one scenario where C++ type-templated string formatters could shine. Especially if you extended them to indicate assumptions about the values at compile time. E.g., possible ranges for integers, whether or not a floating point value can have certain special values, etc.
- vlovich123 2y agoYou’d be surprised. I’m pretty sure std::format is templated. That doesn’t mean that it’s still easy to convince the compiler to delete that code.
- josephg 2y ago> it’s highly unlikely this could be trimmed unless all format strings are parsed at compile time They probably should be passed at compile time, like how zig does it. It seems so weird to me that in C & C++ something as simple as format strings are handled dynamically. Clang even parses format strings anyway, to look for mismatched arguments. It just - I suppose - doesn’t do anything with that.
- vlovich123 2y agoThat’s passed at compile time via template arguments and/or constexpr/consteval. Even still, there can be all sorts of reasons a compiler isn’t able to omit something as deeply integrated as floating point formatting from a generic floating point library. Rust handles this more elegantly with cargo features so that you could explicitly guarantee you’ve disabled floating point altogether in a generically reusable and intentional way (and whatever other features might take up space). It’s also important to note that the floating point code only contributed ~44kib out of 75kib but they stopped once the library got down to ~23kib and then removed the c++ runtime completely to shave off another ~10kib. However, it’s also equally important to remember that these shavings are interesting and completely useless: 1. In a typical codebase this would contribute 0% of overall size and not be important at all 2. A codebase where this would be important and you care about it (ie embedded) is not served well by this library eating up at least 10kib even after significant optimization as that 10kib that is intractible is still too large for this space when you’re working with a max ~128-256kib binary size (or even less sometimes).
- vitaut 2y agoIt is indeed possible to remove unused code with techniques like format string compilation but that's a topic for another post.
- jstimpfle 2y agoCurious what space space you work in? What kind of devices, what are they used for?
- londons_explore 2y agoNot me but a friend. Things like making electronics for singing birthday cards and toys that make noise. But there are plenty of other similar things - like making the code that determines the flashing pattern of a bicycle light or flashlight. Or the code that does the countdown timer on a microwave. Or the code that makes the 'ding' sound on a non-smart doorbell. Or the code that makes a hotel safe open when the right combination is entered. Or the code that measures the battery voltage on a USB battery bank and puts 1-4 indicator LED's on so you know how full it is. You don't tend to hear about it because the design of most of this stuff doesn't happen in the USA anymore - the software devs are now in China for all except high-end stuff.
- furyofantares 2y agoDo any of those need a string formatting library?
- deleted 2y ago[deleted]
- toast0 2y agoHotel safe might, if it logs somewhere (serial port?). The others may have a serial port setup during development, too. If you have a truly small formatter, you can just disable it for final builds (or leave it on, asssuming output is non blocking, if someone finds the serial pins, great for them), rather than having larger rom for development and smaller for production.
- londons_explore 2y agomostly used for debugging with "printf debugging" - either on the developers desk, or in the field ("we've got a dead one. Can you hook up this pin to a USB-serial converter and tell me what it's saying?")
- IshKebab 2y agoIt isn't designed to be small; it's designed to be a fully featured string formatting library with size as an important secondary goal. If you want something that has to be microscopic at the cost of not supporting basic features there are definitely better options. > I can assure you that when writing code for microcontrollers with 2 kilobytes of code space, we don't include a 14 kilobyte string formatting library... No shit. If you only have 2kB (unlikely these days) don't use this. Fortunately the vast majority of modern microcontrollers have way more than that. E.g. esp32 starts at 1MB. Perfectly reasonable to use a 14kB formatting library there.
- londons_explore 2y agoWhen you're designing something that sells for a dollar to retailers, eg. a birthday card that sings, your boss won't let you spend more than about 5 cents on the microcontroller, and probably wants you to spend 1-2 cents if you can.
- swagonomixxx 2y agoI kind of get where you're coming from but at what point do we admit that such use cases are the fringe and not the main?
- deleted 2y ago[deleted]
- edflsafoiewq 2y agoPerhaps a singing birthday card doesn't need to format strings.
- nikbackm 2y agoHow else would you get nice looking logs for debugging it?
- 2y ago
- maccard 2y agoWhat do you use instead? Iostream is… far bigger than this, for example.
- Sharlin 2y agoI presume the sort of custom routines that GP described?
- londons_explore 2y agomost platforms come with their own libraries for this, which are usually a mix of hand coded assembly and C. You #include the whole library/sdk, but the linker strips out all bits you don't use. Even then, if you read the disassembled code, you can usually find within a few minutes looking some stupid/unused/inefficient code - so you could totally do a better job if you wrote the assembly by hand, but it would take much more time (especially since most of these architectures tend to have very irregular instruction sets)
- maccard 2y agoIf you’re just going to use the platform built in, then the size of a third party library doesn’t matter to you.
- criddell 2y agoIf you only have 2 kB of code space, you would likely be doing custom routines in assembly that do exactly what you need and nothing more.
- maccard 2y agoRight - so no matter how small libfmt gets Op isn’t going to use it
- secondcoming 2y agoWould someone writing code for a 2Kb microcontroller even be using full-fledged C++, or just C With Classes?
- londons_explore 2y agoIt's still full fledged C++, you just don't use many of the features, and the compiler leaves out all of the associated code. Pretty easy to accidentally use some iostream and accidentally pull in loads of code you didn't want though.
- astrobe_ 2y agoTo me the only reason to use C++ over C in that case is a the slightly stronger type-checking and maybe some extra syntactic sugar.
- formerly_proven 2y agoavrlibc's small variant of printf (which still has a ton of features) is like 600 bytes.
- jeroenhd 2y agoI don't think the requirements for your specific programming niche should influence the language like that. Your requirements are valid, but they should be served by a bottom of the barrel microcontroller compiler rather than the language spec.
- MobiusHorizons 2y agoIt’s relevant because the author mentions microcontrollers as the reason for focusing on binary size.
- Karliss 2y agoThere are many orders of magnitude difference between smallest and higher end microcontrollers. You can have an 8bit micro with <1k of ram and <8k of flash memory, and you can have something with >8MB of flash memory or more running a RTOS possibly even capable of running Linux with moderate effort. In the later case 14k of formatting library is probably fine.
- jeroenhd 2y agoAll of the optimisation work in this article is done for Linux aarch64 ELF binaries. Besides that, microcontrollers have megabytes of storage these days. To be restricted by a dozen or two kilobytes of code storage, you need to be _very_ storage restricted. I have run into code size issues myself when trying to run Rust on an ESP32 with 2MiB of storage (thought I bought 16MB, but MB stood for megabits, oops). Through tweaking the default Rust options, I managed to save half a megabyte or more to make the code work again. The article also links to a project where the fmt library is still much bigger (over 300KiB rather than the current 57KiB). There are microcontrollers where you need to watch out for your dependencies and compiler options, and then there are _tiny_ microcontrollers where every bit matters. For those specific constraints, it doesn't make a lot of sense to assume you can touch every language feature and load every standard library to just work. Much older language features (such as template classes) will also add hundreds of kilobytes of code to your program already, you have to work around that stuff if you're in an extremely constrained environment. The important thing with language features that includes targets like these is that you can disable the entire feature and enable your own. Sharing design goals between x64 supercomputers and RISC-V chips with literal dozens of bytes of RAM makes for an unreasonably restricted language for anything but the minimal spec. Floats are just expensive on minimum cost chips.
- sixfiveotwo 2y ago> I can assure you that when writing code for microcontrollers with 2 kilobytes of code space, we don't include a 14 kilobyte string formatting library... I'm pretty sure you wouldn't use C++ in that situation anyway, so I don't really see your point.
- usrnm 2y agoIf you get rid of the runtime, which most compilers allow you to do, C++ is just as suitable for this task as C. Not as good as hand-rolled assembly, but usable
- sixfiveotwo 2y agoOkay, vtables in 2kb code space?
- CyberDildonics 2y agoWhere are the vtables coming from if you use no inheritance and no memory allocation?
- sixfiveotwo 2y agoWhy program in C++ if you don't use any of its features? You're just writing C in disguise.
- CyberDildonics 2y agoI don't write anything new and use inheritance anyway. Even in an embedded context you have classes and destructors, operator overloading and templates. You can still make data structures that exist in flat constrained memory.
- sixfiveotwo 2y agoSure, but 2kb code space? What sort of class hierarchy can you possibly use in so little space?
- aseipp 2y agoThe design of any library for a microcontroller and an "equivalent" for general end user application is going to be different in pretty much every major design point. I'm not sure how this is any more relevant to fmt than it is just general complaining out in the open. The code for an algorithm like Dragonbox or Dragon4 alone is already blowing your size budget, so the "optional" stuff doesn't really matter. And that's 1 of like 20 features people want.
- ska 2y agoSure, but there are also microcontrollers with a lot more space. This probably won’t ever usefully target the small ones, but that doesn’t mean it isn’t useful in the space at all.
- fsckboy 2y ago>I can assure you that when writing code for microcontrollers with 2 kilobytes of code space, we don't include a 14 kilobyte string formatting library... then the thing to do is publish the libraries you do use, right, then document what formatting features they support? then other people might discover more and clever ways to pack more features in than you thought of otherwise, I don't get your point.
- dinkumthinkum 2y agoWhat? I mean, you realize that fmtlib is much more complicated than that, right? What you are describing is something very basic, primitive by comparison. I’m also puzzled why you think floats are not used by many programs, that’s kind of mind-boggling. I get that you wouldn’t load it on a microcontroller but you wouldn’t do that with the standard library either.
- progmetaldev 2y agoAs someone coming into this programming space, is it common to be running a debugger that can read the float, as opposed to someone brand new to the space printing out a value for debugging. Not specifically comments like yours, but in general the assumption that others should know their tools as strongly as those profess in limited environment spaces, makes it seem that this is a space only for those with a high level of knowledge. That is absolutely not a knock on your knowledge or ability if that is just how things are, I'm sure there are specific areas of computing that require a vast knowledge of how things work without having a library to debug your issues into the way a simple print statement works. I have worked in limited environments, so I am curious specifically about a microcontroller environment like londons_explore specified. I learned to program in Atari 8-bit systems, and know there are limitations on what you can output. londons_explore had a completely valid comment. I was just looking for a perspective for developers getting involved in microcontroller environments, and how they could best debug their code. We all know that a debugger is better than a "print" statement, but not always the fastest, especially with logic. If my answer is just "debugging", and there isn't another, that satisfies me. I am always looking for unique ways developers solve problems in ANY environment, because I would enjoy being able to work in any environment that's best for the solution being provided. I guess I have been blessed with environments where the client is willing to pay for something higher-level than what is actually required. I enjoy all aspects of development across devices, so I hope nobody took my comment as a challenge, it absolutely was not.
- jcelerier 2y ago> strings are ~4 instructions (test for null terminator, output character, branch back two). that's C strings. You also need to handle (size, data) strings like std::string_view For the record, here's what fmt allows (from the docs): fmt::print(fmt::emphasis::bold | fg(fmt::color::red) , "Elapsed time: {0:.2f} seconds", 1.23); fmt::print("Elapsed time: {0:.2f} seconds" , fmt::styled(1.23, fmt::fg(fmt::color::green) fmt::bg(fmt::color::blue))); fmt::print("{}", fmt::join(std::vector<int>{1, 2, 3}, ", ")); fmt::print("strftime-like format: {:%H:%M:%S}\n", 3h + 15min + 30s); you really think you can replicate that in 50 bytes?