3 ms·

Allocations are only half the story, you also want to control where those allocations are located so that your cache access patterns pull in the right chunks of

Allocations are only half the story, you also want to control where those allocations are located so that your cache access patterns pull in the right chunks of memory.

fiddlerwoaroof 10y ago
I'm not really an expert on high performance Lisp, but my impression is that the compilers can optimize things pretty well and that you can have a fairly low level of control over things like allocation patterns. Edit: looks like I misread what you were saying. Yeah, if you want to pack two things next to each other in memory, that might be tricky in the language defined in the ansi standard. Although I think some implementations give you control over this sort of thing.
fiddlerwoaroof 10y ago
For example, the standard itself gives you some control over code generation via compiler macros and individual implementations often give you much more: https://www.pvk.ca/Blog/2014/03/15/sbcl-the-ultimate-assembly-code-breadboard/ https://www.pvk.ca/Blog/2014/03/15/sbcl-the-ultimate-assembl...
sedachv 10y ago
That's what arrays are for. You can also do structure packing using implementation-specific tricks. SBCL's SB-ALIEN package has a nice interface for doing this: http://www.sbcl.org/manual/#Foreign-Types http://www.sbcl.org/manual/#Foreign-Types A really cool example of SB-ALIEN in action is jiyunomegami's native type extensions for the Vacietis C-to-CL transpiler: https://github.com/jiyunomegami/Vacietis/commits/master https://github.com/jiyunomegami/Vacietis/commits/master (commits 5504079d63eb2745f6b8ef7ec1cf9bb3151994eb onward)
- vvanders 10y ago
  Nice, yeah arrays + structure packing looks like it would do the trick.
lispm 10y ago
That depends on the implementation. This is LispWorks 64bit from http://lispworks.com http://lispworks.com . It uses a generational GC, and each generation has several memory segments. You can tell LispWorks how much generations it should use, how large these segments should be and in which generation/segment to allocate. CL-USER 12 > (room t) > Generation 7: 33904936 (0x2055928) Cons 4774352 (0x48D9D0) Non-Pointer 3213752 (0x3109B8) Other 8048192 (0x7ACE40) Symbol 2881680 (0x2BF890) Function 14569120 (0xDE4EA0) Non-Pointer-Static 5528 (0x1598) Mixed-Static 411600 (0x647D0) Weak 712 (0x2C8) -- Segments: Cons 40C0038800 - 40C04C9000 Non-Pointer 40D0000800 - 40D0314000 Other 40E0038800 - 40E07E8000 Symbol 40F0038800 - 40F02FB000 Function 4100038800 - 4100E20000 Non-Pointer-Static 40B0000800 - 40B000D000 Mixed-Static 400004E800 - 40000C8000 Weak 4110038800 - 4110039000 ================================ > Generation 6: 0 (0x0) > Generation 5: 0 (0x0) > Generation 4: 0 (0x0) > Generation 3: 206544 (0x326D0) Non-Pointer 69648 (0x11010) Other 5040 (0x13B0) Symbol 131856 (0x20310) -- Segments: Non-Pointer 4070000800 - 4071001000 Other 4170018800 - 4171019000 Symbol 4080018800 - 4080819000 ================================ > Generation 2: 8768024 (0x85CA18) Cons 1960288 (0x1DE960) Non-Pointer 2531584 (0x26A100) Other 2720024 (0x298118) Symbol 5088 (0x13E0) Function 700184 (0xAAF18) Non-Pointer-Static 850152 (0xCF8E8) Mixed-Static 64 (0x40) Weak 640 (0x280) -- Segments: Cons 41B0010800 - 41B1011000 Non-Pointer 4180000800 - 4181001000 Other 41C0010800 - 41C1011000 Symbol 41D0010800 - 41D0811000 Function 41E0010800 - 41E0811000 Non-Pointer-Static 4040000800 - 4040136000 Mixed-Static 4090010800 - 4090013000 Weak 41F0010800 - 41F0051000 ================================ > Generation 1: 732480 (0xB2D40) Cons 160240 (0x271F0) Non-Pointer 170344 (0x29968) Other 398200 (0x61378) Function 3248 (0xCB0) Weak 448 (0x1C0) -- Segments: Cons 4200008800 - 4200301000 Non-Pointer 4210000800 - 4210411000 Other 4220008800 - 42203F1000 Symbol 4230008800 - 423001B000 Function 4240008800 - 424003B000 Weak 4250008800 - 4250009000 ================================ > Generation 0: 1956832 (0x1DDBE0) Cons 630224 (0x99DD0) Non-Pointer 361232 (0x58310) Other 923816 (0xE18A8) Function 41560 (0xA258) -- Segments: Cons 4010000800 - 401040B000 Non-Pointer 4030000800 - 403041B000 Other 4020000800 - 402040B000 Symbol 4150000800 - 4150101000 Function 4060000800 - 4060101000 Weak 4190000800 - 4190041000 ================================ Total allocated 45568816 (0x2B75330), total size 178085888 (0xA9D6000) NIL

Allocations are only half the story, you also want to control where those allocations are located so that your cache access patterns pull in the right chunks of memory.

fiddlerwoaroof 10y ago

I'm not really an expert on high performance Lisp, but my impression is that the compilers can optimize things pretty well and that you can have a fairly low level of control over things like allocation patterns. Edit: looks like I misread what you were saying. Yeah, if you want to pack two things next to each other in memory, that might be tricky in the language defined in the ansi standard. Although I think some implementations give you control over this sort of thing.

fiddlerwoaroof 10y ago

For example, the standard itself gives you some control over code generation via compiler macros and individual implementations often give you much more: https://www.pvk.ca/Blog/2014/03/15/sbcl-the-ultimate-assembly-code-breadboard/ https://www.pvk.ca/Blog/2014/03/15/sbcl-the-ultimate-assembl...

sedachv 10y ago

That's what arrays are for. You can also do structure packing using implementation-specific tricks. SBCL's SB-ALIEN package has a nice interface for doing this: http://www.sbcl.org/manual/#Foreign-Types http://www.sbcl.org/manual/#Foreign-Types A really cool example of SB-ALIEN in action is jiyunomegami's native type extensions for the Vacietis C-to-CL transpiler: https://github.com/jiyunomegami/Vacietis/commits/master https://github.com/jiyunomegami/Vacietis/commits/master (commits 5504079d63eb2745f6b8ef7ec1cf9bb3151994eb onward)

vvanders 10y ago
Nice, yeah arrays + structure packing looks like it would do the trick.

lispm 10y ago

That depends on the implementation. This is LispWorks 64bit from http://lispworks.com http://lispworks.com . It uses a generational GC, and each generation has several memory segments. You can tell LispWorks how much generations it should use, how large these segments should be and in which generation/segment to allocate. CL-USER 12 > (room t) > Generation 7: 33904936 (0x2055928) Cons 4774352 (0x48D9D0) Non-Pointer 3213752 (0x3109B8) Other 8048192 (0x7ACE40) Symbol 2881680 (0x2BF890) Function 14569120 (0xDE4EA0) Non-Pointer-Static 5528 (0x1598) Mixed-Static 411600 (0x647D0) Weak 712 (0x2C8) -- Segments: Cons 40C0038800 - 40C04C9000 Non-Pointer 40D0000800 - 40D0314000 Other 40E0038800 - 40E07E8000 Symbol 40F0038800 - 40F02FB000 Function 4100038800 - 4100E20000 Non-Pointer-Static 40B0000800 - 40B000D000 Mixed-Static 400004E800 - 40000C8000 Weak 4110038800 - 4110039000 ================================ > Generation 6: 0 (0x0) > Generation 5: 0 (0x0) > Generation 4: 0 (0x0) > Generation 3: 206544 (0x326D0) Non-Pointer 69648 (0x11010) Other 5040 (0x13B0) Symbol 131856 (0x20310) -- Segments: Non-Pointer 4070000800 - 4071001000 Other 4170018800 - 4171019000 Symbol 4080018800 - 4080819000 ================================ > Generation 2: 8768024 (0x85CA18) Cons 1960288 (0x1DE960) Non-Pointer 2531584 (0x26A100) Other 2720024 (0x298118) Symbol 5088 (0x13E0) Function 700184 (0xAAF18) Non-Pointer-Static 850152 (0xCF8E8) Mixed-Static 64 (0x40) Weak 640 (0x280) -- Segments: Cons 41B0010800 - 41B1011000 Non-Pointer 4180000800 - 4181001000 Other 41C0010800 - 41C1011000 Symbol 41D0010800 - 41D0811000 Function 41E0010800 - 41E0811000 Non-Pointer-Static 4040000800 - 4040136000 Mixed-Static 4090010800 - 4090013000 Weak 41F0010800 - 41F0051000 ================================ > Generation 1: 732480 (0xB2D40) Cons 160240 (0x271F0) Non-Pointer 170344 (0x29968) Other 398200 (0x61378) Function 3248 (0xCB0) Weak 448 (0x1C0) -- Segments: Cons 4200008800 - 4200301000 Non-Pointer 4210000800 - 4210411000 Other 4220008800 - 42203F1000 Symbol 4230008800 - 423001B000 Function 4240008800 - 424003B000 Weak 4250008800 - 4250009000 ================================ > Generation 0: 1956832 (0x1DDBE0) Cons 630224 (0x99DD0) Non-Pointer 361232 (0x58310) Other 923816 (0xE18A8) Function 41560 (0xA258) -- Segments: Cons 4010000800 - 401040B000 Non-Pointer 4030000800 - 403041B000 Other 4020000800 - 402040B000 Symbol 4150000800 - 4150101000 Function 4060000800 - 4060101000 Weak 4190000800 - 4190041000 ================================ Total allocated 45568816 (0x2B75330), total size 178085888 (0xA9D6000) NIL