3 ms·

I appreciate the detailed response! You certainly got the gears turning in my brain. A couple points: > What x86 instructions a Virtual Machine/Emulator/Virtua

by rank0 4y ago

I appreciate the detailed response! You certainly got the gears turning in my brain. A couple points: > What x86 instructions a Virtual Machine/Emulator/Virtual Environment permits to be run directly, that is, which x86 instructions, existing in the guest OS, ran directly on the host CPU (without modification or deletion or "hooking"/"intercepting" or what-have-you), could have a large impact on security... How would I even begin to map out which instructions get passed directly to the host cpu? I mean even if we just look at KVM, there's many other pieces of software that go into a virtualization environment. Wouldn't qemu, virsh, or KVM versions/configuration impact which instructions go right to the hardware? We haven't even gotten to the guest config, or any host hardware details! My instinct says it's extremely difficult to answer our question without a team of engineers. > if you're running an x86 emulator on a non-x86 CPU, the option (from the Virtual Machine/Emulator/Virtual Environment's point-of-view) of simply passing on instructions to the lower-level CPU for direct execution -- no longer exists -- in other words, running on a non-x86 CPU (i.e. ARM, RISC-V, etc.) -- the Virtual Machine/Emulator/Virtual Environment now MUST translate or interpret x86 instructions in the guest -- it can no longer ask the underlying host to run them directly, without modification... Makes sense to me. My follow up is now, does paravirtualization effectively create the same translation requirement? If I have an x86 xen-based hypervisor running x86 guests, does the virtual interface provided by xen give us a similar security control that ARM->x86 provides? (Or at least "good enough")?

peter_d_sherman 4y ago
>I appreciate the detailed response! I appreciate the appreciation! But please remember I am not an expert in this area -- only a fellow interested party. A "student" (for lack of a better term) of sorts. I'll tell you everything I know -- but keep in mind that in this setting I am not unlike Hamlet's Horatio -- that is, "There are more things in Heaven and Earth -- that are dreamt of in my philosophy..." :-) >Wouldn't qemu, virsh, or KVM versions/configuration impact which instructions go right to the hardware? They would indeed! >How would I even begin to map out which instructions get passed directly to the host cpu? It's a great question! If I recall correctly, Bochs has/had a compilation option whereby if a flag was set, Bochs would to (at the expense of a lot of speed!) proceed to create a log or report or tallying of some sort -- such that when you exited it, it would show a list of each x86 instruction type that was used during runtime -- in other words, if you ran an OS in Bochs -- you could then get a list of ALL of the x86 instructions it used! Very handy if you're trying to get a new/alien OS to work on an x86 emulator, if/when in early versions of the emulator there might be bugs in emulating specific x86 instructions... QEMU might have (or have had!) the same facility. I am not a QEMU expert, so I don't know. Also, there's at least 3 possible things that can happen to any given instruction: 1) It can go through unmodified; 2) It can go through modified (for example, jumps may have their target address modified; this is because if code portions are expanded because of other modification, jump target locations will change...) 3) It can be intercepted by the VM environment (and/or Hypervisor), where specific code in the VM environment or Hypervisor is executed to replace or emulate the functionality of the instruction... QEMU might have a way to log/count instructions that it intercepts/modifies, albeit, if this exists, this would probably be a compilation-time switch that would need to be set, and performing this would probably slow down guest considerably... >My instinct says it's extremely difficult to answer our question without a team of engineers. My instinct says that your instinct is extremely correct! >My follow up is now, does paravirtualization effectively create the same translation requirement? Again, I'm not an expert. I would guess that it would not be off-limits for the paravirtualizer to engage in translation, but I would also guess that if the paravirtualizer could avoid it for the most part, it would... but those are only guesses... If you really want to understand "how deep the rabbit hole goes" (AFAIK) -- then I would start by going here: http://pascal.hansotten.com/uploads/pascals/pascal-s%20snepscheut.pas http://pascal.hansotten.com/uploads/pascals/pascal-s%20sneps... Now, what is this mess of code, you might ask? Well for this exercise, IGNORE MOST OF IT!. Do a word search for 'procedure interpret', of which part looks like this: procedure interpret; var pc, sp, j, k, n: integer; i: instr; c: char; h: boolean; begin pc:= 0; h:= false; repeat i:= code[pc]; pc:= pc+1; case i.op of add : begin m[sp+1]:= m[sp+1]+m[sp]; sp:= sp+1 end; neg : m[sp]:= -m[sp]; mul : begin m[sp+1]:= m[sp+1]*m[sp]; sp:= sp+1 end; divd : begin m[sp+1]:= m[sp+1] div m[sp]; sp:= sp+1 end; [...20 or so more lines deleted, for brevity...] end until h end; So what is that, and how is it relevant to this discussion you may ask! Ah! You see, that's a Turing-complete machine implemented 20 or so lines of Pascal Code! (in this case, inside of an early pascal compiler, the reason for which being that if you were running binary instructions (such as compiled output!) on an early computer without memory protection, you could easily crash the computer, but if you ran that same program inside of this emulation environment, then that could no longer happen!) This procedure, procedure "Interpret" (terrible name for it, IMHO) -- actually implements one of the first known and simplest Virtual Machines in existence! (There are simpler ones, but they're more academic, so we'll ignore them for now!). Basically that Virtual Machine, procedure "Interpret" -- could (in theory!) run any program that any of today's computers could, everything from DOOM to Spreadsheets to Operating Systems to what-have-you. Why? Because it's Turing-complete! But... so is the x86 chip! (And every other CPU out there!). SP, incidentally, which is used inside of that code -- can be thought of as x86's ESP (32-bit), or RSP (64-bit). It's a stack pointer, that's all it is! Similarly when you see 'm' and brackets [] -- that's just (virtualized!) memory! 'pc' (program counter) = x86's IP (16 bit)/EIP (32 bit)/RIP (64 bit). OK, so now with all of this background, hopefully you see how on the one hand how easy it is to create a Software Virtual Machine (not a hardware one, which is a different animal!) -- but on the other hand, parsing x86 instructions and doing so correctly might require pages and pages of esoteric code! Then we have the (hardware!) problem that later x86 CPU's allow multiple levels of nested virtualization -- that is, if those later x86 chips were designed correctly, then in theory you could have an almost infinite number of nested virtual machines (subject only to memory availability) -- and that's only in Hardware! A software virtual machine (such as procedure Interpret, or any other Turing-Complete software!) -- can in theory run an almost infinite number of nested (software) virtual machines! One inside the other! Subject only to CPU resource availability (the more you try to run, the slower they get, etc.) So between the two (hardware and software), you could have rabbit holes on top of rabbit holes... Which is why I claim "Socratic Ignorance" ("The only thing I know is that I know nothing"). And then of course there's the age-old philosophic question of "Who guards the guardians?" -- in other words, even though a virtualization environment (or any piece of software or hardware on the stack) can add security -- if it isn't properly written/debugged/audited -- it can take security away. In other words, "Who guards the guardians?" Anyway, it's a broad topic area. I'm not the "go-to guy" for this! But great questions!

by rank0 4y ago

peter_d_sherman 4y ago

>I appreciate the detailed response! I appreciate the appreciation! But please remember I am not an expert in this area -- only a fellow interested party. A "student" (for lack of a better term) of sorts. I'll tell you everything I know -- but keep in mind that in this setting I am not unlike Hamlet's Horatio -- that is, "There are more things in Heaven and Earth -- that are dreamt of in my philosophy..." :-) >Wouldn't qemu, virsh, or KVM versions/configuration impact which instructions go right to the hardware? They would indeed! >How would I even begin to map out which instructions get passed directly to the host cpu? It's a great question! If I recall correctly, Bochs has/had a compilation option whereby if a flag was set, Bochs would to (at the expense of a lot of speed!) proceed to create a log or report or tallying of some sort -- such that when you exited it, it would show a list of each x86 instruction type that was used during runtime -- in other words, if you ran an OS in Bochs -- you could then get a list of ALL of the x86 instructions it used! Very handy if you're trying to get a new/alien OS to work on an x86 emulator, if/when in early versions of the emulator there might be bugs in emulating specific x86 instructions... QEMU might have (or have had!) the same facility. I am not a QEMU expert, so I don't know. Also, there's at least 3 possible things that can happen to any given instruction: 1) It can go through unmodified; 2) It can go through modified (for example, jumps may have their target address modified; this is because if code portions are expanded because of other modification, jump target locations will change...) 3) It can be intercepted by the VM environment (and/or Hypervisor), where specific code in the VM environment or Hypervisor is executed to replace or emulate the functionality of the instruction... QEMU might have a way to log/count instructions that it intercepts/modifies, albeit, if this exists, this would probably be a compilation-time switch that would need to be set, and performing this would probably slow down guest considerably... >My instinct says it's extremely difficult to answer our question without a team of engineers. My instinct says that your instinct is extremely correct! >My follow up is now, does paravirtualization effectively create the same translation requirement? Again, I'm not an expert. I would guess that it would not be off-limits for the paravirtualizer to engage in translation, but I would also guess that if the paravirtualizer could avoid it for the most part, it would... but those are only guesses... If you really want to understand "how deep the rabbit hole goes" (AFAIK) -- then I would start by going here: http://pascal.hansotten.com/uploads/pascals/pascal-s%20snepscheut.pas http://pascal.hansotten.com/uploads/pascals/pascal-s%20sneps... Now, what is this mess of code, you might ask? Well for this exercise, IGNORE MOST OF IT!. Do a word search for 'procedure interpret', of which part looks like this: procedure interpret; var pc, sp, j, k, n: integer; i: instr; c: char; h: boolean; begin pc:= 0; h:= false; repeat i:= code[pc]; pc:= pc+1; case i.op of add : begin m[sp+1]:= m[sp+1]+m[sp]; sp:= sp+1 end; neg : m[sp]:= -m[sp]; mul : begin m[sp+1]:= m[sp+1]*m[sp]; sp:= sp+1 end; divd : begin m[sp+1]:= m[sp+1] div m[sp]; sp:= sp+1 end; [...20 or so more lines deleted, for brevity...] end until h end; So what is that, and how is it relevant to this discussion you may ask! Ah! You see, that's a Turing-complete machine implemented 20 or so lines of Pascal Code! (in this case, inside of an early pascal compiler, the reason for which being that if you were running binary instructions (such as compiled output!) on an early computer without memory protection, you could easily crash the computer, but if you ran that same program inside of this emulation environment, then that could no longer happen!) This procedure, procedure "Interpret" (terrible name for it, IMHO) -- actually implements one of the first known and simplest Virtual Machines in existence! (There are simpler ones, but they're more academic, so we'll ignore them for now!). Basically that Virtual Machine, procedure "Interpret" -- could (in theory!) run any program that any of today's computers could, everything from DOOM to Spreadsheets to Operating Systems to what-have-you. Why? Because it's Turing-complete! But... so is the x86 chip! (And every other CPU out there!). SP, incidentally, which is used inside of that code -- can be thought of as x86's ESP (32-bit), or RSP (64-bit). It's a stack pointer, that's all it is! Similarly when you see 'm' and brackets [] -- that's just (virtualized!) memory! 'pc' (program counter) = x86's IP (16 bit)/EIP (32 bit)/RIP (64 bit). OK, so now with all of this background, hopefully you see how on the one hand how easy it is to create a Software Virtual Machine (not a hardware one, which is a different animal!) -- but on the other hand, parsing x86 instructions and doing so correctly might require pages and pages of esoteric code! Then we have the (hardware!) problem that later x86 CPU's allow multiple levels of nested virtualization -- that is, if those later x86 chips were designed correctly, then in theory you could have an almost infinite number of nested virtual machines (subject only to memory availability) -- and that's only in Hardware! A software virtual machine (such as procedure Interpret, or any other Turing-Complete software!) -- can in theory run an almost infinite number of nested (software) virtual machines! One inside the other! Subject only to CPU resource availability (the more you try to run, the slower they get, etc.) So between the two (hardware and software), you could have rabbit holes on top of rabbit holes... Which is why I claim "Socratic Ignorance" ("The only thing I know is that I know nothing"). And then of course there's the age-old philosophic question of "Who guards the guardians?" -- in other words, even though a virtualization environment (or any piece of software or hardware on the stack) can add security -- if it isn't properly written/debugged/audited -- it can take security away. In other words, "Who guards the guardians?" Anyway, it's a broad topic area. I'm not the "go-to guy" for this! But great questions!